learning spend needs a chain of evidence

Learning budgets are defended with attendance and satisfaction. Almost everyone in the function knows this is inadequate. Very few organisations have replaced it, and the reason is not laziness.

The CIPD's Learning at Work 2023 survey asked 1,108 learning professionals about their practice. Only 50 per cent agreed they had a process in place for assessing learning impact. When the same respondents were asked about their wider organisation, only 24 per cent said their organisation assesses the impact of learning and development. Only 29 per cent said line managers were involved in assessing it. Only 36 per cent said line managers supported their teams in transferring what they had learned back into the workplace.

The academic record shows the same shape. Arthur, Bennett, Edens and Bell published a meta-analysis of organisational training in the Journal of Applied Psychology in 2003, covering 397 effect sizes. Learning criteria accounted for 234 of them. Behavioural criteria accounted for 122. Results criteria accounted for 26.

Training worked, and it worked at similar magnitude across all four criterion levels. The effect sizes were 0.60 for reaction, 0.63 for learning, 0.62 for behaviour and 0.62 for results. The problem was never that training does nothing. The problem is that hardly anyone measures the last link.

be accurate about the two models

Most arguments about learning measurement are conducted with a caricature of both frameworks. It is worth getting them right, because the useful parts are the parts that get dropped.

Donald Kirkpatrick's four levels, first published in the late 1950s, are Reaction, Learning, Behavior and Results. The common criticism, that the model encourages stopping at level one, is not a fair reading of the model. It is a fair reading of how the model gets used.

The New World Kirkpatrick Model adds the parts that make the chain work. It plans backwards from level four to level one, so the business result is defined before the course is designed. It adds required drivers, the processes and accountability in the work environment that reinforce and reward the behaviour. It adds leading indicators, short-term observations that suggest the critical behaviours are producing the intended result. It also offers Return on Expectations, a standard agreed with stakeholders in advance, as an alternative to a monetary ratio.

The Phillips ROI Methodology is often described as Kirkpatrick with a fifth box bolted on. That description misses the two additions that matter. Its six levels of data are Level 0 Input, Level 1 Reaction and Planned Action, Level 2 Learning, Level 3 Application and Implementation, Level 4 Impact, and Level 5 ROI, with intangible benefits reported alongside.

The substantive additions are elsewhere. The methodology includes an explicit step to isolate the effects of the programme from everything else moving at the same time, using control groups, trend-line analysis, forecasting, participant estimates or expert input. It also requires impact data to be converted to monetary value before any ratio is calculated. Skip the isolation step and level five is arithmetic performed on an assumption.

the five-link chain

A credible claim that learning produced a business outcome has five links. Each one is a separate piece of work, and a chain with a missing link is not a weaker argument. It is not an argument.

1. A named business measure, owned outside L&D. Not "improved capability". A number a business leader is already accountable for, written down before design starts. If no leader will put their name to it, the programme has no destination.
2. An observable behaviour that moves it. Stated as an action at a frequency, visible to a third party. "Runs a structured discovery call before proposal" is a behaviour. "Understands consultative selling" is not.
3. A capability the programme builds, tested at the point of use. Not a quiz at the end of the module. A demonstration in conditions resembling the job.
4. A driver in the environment that permits the behaviour. Manager reinforcement, tooling, authority, time. The CIPD numbers on line manager involvement suggest this is the link that breaks most often, and it is the cheapest to fix.
5. A comparison that makes the number arguable. Control group, trend line, or estimation with a stated confidence discount. Without this, you have a correlation and a hope.

instrumenting it without a research department

Each link has a version that a two-person team can execute.

use staggered rollout as a free control group

This is the highest-return move available and it costs nothing but scheduling discipline. Cohort one runs in January, cohort two in April. For three months you have two comparable populations, one trained and one not. Control groups are the most credible way to isolate an effect, and phased rollout is how you get one without denying anybody the programme.

pick a behaviour an existing system already records

Do not build a new observation survey. Find a behaviour that leaves a trace in a system you already run: notes logged in the CRM, tickets reassigned rather than escalated, reviews completed on time, near-misses reported, second interviews scheduled. Behavioural data you have to ask for decays. Behavioural data that falls out of the work does not.

discount your estimates in public

Where no control group is possible, the Phillips approach asks participants or managers to estimate the share of an improvement attributable to the programme, then adjusts that estimate by a stated confidence percentage. Report it in full: a 40 per cent attribution at 70 per cent confidence gives 28 per cent. The discipline is that the discount is visible. A finance director will accept a conservative number with its assumptions shown. They will not accept a confident number with its assumptions hidden.

use the success case method where numbers cannot reach

Robert Brinkerhoff's Success Case Method is built for exactly the programmes that resist quantification. Survey all participants briefly to find who is applying the learning and who is not. Then interview the extremes: the highest adopters and the lowest. You are not looking for an average. You are looking for the conditions that separate a person who used the training from a person who did not. Those conditions are almost always environmental, and they are directly actionable.

agree the evidence standard before you start

Most disputes about learning value are disputes about the evidence standard, held after the results are in. Ask the sponsor one question at the outset: what will you accept as evidence that this worked? Write the answer into a one-page evaluation contract naming the business measure, the behaviour, the comparison method, the reporting date and the decision the evidence will inform. This is Return on Expectations, and it settles the argument in advance.

a triage rule, because not everything deserves a chain

Instrumenting every programme to level four is not a serious plan. It is how measurement initiatives collapse.

Apply a simple rule. Build the full chain for the three programmes with the largest spend or the highest risk. For everything else, stop at link three and measure application: is the behaviour happening in the work, six to twelve weeks out. The meta-analytic evidence supports this triage. Effect sizes at behavioural criteria were essentially the same as at results criteria, which means measuring behaviour is a reasonable proxy rather than a consolation prize.

Reaction data still has a use, and it is worth stating plainly. Perceived relevance predicts whether people attempt the behaviour. Keep the question about relevance. Drop the question about the catering.

the honest constraint

The reason this does not happen is rarely methodological. It is that the business measure sits in one system, the behavioural trace in another, the learning record in a third, and nobody owns joining them. LinkedIn's 2025 Workplace Learning Report found that engagement and retention remain the most common ways organisations measure the business impact of career development, largely because those are the numbers that are already joined up.

Connecting learning records to the work systems where behaviour leaves its trace is the practical problem, and it is what entomo's learning ROI approach is designed to solve.

You do not need a research department. You need one named business measure, one observable behaviour, a staggered rollout, and a written agreement about what counts as proof. Four things, agreed before the first cohort starts. That is the difference between defending a budget and proving a return.

see how entomo does this: learning ROI.
in practice: Japan IT services, skills framework 80% faster.