Reference-class forecasting and base rates

Cerno is a private AI boardroom that runs consequential decisions through a structured process and hands you an Executive Decision Brief. Reference-class forecasting is one of the lenses it applies; here's the method itself, where a small business finds a base rate, and how the method goes wrong.

What is reference-class forecasting?

Developed from the work of Daniel Kahneman and Amos Tversky, it corrects a predictable bias: when we estimate, we take the inside view — building up from the details of our specific plan — and consistently underestimate cost, duration and failure. Kahneman called the underlying error the planning fallacy, and it survives experience: people who have been late on every project they've run still expect the next one to be on time.

The fix is to take the outside view: find a reference class of similar past cases and start from how they actually went. If comparable expansions took 40% longer than planned, your expansion probably will too, however special yours feels. The method was later formalised by Bent Flyvbjerg for large projects, but the logic is the same at any scale.

Why the inside view fails

It isn't stupidity. The inside view fails for structural reasons that apply to careful people too.

You can see your plan; you can't see the surprises. A plan is a list of things that will happen. The things that make projects late are the ones that aren't on the list — and by definition you can't put them on it. The base rate includes them, because it's built from projects where they happened.

Your case always looks different. Every project has specifics, and specifics feel like reasons the average won't apply. But the projects in the reference class had specifics too. "Ours is different" is what everyone in the base rate said.

Optimism is baked into the estimate before you start. The plan you've built is the plan you've built because you think it'll work. Estimating from it is asking the plan's author how good the plan is.

When should you use it?

Use it for any estimate that matters — timelines, budgets, adoption, success odds — especially when you're tempted to think "our situation is different." The more confident and specific your inside-view number, the more you need the base rate as a check.

It matters most on one-way doors, where being wrong about the estimate is expensive to reverse. For a reversible decision, a rough guess and a quick review date will do.

How to run it in 5 steps

1. ›Define the decision as a prediction. "This project will take X months / cost £Y / succeed with Z% odds." If you can't state it as a number or a range, you can't check it.
2. ›Pick a reference class. A set of genuinely comparable past cases — yours and others', broad enough to have a real base rate, narrow enough to be about the same kind of thing.
3. ›Find the base rate. How did that class actually turn out on the dimension you care about? Not how they were planned to — how they did.
4. ›Start from the base rate. Make it your anchor, not your optimistic plan. If the base rate says four months and your plan says three, four is your starting number.
5. ›Adjust — cautiously. Move off the base rate only for specific, evidenced reasons, and less than instinct tells you. "We're more experienced" is worth something; it is rarely worth the whole gap.

Worked example

Considering a market entry, the base rate reframes the confidence:

Where a small business finds a base rate

The standard advice assumes you can look up a study. Most small-business decisions have no study. There are still base rates; they're just closer to home.

Your own records. The most reliable reference class you have is your own past. Pull up the last ten jobs: quoted duration against actual, quoted cost against actual. Most people who do this for the first time discover they have been consistently optimistic by a stable percentage — and that percentage is the base rate for their next quote.

People who see many businesses like yours. Your accountant sees dozens of businesses your size and knows roughly what happens when they take on a second premises. A supplier who serves your competitors knows how many of them tried the thing you're considering. A trade body publishes what it publishes. None of these is a study; all of them are better than your plan.

The local, honest version. Of the businesses like yours that opened a second site, hired a first employee, or moved upmarket within a few miles of you in the last five years — how many are still doing it? You can often count. It's crude, and it's a real base rate.

Three peers. Ask three people who've done the thing how long it actually took and what it actually cost. You'll get three inside views, which together are a small outside view. Ask specifically for the number, not the story — the story is always the plan.

A second example: the quote

A joinery business quotes every fitted-kitchen job at six weeks, because that is how long a fitted kitchen takes when nothing goes wrong. The owner pulls up the last ten jobs. Actual durations: seven, six, nine, eight, six, eleven, seven, eight, seven, ten. Average: just under eight weeks. Not one came in at six.

The inside view — "six weeks, because that's how long the work takes" — was never a forecast. It was a description of the plan. The base rate says eight, and it says that the one time in ten it runs to eleven is not bad luck; it's part of the distribution.

The business now quotes eight weeks, with a note that one job in ten runs longer. Customers who would have been furious at week seven are now pleased at week seven. Nothing about the joinery changed.

Common mistakes

A reference class that's too narrow. "Kitchens we fitted in a house exactly like this one" gives you a class of two. A class that small has no base rate, only anecdotes. Widen until you have enough cases for the average to mean something.

A reference class that's too broad. "All building projects" includes motorways. Narrow until the cases are the same kind of thing — same scale, same sort of customer, same sort of work.

Adjusting away the whole gap. The base rate says eight weeks; your plan says six; you reason that your team is good and settle on six and a half. You have just reinstated the inside view with extra steps. Adjust for specific, evidenced differences, and expect the adjustment to be small.

Reading the base rate as a verdict. "Most first-time market entries fail" is not "don't enter the market". It is "enter in a way that survives being in the majority": a smaller test, a tripwire at the point where entries typically fail, a budget that assumes the typical overrun.

Confusing typical with inevitable. The base rate is the centre of a distribution, not a prophecy. Some jobs come in early. The point is to plan for the centre and know the shape, not to assume the worst.

Which Cerno advisor uses this?