Somewhere in your phone right now is a forecast that says there's a 30% chance of rain tomorrow. Here is the strange part: when meteorologists say 30%, it rains on about three out of ten of those days. That isn't luck. It's one of the few places in public life where people who speak in confident numbers have been checked against reality, again and again, for decades. Almost no one else gets that treatment. Not your boss, not the pundit on TV, and certainly not you.
Think about the last time you said "I'm sure" about something that hadn't happened yet. This launch will be fine. She'll say yes. I'll be done by Friday. We'll never speak again after this. Did you ever go back and check how often "I'm sure" was right?
Most of us haven't. And that gap, between how certain we feel and how often we're correct, is where a surprising amount of quiet damage happens.
The gap has a name: calibration
Calibration is simple to define. If you could gather every time you said you were 90% confident, about nine in ten of those things should have turned out true. If only six did, you're overconfident. If nearly all did, you're underconfident, which is rarer, but it exists.
The classic finding in this field comes from decades of experiments, including work by Sarah Lichtenstein, Baruch Fischhoff and Paul Slovic. They asked people general-knowledge questions and had them rate how sure they were in their answers. The pattern was consistent: confidence ran higher than accuracy. In one well-known version, answers people labeled as certain turned out wrong a surprisingly large share of the time. The feeling of certainty simply isn't a reliable gauge.
Why not? Because certainty is a feeling, produced by how easily a story comes to mind. A tidy story feels true. It doesn't need to be checked to feel that way.
Why experts aren't exempt
It would be comforting to think this is a problem for amateurs. Philip Tetlock's long study of expert political judgment, published in 2005, suggests otherwise. He collected thousands of forecasts from professionals over many years and scored them. The headline result, in plain terms: experts were often not much better than simple baselines, and many were more confident than their records justified. The ones who did worse tended to be those with one big theory of how the world works. The ones who did better were more like foxes than hedgehogs, in Isaiah Berlin's old image: they held many small ideas loosely and changed their minds.
Tetlock's later project, the Good Judgment Project, ran forecasting tournaments for a US intelligence research program. A small group of "superforecasters" consistently beat the field. What set them apart wasn't genius or insider access. It was habits: breaking questions into pieces, thinking in probabilities rather than yes or no, updating in small steps, and, crucially, keeping score.
Why the forecasters got good
Back to the weather. Research on forecaster accuracy going back to the 1970s (Allan Murphy and Robert Winkler studied this) found professional weather forecasters to be remarkably well calibrated. The explanation is nearly boring: they make a prediction every day, the outcome arrives within hours, and the result is recorded. A Brier score, a method for grading probability forecasts developed by Glenn Brier in 1950 for exactly this purpose, turns each forecast into a number.
Prompt, unambiguous, repeated feedback. That's the whole secret.
The rest of us live in the opposite environment. Our predictions are vague ("it'll probably work out"), the outcomes arrive months later, and by then memory has quietly rewritten what we thought. Hindsight makes the result feel like we expected it all along. You can't learn from feedback you never received, and you can't receive feedback on a prediction you can't recall making.
What a prediction log actually looks like
The fix is not a spreadsheet of statistics. It's a habit of writing down small guesses, with numbers, before you know the answer. Each entry has four parts:
- The claim, specific enough that it will clearly be true or false. Not "the meeting goes well" but "my manager approves the budget request in Thursday's meeting."
- Your confidence, as a percentage from 50 to 99. (Below 50 just means you'd flip the claim.)
- A resolve-by date, so the question can actually close.
- One line on why, which is the part you'll be glad to have later.
When the date arrives, you mark it true or false. That's all. After twenty or thirty entries, group them: everything you rated 60-70%, everything you rated 80-90%, and so on. Count how many came true in each group.
The first time most people do this, they find something humbling and useful. Their 90% bucket comes in at 70%, say. That isn't a verdict on their intelligence. It's a correction factor. Now when your gut says "I'm sure," you have actual evidence for how much to trust it, and you can adjust.
What changes when you do this
The benefits are less about winning predictions and more about how you treat uncertainty.
You start to notice which kinds of things you're overconfident about. For many people it's their own timelines and other people's reactions. For others it's money, or health, or how long a feeling will last. A pattern appears, and patterns can be worked with.
You also get gentler about being wrong. A 75% prediction that fails isn't an embarrassing mistake; it was supposed to fail one time in four. If your 75% predictions fail a quarter of the time, you're doing it right. Probabilistic thinking quietly loosens the all-or-nothing relationship many of us have with being correct.
And you get better at disagreeing without a fight. "I'd say 60%, you say 90%, let's write it down" is a much kinder conversation than "you're wrong."
Your next moves
- Write five predictions today, about things that resolve within two weeks. Use real stakes from your life: whether a reply arrives by Friday, whether you'll finish a task by a given day, whether a friend will confirm plans. Give each a percentage.
- Make each claim checkable. Rewrite any entry that you couldn't clearly mark true or false. If you'd have to argue about whether it counts, it's too fuzzy.
- Add the reason in one sentence. "I'm at 80% because she's replied within a day every time so far." Later you'll see which reasons held up.
- Put the resolve dates on your calendar. Set a recurring Sunday reminder, five minutes, to mark what came true. Without a trigger, the log quietly dies.
- After 20 resolved entries, sort and count. Group by confidence level and compare each group's confidence to its actual hit rate. Write one sentence about the biggest gap, and carry that adjustment into your next batch.
Where this lives
The log only works if adding an entry takes seconds and finding the old ones is effortless. If recording a guess takes more effort than the guess itself, you'll stop at day four. A fast notes app with a simple list or a light table (claim, percent, date, outcome) is plenty. Pagebox opens in under a second, works offline, and keeps a lightweight database where the columns above fit naturally, so you can log a prediction in the time it takes to have the thought, then sort and count when the dates come due.
You don't need the app to do this; a paper notebook works. But if you'd like somewhere quick to keep it, you can look at Pagebox. Either way, start with five guesses today. The forecaster's advantage was never talent. It was a world that told them, every day, how often they were right.