"The computer lies. Thousands of hours of simulation give persistently wrong results." Stanford mathematician Persi Diaconis said that in a lecture at the University of Washington on 1 November 2024, titled "The Value and Pitfalls of Proof." It sounds like a broadside against all simulation. It isn't — and the honest version is more interesting than the one going around. It's worth a look at what Diaconis actually studies, and what he actually said.

Who Diaconis is — and why you listen

Diaconis isn't a contrarian dropping a hot take. He's the world's leading figure on how fast a random process "settles" — how long it takes to forget where it started. His most famous everyday result: how many riffle shuffles randomize a deck of cards? About seven. After six it looks shuffled but isn't — patterns survive that a sharp player can exploit.

That's the whole idea in one image: something can look finished before it is.

What "the computer lies" actually means

Diaconis's field is rates of convergence. A simulation that samples from a probability distribution can take far longer to settle than intuition suggests — sometimes astronomically longer. And here's the trap: during that long run-up, the simulation looks stable and correct. It reports tidy numbers. The numbers are wrong.

That's what "the computer lies" means — not a coding bug, but a process that hasn't finished and pretends it has. It's called pseudo-convergence.

The half that usually gets cut

Now the part that almost always disappears in the retelling. In the same lecture Diaconis says the opposite too: people who prove theorems can prove useless, irrelevant things that choke off research and miss what's interesting. His actual thesis is balanced — simulation and proof each have value and pitfalls. There is room for both.

So he isn't saying "scrap simulation, prove instead." He's saying: know which tool lies to you, and when. Quote only the "computer lies" half and you turn a balanced mathematician into a crusade.

What this means for backtests — and what it doesn't

Here's where honesty matters, because the jump "every backtest is a simulation, so this applies" is too fast. Three different things get mixed up:

A historical backtest doesn't "mix badly." It replays a strategy over the real past. It lies differently — through look-ahead, overfitting, samples too small, regime dependence, ignored costs. Diaconis's specific pathology (slow mixing) rarely touches it.

But the meta-lesson lands fully: more compute doesn't manufacture truth, and "looks stable" isn't the same as "is right."

The honest carry-over

The tempting conclusion "proof instead of simulation" is a false choice. For real strategies there's no closed-form proof of profit — that's exactly why you simulate. A backtest is a simulation; you don't abandon it. You do two honest things instead:

First, simulate so it can't lie about the past. Look-ahead-free, point-in-time — the past measured with the data actually available then.

Second, prove what's provable. Not that a strategy makes money — that the method is clean. A prefix-invariance test is exactly that kind of proof: it guarantees, mechanically and across the whole history, that the calculation can't peek at the future. That's Diaconis's "value of proof," applied where it fits — to the integrity of the method, not the profitability of the trade.

And small samples stay what they are: under 30 cases is an anecdote. Run a thousand backtests and you don't get truth a thousand times over — you get a thousand chances to fool yourself. Which is why performance measures exist that penalise the number of tries.

Why the pattern matters

Why this matters beyond one lecture: the same idea keeps getting flattened. Either into "proof beats simulation" — a crusade Diaconis didn't call for. Or bolted onto the backtest, where the specific pathology doesn't apply. Both borrow a real mathematician's authority for a claim he never made.

The honest reading is narrower and more useful: compute is not truth. Looking converged is not being converged. And a wrong method, run faster, is still wrong.

FAQ

Did Diaconis say we should scrap simulations? No. His thesis is balanced: simulation and proof each carry value and pitfalls. There's room for both.

Does his convergence result apply to every backtest? No. It's about Markov-chain simulations. A historical backtest has no mixing-time problem — it has look-ahead, overfitting, and small samples instead. The meta-lesson (compute isn't truth) still holds.

What is pseudo-convergence? When a simulation run looks stable and correct even though it hasn't settled yet. The numbers look solid and are wrong.

This post is an analytical read, not investment advice.

Study the Past — Improve your Future. 🥋