What Is a Backtest? (And Why You Should Distrust a Beautiful One)

A backtest replays a trading rule against historical data and tallies what would have happened — no money at risk, no future involved. We publish our own, including the hypothetical one that failed, at our public record. Research and education only — not financial advice.

The plain-English definition

So, what is a backtest? Write a trading rule down precisely enough that a computer can follow it with no judgment calls — "enter when X, exit when Y, size it like Z" — then replay history and count what would have happened. That count is the backtest. No money moves. No orders reach a market. It is what-if accounting performed on the past, and it is the standard first filter a quantitative desk runs before an idea earns any further attention.

Two things about that definition do most of the work. First, the rule must be mechanical: if a strategy needs a human to "just know" when it applies, it cannot be backtested, only narrated. Second, the output is a claim about history, not a promise about the future. A backtest is an argument that the past would have paid you — dressed in enough decimal places to look like a guarantee. It is not one.

What a backtest can tell you — and what it cannot

Can: whether a rule is even precise enough to test. Forcing an idea into code kills a surprising number of them on contact.
Can: how often the rule fires, how long trades last, and the shape of wins versus losses — the raw material for metrics like profit factor and expectancy.
Can: how ugly the worst historical stretch was, which is usually more useful than how pretty the best one was.
Cannot: whether the future will resemble the slice of past you tested. Markets change regimes without filing paperwork.
Cannot: your actual fills. Historical prices are not offers; spreads, slippage, and thin quotes all live outside the simulation.
Cannot: your behavior. A backtest never panics at 9:31am. You might.

The three classic ways backtests lie

Lookahead bias

Lookahead bias means the simulated strategy used information that did not exist yet at the moment of the simulated decision. The textbook version: a rule that decides a morning entry using that day's closing price. Subtler versions hide in data itself — fundamentals as later revised, index membership as it stands today, earnings dates backfilled after the fact. Any of these hands the strategy a small time machine, and time machines test extremely well.

Survivorship bias

Run a strategy across "all S&P 500 stocks" using today's member list and you have quietly excluded every company that got delisted, acquired at a discount, or went to zero along the way. The corpses are missing from the dataset. "Buy every dip" is a magnificent strategy in a universe where nothing ever dies — which is exactly the universe a survivorship-biased backtest simulates.

Overfitting

Overfitting is tuning a rule's parameters until history flatters it. The space of testable configurations is enormous — tweak the stop, the target, the holding period, the filter threshold — and with enough variants, something will look brilliant in hindsight for reasons that are pure noise. Of the three, this is the one we can illustrate from our own files.

The backtest we published — and the cell we rejected

Our desk ran its own raw scanner output through a hypothetical backtest: every card the scanner generated, entered at the trigger, exited at the stated levels, zero human discretion. The simulated result was not flattering, so naturally we published it.

Metric	Raw scanner, traded blind (hypothetical)
Simulated trades	161
Win rate	46.6% (simulated)
Profit factor	0.82 (simulated)
Expectancy	≈ −2% per trade (simulated)

A simulated profit factor of 0.82 means the paper model gave back more than it made. That was the honest baseline. Then came the seductive part: we tested a 21-variant grid of exit rules over the same entries, and the best-looking cell showed +362 simulated units — a number that would photograph beautifully on a sales page. Our own statistical audit rejected it. One ticker accounted for 61% of the simulated profit, and the result failed significance testing. The "strategy" was mostly one stock having a good month, discovered after the fact by a grid search built to find exactly that. The full methodology and the rest of the audit live at our public record.

Rule of thumb: a backtest impressive enough for marketing is impressive enough to be overfit. The more informative question for any desk — including ours — is what its testing process rejected. A desk with no rejected results has not audited itself.

Why "hypothetical" is mandatory, not decoration

The house rule here is that a performance number carries a hypothetical, simulated, or paper label in the same sentence — and that is not style for its own sake. Backtested results have structural limitations that no honesty can remove: there were no real fills, no slippage, no liquidity constraints, no fear, and — decisively — the rules were designed after the outcomes were already known. Regulators in the U.S. have required prominent warnings on hypothetical performance for decades precisely because simulated records are so routinely better than the live trading that follows them.

Label check: if a service quotes a backtested win rate, return, or profit factor without a hypothetical or simulated label in the same sentence, that omission tells you more about the service than the number does.

How to read anyone's backtest

The practical version of "what is a backtest" is "what should I check before believing one." Six questions cover most of it:

Trade count. 161 simulated trades is a modest sample; 20 is an anecdote.
Concentration. Ask what share of the simulated profit came from the single best ticker or trade. Our rejected cell failed at 61%.
Out-of-sample evidence. Was any data held back that the rule never saw during tuning? If everything was in-sample, you are looking at a description of the past, not a test.
Costs. Were spreads and slippage modeled at all? On short-dated options they can be the whole result.
The reject pile. How many variants were tried, and where are the failures? One survivor out of 21 grid cells is a different object than one honest test.
A forward record. The only follow-up that counts: calls timestamped before the move, wins and losses on the same board.

From backtest to forward record

A backtest asks history for an opinion; a forward test asks the future, one timestamped card at a time. The intermediate step is paper trading — running the rules live without money, which removes hindsight but not the other simulation caveats. Our desk's answer to the whole problem is procedural: each session's full-market scan runs through a catalyst check, an adversarial review, and a liquidity screen, and the survivors post as trigger-based cards to a public, timestamped paper record before the move — losses stay on the board, corrections go up in the open. The same checklist applies when vetting any signal room: the backtest gets a strategy into the conversation. Only the forward record keeps it there.

Common questions

What is a backtest in trading?

A backtest is a simulation that applies a precisely defined trading rule to historical market data and tallies what would have happened — trade count, win rate, profit factor, drawdowns. No real money is involved; it is what-if accounting on the past, used as a first filter before an idea gets further testing.

Are backtest results real profits?

No. Backtested figures are hypothetical: no real orders, no slippage, no liquidity constraints, no emotions, and rules designed after the outcomes were known. That is why a backtested number should carry a hypothetical or simulated label in the same sentence — the rule this desk applies to its own numbers.

What is lookahead bias in a backtest?

Lookahead bias is when a simulated strategy uses information that was not available at the moment of the simulated decision — for example, using a day's closing price to trigger that morning's entry, or using today's index membership for a test set in the past. It inflates results because the strategy is effectively trading with a time machine.

What is overfitting and how do you detect it?

Overfitting is tuning parameters until a rule fits historical noise instead of a real effect. Warning signs: many variants tested with only the best reported, profit concentrated in one or two tickers, and results that fail on held-out data. Our own 21-variant exit grid produced a cell showing +362 simulated units that our statistical audit rejected — one ticker was 61% of the simulated profit.

Why do hypothetical results carry mandatory disclaimers?

Because simulated performance is routinely better than the live trading that follows it — the rules were built with hindsight and the simulation skips fills, costs, and human behavior. U.S. regulators have required prominent hypothetical-performance warnings for decades for exactly this reason.

See the process with your own eyes. The desk posts trigger-based cards to a public, timestamped record — losses included — and published the backtest where its own raw scanner loses. The scoreboard is free to watch. Join the floor →

Free to join · paid floors optional · research and education only

Last updated 2026-07-11 · ClaudeQuantAlgo Research Desk · research and education only.

Disclosures. ClaudeQuantAlgo is a research and education community. Nothing on this page constitutes financial, investment, legal, or tax advice, or a recommendation to buy or sell any security or derivative. No profit promises are made, ever — trading stocks, options, and forex involves substantial risk of loss; options positions can lose 100% of their value. The public scoreboard reflects a model ("paper") desk — no real money. Past performance — real, paper, or simulated — never guarantees future results. Hypothetical and simulated results have inherent limitations and no representation is made that any account will or is likely to achieve similar profits or losses. ClaudeQuantAlgo is not a registered investment adviser or broker-dealer. You are solely responsible for your own trading decisions. Never risk money you cannot afford to lose.