Backtesting ArenaBacktesting Arena
Evidence

Falsification Register

Every question this platform fixed in advance and then measured is listed here — with its outcome, including the claims that fell. Most of them fell. That is the price a claim has to pay before it gets to be one.

11
Entries
6
fell
1
held
2
not answerable
2
dissolved

Locked provably before the first outcome number: 5 of 11. 5 were locked in the same commit as their result, 1 recorded after the fact without a lock — they are listed anyway, marked, instead of silently missing.

dissolvedrule: noise_floormultiday-grid-3d-v1

Is the lead of the 3-day interval (and the 2d-versus-3d gap) a finding — or smaller than the noise of where the 3-day candle grid happens to be anchored?

The question dissolved on re-measurement (rule: noise_floor). 1 of 1 decision gates not met.

Preregistration:
2026-07-30 · 46af74c · recorded after the fact
Result:
2026-10-12 · 46af74c
Data window:
2017-08-17 – 2026-07-29
n:
15 (effective 5)
Decision gates (1)
  • not met · |median gross CAGR 2d − 3d| ≥ mean grid-phase spread · spread 6.66 pp (max 12.30) · 0.30
Limits (5)
  • Recorded after the fact: no preregistration; the reading (gap below the grid-phase spread ⇒ nothing to explain) was set with the first measurement on 2026-07-30.
  • Grid phase re-run on 2026-10-12 with the fixed end date 2026-07-29: mean spread 6.66 pp, max 12.30 pp over 5 strategies on BTCUSDT — identical to the first measurement.
  • Interval gap measured anew on 2026-10-12 (docs/research/multiday-grid/interval-distance-v1.json): 217971 runs, 4305 combinations present in all intervals; median gross 2d -2.10 %, 3d -1.80 % ⇒ gap 0.30 pp. The first measurement used a much smaller pool and is not reproducible; today 3d sits above 2d.
  • Two different populations: the spread comes from five strategies on BTCUSDT, the gap from all crypto pairs. The comparison asks whether a gap of this size could be produced by an arbitrary anchor alone.
  • Weekly and monthly intervals rank above 3d gross today, but with a median of about two trades or fewer per cell; they are not part of the 2d/3d question.
Full record →
fellrule: prereg_testmaxpain-distance-v1

Does the distance between spot and max pain three days before a Deribit BTC expiry relate to the move until settlement — does price get pulled toward max pain, or run away from it?

The claim fell (rule: prereg_test). Spearman ρ +0.339, 95 % interval [−0.038; +0.647], permutation p 0.060 on 31 overlap-free cases (α 0.05).

Preregistration:
2026-09-29 · c34dea2 · locked before the result
Result:
2026-09-29 · 199d384
Data window:
2026-05-31 – 2026-09-28
n:
122 (effective 31)
Decision gates (2)
  • met · overlap-free n ≥ 30 · 31
  • not met · two-sided permutation p < 0.05 · 0.0600
Limits (5)
  • Short, one-sided window: sample A covers 2026-05-31 to 2026-09-28; 122 settled expiries, 31 overlap-free.
  • Large expiries (monthly + quarterly): n = 4 — descriptive only; for the big expiries that max-pain claims usually refer to, this sample does not pose the question.
  • Only the overlap-free expiries enter the test; the descriptive distance bands and expiry classes count overlapping 78-hour windows several times.
  • Different window from the max-pain magnet study (6 h on expiry day) — the two results are not to be read against each other.
  • A finding needs sample A and sample B (expiries from 2026-09-30) in the same direction under α; sample B is not evaluated yet.
Full record →
fellrule: prereg_testbmsb-weekly-v1

H1 (the preregistered, sceptical claim): on the weekly chart, does the Bull Market Support Band rule fail to beat like-for-like buy & hold on the majority of the crypto top 50?

The claim fell (rule: prereg_test). 1 of 3 locked decision gates not met.

Preregistration:
2026-09-28 · c72b8d6 · locked before the result
Result:
2026-09-28 · b1491a6
Data window:
2023-08-14 – 2026-09-27
n:
50 (effective 48)
Decision gates (3)
  • met · pairs with a shifted evaluation window ≤ 16 · 16
  • not met · "beats" below half of the decided pairs (tie band ±0.05 pp) · 24 beats of 48 decided pairs · 24
  • met · BTC distinguishable from the median of the other pairs (pair-selection noise floor, 1w) · noise floor 6.58 pp · 10
Limits (5)
  • "Fell" here means the sceptical H1 did not pass its locked rule: 24 beats, 23 loses, 1 tie (UUSDT), 2 without a trade — exactly half of the decided pairs, which is not "below half". It does NOT mean the rule beats buy & hold; one pair either way flips the outcome.
  • The §6 wording ("does not beat on the majority") would read the same count as held; the operational §8a rule decides, and it was written after run 1 had shown uncounted values from the same study (before the run-2 result).
  • Descriptive: on pairs with negative buy & hold the rule beats 14 of 15, on pairs with positive buy & hold 10 of 32 — the profile of a trend filter (less loss in a fall, less gain in a rise), not an edge.
  • H2: BTC 24.8 % against the median of the other pairs 14.8 % — distinguishable (10 pp > 6.58 pp). H3: trade events per pair median 14, range 0–20; no pair reaches 30 round trips.
  • Universe = today's top 50 by market cap (survivorship: coins that died before are missing). The runs were API jobs on 2026-09-28; the count was first done by hand and re-done by script from the job export (cb48e1a) with identical decision numbers.
Full record →
fellrule: prereg_testetf-divergenz-v1

What followed for the price when price and spot-ETF net flow moved in opposite directions on the same day?

The claim fell (rule: prereg_test). 8 of 8 locked decision gates not met.

Preregistration:
2026-09-28 · 1d667c7 · locked before the result
Result:
2026-09-28 · a3b292e
Data window:
2024-01-11 – 2025-09-25
n:
550
Decision gates (8)
  • not met · two-sided binomial p < 0.00625 · BTC · down/up · 1d · 0.72
  • not met · two-sided binomial p < 0.00625 · BTC · down/up · 6d · 0.31
  • not met · two-sided binomial p < 0.00625 · BTC · up/down · 1d · 0.56
  • not met · two-sided binomial p < 0.00625 · BTC · up/down · 6d · 0.97
  • not met · two-sided binomial p < 0.00625 · ETH · down/up · 1d · 0.76
  • not met · two-sided binomial p < 0.00625 · ETH · down/up · 6d · 0.19
  • not met · cell too thin to test (n 15) · ETH · up/down · 1d
  • not met · cell too thin to test (n 15) · ETH · up/down · 6d
Limits (4)
  • Retrospective sample A only; BTC 2024-01-11 to 2025-09-25 (428 periods), ETH 2024-07-23 to 2025-09-25 (122 periods). Sample B (forward) is due later.
  • Smallest p across the 6 testable comparisons: 0.19; 2 of 8 comparisons too thin to test.
  • 6-day forward windows overlap between consecutive days; the binomial test treats periods as independent.
  • The result file was last rewritten by the archive move 33e2faf (only its prereg path field changed); the numbers are those of the first result commit a3b292e.
Full record →
heldgrid-leverage-sweep-v1

How often is a long grid with isolated margin at 1×, 2×, 3× or 5× leverage liquidated within 90 days — and what does the leverage cost when it is not?

The claim held. 0 of 3 locked decision gates not met.

Preregistration:
2026-09-12 · 4d1696d · locked before the result
Result:
2026-09-13 · 748be3a
Data window:
2017-11-16 – 2026-07-02
n:
70
Decision gates (3)
  • met · ≥ 30 valid windows per asset (§9 abort) · BTCUSDT · 35
  • met · ≥ 30 valid windows per asset (§9 abort) · ETHUSDT · 35
  • met · 5× liquidated in more than 0 valid windows on at least one asset (§9 falsification) · BTCUSDT + ETHUSDT · 1
Limits (6)
  • The claim tested is the content thesis "leverage = liquidation risk within 90 days": it held because the locked falsification (no liquidation at 5× on either asset) did not fire — not because leverage paid off or was safe.
  • "Held" rests on 1 liquidation at 5× across BTCUSDT 1/35 and ETHUSDT 0/35 (90 % Wilson interval BTCUSDT 0.6 %–11.8 %, ETHUSDT 0.0 %–7.2 %).
  • H1 (rate > 0 at 5× on BOTH assets) is not supported: ETHUSDT had no liquidation at 5×. H2 (no return gain from leverage) is decided by Holm p-values that are only in log.md, not in a machine-readable file.
  • Long-only grid with a cash start; superseded as the platform default by the initial-position variant (2026-10-07, separate run). 15-minute candles with an open→low→high→close path, no slippage.
  • Amendment A1 (potency arm before the first run) first appears in the result commit 748be3a, not in the lock commit.
  • The numbers are from the re-run after the sell-tolerance fix (3052b0c, locks unchanged); the pre-fix run moved 86 of 280 rows but not the liquidation count.
Full record →
dissolvedrule: noise_floormaxpain-magnet-v1

Does the max-pain strike pull spot toward it into a Deribit BTC expiry — does settlement land closer to max pain than price movement alone would put it?

The question dissolved on re-measurement (rule: noise_floor). 1 of 3 locked decision gates not met; post-hoc control outside the locks: 1 of 1 not met.

Preregistration:
2026-08-26 · a30b6cf · locked in the result commit
Result:
2026-08-30 · a30b6cf
Data window:
2026-05-28 – 2026-08-29
n:
94
Decision gates (4)
  • met · pooled n ≥ 30 · all expiries · 94
  • not met · large expiries (weekly + monthly + quarterly) n ≥ 30 · large expiries, descriptive only · 14
  • met · |z| ≥ 1.96 against the locked DVOL null · locked null (D3) · -3.02
  • not met · |z| ≥ 1.96 against the realized window-volatility null · post-hoc control arm, outside the locks · -1.70
Limits (5)
  • Lock and result in the same commit — the preregistration cannot be shown to precede the first outcome number (spec D3).
  • The locked test cleared the threshold (z −3.02, mean PIT 0.410, magnet direction). The 02:00–08:00 UTC hours carry on average 72 % of the volatility the DVOL null assumes, so the locked null was too wide; with the realized window volatility the rest is z −1.70 (mean PIT 0.449), inside the band.
  • The control arm was added after the result, outside the locks (declared 2026-08-31, scripts/maxpain-study/control-arm.ts, docs/research/maxpain/control-arm-results.json) — the outcome rests on a post-hoc correction of the null, which is why it is booked as dissolved, not as fell.
  • Daily expiries alone (n 80): z −2.17 against the window-volatility null — just outside the band, with 60 trailing days as the volatility estimator not robust.
  • The numbers are from the re-run after the settlement-price correction (1be1b9c, locks unchanged); the first published figure in a30b6cf used a daily close instead of the Deribit delivery price and was an input artefact. Large expiries n = 14: anecdotes, the question for them is not posed.
Full record →
not answerablerule: min_n_per_cellfundamental-score-v1

Did tokens with a high fundamental score lead tokens with a low score over the following months — measured within the same snapshot date?

The question could not be answered with this sample (rule: min_n_per_cell). 1 of 2 locked decision gates not met.

Preregistration:
2026-08-26 · 9bf6baf · locked in the result commit
Result:
2026-08-26 · 9bf6baf
Data window:
2023-01-26 – 2024-12-01
n:
2840 (effective 1240)
Decision gates (2)
  • met · observation-weighted price coverage ≥ 40 % · 0.44
  • not met · share of snapshot dates with a tercile below n = 10 ≤ 1/3 · 10 of 24 dates · 0.42
Limits (4)
  • Lock and result in the same commit — the preregistration cannot be shown to precede the first outcome number (spec D3).
  • The study design (spec §5.2/§5.3) was locked earlier, in commit bb58457 (2026-07-29); only the operative choices L1–L12 share the result commit.
  • Regime A2: price coverage 43.7 % (1240 of 2840 observations) — any statement would cover listed survivors only.
  • The aborted horizons were still computed and are in the result file; per the lock they carry no verdict.
Full record →
fellrule: placeboconsensus-ampel-v1

Does a majority traffic light over a preregistered committee of five signals, traded with hysteresis rules on BTCUSDT, end with more net than holding — and does the rule carry that, or the market phase?

The claim fell (rule: placebo). 7 of 9 locked decision gates not met.

Preregistration:
2026-08-05 · 560d6e8 · locked in the result commit
Result:
2026-08-05 · 560d6e8
Data window:
2018-03-04 – 2026-07-31
n:
3072
Decision gates (9)
  • not met · net multiplier > buy-and-hold net multiplier · W · 0.73
  • not met · placebo p < 0.05 · W
  • not met · DSR verdict "pass" · W
  • met · net multiplier > buy-and-hold net multiplier · S · 8.00
  • not met · placebo p < 0.05 · S
  • not met · DSR verdict "pass" · S
  • met · net multiplier > buy-and-hold net multiplier · H · 11.57
  • not met · placebo p < 0.05 · H
  • not met · DSR verdict "pass" · H
Limits (7)
  • Lock and result in the same commit — the preregistration cannot be shown to precede the first outcome number (spec D3).
  • Arms with fewer than 30 round trips (anecdote): S (9).
  • (de, from the script) Slippage = 0 — alle Zahlen sind Obergrenzen.
  • (de, from the script) Nackte Signal-Semantik ohne Entry-Filter/Trailing-Stop — die Live-Ampel kann abweichen.
  • (de, from the script) Wuguan-Benchmark hat auf Binance-Daten erst ab ~2021 ein volles 200W-Gate (erster Trade wird ausgewiesen).
  • (de, from the script) Einzel-Asset, Einzel-Intervall — Fenster-Rausch-Schranke (noise-axes: 2,74–4,58 pp) gilt für jede Interpretation kleiner CAGR-Differenzen.
  • Out-of-sample runs of the same locked design on ETH and SOL (addendum 2026-08-16) are separate results and not part of this entry.
Full record →
not answerablerule: min_n_per_cellvola-episoden-v1

Do phases of low Bitcoin volatility resolve upward or downward — and does that depend on the market phase (above or below the 200-week average)? (F2; F1, whether volatility expands afterwards, is reported descriptively.)

The question could not be answered with this sample (rule: min_n_per_cell). 1 of 3 locked decision gates not met.

Preregistration:
2026-07-30 · 25b8237 · locked in the result commit
Result:
2026-07-30 · 25b8237
Data window:
2011-04-24 – 2025-11-22
n:
47
Decision gates (3)
  • met · ≥ 30 episodes in total (§9.1) · 47
  • met · ≥ 30 episodes per phase cell (§9.2) · F2 · above the 200-week average · 34
  • not met · ≥ 30 episodes per phase cell (§9.2) · F2 · below the 200-week average · 3
Limits (5)
  • Lock and result in the same commit — the preregistration cannot be shown to precede the first outcome number (spec D3).
  • Phase cells: 34 above / 3 below the 200-week average, 10 episodes in the warm-up before the 200-week average exists — F2 is not answerable by the locked rule.
  • F1 (does volatility expand afterwards) has no locked decision threshold; the script's verdict on it is descriptive and does not decide this entry.
  • Forward windows of 90 and 180 days overlap between neighbouring episodes; the effective n for the longer horizons is lower than the episode count (RESULTS.md).
  • The input was a live price table; a re-run today would not reproduce this sample.
Full record →
fellrule: placebobtc-rotation-v1

Does a three-state rotation (alt / BTC / USDT) driven by standard signals end with more BTC, net of fees, than simply holding BTC — and is it the rule that carries the result, or the market phase?

The claim fell (rule: placebo). 3 of 4 locked decision gates not met.

Preregistration:
2026-07-27 · e2ec7a5 · locked in the result commit
Result:
2026-07-27 · e2ec7a5
Data window:
2017-09-07 – 2026-06-30
n:
45
Decision gates (4)
  • not met · median net multiplier across pairs > 1.00 (beats holding BTC)
  • met · median net multiplier > median alt-hold in BTC
  • not met · median placebo p < 0.05 across cells
  • not met · DSR verdict "pass" in the majority of cells
Limits (4)
  • Lock and result in the same commit — the preregistration cannot be shown to precede the first outcome number (spec D3).
  • 45 cells (9 pairs × signals × intervals) share the same calendar years — they are not independent; the DSR counts all 45 as trials.
  • Median net multiplier per signal × interval ranges from 0.28 to 3.09; 15 cells have fewer than 30 state changes (anecdotes).
  • Native /BTC pairs only; synthetic pairs are not part of this measurement. A top pair delisted since 2019 would be invisible (survivorship residue, empty for majors).
Full record →
fellrule: placebowyckoff-spring-v1

Does price after a mechanically detected Wyckoff spring develop differently from price over the same period without the condition — and does the Wyckoff structure add anything over a trivial oscillator rule?

The claim fell (rule: placebo). 4 of 7 locked decision gates not met.

Preregistration:
2026-07-25 · b4c54e0 · locked before the result
Result:
2026-07-25 · df4a34d
Data window:
2017-08-17 – 2026-06-30
n:
395 (effective 332)
Decision gates (7)
  • met · ≥ 30 deduplicated events per variant · V0 · 332
  • met · ≥ 30 deduplicated events per variant · V1 · 197
  • not met · ≥ 30 deduplicated events per variant · V2 · 20
  • met · ≥ 30 deduplicated events per variant · PLACEBO · 312
  • not met · V0 spring interval does not overlap the placebo interval · h=5
  • not met · V0 spring interval does not overlap the placebo interval · h=10
  • not met · V0 spring interval does not overlap the placebo interval · h=20
Limits (4)
  • Forward windows overlap (5/10/20 days on daily bars); the block bootstrap with block length h accounts for it, the event counts do not.
  • 4 of 9 spring arm × horizon cells exceed one standard error: V0 5d −1.18 % (n 332), V0 10d −1.06 % (n 325), V1 20d +2.09 % (n 194), V2 20d +6.98 % (n 20) — mean log-return differences; cells under 30 events are anecdotes.
  • Deflated Sharpe with n_trials = 12; ten USDT pairs from a frozen 2020 universe, one data source.
  • n_raw / n_effective refer to the price-structure arm V0 (raw / deduplicated per pair and bar).
Full record →

How the register works

Each entry is an immutable knowledge object: the database rejects any later edit. A correction is filed as a new entry that points to the old one. Every number comes from the study script’s own output file, not from a transcript. The lock commit is the proof of preregistration — whether it precedes the result can be checked with git.

For agents and scripts (free API key): GET /api/v1/knowledge/study/{slug}, the balance at /api/v1/knowledge/study/GLOBAL, via MCP through arena_get_knowledge.

An entry describes a test, not a trading signal. “Fell” means the claim did not pass the check fixed in advance — not that the opposite is proven.

Falsification register — current state

11 tested questions with their outcome: 6 fell, 1 held, 2 not answerable, 2 dissolved, 0 open. 5 of 11 were locked provably before the first outcome number; 5 were locked in the same commit as their result; 1 recorded after the fact without a lock.

Entries
11
Fell
6
Held
1
Not answerable
2
Dissolved
2
Locked before result
5 / 11
Latest run
2026-10-12

multiday-grid-3d-v1: The question dissolved on re-measurement (rule: noise_floor). 1 of 1 decision gates not met.

maxpain-distance-v1: The claim fell (rule: prereg_test). Spearman ρ +0.339, 95 % interval [−0.038; +0.647], permutation p 0.060 on 31 overlap-free cases (α 0.05).

bmsb-weekly-v1: The claim fell (rule: prereg_test). 1 of 3 locked decision gates not met.

etf-divergenz-v1: The claim fell (rule: prereg_test). 8 of 8 locked decision gates not met.

grid-leverage-sweep-v1: The claim held. 0 of 3 locked decision gates not met.

maxpain-magnet-v1: The question dissolved on re-measurement (rule: noise_floor). 1 of 3 locked decision gates not met; post-hoc control outside the locks: 1 of 1 not met.

fundamental-score-v1: The question could not be answered with this sample (rule: min_n_per_cell). 1 of 2 locked decision gates not met.

consensus-ampel-v1: The claim fell (rule: placebo). 7 of 9 locked decision gates not met.

vola-episoden-v1: The question could not be answered with this sample (rule: min_n_per_cell). 1 of 3 locked decision gates not met.

btc-rotation-v1: The claim fell (rule: placebo). 3 of 4 locked decision gates not met.

wyckoff-spring-v1: The claim fell (rule: placebo). 4 of 7 locked decision gates not met.