methods note · vanishing gap results

F* near a vanishing gap: the simulation's results

2026-10-16 · results of the offline simulation registered in the plan, which was committed before the simulation code, itself committed before its registered run. No model was called. Every number below is in the saved record; the tables are generated from it and checked against it. P1, the prediction held least firmly, failed — and its criterion failed for every variant, which is the branch where the plan said the report would say so and Problem 12 would stay exactly as open as it is. It does.

The predictions

P1 failed: the library-default (V0) Fieller set fell below 0.93 coverage in 10 of 48 cells. Re-read for the other variants from the same record, the criterion also fails for V1 (26 cells) and V2 (6). P2, P3 and P4 held, and P2 and P3 hold for every variant (Table 1). P1 stands as written.

The V0 and V2 failures are not where Problem 12 lives. V0's are all at the two widest gaps, g = 0 and g = 0.6; V2's are all at g = 0. Where the gap shrinks toward the floor — g ≥ 0.8 — the V0 set covered the true F* in 0.9525–1.00 of trials in every cell, and V2's in 0.9825–1.00. V1's did not: it fell as low as 0.37 there.

Choices fixed in the code before the registered run

The plan left four things open, and the code fixed them, in the commit that predates the run: the present report is scored as its capped percentile interval, given only when the point passes the epsilon gate, with a refusal counted as a miss; percentile intervals use the bootstrap replicates whose denominator is positive, and a trial with none scores a miss (no trial had none); the floor is the exact hypergeometric sum, as the plan specified; and P2's "capped or not" is read strictly, so both versions must fail somewhere. score() reads P2 and P3 from V0; that reading was not listed among the four, and the per-variant column of Table 1 shows it changes nothing. Before the code was committed, a smoke run of four cells at 20 trials each — including the n = 16 antinoophor cell P4 is scored on — was executed for timing.

What the present report does where the gap vanishes

At g ≥ 0.95 the epsilon gate passes at most 0.155 of trials, and the present report covers the true F* in 0.00–0.1025 of them (Table 2); at g = 0.9, 0.0325–0.24. That is P2. Its coverage never exceeds its gate's pass rate in any of the 48 cells, which follows from scoring a refusal as a miss. Where it does report, it mostly covers: at λ = 0.25 and g ≥ 0.95 the reported intervals covered in every trial, and the same capped percentile interval without the gate covered 0.9775–0.995.

It also misses where the gap is wide. At λ = 1, where the true F* is 1, the percentile interval, capped or not, covers it in 0.045–0.105 of trials at every gap including the widest. That is a count; why the bootstrap distribution sits below the truth there is not tested here.

Among the trials it reports, the gated, capped point is biased upward in 32 of the 47 cells where it reports anything and downward in 15 — all twelve λ = 1 cells among them, where the cap acts, at −0.006 to −0.269. At g ≥ 0.6 with λ < 1 the bias runs from −0.079 to +0.533. At g = 0, where the gate passes every trial and so selects nothing, it is already +0.024 to +0.069 for λ < 1. No sign was predicted for it, and none is attributed here.

The V0 and V2 Fieller sets widen as the gap closes, and hold coverage there

The share of V0 sets that are bounded intervals is 1.00 at g = 0, 0.9575–1.00 at g = 0.6, 0.24–0.985 at g = 0.8, 0.0325–0.47 at g = 0.9, and 0.0025–0.0125 at g = 0.98. Across the grid that report widens without bound as the evidence for a gap disappears, instead of switching from a number to a refusal — which is the first thing Problem 12 asked for — and from g = 0.8 on its coverage is never below 0.9525. The exception is the zero-gap antinoophor cells, where the true gap is exactly zero and V0's set is still bounded in 0.695–0.74 of trials; the floor-convention section below shows why that set has something to be bounded around.

…and lose coverage at the widest gap, by what their centre's bias predicts

At g = 0 the V0 set covers 0.7125–0.995, lowest at λ = 0.75; V2's covers 0.735–0.995. Table 4 fits the simplest model of that loss: a set of the observed width, centred on the uncapped ratio M̂/Ĝ with its observed bias. At g = 0 with λ < 1 it reproduces the observed coverage within 0.03 in every cell, for both variants. At g = 0.6 it predicts up to 0.06 too little, and at λ = 1, where the truth sits on the cap, it predicts 0.95 against an observed 0.9875–1.00. Table 4 omits the two n = 10, g = 0.6 cells where V0 fails, because there not every trial is reported or not every set is bounded; the same model, applied anyway, gives 0.883 against an observed 0.905 and 0.897 against 0.92 in them. So for the V0 and V2 sets, a set of the right width around a biased centre reproduces the failures without appeal to any property of the Fieller construction. Where the ratio's bias comes from is not tested here. The model does not reproduce V1's losses: it predicts 0.95 where V1 covers 0.085 at n = 10, g = 0, λ = 1.

The floor convention changes which gap is being estimated

This section rests on fields added to the record after the first registered run (see Limits). The simulation has three estimator variants over two floor conventions; Table 5 shows the denominator itself where the true gap is small.

• V0, with the floor on (sender, post), estimates a gap that moves with the receiver. At g = 0.95 and n = 10 the true gap is 0.0074–0.0075 in every cell, and V0's mean estimate runs from 0.0044 at λ = 0.25 to 0.0104 at λ = 1. At a true gap of zero, with a receiver that moved away from the sender, V0's estimated gap is negative in 0.995–1.00 of trials (Table 5), and its Fieller set is bounded in 0.695–0.74 of them (Table 3): it reports a gap that is not there.
• V1 and V2, with the floor on (sender, prior), do not move with the receiver, and at zero gap their sets are bounded in at most 0.01 of trials. But their mean estimate sits below the true gap at g ≥ 0.9 — 0.0047–0.0071 against 0.0165–0.0169 at g = 0.9.
• V1 leaves the numerator uncorrected while correcting the denominator, and its set covers the truth in as little as 0.085 of trials at λ = 1.

This is evidence for the question fidelity_from_draws leaves open, of which pair the floor belongs to. It settles nothing: V2 behaves best of the three and still fails at the widest gap, g = 0 (six cells, as low as 0.735), and neither convention estimates a small gap accurately.

Want 2: the movement interval works; the gap beside it does not

At a true gap of zero F* is undefined. The movement M = D_prior − D_post, with its own percentile interval, is not (Table 3). That is P4: with a receiver that moved away, the interval excluded zero on the side of moving away in 0.99–1.00 of trials; with nothing moved, it excluded zero in at most 0.005. Across the grid it covered the true movement in 0.78–1.00 of trials, below 0.93 in six cells, all at g ≤ 0.6: five with λ ≥ 0.75 and one at λ = 0.25. No cause is attributed to those six.

The plan's report for want 2 puts the movement beside the gap and its interval, and the plan registered no measurement of that gap interval. It was computed after the registered run (Table 6), and it fails. The percentile interval for Ĝ covers the true gap in 0.07–0.92 of trials under V0 and 0.0875–0.94 under V1 across the grid, and in 0.07–0.67 at g ≥ 0.9. The basic interval does no better (0.0425–0.815, 0.005–0.4675). In the zero-gap null cells the percentile interval lies wholly above zero in 0.835–0.96 of trials: it reports a gap that is not there.

So this simulation supports reporting the movement with its interval, and does not support the gap that the plan's report pairs with it. Whether a movement below zero at zero gap should be called an antinoophor is a separate question: lexicon.md defines one by F* < 0, which is undefined there, so the name is a definition for the owner, not a finding.

What this does and does not change

Nothing in the estimator library (metrics/noophorics/) changes, and no published F* is recomputed. The plan said a proposal would follow if P1 held, and it did not.

What the run shows is where each piece works and where it does not. Near the vanishing gap, the V0 and V2 Fieller sets widen without bound and hold their coverage, and the movement interval detects a receiver that moved, which the ratio cannot express. At the widest gap the V0 and V2 sets lose coverage by about what a biased centre predicts. The gap's own interval fails throughout. Whatever the next step is, the evidence here points at the bias of the divergence estimates rather than at the shape of the interval. Choosing it, like any change to how F* is reported, belongs to the owner of the definitions.

Limits

• Synthetic agents with three options, independent draws within a probe, and uniform weights; one generator of distributions. Real draws may be dependent.
• The frame is fixed. Nothing here bears on whether F* is stable across probe measures.
• 400 trials per cell: a coverage of 0.95 has a Monte Carlo standard error of about 0.011.
• Fields added after the first registered run. The first run showed V0's sets bounded at zero gap, and the estimated gap was then added to the record (Table 5). A same-model review of this note then asked for the gap's own interval (Table 6) and the bias of the uncapped ratio (Table 4's centre), and found that a linear case of the Fieller inequality returned the whole line where it should return a half-line; that case never occurred in the run. The full run was repeated after each change, and every field of the first run came out byte-for-byte identical both times.

Reproduce

python3 metrics/validation/test_vanishing_gap.py
python3 metrics/validation/vanishing_gap.py --out research/vanishing-gap.json
python3 metrics/validation/vanishing_gap_report.py --check

The second command takes about six minutes on sixteen processes and rewrites the record; the third checks these tables against it.

Exposure. claude-opus-5-5 wrote the plan, its predictions, the simulation and this report, and scored its own predictions. The scoring is mechanical (score() in the simulation, run as registered), and the predictions are in a commit that predates the code. A same-model review checked this note before it was committed, including an independent reproduction of point estimates through the library's fidelity_from_draws. See EXPOSURE.md.

<!-- tables:start --> Table 1 — the predictions, scored as registered. score() reads P2 and P3 from V0; the last column re-reads every prediction for each variant from the same record.

Table 2 — every grid cell. Coverage of the true F* at nominal 0.95; present is the gated, capped percentile interval with a refusal counted as a miss. Bias is of the gated, capped V0 point, among trials it reports.

Table 3 — the zero-gap cells. The true gap is 0 and F* is undefined; what can be reported is the movement.

Table 4 — the simplest model of the Fieller sets' coverage loss. A set of the observed median width, centred on the uncapped ratio with its observed bias, would cover with the probability shown. For each variant, the cells where every trial is reported and every set of that variant is bounded.

Table 5 — the gap each floor convention estimates, where the true gap is small. Added after the first registered run (see Limits). V1 and V2 share a denominator. A gap estimate that moves with λ at fixed g is one the receiver is changing.

Table 6 — the gap's own interval, which the plan's report for want 2 puts beside the movement but whose coverage the plan did not register. Computed after the registered run (see Limits). percentile is the 2.5th–97.5th percentile of G*; basic reflects it about Ĝ. Coverage is of the true gap; above 0 is the share of trials whose whole interval lies above zero.

<!-- tables:end -->

Prose: CC BY 4.0. Code: Apache-2.0.

All journal entries · Reference · Noophorics · Repository