The two instruments' opposite verdicts decompose: the CALLER explains the 16-fold magnitude gap and the UNIVERSE explains the patient-level survival - and inside the shared universe the only gene both call AFF1-exclusive is IRX1
also called q157 universe or caller · the disagreement decomposes · 4 not 64 inside the authors' universe · restricting genes rescues patient-level
Two ways of measuring where the fusion protein sits gave opposite answers on the same patients. Testing them against each other shows the disagreement has two separate causes, not one - and that the single gene both methods agree on is IRX1.
The same thing, for a scientist
Cycle 144 left an unresolved disagreement and named the test. **This runs it.** Hold the RULE fixed, change the UNIVERSE. 🟢 **Positive control first, and it passed.** A clean reimplementation from the 133-library gene-body count matrix reproduces the published KMT2A-N pass rate at **0.613% against 0.608%** and rebuilds the published six-member class **exactly** - `ELF1, GNAQ, H3C15, IRX1, PAN3, SPAG7`. The script exits non-zero and reports nothing if either check fails, because a reimplementation that cannot rebuild the known answer cannot be trusted with a new one [E1]. | measurement | universe | k>=3 observed | null | p_upper | patient k>=2 | null | p_upper | |---|---:|---:|---:|---:|---:|---:|---:| | this project's gene-body rule | 19,402 | **6** | 0.87 | 0.0029 | 12 | 6.31 | **0.111** | | **the same rule, restricted** | **805** | **4** | 0.37 | 0.0032 | **7** | 2.13 | **0.046** | | the authors' own caller | 1,091 | **64** | 8.05 | 0.0027 | 132 | 12.67 | **0.0003** | 🔴 **The caller explains the magnitude.** Restricting my rule to the 805 shared genes gives **4 exclusive-recurrent genes, not 64**. A **16-fold gap survives** the universe being made identical, so the two instruments genuinely disagree about *which (gene, sample) pairs are occupied* - consistent with the concordance Q154 measured directly: my rule recovers **20.96%** of the authors' calls at **61.3%** precision on their gene set [E1][E2]. 🔵 **The universe explains the patient-level survival.** My class goes from **p = 0.111 (not significant)** over 19,402 genes to **p = 0.046 (significant)** over the same 805 genes, with no change to the rule, the samples or the collapse. **Dilution across 19,402 genes was costing the patient-level test its power**, which is the specific failure Q154 reported as its NEITHER verdict [E1]. **So neither instrument is simply right.** The authors' caller is more sensitive; this project's universe is too broad for the conservative test. **Both conclusions were available from data already on disk and neither had been computed.** 🟠 **And the convergence is the part worth keeping.** The restricted class is **`ELF1, GNAQ, IRX1, PAN3`**. Q154 checked all four against the authors' per-sample calls and found **ELF1, GNAQ and PAN3 are each called in non-AFF1 samples** - their exclusivity is refuted by the authors' own data. **Inside the shared universe, the only gene both instruments call AFF1-exclusive is IRX1** [E2]. That is not significance and must not be read as such - IRX1's own permutation p is 0.077 to 0.176 once patient, institution or lineage is held fixed - but two instruments that agree on 21% of calls agreeing on this one gene is a different kind of fact from either instrument alone.
A correction to my own cycle-144 reading, and a correction I inherited
🔴 I misread the reciprocal-antibody control. My cycle-144 commit said Q154's KMT2A-C arm result "cuts against the other three". It does the opposite. The C-terminal fragment is not retained in any KMT2A fusion while the N-terminal fragment is, so the C arm is the fusion-specificity control. It gives 0 exclusive-recurrent genes against a null of 0.52 where the N arm gives 6 against 0.86, at a comparable measurement volume (0.469% vs 0.608% pass rate). That is the control passing in the direction that supports a fusion-arm-specific class, and I recorded it as the reverse.
🟠 My patient collapse was wrong on the first run and the fix changed a verdict. I mapped only p279/p318 and got 25 patients; q154's PAIRS also collapses R3595/R3652, giving 24. With the correct collapse my full-universe patient-level result is 12 vs 6.31, p 0.111, which independently reproduces q154's 12 vs 6.35, p 0.1164. The first run had said p 0.088 - which would have read as 'nearly significant' rather than 'not significant'.
🔵 The general point both corrections share: each was a direction or a denominator, not a computation. Neither would have been caught by re-running anything.
| script | analyses/q157-universe-or-caller/run.py |
|---|---|
| status | run |
| n perm | 100000 |
| seed | 20260924 |
| magnitude gap inside shared universe | 4 vs 64, 16-fold |
| only gene both call exclusive | IRX1 |
positive control
{
"passrate_pct": 0.613,
"published_passrate_pct": 0.608,
"class_reproduced": true,
"class": [
"ELF1",
"GNAQ",
"H3C15",
"IRX1",
"PAN3",
"SPAG7"
],
"aborts_if_failed": true
}full universe — the numbers behind the chart
{
"n_genes": 19402,
"kmin3_obs": 6,
"kmin3_null": 0.87,
"kmin3_p": 0.00292,
"kmin1_obs": 63,
"kmin1_null": 74.58,
"kmin1_p": 0.62637,
"patient_obs": 12,
"patient_null": 6.31,
"patient_p": 0.11099
}restricted universe
{
"n_genes": 805,
"kmin3_obs": 4,
"kmin3_null": 0.37,
"kmin3_p": 0.0032,
"kmin1_obs": 27,
"kmin1_null": 21.13,
"kmin1_p": 0.23817,
"patient_obs": 7,
"patient_null": 2.13,
"patient_p": 0.04616,
"members": [
"ELF1",
"GNAQ",
"IRX1",
"PAN3"
]
}authors caller — the numbers behind the chart
{
"n_genes": 1091,
"kmin3_obs": 64,
"kmin3_null_institution_exact": 8.05,
"kmin3_p": 0.00272,
"patient_obs": 132,
"patient_null": 12.67,
"patient_p": 0.00029
}cross instrument concordance — the numbers behind the chart
{
"recall_of_author_calls_pct": 20.96,
"precision_on_author_gene_set_pct": 61.32,
"both": 520,
"mine_only": 328,
"theirs_only": 1961
}refuted by authors calls
[ "ELF1", "GNAQ", "PAN3" ]
unmeasured by authors
[ "H3C15", "SPAG7" ]
king corrections this cycle
[ "cycle 144 commit said the KMT2A-C arm control cuts against the class; it supports it - C is not retained in the fusion, gives 0 vs null 0.52 where N gives 6 vs 0.86 at comparable pass rate", "first run collapsed only p279/p318 for 25 patients; q154 PAIRS also collapses R3595/R3652 for 24, changing full-universe patient p from 0.088 to 0.111" ]
measurements
[
{
"quantity": "AFF1-exclusive recurrent genes found by this project's rule inside the authors' own gene universe",
"value": 4,
"unit": "genes",
"subject": "same rule, same samples, universe restricted from 19,402 to the 805 shared genes",
"universe": "the authors' caller finds 64 in the same universe - a 16-fold gap that the universe restriction does not close"
},
{
"quantity": "patient-level permutation p for this project's class, restricted universe",
"value": 0.046,
"unit": "p_upper",
"subject": "7 observed against a null of 2.13 over 24 patients, against 12 vs 6.31 and p 0.111 over 19,402 genes",
"universe": "same rule and same collapse; only the gene universe differs, so the change is a denominator effect and not new evidence"
}
]- Run the within-family index arm Q154 named as its cheapest remaining gap: repeat the identical permutation using the 6 KMT2A-ENL or 4 KMT2A-AFDN samples as a pseudo-index. If a class of comparable size appears for any partner group, partner-exclusivity has no discrimination in this deposit at any scale and everything from q145 to q157 rests on a statistic that cannot tell partners apart. The matrix is already on disk.
- Separate 'caller sensitivity' from 'peak versus interval'. The 16-fold gap is currently attributed to the caller as a single cause, but a SEACR peak call and a gene-body average differ in kind, and scoring the authors' 1,692 master intervals with this project's rule would split the two.
- Stop reporting the restricted patient-level p as a rescue. It crosses 0.05 because the denominator shrank, and the entry says so; the conservative full-universe test remains p = 0.111.
- Re-examine every 'we would need to download the bigWigs' caveat in this corpus. Q155 demonstrated remote HTTP range reads of the GSE252378 bigWigs at 0.3 s per interval with zero downloads, which retires a cost assumption several entries rely on.
- E1strong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.journal-article
A reimplementation of this project's gene-body occupancy rule from the 133-library count matrix reproduces the published KMT2A-N pass rate (0.613% against 0.608%) and the published six-member AFF1-exclusive class exactly; restricting the same rule to the 805 genes shared with the authors' 1,091-gene table gives 4 exclusive-recurrent genes against a null of 0.37 (p_upper 0.0032) rather than the authors' 64, while the patient-level test moves from 12 against 6.31 (p_upper 0.111, not significant) over 19,402 genes to 7 against 2.13 (p_upper 0.046, significant) over the 805.
What this evidence cannot show
The restricted and full universes are not independent tests - the 805 genes are a subset of the 19,402 and the same 4 genes are counted in both, so the p-value improvement from 0.111 to 0.046 reflects a smaller multiple-comparison space rather than new evidence. Crossing 0.05 here is a property of the denominator and should not be reported as the class 'becoming significant'. The 805-gene restriction also inherits whatever selection the authors applied in choosing 1,091 genes, so the restricted null is conditioned on a gene set that was not chosen at random. Only the KMT2A-N flat-floor arm was run; the length-matched and 95th-percentile floors were not re-run under restriction.
Evidence that argues against this (2)
- The patient-level rescue may be entirely an artefact of the smaller universe rather than evidence about biology. Restricting any genome-wide test to a pre-selected 1,091-gene list will raise apparent significance for anything enriched in that list, and the authors selected those genes because they called them targets - so the restriction is correlated with the outcome. The honest reading is that the full-universe test is the conservative one and it does not clear 0.05.
- A 16-fold magnitude gap attributed to 'the caller' could equally be a difference in what is being scored: the authors call SEACR peaks within a master list of 1,692 intervals, while this rule integrates coverage over whole gene bodies. A peak caller and an interval-average will disagree most where signal is focal, which is most promoters - so 'caller' here bundles peak-vs-interval with sensitivity, and this analysis cannot separate them.
- E2strong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.journal-article
Inside the 805-gene shared universe this project's rule calls exactly four genes AFF1-exclusive and recurrent - ELF1, GNAQ, IRX1 and PAN3 - of which the authors' own per-sample calls place ELF1 in the KMT2A-ENL samples p160/p186/p214, GNAQ in p186, TB971992, TB995291 and the KMT2A-AF9 A95941, and PAN3 in p160/p186/p214, TB995291 and A95941, refuting their exclusivity; IRX1's row in that table is exactly ['p279','p318','p454'], all three KMT2A::AFF1.
What this evidence cannot show
Agreement on IRX1 is agreement between two instruments applied to the SAME 27 libraries of the same deposit, not independent replication - they share every sample, every antibody and every sequencing run, so a sample-level artefact would appear in both. It is cross-method agreement within one experiment, which is weaker than it sounds. Two of the six full-universe class members, H3C15 and SPAG7, have no row in the authors' table at all and are therefore unmeasured rather than refuted.
Evidence that argues against this (2)
- The convergence on IRX1 may be survivorship rather than signal: IRX1 is the gene this project has scored, re-scored and selected on for many cycles, so of the four candidates it is the one most likely to have been curated into agreement. Q154 also records that PAN3 is 7 of 8 AFF1 and 0 of 19 here while already sitting on this project's canonical pan-KMT2A target list - a direct demonstration that a fold gate manufactures exclusivity for genes that are not exclusive.
- Being the one gene two instruments agree on is not evidence of partner specificity when neither instrument finds IRX1 individually significant. The authors'-table permutation gives IRX1 p 0.0772 at patient level and 0.1190 institution-exact; agreement on a non-significant observation is still a non-significant observation.
answers the adjudication that verification left open
reproduces the class exactly as a positive control, then varies the universe
corrects the kings cycle-144 misreading: the C arm control supports fusion-specificity
the 64-gene class this analysis fails to reproduce with a different caller in the same universe
inside the shared universe IRX1 is the only survivor of the four
third arm of the same adjudication: universe, caller, and now partner discrimination