Oncofusion

start / graph / ana-q171-the-two-instrument-intersection-does-not-beat-a-patient-level-null-and-the-published-count-was-four-not-one

Analysisstrong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.

The permutation cycle 155 asked for: the two-instrument exclusive intersection is four genes not one, 34 of the 42 could never have been in it, and the surviving IRX1 claim gives p=0.092 against a patient-level null

also called q171 · IRX1 permutation · the test that finished the occupancy line

We had one encouraging result left about IRX1: two different ways of reading the same experiment both said it was switched on only in this fusion. We finally ran the fairness test we had been promising - shuffle which patients count as having the fusion, and see how often the same pattern shows up by luck. It shows up about one time in eleven, which is not rare enough to mean anything. Two other things also went wrong: there were really four such genes, not one, and three quarters of the genes we were comparing against could never have qualified in the first place. This closes the line of work.

The same thing, for a scientist

Cycle 155 produced the occupancy line's only positive result in several cycles and wrote down the condition for retracting it: *"if the permutation shows one hit is the expected number, the line is finished - and I should say so rather than find a sixth criterion."* This is that permutation. **It fails on three independent counts, and the line is finished.** 🔴 **1. The published count is wrong: four, not one.** At the stated criterion - called by the gene-body rule in >=1 AFF1 sample and in **zero** non-AFF1 samples - the hits among the 42 are **IRX1, ANGPT1, FOXL2 and FOXL2NB** [E1]. Cycle 155's table displayed only genes with **two or more** AFF1 calls, so the three single-call genes were invisible, and the prose generalised from the filtered table to the whole set. 🔴 **2. The denominator could mostly not fail.** Only **8 of the 42** are called by the gene-body instrument in **any** of the 26 permutable samples. The other **34 fail the '>=1 AFF1 call' criterion by construction** [E1]. 'The only one of 42' is a ratio whose denominator is 80.9% inert; the honest ratio is **4 of 8**. 🔴 **3. Three of the four hits are one observation.** ANGPT1, FOXL2 and FOXL2NB have an **identical call vector** - each called in **TB051939 alone** - and **FOXL2 and FOXL2NB are 79 bp apart** on chr3 [E1]. The four hits are **two distinct call patterns.** 🟠 **What survives, and what it is worth.** IRX1 is genuinely the only gene with **>=2** AFF1 calls and zero non-AFF1 calls. Permuting which samples the gene-body instrument treats as AFF1 (20,000 draws, authors' 42 held fixed) gives **p = 0.031** at that threshold [E2]. **That figure is anticonservative and should not be quoted.** IRX1's three calls are **p279, p318 and p454**, and **p279 and p318 are the same patient** - which is exactly why IRX1 sits in a set defined as *two*-patient. A sample-level null can hand a random gene three calls from three different patients; IRX1 does not have that. 🔴 **Under a patient-level null - 7 of 24 patients permuted, samples expanded from patients - every threshold gives p = 0.092 to 0.112** [E3]: | unit | t | observed | null mean | P(>=obs) | |---|---:|---:|---:|---:| | gene | 1 | 4 | 1.269 | 0.092 | | gene | 2 | 1 (IRX1) | 0.099 | 0.092 | | gene | 3 | 1 (IRX1) | 0.099 | 0.092 | | distinct call vector | 1 | 2 | 0.684 | 0.112 | | distinct call vector | 2 | 1 (IRX1) | 0.099 | 0.092 | **Nothing reaches 0.05 at any threshold, under either unit of analysis.** The entire distance between 0.031 and 0.092 is patient structure.

Go deeper
Why the sample-level p-value was the wrong one, stated precisely

The two nulls differ in one respect. The sample-level null draws 8 of 26 libraries; the patient-level null draws 7 of 24 children and takes all their libraries. The true AFF1 arm is 8 libraries from 7 children, because p279 and p318 are the same child sampled twice.

Under the sample-level null a random gene can collect three calls from three unrelated children and clear the t>=3 bar. IRX1 cannot do that - it has three libraries but two children. Scoring IRX1 against a null that is allowed a structure IRX1 does not have makes IRX1 look rarer than it is. Correcting it moves p from 0.031 to 0.092, and that single change is the whole result.

This is the project's standing rule restrict the null with the population applied to a case where the restriction was not obviously load-bearing until it was measured.

The threshold curve, and why the usual warning does not apply here

This project treats a verdict that moves with its cutoff as a verdict about the cutoff, so every threshold was scored rather than the one cycle 155 implicitly used.

tsample-level ppatient-level p
10.0900.092
20.0310.092
30.0310.092
41.000 (obs 0)-

Under the sample-level null the warning fires loudly - 0.090 at t>=1 and 0.031 at t>=2, with the significant value at exactly the threshold the published table happened to use, which is the textbook shape of a threshold artefact.

Under the patient-level null the curve is flat at ~0.09 and the warning becomes moot, for the reason that matters: there is nothing left to select. A researcher free to choose any cutoff still cannot reach 0.05. That is a stronger negative than a single non-significant p-value, because it forecloses the obvious next move.

Reported separately, since it is the sharpest single sentence: across 20,000 sample-level permutations, some gene reaches 3 AFF1 calls with zero non-AFF1 calls in 3.08% of relabellings - and that 3.08% is the optimistic figure.

What this does and does not retract

Retracted: that the two instruments' agreement on IRX1 is a discriminating result. It is not, at any threshold, under a null that respects patient structure.

Not retracted: the underlying calls. IRX1 really is called by the gene-body rule in p279, p318 and p454 and in no non-AFF1 sample; CLEC14A really does have more AFF1 calls and also two non-AFF1 calls. Those numbers are unchanged and cycle 155's table remains correct for what it displays. What fails is the inference from them.

Not addressed: IRX1's standing on grounds that never came from this deposit - the published expression subgroup and the prior mechanistic literature, which is why IRX1 was pursued in the first place. This analysis says the occupancy argument for IRX1 is dead, not that IRX1 is uninteresting. Keeping those two apart is the point; conflating them is how the line survived as long as it did.

Newly owed: ANGPT1, FOXL2 and FOXL2NB are exclusive by the literal criterion and have no entry, which is the same omission this project already recorded for CLEC14A, ROMO1 and DUSP6. Their evidence is weak and shared - one sample, one call vector, two of the three genes 79 bp apart - and that judgement should be written down rather than left implied.

Facts and figures
scriptanalyses/q171-permute-the-two-instrument-intersection/{run,detail,thresh,patient}.py
statusexecuted
answers open question fromana-q170-irx1-is-the-only-one-of-fifty-eight-that-both-instruments-call-exclusive-and-three-genes-that-nearly-match-it-have-no-entry
correctsana-q170-irx1-is-the-only-one-of-fifty-eight-that-both-instruments-call-exclusive-and-three-genes-that-nearly-match-it-have-no-entry
nperm20000
seed20260924
n two patient exclusive58
n scored42
n ever called by gene body8
n cannot be a hit by construction34
observed hits t14
n distinct call patterns among hits2
p sample level t20.0308
p patient level t20.0916
p patient level t10.0923
p vector level t10.1115
p any gene reaches 3 aff1 zero non sample level0.0308
verdictthe occupancy line is finished on its own pre-registered acceptance condition; no threshold under a patient-respecting null reaches 0.05
measurements
permutation_p0.031permutation_p: 0.0308fraction0.031fraction: 0.0308permutation_p0.092permutation_p: 0.0916count2.000count: 2count4.000count: 4count34.0count: 34
observed genes t1
[
  "IRX1",
  "ANGPT1",
  "FOXL2",
  "FOXL2NB"
]
irx1 samples
[
  "p279",
  "p318",
  "p454"
]
irx1 patients
[
  "p279",
  "p454"
]
measurements — the numbers behind the chart
[
  {
    "quantity": "permutation_p",
    "subject": "IRX1 t>=2, patient-level null",
    "universe": "42 two-patient AFF1-exclusive genes, 7 of 24 patients permuted, 20000 draws",
    "value": 0.0916
  },
  {
    "quantity": "permutation_p",
    "subject": "IRX1 t>=2, sample-level null (anticonservative)",
    "universe": "42 two-patient AFF1-exclusive genes, 8 of 26 samples permuted, 20000 draws",
    "value": 0.0308
  },
  {
    "quantity": "count",
    "subject": "genes both instruments call exclusive at t>=1",
    "universe": "42 two-patient AFF1-exclusive genes in the gene-body table",
    "value": 4
  },
  {
    "quantity": "count",
    "subject": "genes the gene-body instrument never calls in any sample",
    "universe": "42 two-patient AFF1-exclusive genes in the gene-body table",
    "value": 34
  },
  {
    "quantity": "count",
    "subject": "distinct call vectors among the 4 hits",
    "universe": "4 genes both instruments call exclusive",
    "value": 2
  },
  {
    "quantity": "fraction",
    "subject": "sample-level permutations in which some gene reaches 3 AFF1 calls with 0 non-AFF1",
    "universe": "20000 relabellings of 8 of 26 samples",
    "value": 0.0308
  }
]
Still open
  1. Write ANGPT1, FOXL2 and FOXL2NB - or record the refusal. They meet the literal criterion and have no entry, exactly as CLEC14A, ROMO1 and DUSP6 did. The honest note is that all three rest on one library and share a call vector, and that note is itself the entry.
  2. Is TB051939 simply the library with the most calls? Three of the four hits are its alone. If one library drives the t>=1 statistic, the criterion is measuring library quality and the whole four-gene set collapses further.
  3. Stratify the patient-level null by institution and lineage. This project has already found one case where a composition confound reversed the sign of a z=-7.13 result inside every stratum, and neither was balanced here. The conclusion would have to move a long way to change, but the direction of the residual bias is currently unknown.
  4. Does the conclusion survive a sensitivity sweep over the 2.0x occupancy threshold and the width-matched negative-locus rule? The null's strongly non-unimodal shape suggests the answer depends on a few libraries, and no sweep was run.
Every source behind this entry (3)
  1. E1strong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.journal-article

    Of the 58 two-patient AFF1-exclusive genes, 42 are in this project's gene-body interval table; only 8 of those 42 are called by the gene-body rule in any of the 26 permutable samples, so 34 cannot satisfy a '>=1 AFF1 call' criterion by construction. Four satisfy the full exclusivity criterion - IRX1 (3 AFF1, 0 non-AFF1), ANGPT1 (1, 0), FOXL2 (1, 0) and FOXL2NB (1, 0) - not one. ANGPT1, FOXL2 and FOXL2NB share an identical call vector, each called in TB051939 alone, and FOXL2 (chr3:138944223-138947137) and FOXL2NB (chr3:138947216-138953990) are 79 bp apart.

    PMID 39472576DOI
    What this evidence cannot show

    The gene-body interval table is this project's own construction and recovers only 20.96% of the authors' calls, so '34 never called' is partly a statement about an insensitive instrument rather than about the loci. The 79 bp figure is the gap between the two interval records used here, not a curated gene-structure claim.

    Evidence that argues against this (2)
    • The 34 never-called genes are not evidence that those loci are unoccupied - they are unscored by an instrument with roughly a fifth of the authors' sensitivity. Treating the honest denominator as 8 makes the ratio 4/8, which is a far weaker sample than 4/42 sounds, and no statistic computed on 8 genes should be described as robust.
    • ANGPT1 is on chr8 and FOXL2/FOXL2NB on chr3, so calling all three 'one observation' rests on the identical call vector rather than on physical linkage. Two unlinked loci called in the same single library could be genuinely independent events that happen to coincide because that library is the one with signal - which is itself the point, but it is a library-level confound, not proof of redundancy.
  2. E2strong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.journal-article

    Permuting which 8 of 26 samples the gene-body instrument treats as AFF1 (20,000 draws, seed 20260924, the authors' 42 held fixed) gives a null mean of 1.259 hits at t>=1 with P(>=4) = 0.090, and a null mean of 0.032 at t>=2 with P(>=1) = 0.031. Across the same 20,000 relabellings, some gene reaches 3 AFF1 calls with zero non-AFF1 calls in 616 draws, 3.08%.

    PMID 39472576DOI
    What this evidence cannot show

    This null treats the 26 libraries as exchangeable, which they are not: it ignores patient structure, and it also ignores institution, lineage and library depth, none of which were balanced. It is reported because it is the figure a straightforward analysis would produce, and because the gap between it and E3 is the finding.

    Evidence that argues against this (2)
    • This p-value is anticonservative and is superseded by E3. Quoting 0.031 as the result would be the error this entry exists to correct - it is retained here only to show what the uncorrected analysis yields, and any downstream use of 0.031 is a misreading of this entry.
    • The null distribution is strongly non-unimodal - 45.5% zero, 23.1% one, 0.7% two, 21.8% three - which indicates the statistic is driven by a handful of libraries rather than by 42 genes behaving independently. A permutation p-value computed on such a lumpy null is unstable to small changes in the call rule, and no sensitivity analysis over the 2.0x occupancy threshold was run.
  3. E3strong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.journal-article

    Permuting which 7 of 24 patients are AFF1 and expanding to their samples (20,000 draws, same seed) gives P(>=observed) = 0.0916 at t>=1, t>=2 and t>=3 scoring genes, and 0.1115 at t>=1 scoring distinct call vectors. IRX1's three gene-body calls - p279, p318 and p454 - collapse to two patients, because p279 and p318 are one child. No threshold under either unit of analysis reaches p = 0.05.

    PMID 39472576DOI
    What this evidence cannot show

    Patient collapse uses this project's three recorded replicate pairs; if any further pair of libraries in the deposit is the same child and is not on that list, 24 is an overcount and the true p-value is larger still, not smaller. With 24 patients and 7 drawn there are 346,104 distinct assignments, so 20,000 draws sample the space rather than enumerate it - the Monte Carlo standard error on 0.092 is about 0.002, which does not affect the conclusion.

    Evidence that argues against this (2)
    • A p-value of 0.092 is not evidence of absence. This analysis shows the observation is compatible with chance at n=8 scoreable genes and 24 patients, not that IRX1 is unexceptional - the deposit was already shown to be underpowered for this question, and an underpowered test returning a null result is the expected outcome whether the effect is real or not.
    • Permuting patient labels breaks the correspondence between the two instruments but also breaks any genuine biological structure shared by the AFF1 arm, so the null may be too permissive in a different direction: if AFF1 samples resemble each other for reasons unrelated to the fusion (institution, lineage, handling), a random 7-patient draw is less internally consistent than the real arm and will under-produce hits. Institution and lineage were not stratified here, and this project has previously found a composition confound that reversed a z of -7.13 inside every stratum.
Connected to