Oncofusion

start / graph / concept-q148-a-uniform-null-and-a-coverability-matched-null-disagreed-on-eighteen-of-twenty-three-samples

Explainerstrong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.Q148-what-the-hoxa-negative-blast-occupies

A uniform permutation null and a coverability-matched one disagreed on 18 of 23 samples for the same gene set - the null, not the data, decided the verdict

To ask whether a set of genes is unusually busy you have to compare it with something, and here the choice of comparison decided the answer. Against randomly picked genes the set looked busy in 18 of 23 patients; against genes that are equally easy to read, in none of them.

The same thing, for a scientist

Same 40 promoters, same 23 libraries, same statistic - the class mean share - and two nulls: - **N2**, uniform: 10,000 random promoter sets of size 40 drawn from the 19,404. The class clears it in **18 of 23** samples. - **N4**, matched: 10,000 sets of size 40 drawn to match the class's **coverability decile** composition, where coverability is the median share across the KMT2A-N libraries of the **other 26 samples** (leave-the-sample-out, so the matching variable is not built from the sample being tested). The class clears it in **0 of 23**. **The disagreement is total on this class and partial on the others** - the canonical 827-promoter class goes 21 of 23 under N2 to 16 of 23 under N4, and the 37-promoter B-lineage class goes 3 of 23 to 7 of 23 (it *gains* calls, because several samples' B-lineage class sits above equally coverable promoters while below an average one). So N4 is not a blanket subtraction; it reorders. **The lesson is not that matched nulls are better.** It is that a curated gene set is never a random draw from the genome, so a uniform null tests a hypothesis nobody holds - *is this set above an arbitrary set* - while the hypothesis actually at issue is *is this set above a comparable set*. Reporting only the uniform null on a curated set manufactures significance from the curation.

Go deeper
How to tell in advance which null you need

Ask what property got the genes into the set. If the set was assembled by a differential test on coverage-derived signal, then coverage is a property the set was selected on, and a null that does not match on coverage is testing the selection.

Here the tell was visible before the nulls disagreed: the GMP-like class's matched-null mean is 1.75-2.12x an average promoter in most libraries. A random set of 40 promoters with the same coverability profile is already at ~1.9x. So an observed 1.3-2.6x was never going to be evidence of anything. Print the matched null's mean beside the observation and the question answers itself.

Where this sits in the corpus's existing rules

This is the same family as the repo's standing rule that a proportion is not a result until its null is written down, and the sharper version that a category whose base rate is 50% by construction carries no information. What this case adds: writing down a null is not enough if the null is the wrong population. The pre-registration here did write one down, in advance, with 10,000 draws and both tails labelled - and it still produced 18 false positives, because it drew from the whole genome rather than from the sample space the gene set came out of.

Facts and figures
class under testAML_specific (GMP-like), 40 promoters
n samples23
uniform null calls18
matched null calls0
scriptanalyses/q148-what-the-hoxa-negative-blast-occupies/matched_null_summary148.py
statusrun
result summary18 versus 0 on identical data; the null decided the verdict
other classes
{
  "Oncoprotein_Target": {
    "uniform": 21,
    "matched": 16,
    "n_promoters": 827
  },
  "ALL_specific": {
    "uniform": 3,
    "matched": 7,
    "n_promoters": 37
  }
}
matched null mean range for the gmp class
[
  1.75,
  2.12
]
measurements
[
  {
    "quantity": "samples_called_enriched",
    "subject": "GMP-like 40-promoter class under a UNIFORM permutation null",
    "value": 18,
    "units": "samples",
    "n": 23,
    "universe": "GSE252378 untreated non-circular samples, KMT2A-N libraries, 19,404 promoters",
    "null": "10,000 uniform draws of 40 promoters",
    "note": "the pre-registered null"
  },
  {
    "quantity": "samples_called_enriched",
    "subject": "the SAME class under a COVERABILITY-MATCHED null",
    "value": 0,
    "units": "samples",
    "n": 23,
    "universe": "identical samples, libraries and promoters",
    "null": "10,000 decile-matched draws, matching variable = median share across the other 26 samples' KMT2A-N libraries",
    "note": "the two nulls disagree on 18 of 23 samples; the data are identical"
  }
]
How this could bite us back
  • A negative produced by a matched null is easy to over-trust. If accessibility is part of how the fusion chooses its targets, matching on accessibility removes the mechanism along with the confound.
Still open
  1. How many gene-set enrichment claims in this corpus rest on a uniform null over a curated set? The count has not been taken, and this case suggests it is the most consequential methodological audit available.
  2. What is the right matching variable when no IgG or input control is usable? Coverability from sibling samples is one answer; mappability, CpG content and promoter width are others, and they would not agree.
Every source behind this entry (1)
  1. E1strong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.dataset

    For the same 40-promoter class in the same 23 libraries, a uniform permutation null returns 18 of 23 enriched and a coverability-matched null returns 0 of 23.

    What this evidence cannot show

    The matched null is post-hoc; the pre-registration specified an IgG-decile match, which could not be computed because IgG promoter shares are 9%-98% zero in the 25 samples with an IgG library. Choosing a matching variable after seeing the uniform result is itself a degree of freedom, and the defence is that the matched null was applied to all three classes and reordered rather than flattened them.

    Evidence that argues against this (1)
    • Matching on coverability can match on the signal itself when the signal IS accessibility, in which case the matched null is over-corrected and 0 of 23 is an underestimate. The check that argues against that here is that the canonical class still clears the matched null in 16 of 23 and the B-lineage class gains calls under it. Q148 (Oncofusion Knowledge Graph) 2026
Connected to