An activity-stratified permutation null for promoter-set overlap, and the check that its hypergeometric shortcut is not an instrument bug
To know whether two lists of genes overlap more than expected, you need to know what 'expected' means. Because busy genes are busy in every sample, the simple textbook expectation is far too low, so we built a fairer one and then proved our fast version of it gives the same answer as the slow honest version.
The same thing, for a scientist
The independence expectation `ab/N` assumes promoters are exchangeable. They are not: promoter activity clusters, and both members of any pair call preferentially inside the covered, active set, so the true chance floor is higher. The null built here bins the universe into deciles of joint coverage `sqrt(tot_A*tot_B)` (tot = nN+nC, which is simultaneously an activity proxy and the denominator that sets binomial power) and permutes one sample's call labels **within** each decile, 2,000 draws, seed 20260924. Set sizes, per-stratum call counts and the activity distribution of the calls are preserved exactly; only the promoter-identity correspondence is broken. Because a within-stratum permutation's intersection is exactly a sum of per-stratum hypergeometric draws, the null is sampled directly rather than by shuffling arrays - and that equivalence was VERIFIED against 200 literal label permutations rather than assumed: medians 0.2461 versus 0.2458, SDs 0.0046 versus 0.0049.
How much the structure correction moves the floor
| universe | independence null | activity-stratified null | ratio |
|---|---|---|---|
| U_full (19,404, unfiltered) | 0.125 | 0.217 | 1.73x |
| U_pair (10,248) | 0.235 | 0.247 | 1.05x |
| U_27 (4,646, 27-way intersection) | 0.267 | 0.272 | 1.02x |
The correction is large exactly where the universe is unfiltered and negligible where the universe is already a coverage filter. A 27-way nN+nC >= 20 intersection has already done most of the stratification, which is why q144's independence null was nearly right for its own universe and would have been badly wrong in q142's. This is a general point: a coverage filter and a matched null are partial substitutes, and a beat that has neither is exposed twice.
Why coverage and not some other covariate
Three candidates were available. H3K4me3 exists for only 3 of 27 samples, so it cannot define a cohort-wide null. IgG exists for 25 of 27 but is a background channel whose own entry in this corpus records it failing specificity checks. Total promoter fragments nN+nC exists for all 27, is a direct accessibility/activity proxy in AutoCUT&RUN 2, and is the binomial denominator in the calling rule, so stratifying on it matches on the two things that decide whether a promoter can be called: how active it is and how much statistical power there is at it. Matching on the ratio itself would be wrong - the ratio is the biology under test.
The instrument check, because a failed check is a claim too
The fast path could have been wrong in a way that looked like data. So the same pair was scored twice: 2,000 hypergeometric draws against 200 literal within-stratum label permutations built by sampling promoter indices without replacement.
hypergeometric shortcut median 0.2461 sd 0.00461
literal permutation median 0.2458 sd 0.00488
Difference of medians 0.0003. EQUIVALENT. Had this diverged, every excess figure in this beat would have been a property of the shortcut.
| script | analyses/q146-one-universe-jaccard/run146.py |
|---|---|
| status | run |
| result summary | Activity-stratified (joint-coverage decile) label-permutation null, 2,000 draws, seed 20260924. Raises the chance floor 1.73x over independence in the unfiltered universe and 1.02-1.05x in coverage-filtered universes. Hypergeometric shortcut verified against 200 literal permutations (medians 0.2461 vs 0.2458). |
| n perm | 2000 |
| n strata | 10 |
| seed | 20260924 |
| null independence U full | 0.1252 |
| null stratified U full | 0.2169 |
| null independence U pair | 0.2351 |
| null stratified U pair | 0.2468 |
| null independence U 27 | 0.2674 |
| null stratified U 27 | 0.272 |
inputs
[ "GSE252378 promoter counts" ]
equivalence check
{
"fast_median": 0.2461,
"slow_median": 0.2458,
"verdict": "EQUIVALENT"
}- If the excess overlap this corpus reports between fusion target sets is largely a coverage artefact, then cross-sample target lists used to prioritise a drug target are less reproducible than their Jaccards suggest.
- Ten deciles is a choice. Does the floor keep rising with finer stratification, and if so is there a number of strata at which the null absorbs the entire observed overlap? That would be the point at which the excess is a coverage artefact rather than shared biology.
- A promoter-level permutation ignores genomic clustering of called promoters (HOXA, for example, is a cluster of adjacent promoters). A block or chromosome-restricted permutation would be a stricter null. How much stricter?
- E1established confidence 5 of 5: Multiple independent lines of evidence; treated as settled in the field.paper
Permutation frameworks for genomic-region association exist precisely because genome organisation makes uniform placement an invalid null, and they work by restricting the randomisation space rather than by changing the statistic.
regioneR: an R/Bioconductor package for the association analysis of genomic regions based on permutation tests · Gel B, Diez-Villanueva A, Serra E, Buschbeck M, Peinado MA, Malinverni R · Bioinformatics 32:289-291 2015What this evidence cannot show
regioneR randomises interval positions on the genome; this beat permutes labels over a fixed promoter set, which is the discrete analogue and not the same operation.
Evidence that argues against this (1)
- A restriction chosen after seeing the data can be tuned until a result appears; the stratification here was named in the pre-registration before any cross-sample overlap was computed, which is the only protection available. Heger A et al. 2013
- E2strong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.paper
Promoter fragment recovery in AutoCUT&RUN/CUT&Tag of KMT2A-rearranged leukaemia tracks promoter activity and accessibility, which is what makes total promoter coverage a usable stratification variable.
Automated CUT&Tag profiling of chromatin heterogeneity in mixed-lineage leukemia · Janssens DH, Meers MP, Wu SJ, Babaeva E, Meshinchi S, Sarthy JF, Ahmad K, Henikoff S · Nature Genetics 53:1586-1596 2021What this evidence cannot show
That paper establishes the platform and its promoter-level behaviour in KMT2A-rearranged samples; it does not validate coverage as a stratification variable for an overlap null, which is this beat's inference.
Evidence that argues against this (1)
- If total coverage were partly driven by the oncoprotein signal itself, stratifying on it would remove some of the biological signal along with the confound, making every excess figure here conservative rather than merely corrected. Janssens DH et al. 2024
the null used for every excess figure in the headline
measured: 1.73x flattery in the unfiltered universe, 1.02x in a coverage-filtered one
the depth dependence is why the null must match on coverage and the calls must be depth-matched
the corrected floors are the activity-stratified ones
the null whose survival is being tested
the worked implementation of the second floor