A partner-exclusive promoter set needs a label-permutation null, not an independence one
also called exclusion-pattern null · present in k of A absent in all of B
When you claim a set of genes is switched on only in one kind of leukaemia, the honest comparison is to shuffle the labels on the patients and see how many genes look 'only in that group' by accident. Doing the arithmetic instead, as if each gene were independent, gives an answer that is sixteen times too big in the one dataset where both were tried.
The same thing, for a scientist
The pattern 'occupied in >=k of index arm A and 0 of comparator arm B' is the shape of almost every partner-specificity claim in this project. Its null cannot be computed from marginal pass rates: the exclusion term is a product over |B| samples whose occupancy is positively correlated, so an independence calculation runs high, by 16.6x at |B| = 19 in GSE252378. The correct null holds the binary occupancy matrix fixed and re-draws the index arm, which preserves each sample's pass rate, each promoter's activity and each promoter's cross-sample correlation while destroying only the partner labels.
The recipe
- Build the binary occupancy matrix once, promoters x samples, with the rule you actually intend to publish.
- Do not touch it again. Permute only the sample labels: draw |A| of the N samples as a pseudo-index arm, re-apply your exact criterion, count.
- Report both tails and the full range, not just the 95th percentile. In this beat the observation (17) was above the null mean (9.1) and the interesting statement was that it was nowhere near the 95th (21).
- Add a pass-rate-matched restriction if your real index arm has an unusual depth or enrichment profile. Here matching raised the null from 9.1 to 12.0 and the p-value from 0.11 to 0.20 - matching made the negative stronger.
- State how many permutations qualify after matching. If fewer than a couple of hundred, the matched null is underpowered and should be reported as such rather than quoted.
Why the independence shortcut is tempting and wrong
It is tempting because it is one line and because the folklore says clustering makes it conservative. The folklore is about the co-occurrence half. The exclusion half is a product of |B| terms, each of which is too close to 1 when occupancy is shared, and it swamps the other. See ana-q152-an-independence-null-overstates-recurrence-sixteenfold-in-this-matrix-and-is-not-a-lower-bound for the measured numbers.
applies to
[ "partner-specific promoter sets", "drug-specific gained-site sets", "patient-exclusive peak sets" ]
measurements
[
{
"quantity": "independence_null_error_factor",
"subject": "the exclusion-pattern null, independence versus label permutation",
"value": 16.6,
"units": "fold overstatement",
"n": 1,
"universe": "GSE252378, 19,404 promoters, index arm 8, comparator arm 19",
"null": 1,
"note": "measured once; direction expected to generalise, magnitude is |B|-dependent"
}
]- Should this project's existing partner-specificity claims (q147, q148, q149) be re-scored against a label-permutation null, and which of them were built on an independence intuition rather than an explicit null?
- E1strong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.dataset
In GSE252378 the independence expectation for 'occupied in >=3 of 8 AFF1 and 0 of 19 non-AFF1' is 150.9 promoters while the label-permutation null on the identical matrix is 9.1, a 16.6-fold discrepancy in the direction that makes an observation look significant.
Q152 independence versus label-permutation nulls on the same occupancy matrix · Iqbal Lab oncofusion knowledge graph, beat Q152 · this project 2026What this evidence cannot show
One dataset, one criterion shape. The direction of the error should generalise wherever occupancy is positively correlated across samples, but the magnitude is specific to |B| = 19 and to these pass rates.
Evidence that argues against this (1)
- A label permutation destroys the partner labels but also destroys any real shared biology within the index arm, so it can be too permissive in the opposite direction: if the eight AFF1 samples genuinely share a programme, permuted arms mixing partners will look more heterogeneous and produce more exclusive promoters, raising the null above what a fair comparison would give. A within-family or partner-matched permutation is the harder and untried test. Iqbal Lab oncofusion knowledge graph et al. 2026
the measured 16.6x error is the evidence for the concept
the jaccard-null concept is the same family of failure
the concept's recipe is what produced the branch-A verdict
the label permutation is used here exactly as that concept prescribes, with no analytic independence bound anywhere
four stratified label permutations, no analytic independence bound anywhere