The two beats' opposite answers are produced by the calling rule, not by the promoter universe: three universes give one answer, two rules give two
Two earlier analyses of the same data reached opposite conclusions. Running every combination shows the difference came from how they decided what counts as a target gene, not from which genes they looked at.
The same thing, for a scientist
q142 and q144 differed in three named ways: promoter universe (19,404 unfiltered versus 4,646 covered in all 27 samples), calling rule (a quantile rule that calls exactly 971 promoters in every sample versus the paper's absolute N/C > 1.75, p < 1e-5), and comparator (two unrelated infants versus a 351-pair cohort median). Scoring the p279/p318 pair in all six cells of {3 universes} x {2 rules}, with one depth-matching procedure and one activity-stratified null throughout, the direction of the answer is determined **entirely by the rule**: the same-patient pair falls below the unrelated KMT2A::AFF1 median excess in all three absolute-rule cells (percentiles 37, 37, 48) and above it in all three quantile-rule cells (74, 70, 70). Changing the universe moves the raw Jaccard by up to 0.11 and moves the direction not at all.
The factorial
Excess over the activity-stratified null, X = J_obs / median(J_null), with the same-patient value against the median over 27 unrelated KMT2A::AFF1 sample pairs scored identically.
| universe | rule | J | X (same patient) | X median, unrelated | percentile | direction |
|---|---|---|---|---|---|---|
| U_pair (10,248) | paper 1.75 / 1e-5 | 0.328 | 1.329 | 1.361 | 37 | q142 |
| U_27 (4,646) | paper 1.75 / 1e-5 | 0.397 | 1.459 | 1.525 | 37 | q142 |
| U_full (19,404) | paper 1.75 / 1e-5 | 0.283 | 1.305 | 1.312 | 48 | q142 |
| U_pair (10,248) | q142 quantile | 0.196 | 3.668 | 2.926 | 74 | q144 |
| U_27 (4,646) | q142 quantile | 0.212 | 4.461 | 3.943 | 70 | q144 |
| U_full (19,404) | q142 quantile | 0.164 | 2.689 | 1.903 | 70 | q144 |
Read the columns, not the rows. The universe changes J and leaves the verdict alone. The rule changes the verdict in every universe.
Why the rule and not the universe
The absolute rule lets call-set size track library depth and biology: it calls 4,347 promoters in p279 and 3,539 in p318, i.e. 42% and 35% of the pair universe. Two sets each covering ~40% of a universe overlap heavily by construction, so the raw Jaccard is large (0.33-0.40) and the excess over any null is compressed toward 1 - there is little room above a chance floor of 0.25.
The quantile rule fixes both sets at the top decile of high-N promoters: ~5% of the universe, 512 and 513 calls. The chance floor is then 0.05 and an observed 0.196 is 3.7x it. The quantile rule is measuring the agreement of the STRONGEST calls; the absolute rule is measuring the agreement of ALL calls including the marginal ones. Those are different questions and it is not surprising that they order the pairs differently.
The consequence for the corpus: a Jaccard on this data cannot be quoted without its rule, and an entry that reports one without the other is not reproducible. Neither q142 nor q144 was wrong arithmetically; they answered two questions and the corpus recorded them as one.
The comparator, which is the third divergence and the largest
q144 compared 0.400 to a cohort median of 0.132 over 351 pairs. That median is dominated by cross-fusion pairs (six partners) and by pairs with far smaller call sets - one sample calls zero. Restricting to KMT2A::AFF1 and normalising each pair against its own null removes that, and the same-patient pair's advantage disappears. The comparator, not the arithmetic, was carrying q144's conclusion.
| script | analyses/q146-one-universe-jaccard/run146.py |
|---|---|
| status | run |
| result summary | 3x2 factorial: direction of the same-patient-versus-unrelated comparison is set by the calling rule in every universe (3 cells q142, 3 cells q144). Universe changes raw J by up to 0.11 and never changes the direction. |
inputs
[ "GSE252378 promoter counts", "analyses/q146-one-universe-jaccard/results.json" ]
cells — the numbers behind the chart
{
"U_pair|paper": {
"J": 0.3281,
"X": 1.329,
"pctile": 37
},
"U_27|paper": {
"J": 0.3969,
"X": 1.459,
"pctile": 37
},
"U_full|paper": {
"J": 0.2829,
"X": 1.305,
"pctile": 48.1
},
"U_pair|quantile": {
"J": 0.196,
"X": 3.668,
"pctile": 74.1
},
"U_27|quantile": {
"J": 0.2115,
"X": 4.461,
"pctile": 70.4
},
"U_full|quantile": {
"J": 0.1642,
"X": 2.689,
"pctile": 70.4
}
}- Any target list this corpus quotes from this deposit inherits the rule that produced it. A drug-target shortlist built on the quantile rule and a resistance-mechanism claim built on the absolute rule are not comparable, and joining them would fabricate agreement.
- Is there a principled reason to prefer the strongest-calls question (quantile) or the all-calls question (absolute) for a lineage-switch comparison, or is the honest answer that both must be reported side by side forever?
- The two rules are confounded with set size (5% versus 40% of the universe). Does an absolute rule tuned to call 5% of promoters reproduce the quantile rule's ordering, which would show the divergence is about set size rather than about threshold type?
- E1established confidence 5 of 5: Multiple independent lines of evidence; treated as settled in the field.paper
The absolute rule re-implemented here (N/C > 1.75, binomial p < 1e-5) is the paper's own; the quantile rule is q142's re-implementation of a top-decile-of-high-N criterion.
MLL oncoprotein levels influence leukemia lineage identities · Janssens DH, Duran M, Otto DJ, Wu W, Xu Y, Kirkey D, Mullighan CG, Yi JS, Meshinchi S, Sarthy JF, Ahmad K, Henikoff S · Nature Communications 15:9341 2024What this evidence cannot show
The paper's rule is applied to its own peak intervals with a deposited-elsewhere CD34+ calibrator; this re-implementation applies it to refGene promoters without a calibrator, which is why absolute counts differ from the published 1,692.
Evidence that argues against this (1)
- Neither rule is the paper's full pipeline, so this factorial bounds the sensitivity of the comparison to the rule, not the correctness of the paper's own site list. Janssens DH et al. 2024
- E2established confidence 5 of 5: Multiple independent lines of evidence; treated as settled in the field.paper
The randomisation space, not just the statistic, determines whether an interval-overlap result is significant, so two analyses with different call sets are testing different nulls even when they report the same statistic.
GAT: a simulation framework for testing the association of genomic intervals · Heger A, Webber C, Goodson M, Ponting CP, Lunter G · Bioinformatics 29:2046-2048 2013What this evidence cannot show
GAT's restriction variables are genomic (isochore, chromosome), not assay-derived; the analogy to a coverage-stratified restriction is by argument, not by that paper's demonstration.
Evidence that argues against this (1)
- regioneR's authors make the opposite emphasis - that randomisation-based approaches avoid needing an explicit model - which can be read as implying the choice of restriction matters less than the avoidance of a parametric null. Gel B et al. 2015
the factorial that locates the divergence behind the headline
q142's quantile rule reproduced in three universes
q144's absolute rule reproduced in three universes
the quantity that varies here is call-set size, and it is what the rule decides
the same six cells, scored against a different comparator set
the factorial that measures the rule-driven direction flip
the same six cells scored on q142's own comparator
a candidate explanation of the divergence, cleared