Oncofusion

start / graph / concept-q147-a-spearman-over-exact-zeros-is-tie-degenerate-and-argsort-picks-one-value

Explainerestablished confidence 5 of 5: Multiple independent lines of evidence; treated as settled in the field.Q147-hoxa-irx1-across-partners

A Spearman over exact zeros is tie-degenerate, and argsort-of-argsort silently picks one value from a range

Ranking numbers that are all exactly zero has no single right answer. A common one-line way of computing rank correlation quietly picks one of the possible answers, and which one it picks depends on the order the samples happen to be in.

The same thing, for a scientist

`np.argsort(np.argsort(x))` is the idiomatic one-line rank, and this corpus uses it in at least q145's `controls145.py`. **It breaks ties by input order rather than averaging them**, which for occupancy data is not a corner case: a promoter that reads exactly 0.000 in several libraries is the normal outcome of a CUT&RUN experiment at an unoccupied locus. Measured on the actual vectors. At 2 kb promoter scale, three of the eight KMT2A::AFF1 IRX1 values are **exactly 0.0**. Enumerating every arbitrary tie-break ordering of those three gives **four distinct rho values spanning -0.5714 to -0.3810**, while the **tie-corrected (average-rank) value is -0.4880** [E1]. q145's committed promoter-scale figure is **-0.571** - the most negative of the four attainable values [E1]. Against HOXA9 alone the same enumeration spans **-0.6429 to -0.4524** with a tie-corrected value of -0.5611 [E1]. At gene-body scale, where q145's headline lives, **there are no exact ties** and rho = -0.71429 is a single value with no degeneracy [E1]. So the defect bites the promoter-scale ladder rung and not the headline - which is the kind of thing only enumeration tells you.

Go deeper
What to do instead
def avgrank(a):                     # average ranks, ties shared
    a = np.asarray(a, float)
    o = np.argsort(a, kind='mergesort')
    r = np.empty(len(a), float); i = 0
    while i < len(a):
        j = i
        while j + 1 < len(a) and a[o[j+1]] == a[o[i]]: j += 1
        r[o[i:j+1]] = (i + j) / 2.0; i = j + 1
    return r

scipy.stats.spearmanr and scipy.stats.rankdata already do this; the hand-rolled argsort(argsort(...)) does not, and the two differ only when ties are present, which is why it passes every smoke test.

Three rules follow:

  1. Count the exact ties and print the count beside the rho. n_exact_zero = 3 of 8 is the warning label.
  2. When ties are present, report the tie-corrected value and the arbitrary-tie-break range. The range is cheap to enumerate at these n and it is the honest error bar on the instrument, separate from the sampling error bar.
  3. Do not read a tie-broken rho as more extreme evidence. Here the arbitrary choice moved the value by 0.19 in rho, and in the favourable direction.
Why this is the same shape as two other traps in this corpus

It joins the family where the instrument supplies an answer the data did not:

  • a fallback regex that finds an accession inside a subunit map and reports it as the entry's own (rule 69b.3);
  • a flat control reaching Spearman -0.619 on values that are all <= 0.09x, which is why this beat put an absolute floor on every locus entering a correlation;
  • and this: a tie-break rule inventing an ordering among identical zeros.

In all three the code runs, nothing errors, and the number is a property of the procedure.

Facts and figures
promoter scale tie corrected-0.488
promoter scale distinct values4
n exact zeros of 83
hoxa9 only tie corrected-0.5611
gene scale tie corrected-0.7143
bad idiomnp.argsort(np.argsort(x))
good idiomscipy.stats.rankdata / average-rank helper
promoter scale tiebreak range
[
  -0.5714,
  -0.381
]
hoxa9 only tiebreak range
[
  -0.6429,
  -0.4524
]
gene scale tiebreak range
[
  -0.7143,
  -0.7143
]
analyses
[
  "ana-q147-irx1-fusion-occupancy-is-confined-to-kmt2a-aff1"
]
Still open
  1. How many stored rank correlations in this corpus were computed on vectors containing exact zeros? The tie count was never recorded beside any of them, so the exposure is unknown.
Every source behind this entry (1)
  1. E1established confidence 5 of 5: Multiple independent lines of evidence; treated as settled in the field.dataset

    Enumerated over all arbitrary tie-break orderings of the exactly-zero values: at 2 kb promoter scale the eight-sample KMT2A::AFF1 Spearman(HOXA, IRX1) spans -0.5714 to -0.3810 (four distinct values, three exact zeros among the eight IRX1 values) with a tie-corrected value of -0.4880; against HOXA9 alone it spans -0.6429 to -0.4524 with a tie-corrected value of -0.5611. At gene-body scale there are no exact ties and rho = -0.71429 is unique. q145's committed promoter-scale value, -0.571, is at the extreme of the attainable range.

    What this evidence cannot show

    The enumeration is exhaustive and exact, so there is no sampling uncertainty in the range itself. It is a statement about one vector of eight numbers and does not generalise to a claim about how often this happens in the corpus - that has not been measured. The comparison to q145's -0.571 is to a figure in its `body_md` scale ladder, not to its headline.

    Evidence that argues against this (1)
    • Whether q145's promoter-scale -0.571 came from argsort tie-breaking or from a slightly different HOXA aggregation cannot be settled from the committed artefacts alone: -0.5611 (tie-corrected, HOXA9 only) rounds to -0.56 and -0.5714 (tie-broken) to -0.57, and both are consistent with the text. The measured fact - that the value is degenerate over a 0.19-wide range - holds either way. Iqbal Lab oncofusion knowledge graph et al. 2026
Connected to