RUN, by Q156, within the hour: the whole-table displacement audit - drift is 1.83%, the real blind spot is one row per symbol at 14.69%, and the case that motivated it is not displaced
also called Q155 displacement audit · promoter table release audit
The check that would say how often this project's gene-start list is out of date. It was specified here as a plan and the coordinator ran it immediately. The list is mostly fine - 82% of entries are on exactly the right spot - and the audit found a different problem instead.
The same thing, for a scientist
🟢 **PLANNED HERE, RUN BY Q156 THE SAME CYCLE.** This entry specified the audit; the coordinator executed it against the current UCSC hg38 `ncbiRefSeqCurated` (fetched 2026-09-24, HTTP 200, 4,801,131 bytes) over all 19,404 rows of `promoters.tsv`, with identities persisted [E3]. | | NM_ only | all curated | |---|---:|---:| | TSS identical to a current TSS | **15,839 (81.63%)** | 15,927 (82.08%) | | shifted, windows still overlap | 2,688 | 2,683 | | **displaced, window disjoint** | **355 (1.83%)** | 360 (1.86%) | | symbol absent from current release | 522 | 434 | | moved chromosome | **0** | **0** | | **symbol has multiple non-overlapping TSSs** | **2,851 (14.69%)** | 2,960 (15.25%) | | row window contains a foreign current TSS | 2,754 (14.19%) | 4,391 (22.63%) | **Three results, in order of importance.** 🔴 **The multi-TSS rate, 14.69%, is 8x the drift rate and is the real second blind spot.** Median span 21,291 bp; 1,939 symbols with an alternate TSS more than 10 kb from the scored one. It is a schema property and refreshing the annotation cannot fix it. 🟢 **Drift is small and bounded: 1.83%, with 81.63% TSS-identical and zero chromosome moves.** That is a reassuring negative for every promoter-scale call this project has made, and it refutes `hyp-q155-window-displacement-against-the-current-annotation-is-systematic-not-a-one-off-at-aff1`. 🟠 **Q153's sharing census reproduces across two annotation releases and two implementations**: 14.19% against 14.37% within-table, 22.63% against 21.60% full-annotation. The 21.6% this beat uses as a null is not a snapshot artefact. **And the audit corrected the entry that asked for it.** C4orf36's `d_nearest` is **0** - the row is not displaced. See `gene-c4orf36`.
The controls this audit needed, and which of them it got
Specified before the run, and reported here against what Q156 actually did:
- A spike - require the audit to flag C4orf36 at 42,310 bp. 🔴 This control would have been actively harmful, because the spike value was wrong. Q156 did not use it, and instead reported C4orf36's true
d_nearestof 0, which is how the error was found. Spiking an audit with an unverified known-positive can only confirm your own mistake. - A negative spike - a stable 5' end near zero. Satisfied: IRX1
d_nearest0, IRX2 0, AFF1 9 bp. - Count and name what the join dropped. Satisfied: 522 symbols absent from the current release are reported as their own category, not folded into displacement.
- Do not fold retirement into displacement. Satisfied - they are separate rows in the output.
- Report the load-bearing subset separately. Partly: three of the ~20 load-bearing genes are named with
d_nearestand the rest are not. This is the one control still outstanding. - Identities persisted, per the standing rule that a count you cannot re-derive is a claim. Satisfied:
identities_persisted: true, withresults_NM_only.json,results_all_curated.jsonandresults_summary.json. - A control nobody specified and the run supplied anyway: NM_-only as well as all-curated, so an NM_-derived table is not scored against NM_+NR_ and made to look worse than it is.
| script | analyses/q156-promoter-window-displacement-audit/run.py (executed by the coordinator, not by this beat) |
|---|---|
| status | run |
| result summary | RUN by Q156 the same cycle. Drift 1.83% (355 of 19,404), 81.63% TSS-identical, 0 chromosome moves. The larger blind spot is one row per symbol at 14.69% (2,851 symbols, median span 21,291 bp, 1,939 with an alternate TSS >10 kb away). C4orf36 d_nearest = 0, refuting the displacement claim this entry's beat had made. Q153's sharing census reproduced at 14.19% / 22.63% against 14.37% / 21.60%. |
| cost tier | low |
| no wet work | true |
| outstanding | The load-bearing-subset breakdown: three of ~20 genes are reported with d_nearest, the rest are not. And the 355 displaced rows have not been re-scored for occupancy. |
inputs
[ "C:/Temp/q145irx1/promoters.tsv (19,404 rows, hg38 refGene snapshot dataTime 2020-08-17)", "UCSC hg38 ncbiRefSeqCurated, current release, downloaded in bulk", "C:/Temp/q147part/promoter_matrix.tsv for the re-scoring step" ]
outputs required
[ "per-row d_nearest, overlaps, foreign_tss_in_window, n_models - PERSISTED AS IDENTITIES, not as counts", "the list of symbols dropped by the join because they have no current row", "the whole-table rate AND the rate among the ~20 load-bearing promoter genes, reported separately", "the count of occupancy calls that change when a displaced row is re-scored at its current window" ]
measurements — the numbers behind the chart
[
{
"quantity": "pct_rows_tss_identical",
"subject": "promoter-table rows whose TSS is identical to a current curated TSS for the same symbol",
"value": 81.63,
"units": "percent of rows",
"n": 19404,
"universe": "promoters.tsv (2020 refGene snapshot) joined against current UCSC hg38 ncbiRefSeqCurated, NM_-only (Q156)",
"null": null,
"note": "82.08% under the all-curated comparison"
},
{
"quantity": "pct_rows_displaced",
"subject": "rows whose 2 kb window is disjoint from every current window for the same symbol",
"value": 1.83,
"units": "percent of rows",
"n": 19404,
"universe": "same join, NM_-only; 355 rows; 0 rows moved chromosome (Q156)",
"null": 7.2,
"note": "null is Q153's blind spot 1 at 7.2 percentage points; drift is four times smaller"
},
{
"quantity": "pct_symbols_multi_tss",
"subject": "symbols with more than one non-overlapping current TSS",
"value": 14.69,
"units": "percent of rows",
"n": 19404,
"universe": "same join, NM_-only; 2,851 symbols, median span 21,291 bp, p90 163,303 bp, 1,939 with an alternate TSS >10 kb away (Q156)",
"null": 1.83,
"note": "the null is the drift rate from the same audit; this is 8x it and is not fixable by refreshing the annotation"
},
{
"quantity": "pct_rows_with_a_foreign_tss_in_window",
"subject": "Q153's sharing census, recomputed on the current release by a different implementation",
"value": 22.63,
"units": "percent of rows",
"n": 19404,
"universe": "same join, all-curated (Q156); the NM_-only figure is 14.19%",
"null": 21.6,
"note": "the null is Q153's own full-annotation figure of 21.60% on the 2020 release; within-table 14.19% against Q153's 14.37%. Two releases, two implementations, the same two numbers"
}
]outputs delivered
[ "results_NM_only.json", "results_all_curated.json", "results_summary.json" ]
- If this audit is quoted as 'the promoter table is validated' it will be quoted wrongly. It bounds one of three known blind spots at a reassuringly low rate and measures a second at eight times that rate; Q153's first blind spot at 7.2 percentage points is untouched by either.
- And the spike control this entry specified would have propagated my own error into the audit. An audit spiked with an unverified known-positive can only confirm the mistake that produced the spike.
- The load-bearing subset. Q156 names IRX1, IRX2 and AFF1; the other ~17 genes carrying this project's promoter claims have not been reported with d_nearest, and that is the number that decides whether anything needs re-running.
- The 522 symbols absent from the current release are a larger group than the 355 displaced ones and nobody has looked at them. A row whose symbol no current annotation claims is measuring a coordinate with no owner.
- Re-scoring the 355 displaced rows and the 2,851 multi-TSS symbols at their unscored TSSs. C4orf36 goes from 0/27 to 23/27 that way; the whole-table version is the same machinery.
- E1established confidence 5 of 5: Multiple independent lines of evidence; treated as settled in the field.database
WITHDRAWN. This item previously read: 'The one row checked so far, C4orf36, is displaced by 42,310 bp ... so the audit has at least one known-positive case to be spiked with.' C4orf36 is NOT displaced: its row TSS 86,892,422 is one of the symbol's three current curated TSSs and carries 13 of its 20 curated transcripts, so d_nearest = 0. The 42,310 bp came from a UCSC track query restricted to chr4:86,915,000-86,945,000, which cannot contain that transcript.
UCSC Genome Browser hg38 ncbiRefSeqCurated track, chr4:86,800,000-87,000,000 · UCSC Genome Browser, api.genome.ucsc.edu · UCSC Genome Browser 2026What this evidence cannot show
Re-fetched over a 200 kb window specifically to check the figure the 30 kb window had produced. The withdrawn claim is kept in the claim text rather than deleted, so a reader can see what was asserted as well as what happened.
Evidence that argues against this (1)
- The withdrawn spike would have been actively harmful: an audit tuned until it recovers an unverified known-positive can only confirm the mistake that produced it. Q156 did not use the spike, which is why the error surfaced rather than being validated. UCSC Genome Browser et al. 2026
- E2strong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.dataset
Q153's sharing census - 14.37% within-table and 21.60% against the full annotation, computed on the 2020 refGene snapshot - reproduces on the current release with a different implementation at 14.19% and 22.63%, so it is not a snapshot artefact; the AFF1 window is a worked case where the current release adds a third TSS the 2020 census could not see.
21.6 percent of this project's 19,404 promoter windows contain another gene's TSS · Iqbal Lab oncofusion knowledge graph, beat Q153 · this project 2026What this evidence cannot show
The reproduction figures are Q156's and are quoted, not re-derived here. The two implementations differ in their transcript filter (NM_-only vs all-curated), which is why the pair 14.19/22.63 is compared against the pair 14.37/21.60 rather than a single number against a single number.
Evidence that argues against this (1)
- Two implementations agreeing is weaker than it looks when both are joins of the same table against a RefSeq-derived annotation; they share the promoter table, the window convention and the coordinate system. It rules out a snapshot artefact and an implementation slip, not a shared assumption. UCSC Genome Browser et al. 2026
- E3strong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.dataset
The audit specified in this entry was run by Q156 over all 19,404 rows against the current UCSC hg38 ncbiRefSeqCurated: 81.63% TSS-identical, 1.83% displaced with a disjoint window, 0 moved chromosome, 522 symbols absent from the current release, 14.69% of symbols with multiple non-overlapping TSSs (median span 21,291 bp), and a foreign-TSS-in-window rate of 14.19% NM_-only and 22.63% all-curated that reproduces Q153's 14.37% and 21.60%.
The promoter table's second blind spot is not annotation drift at 1.8% - it is one row per symbol at 14.7%, and the case that motivated the question is not displaced at all · Iqbal Lab oncofusion knowledge graph, beat Q156 (coordinator) · this project 2026What this evidence cannot show
Q156's whole-table figures were run by the coordinator and are QUOTED here, not re-derived; only its C4orf36 result was independently re-checked by this beat, against UCSC ncbiRefSeqCurated over a 200 kb window. The 1.83% and 14.69% should be re-run before being used as load-bearing quantities elsewhere.
Evidence that argues against this (1)
- That the C4orf36 row is not displaced makes the situation WORSE, not better. A displaced row can be fixed by refreshing the annotation; a symbol with three legitimate TSSs and a one-row-per-symbol schema cannot be, and Q156 measures that class at eight times the displacement rate. UCSC Genome Browser et al. 2026
executes the audit that entry specified and left PLANNED
the planned analysis that would confirm or falsify it
would re-run Q153's census against the current annotation rather than the 2020 snapshot it was computed on
executed the audit this entry planned, the same cycle, with identities persisted