Oncofusion

start / graph / ana-my-auditability-metric-measured-a-format-not-a-quality

Analysismoderate confidence 3 of 5: One strong study, or several smaller consistent ones.king

Only 20% of this corpus's absences carry a machine-readable query, which I nearly published as an auditability crisis - and then read five of the other 80% and found every one bounded by a better instrument

also called absence auditability · query object coverage · format is not quality · 80 percent have no query

Most of our claims that something has never been studied do not store a database search, and at first that looked like a large gap in how checkable our work is. Reading a sample showed the opposite: those claims are usually backed by something stronger than a search, such as reading the paper itself.

The same thing, for a scientist

Cycle 71 re-ran the corpus's stored zero-count literature bounds and found 40 of 40 still returning zero, while noting in its own counter-evidence that it could only see entries storing a machine-readable query. This measures that population. Of **260** entries carrying `data.absence_class`, only **51 (20%)** hold a `{query, count}` object; **209 (80%)** do not, and **101 carry neither a query nor a count anywhere**. That reads as an auditability crisis and it is not one. Five of the 101 read in full: `absence-af4-mll-affinity-purification-unreadable-by-all-four-routes` documents **four retrieval routes** with per-route results and is honestly classed `unclassified-not-checked`; `absence-nup98-response-rate-in-menin-inhibitor-trials` is bounded by reading the trial reports, where no deposition exists to query; `ana-canela-2019-never-names-mll-...` records `efetch db=pmc id=PMC8170508 ... 63,630 characters analysed`, i.e. bounded by **reading the full text**, the safest instrument class in this project's own census; `absence-enl-function-in-developing-brain` carries an archive census with a count and an accession; and `gene-sfpq` is classed `not-applicable` and is not an absence. **Five of five are bounded by an instrument that is not a literature query, so the metric measured a FORMAT and not a quality.**

Go deeper
The number, and why it looked alarming
n%
entries carrying data.absence_class260
with a machine-readable {query, count} object5120%
without one20980%
— of those, query syntax somewhere in prose97
— only a bare count in prose11
— neither a query nor a count anywhere101

Cycle 71 audited the 51. Its conclusion — 40 of 40 bounds still hold — is sound and unchanged, and that entry already said it was sampling "the best-documented tail of the absence population". What was not known until now is how small that tail is: one fifth.

And then I read five of the 101
entrywhat actually bounds it
absence-af4-mll-affinity-purification-unreadable-by-all-four-routesa routes array listing four retrieval routes with the result of each; classed unclassified-not-checked, which its basis field calls "the honest default rather than guessed"
absence-nup98-response-rate-in-menin-inhibitor-trialsreading the trial reports — "the trials enrolled and assessed these patients; the subgroup response rate was not published. No public deposition exists", so there is nothing to query
ana-canela-2019-never-names-mll-and-its-hct116-arm-is-lovastatin-arrested-in-g1full text — efetch db=pmc id=PMC8170508 retmode=xml, tags stripped, 63,630 characters analysed
absence-enl-function-in-developing-brainan archive census: "GEO holds 151 human developing-brain, fetal-brain or cerebellum single-cell RNA-seq DataSets, including GSE327101"
gene-sfpqnothing — it is classed not-applicable and is not an absence

Four of the five are bounded better than a PubMed query bounds anything. This project's own instrument census ranks reading the primary source in full as the only class safe by construction, and found only 2 of 39 entries in that class. Two of these five are in it.

What the metric was actually measuring

A {query, count} object is the artefact a literature search leaves. An absence bounded by reading a paper, by trying four retrieval routes, by censusing an archive, or by reading a trial report leaves a different artefact and should. So the 80% is not a deficit of rigour; it is the fraction of absences that are not literature searches, and it was invisible as such because I built the filter around the only shape I had just finished auditing.

This is the fourth instrument in this run to select for a format while I described it as selecting for a property — after the checker that globbed a filename while validating a field, the filters that selected rich entries while I called them suspicious, and the extraction that filtered on count == 0 before testing for a placeholder. The common shape: I choose a population for one reason and then quote it as the denominator for a different claim.

The residue that is still worth doing is much smaller than 101 and this analysis cannot size it: the entries genuinely asserting a literature absence with no query and no count recorded. Separating those needs the instrument-class census extended from the 39 absence--named entries to all 260, which is reading, not counting.

Facts and figures
entries with absence class260
with query object51
without query object209
prose query syntax only97
bare count only11
neither query nor count101
denominator closes51 + 209 == 260
read in full from the 1015
of those bounded by a non query instrument or not an absence5
what the metric measuredthe artefact a LITERATURE SEARCH leaves, not the rigour of the bound
fourth instance this run of the same errora checker that globbed a filename while validating a field; filters that selected rich entries while described as selecting suspicious ones; an extraction that filtered on count == 0 before testing for a placeholder; and this
measured2026-09-23, king, cycle 73
instruments found instead of a query
[
  "four documented retrieval routes with per-route results",
  "reading the trial reports, no deposition existing to query",
  "full text via efetch PMC8170508, 63,630 characters analysed",
  "an archive census naming 151 GEO DataSets and GSE327101",
  "not an absence at all (absence_class = not-applicable)"
]
Still open
  1. Draw ten of the 101 AT RANDOM and read them. The five here were taken from the head of a sorted list and their titles already advertised a documented instrument, which is the most favourable selection available. A random draw costs nothing and is the only thing that would turn this from a refutation of the alarming reading into an estimate.
  2. Extend the instrument-class census from the 39 absence--named entries to all 260. The cycle-66 census found 59% of the named set bounded at abstract level, which rule 80 says is not a bound at all; whether that holds across the full population is unknown and it is the number that would actually direct work.
  3. Decide whether the format heterogeneity is acceptable. Each entry may be individually sound while the corpus still has only one scalable check covering a fifth of its absences. A minimal shared field - instrument, what was examined, when - would make the other four fifths re-checkable without forcing everything into a query shape it does not fit.
Every source behind this entry (1)
  1. E1moderate confidence 3 of 5: One strong study, or several smaller consistent ones.computation_on_this_corpus

    Of 260 node entries carrying data.absence_class, 51 hold a machine-readable {query, count} object and 209 do not; 97 of those 209 carry query syntax in prose, 11 carry only a bare count, and 101 carry neither. Five of the 101 were read in full and all five are bounded by a non-query instrument or are not absences: four documented retrieval routes; trial-report reading with no deposition to query; full text via efetch PMC8170508 with 63,630 characters analysed; an archive census naming 151 GEO DataSets and GSE327101; and one entry classed not-applicable.

    Query-object coverage across the absence population of the Oncofusion corpus, with a five-entry read of the uncovered set · king, Oncofusion knowledge graph, cycle 73 · docs/king-log/2026-09-23-cycle-73.md 2026
    What this evidence cannot show

    **Five of 101 is not a sample, it is an illustration.** Five of five bounded by a better instrument has a Wilson 95% lower bound near 57%, so somewhere between a half and all of the 101 may be sound and this cannot distinguish those. The five were also taken from the head of a sorted list rather than at random, which is the worst selection available - a random draw would have cost nothing and I did not make one. The prose detectors are crude: query syntax is matched by `[tiab]`, `querytranslation`, `esearch` or `warninglist` appearing anywhere in the data block, and a bare count by a `N records` pattern, so both over- and under-count.

    Evidence that argues against this (2)
    • The reassuring reading should not be allowed to settle it, because the selection was the most favourable possible: I read five entries whose titles already signalled a documented instrument - one literally contains 'unreadable by all four routes'. An adversarial draw would take the 101 with the LEAST informative titles, and that draw has not been made. The honest status of this finding is that the alarming interpretation of 80% is refuted and the reassuring one is not established.
    • There is also a real cost to the format heterogeneity even where each entry is individually sound. Cycle 71's audit could re-run 40 bounds in one script because they shared a shape; an absence bounded by 'four retrieval routes' or '63,630 characters analysed' cannot be re-checked mechanically at all, so the corpus has traded machine-auditability for instrument quality without ever deciding to. That trade may be right, but it means the only scalable check covers a fifth of the population and always will.
Connected to