Confidence in this graph carries no information about whether anyone read the paper
also called reading depth · abstract-only citation · declared shallowness
We count a paper as read if any entry cites it, which treats a passing mention like a careful reading. Nearly a third of our citations say in their own notes that only the abstract was seen, and our confidence labels do not track that at all.
The same thing, for a scientist
Of **3,003 paper-citing evidence items**, **731 (24.3%) declare that only the abstract was read**, using this project's own caveat convention — and that is a **floor**, because silence is not evidence of depth. **52.6% of the 1,155 distinct papers are cited in exactly one entry.** The reassuring finding: the widest-reach papers ARE read deeply — the 2023 recombinome is 1.3% declared-shallow across 76 items, the MEN1 resistance paper 0.0%, AUGMENT-101 4.3%. The finding that is not reassuring: **declared depth does not vary with stated confidence.** `established` items are 25.7% abstract-only against `strong` 30.0% and `moderate` 30.9% — flat.
The load-bearing beams are read; the confidence labels are not earning their keep
Two results point in opposite directions and both matter.
The papers this graph rests on hardest are the ones it read hardest. ana-load-bearing-citations found the 2023 KMT2A recombinome supporting 76 items across 70 entries, and asked what happens if it is misread. This says 1.3% of those items declare abstract-only. The next four widest — 28, 25, 23 and 18 entries — run 10.7%, 0.0%, 4.3% and 0.0%. Whatever else is true, the graph did not build its main beams out of abstracts.
But established is not a deeper-read label than moderate.
| confidence | items | declared abstract-only |
|---|---|---|
| established | 533 | 25.7% |
| strong | 1,597 | 30.0% |
| moderate | 653 | 30.9% |
| weak | 88 | 20.5% |
That is flat, and weak is the least shallow band of all. Whatever the confidence field is tracking — how settled the claim seems, how many sources agree, how the writer felt — it is not tracking how much of the source was read. A reader who treats established as 'someone checked this carefully' is reading something the label does not say.
Beat variation is enormous, and partly an artefact of age
| beat | paper items | declared abstract-only |
|---|---|---|
| Q4c Marrow niche | 79 | 100.0% |
| Q3b Histone marks | 148 | 98.6% |
| Q11d KMT2A-side subtraction | 90 | 77.8% |
| Q5 Interactome | 359 | 60.7% |
| Q6b Resistance saturation | 35 | 60.0% |
The convention spread through the standing brief over several cycles, so a beat written recently declares more and a beat written early declares less, independently of how it was actually researched. Some of this table is honesty, not shallowness.
One entry in it deserves naming. Q3b histone marks is 98.6% declared abstract-only across 148 items, and it is the beat that produced the CRISPR-ChIP partition of H3K79 methylation — which the cycle-48 log called the single highest-value question in the graph, and which a beat is checking right now. That question turns on which fusion the experiment used, and the abstract does not say. This is what it looks like when a graph's most important open question sits on its least deeply read beat.
Correction: the classifier counted its own negations
Corrected 2026-09-19. The classifier behind this number had no negation handling: a caveat reading “Full text read (PMC4523210), not abstract only” contains the literal words “abstract only” and was counted as a declaration of shallow reading. 165 of 888 matches (18.6%) were negations of exactly this kind. The pattern now requires that no phrase asserting the full text WAS read appears, with a negative lookbehind so that “not read in full” is not itself cancelled. Found by reading the top item of the reading queue this number produced.
The headline moves from 29.1% to 24.3% (731 of 3,003 paper-citing items). The direction and the argument are unchanged; the level was inflated by about a fifth.
What makes this worth a section rather than a silent edit is where the correction came from. The number was used to build a reading queue; the first item read off that queue was pt-story-congenital-aml-kmt2a-mllt1-subdural-haemorrhage E1, whose caveat opens “Full text read (PubMed Central, PMC4523210), not abstract only”. The instrument was only caught because someone acted on its output. A measure that is reported and never used is never tested.
- Condition the declaration rate on the cycle an entry was written, to separate honesty from shallowness. The beat table currently confounds them.
- Decide what `confidence` means in the schema. It is documented as a property of the claim and read by humans as a property of the checking; those diverge and the field is load-bearing in the go/no-go and trial-design entries.
- The 607 papers cited in exactly one entry are the cheapest place to look for a citation that does not support what it is attached to — the failure this project has already made twice with plausible-looking PMIDs.
- E1established confidence 5 of 5: Multiple independent lines of evidence; treated as settled in the field.analysis
Of 3,003 evidence items citing a PMID, 731 (24.3%) declare in their own caveats that only the abstract was read, using this project's caveat convention. 45.6% of paper-citing claims state a quantity. Of 1,155 distinct papers cited, 607 (52.6%) appear in exactly one entry.
Reading-depth audit: self-declared abstract-only citation across the Oncofusion knowledge graph · Oncofusion knowledge graph (internal analysis) 2026What this evidence cannot show
Run by the coordinator on 2026-09-22 over all 853 entries. An item counts as self-declared shallow if its caveats match this project's own convention — 'abstract read', 'full text not read', 'abstract-level', 'esummary only', 'title-only', 'no abstract'. Only the 3,003 items citing a PMID are in scope; database, registry and internal-analysis citations are not. **Corrected 2026-09-19.** The classifier behind this number had no negation handling: a caveat reading *“Full text read (PMC4523210), not abstract only”* contains the literal words “abstract only” and was counted as a declaration of shallow reading. **165 of 888 matches (18.6%) were negations of exactly this kind.** The pattern now requires that no phrase asserting the full text WAS read appears, with a negative lookbehind so that “not read in full” is not itself cancelled. Found by reading the top item of the reading queue this number produced.
Evidence that argues against this (2)
- Self-declaration gives a FLOOR on shallow reading and cannot give a rate. Silence is not evidence of depth: an item whose caveats say nothing may have been read closely, skimmed, or written from a search-result snippet, and this measure cannot tell those apart. The true fraction is higher than reported by an unknown amount. The honest headline is therefore 'at least 24.3%', and a reader should assume the real figure is materially higher. Oncofusion knowledge graph (internal analysis) 2026
- Abstract-only citation is not automatically a defect and this measure cannot separate the cases. Citing a paper for the existence of a result, for a trial's registration, or as one of several converging sources needs the abstract and no more. What the number cannot show is how often an abstract was used where methods were needed — which is the question worth asking and the one this does not answer. Oncofusion knowledge graph (internal analysis) 2026
- E2strong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.analysis
Declared reading depth does not vary with stated confidence: items labelled `established` are 25.7% declared abstract-only (n=533), `strong` 30.0% (n=1,597), `moderate` 30.9% (n=653) and `weak` 20.5% (n=88) — with `weak` the least shallow band of the four.
Reading-depth audit: self-declared abstract-only citation across the Oncofusion knowledge graph · Oncofusion knowledge graph (internal analysis) 2026What this evidence cannot show
Run by the coordinator on 2026-09-22 over all 853 entries. An item counts as self-declared shallow if its caveats match this project's own convention — 'abstract read', 'full text not read', 'abstract-level', 'esummary only', 'title-only', 'no abstract'. Only the 3,003 items citing a PMID are in scope; database, registry and internal-analysis citations are not.
Evidence that argues against this (2)
- This is arguably correct behaviour rather than a defect. Confidence in this graph is defined as how settled a CLAIM is, not how thoroughly a SOURCE was read, and a well-replicated fact stated in an abstract can honestly be `established` on an abstract reading. The finding is that the two are independent, and whether that is a problem depends on what a reader assumes `established` means — which the schema does not say. Oncofusion knowledge graph (internal analysis) 2026
- The convention is also unevenly applied. It spread through the standing brief over several cycles, so entries written early carry it less often than entries written late, and a beat can look deep purely for being old. Beat-level comparisons are therefore partly a measure of when the beat was written. If the convention is applied more by recent beats, and recent beats also write more `moderate` claims, the flatness could be a composition effect rather than a real independence. Testing that needs the declaration rate conditioned on cycle, which this run does not compute. Oncofusion knowledge graph (internal analysis) 2026
- E3strong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.analysis
The graph's widest-reach papers are among its least shallowly cited. The 2023 KMT2A recombinome, supporting 77 items across 71 entries, is 2.6% declared abstract-only; the next widest-reach papers run 16.1%, 0.0%, 0.0% and 0.0% across 31, 25, 23 and 18 entries respectively.
Reading-depth audit: self-declared abstract-only citation across the Oncofusion knowledge graph · Oncofusion knowledge graph (internal analysis) 2026What this evidence cannot show
Figures regenerated 2026-09-19 after the shallow-reading classifier was corrected for negation handling; the previous version of this item read 76 items / 70 entries / 1.3%. **The citation was also wrong in kind.** This is a claim about THIS GRAPH's citation behaviour, and it carried the recombinome paper as its source — a paper that says nothing about how this project cites it. It now cites the analysis that measured it. Surfaced by `scripts/reading-queue.py`, which flagged the item for low claim-to-abstract overlap; the overlap was low because the abstract could not possibly contain the claim.
Evidence that argues against this (3)
- Low declared shallowness on a heavily reused paper may reflect the convention being applied less often when a citation is routine, rather than the paper being read more. A source cited seventy times is exactly the one an agent is least likely to re-examine before citing, and most likely to copy a block for — which this project has separately had to forbid. Oncofusion knowledge graph (internal analysis) 2026
- Depth of reading is not the risk `ana-load-bearing-citations` identified for this paper. Its risk is concentration: 70 entries moving together if the census is superseded, which careful reading does not mitigate. Oncofusion knowledge graph (internal analysis) 2026
- A citation that is wrong in kind is invisible to every check this project runs. `verify-citations.py` confirms the PMID resolves and the title matches, which it did; `verify-claims.py` scores the claim against the abstract and ranked it low, which was read as 'the abstract cannot contain this' rather than 'this paper is not the source'. The two readings are indistinguishable from the score alone. Oncofusion knowledge graph (internal analysis) 2026
- E4moderate confidence 3 of 5: One strong study, or several smaller consistent ones.analysis
Declared shallowness varies from 0% to 100% across beats. Q4c marrow niche is 100% over 79 paper items and Q3b histone marks 98.6% over 148 — and Q3b is the beat that produced the CRISPR-ChIP partition of H3K79 methylation, recorded as the graph's highest-value open question.
Reading-depth audit: self-declared abstract-only citation across the Oncofusion knowledge graph · Oncofusion knowledge graph (internal analysis) 2026What this evidence cannot show
Run by the coordinator on 2026-09-22 over all 853 entries. An item counts as self-declared shallow if its caveats match this project's own convention — 'abstract read', 'full text not read', 'abstract-level', 'esummary only', 'title-only', 'no abstract'. Only the 3,003 items citing a PMID are in scope; database, registry and internal-analysis citations are not.
Evidence that argues against this (2)
- The convention is also unevenly applied. It spread through the standing brief over several cycles, so entries written early carry it less often than entries written late, and a beat can look deep purely for being old. Beat-level comparisons are therefore partly a measure of when the beat was written. Q3b was written in cycle 48, after the convention was fully established in the brief, and Q4c earlier; comparing them on declaration rate compares two different reporting regimes as much as two research efforts. Oncofusion knowledge graph (internal analysis) 2026
- A beat being abstract-read does not make its findings wrong, and Q3b's central result illustrates the distinction: the partition is reported in the abstract and is citable from it, while the question that decides its relevance — which fusion was used — is not in the abstract at all. The problem is not depth in general but depth at the specific point where the graph needs it. Oncofusion knowledge graph (internal analysis) 2026
both measure whether a quality number is reporting what its name implies
both measure whether a quality number reports what its name implies