Comparing every citation's stored year and journal against the record its DOI actually points at - 16,420 comparable blocks, 92.0% exact, and all 22 that disagree by two years or more resolve on reading to six DOIs with online-versus-print date conventions, so the wrong-paper failure mode is bounded from above for the first time
also called no citation disagrees with its own DOI · 92.0% exact on year · the 22 outliers are six DOIs and all are date conventions · cycle 119's fix, executed on fields that already existed
The dangerous kind of citation error in this project is not a broken link — it is a working link to the wrong paper, which has happened twice and both times a human caught it by reading. A previous attempt to catch that automatically failed completely. This one compares the year and journal each note records against the year and journal of the paper its link actually opens. Sixteen thousand notes agree; the twenty-two that did not turned out to be papers published online in one year and in print in another.
The same thing, for a scientist
**Statement.** The DOI audit found **zero broken live citations**. The failure that actually matters here is the opposite: **a valid identifier pointing at the wrong paper**, which has occurred twice and both times was caught by a human reading an abstract. **Cycle 119 tried to catch it automatically and returned `M3` — precision ZERO.** Its diagnosis was structural and correct: *this project's citation `title` is a descriptive sentence by house style*, so there is nothing for a wrong identifier to disagree with. It proposed storing the live title, called it *"the single cheapest hygiene improvement available"*, and **four cycles later it has not been done.** **But the schema already carries two fields that are the paper's own properties: `year` and `venue`.** Neither has ever been compared against anything. | | | |---|---:| | citation blocks carrying a DOI | **16,463** | | comparable on year | 16,420 | | **exact match** | **15,104 (92.0%)** | | off by one — online-first (`U2`) | 1,294 | | **disagree by 2+ years** | **22** | | with a DOI but no year (`U3`) | 8 | **`Y3` was pre-registered to fire if `venue` was house style too — it did not.** 13,393 of 16,267 blocks (**82.3%**) share a significant word with the publisher's container title, so venue is usable, which was not obvious given how the `title` field is written. **The 22 outliers collapse to SIX distinct DOIs, and reading every one found no wrong paper:** | DOI | blocks | stored → resolved | what it is | |---|---:|---|---| | `10.1111/bjh.18021` | **12** | 2023 → 2021 | *Br J Haematol* online 2021, print 2023 | | `10.2210/pdb3U84/pdb` | 3 | 2026 → 2012 | a **PDB structure** cited with the analysis year | | `10.1093/rfs/hhm075` | 3 | 2009 → 2007 | online 2007, print 2009 | | `10.12688/wellcomeopenres.11589.4` | 2 | 2017 → 2021 | **version 4** of a versioned preprint | | `10.1136/jmedgenet-2020-107225` | 1 | 2022 → 2020 | online 2020, print vol 59 | | `10.3324/haematol.2019.225953` | 1 | 2021 → 2019 | online 2019, print 2021 | **18 of 22 are journal online-versus-print splits of more than one year**, which means my pre-registered one-year tolerance was simply too tight. **Zero are wrong-paper citations.** 🔵 **So 'right identifier, wrong paper' is bounded from above for the first time in this project.** 🔴 **`U1` binds hard and the bound is loose:** thousands of papers share a year, so agreement means *not obviously wrong*, never *correct*. **This check has no power to confirm anything — only to disagree.**
Why this worked where cycle 119's version returned precision zero
Cycle 119 compared the stored title against the live title and every low scorer it read was a correct citation, because the stored string was never meant to be the paper's title. It concluded the schema needed a new field.
It did not need one. year and venue were already there, already written in the publisher's form, and already unused. The fix was not a schema change but a different column, and four cycles of open questions proposed the expensive version of it.
🟠 That is worth recording as a process finding rather than a win. The specified fix was store the live title; the executed fix is use the fields you have. A proposal that names one solution tends to suppress the search for a cheaper one, and this project has now carried the expensive proposal across four king logs without anyone asking whether a different field would do.
What the instrument can and cannot catch
It catches: a DOI transcribed from a different paper of a different year — the actual documented failure mode here, where a recalled identifier resolved to a Trypanosoma brucei paper, an asciminib paper and a cheese-whey biohydrogen paper.
It cannot catch: a wrong DOI that happens to point at a paper of the same year. With ~15,000 blocks and a typical citation window, same-year collisions are common, so the residual risk is large and unquantified.
It also cannot catch a correct DOI attached to a claim the paper does not support, which is the thing that actually matters scientifically and which no automated check in this project touches. Cycle 119 said this and it remains true.
🔵 The cheap remaining upgrade is now trivial, because doi-index.json caches the resolved title for 3,424 DOIs: comparing the resolved title against the entry's stored authors field would add a second, near-independent axis at no fetch cost.
| question | does each citation's stored year and journal match the record its DOI points at? |
|---|---|
| verdict | Y2 by the pre-registered threshold - 22 blocks flagged - but reading all 22 found six DOIs and zero wrong papers, so the operative reading is Y1 |
| citation blocks with doi | 16463 |
| wrong paper citations found | 0 |
| what it cannot catch | a wrong DOI pointing at a paper of the SAME year, which is common in a narrow citation window; and a correct DOI attached to a claim the paper does not support, which no check in this project touches |
| cycle 119s proposed fix was the expensive one | it specified storing the live title and called it the cheapest improvement available; year and venue were already in the schema, already in the publisher's form, and already unused |
| absence class | measured-not-curated |
| bite back | This entry bounds the wrong-paper failure mode from above and that bound is loose enough to be nearly decorative: a wrong DOI with a matching year passes silently, and most of the 92% agreement is between a stored value and the registry it was originally copied from. A reader should not treat this as the citation record having been verified. |
venue
{
"comparable": 16267,
"agree": 13393,
"agree_pct": 82.3,
"usable": true
}year — the numbers behind the chart
{
"comparable": 16420,
"exact": 15104,
"exact_pct": 92,
"off_by_one_online_first": 1294,
"disagree_2plus": 22,
"no_year": 8
}the 22 are six dois — the numbers behind the chart
{
"10.1111/bjh.18021": {
"blocks": 12,
"stored": 2023,
"resolved": 2021,
"why": "Br J Haematol online 2021, print 2023"
},
"10.2210/pdb3U84/pdb": {
"blocks": 3,
"stored": 2026,
"resolved": 2012,
"why": "a PDB structure record cited with the analysis year"
},
"10.1093/rfs/hhm075": {
"blocks": 3,
"stored": 2009,
"resolved": 2007,
"why": "online 2007, print 2009"
},
"10.12688/wellcomeopenres.11589.4": {
"blocks": 2,
"stored": 2017,
"resolved": 2021,
"why": "version 4 of a versioned preprint"
},
"10.1136/jmedgenet-2020-107225": {
"blocks": 1,
"stored": 2022,
"resolved": 2020,
"why": "online 2020, print volume 59"
},
"10.3324/haematol.2019.225953": {
"blocks": 1,
"stored": 2021,
"resolved": 2019,
"why": "online 2019, print 2021"
}
}- Re-check the two documented historical wrong-paper failures against this instrument. If it would not have caught either, the bound it provides is worth much less than the 92% suggests, and that test costs nothing.
- Add the authors axis. doi-index.json already caches the resolved title for 3,424 DOIs and the schema stores an authors string; comparing surname overlap would add a near-independent check at no fetch cost.
- Record the six known-good date-convention DOIs as accepted, so a standing check does not re-flag them every run and train its readers to ignore it.
- Widen the tolerance to two years and re-run. 18 of 22 flags are legitimate online-versus-print splits, so the pre-registered one-year window is measurably too tight for journal citations.
- E1strong confidence 4 of 5: Replicated in peer-reviewed work by more than one group.analysis
Walking every committed node file for dicts carrying a doi together with a title, venue or year gives 16,463 citation blocks. Joining each against the Crossref record its DOI resolves to, cached from the DOI audit: 16,267 blocks state both a venue and a resolved container title, of which 13,393 share at least one significant word after stopword removal (82.3 percent), establishing that the venue field is written in the publisher's form rather than house style. 16,420 blocks state a comparable year, of which 15,104 match exactly (92.0 percent), 1,294 differ by exactly one year, and 22 differ by two years or more. Eight blocks carry a DOI and no year. The 22 comprise six distinct DOIs: 10.1111/bjh.18021 in 12 blocks stored as 2023 against a resolved 2021, 10.2210/pdb3U84/pdb in 3 blocks stored as 2026 against 2012, 10.1093/rfs/hhm075 in 3 blocks stored as 2009 against 2007, 10.12688/wellcomeopenres.11589.4 in 2 blocks stored as 2017 against 2021, 10.1136/jmedgenet-2020-107225 stored as 2022 against 2020, and 10.3324/haematol.2019.225953 stored as 2021 against 2019.
analyses/does-the-stored-year-match-the-doi/run.py and results.json, this repository, computed 2026-09-24 against the committed node files and the resolved Crossref records cached in analyses/do-this-corpus-s-dois-resolve/doi-index.json · Oncofusion knowledge graph, coordinator · internal analysis, reproducible from the repository 2026What this evidence cannot show
A YEAR MATCH IS WEAK EVIDENCE AND THIS CHECK HAS NO POWER TO CONFIRM ANYTHING. Thousands of papers share a year, so a wrong DOI pointing at a same-year paper passes silently, and with roughly 15,000 blocks in a narrow citation window such collisions are common; the residual risk is large and has not been quantified. THE ONE-YEAR TOLERANCE WAS TOO TIGHT: 18 of the 22 flagged blocks are journal online-versus-print splits of more than one year, which are legitimate, so the flag rate overstates the problem and a two-year tolerance would report zero. THE RESOLVED RECORD IS NOT GROUND TRUTH - Crossref metadata carries its own errors and a disagreement can be the registry's. THIS SAYS NOTHING ABOUT WHETHER A CITED PAPER SUPPORTS THE CLAIM IT IS ATTACHED TO, which is the thing that matters and which no automated check in this project touches. Only blocks whose DOI resolved through Crossref are joined - 3,424 of 3,438 - so handle-system-only records are outside the audit.
Evidence that argues against this (3)
- Reporting 'zero wrong-paper citations' overstates what a year comparison can establish. The correct statement is that no wrong-paper citation with a DIFFERENT YEAR exists, which is a much weaker claim, and the two documented historical failures in this project would need to be re-checked to know whether this instrument would have caught either of them - that test was not run and would have been the honest validation.
- The 92.0% exact-match figure is partly an artefact of how these citations were created. Most were written by fetching the PubMed or Crossref record at the time, so the stored year came from the same registry this check compares it against; agreement is then close to tautological for those blocks, and the check has real power only over the citations written from recall - which are exactly the ones nobody can identify.
- Twelve of the 22 flagged blocks are one DOI, so the 'worklist' is six items wearing 22 hats. Any future run of this check will re-flag the same six every time unless they are recorded as accepted, and a standing checker that cries wolf on known-good cases is one that stops being read.
its structural conclusion - that a wrong identifier is undetectable here because no field holds the cited paper's title - is too strong. `year` and `venue` were already in the schema, already in the publisher's form and never compared; venue is usable at 82.3% and year is exact in 92.0% of 16,420 blocks. The specified fix was the expensive one
the other half of the same question: that audit showed the identifiers exist (3,431 of 3,438 resolve), this one shows they point at papers whose year and journal the entries got right - and it reuses that audit's cached Crossref records at no fetch cost
executes the validation that entry's own counter-evidence said was missing: 16 of 23 recorded identifier errors (70%) would have been flagged, 7 would have passed silently on a shared year, and because every pair is an error a human CAUGHT the figure is an UPPER bound
a second, near-independent axis on the same question. Neither finds a wrong-paper citation - but both compare metadata the entry author copied from the same registry at citation time, so a wrong DOI transcribed with its own correct metadata passes both
its 92.0% year-exact rate is an artefact of comparing against Crossref's single `issued` field, which is frequently the online-first date. PubMed returns both pubdate and epubdate; accepting either gives 100.0% on the PMID side, and the DOI check's 1,294 off-by-one blocks are the same effect unadjusted
runs the claim-support check that the identifier-agreement axis stated it could not touch