Analysismoderate confidence 3 of 5: One strong study, or several smaller consistent ones.king
18% of this graph's evidence is the same finding said again
also called evidence redundancy · independent evidence audit
Every cycle this project reports how many pieces of evidence it holds, and the number keeps going up. It turns out about one in six of those pieces is the same result from the same paper, written out again in a different entry.
The same thing, for a scientist
2235 PMID-backed evidence items across 926 distinct papers resolve to **1822 distinct (paper, finding) pairs** once each paper's claims are clustered by token-set Jaccard >= 0.6. That is 413 restatements, **18.5%**. The redundancy is not uniform: it concentrates almost entirely on entries for members of protein COMPLEXES, where the evidence is about the complex and the one-entry-per-thing rule requires a separate entry per subunit.
Go deeper
The number, and what it does and does not mean
| count | |
|---|---|
| evidence items (all) | 2906 |
| PMID-backed items | 2235 |
| distinct papers | 926 |
| distinct (paper, finding) pairs | 1822 |
| restatements | 413 (18.5%) |
Restatement is not misconduct and mostly not error. Every entry needs its own evidence, and one paper legitimately supports many entries. What the number measures is how far the headline evidence count overstates the amount of independent published support the graph rests on. The honest version of "we hold 2235 pieces of evidence" is "we hold 1822 findings from 926 papers".
The redundancy has one cause, and it is a rule this project chose deliberately
The worst cases are not scattered. They are protein-complex subunits:
| PMID | cited | distinct findings | largest identical cluster |
|---|---|---|---|
| 20159607 | 14 | 3 | 12 |
| 20541448 | 15 | 5 | 10 |
| 39121437 | 23 | 15 | 5 |
| 15199122 | 12 | 5 | 4 |
| 20159561 | 15 | 9 | 4 |
| 24389101 | 7 | 1 | 7 |
PMID 24389101 is cited seven times and carries one finding, repeated in all seven: gene-ash2l, gene-dpy30, gene-rbbp5, gene-wdr5 and three subtraction entries. PMID 20159607 is cited fourteen times across three findings, one cluster holding twelve copies, across the same WRAD subunit entries.
The cause is structural. The PI's standing rule is one entry per thing, never a summary entry covering several. WRAD is four proteins, so it is four entries. But the published evidence is about the complex — MM-401 blocks the MLL1-WDR5 interaction and prevents complex assembly — so each subunit entry carries a copy of the same complex-level result. The rule and the evidence sit at different levels of description, and copying is what happens in between.
This is worth knowing rather than fixing. Collapsing the copies would mean summary entries, which is the thing the rule exists to prevent.
Facts and figures
| script | analyses/evidence-redundancy/run.py |
|---|---|
| status | executed |
| run date | 2026-09-19 |
| n entries | 653 |
| n evidence items | 2906 |
| n pmid backed items | 2235 |
| n distinct pmids | 926 |
| n distinct findings | 1822 |
| n restatements | 413 |
| restatement fraction | 0.1848 |
| jaccard threshold | 0.6 |
| headline | 2235 evidence items are 1822 findings from 926 papers; 18.5% restatement, concentrated on complex subunits. |
| caveat | Textual similarity only. Two genuinely different findings of one paper written in similar words merge and are undercounted as duplicates; two restatements written differently do not merge and are overcounted as independent. Restatement is also not an error - each entry needs its own evidence - so this measures how much the headline count overstates independent support, not how much is wrong. |
worst — the numbers behind the chart
[
{
"pmid": "20159607",
"citations": 14,
"distinct_findings": 3,
"restatements": 11,
"largest_cluster": 12,
"example": "Menin recruits both wild-type MLL and the oncogenic MLL-AF9 fusion protein to HOX gene loci to activate transcription, and the wild-type Mll allele is required for MLL-AF9-induced leukemogenesis and f",
"entries": [
"ana-kmt2a-lost-module-ledger",
"exp-cyp33-induction-allele-resolved-cutrun",
"exp-turboid-redesign-degron-timed-proximity-map",
"exp-wildtype-allele-dosage-vs-fusion-occupancy",
"gene-ash2l",
"gene-dpy30",
"gene-rbbp5",
"gene-wdr5",
"hyp-fusion-is-exempt-from-restraint-not-free-of-it",
"hyp-kmt2a-half-loses-abundance-control-not-activity-control",
"mech-kmt2a-loss-pattern-restraint-or-not",
"mechanism-set-loss-neomorph-not-haploinsufficiency"
],
"title": "MLL-AF9-induced leukemogenesis requires coexpression of the wild-type Mll allele.",
"authors": "Thiel AT, Blessington P, Zou T, et al.",
"venue": "Cancer Cell",
"year": 2010,
"title_provenance": "Title/authors/venue backfilled from PubMed esummary on 2026-09-20 because this citation stored a bare PMID with no title, which made it unscoreable by verify-citations.py. BACKFILLING IS NOT VERIFICATION: it confirms the PMID resolves to a real paper, not that this is the right paper for the claim. That still requires reading."
},
{
"pmid": "20541448",
"citations": 15,
"distinct_findings": 5,
"restatements": 10,
"largest_cluster": 10,
"example": "The essential MLL1 recruitment domain includes both the CXXC domain and the PHD fingers and is controlled by interactions with the PAF elongation complex and H3K4me2/3. MLL1 fusion proteins lack the P",
"entries": [
"ana-kmt2a-lost-module-ledger",
"exp-cyp33-induction-allele-resolved-cutrun",
"exp-turboid-redesign-degron-timed-proximity-map",
"exp-wildtype-allele-dosage-vs-fusion-occupancy",
"gene-ash2l",
"gene-dpy30",
"gene-paf1",
"gene-rbbp5",
"gene-wdr5",
"hyp-fusion-is-exempt-from-restraint-not-free-of-it",
"mech-kmt2a-loss-pattern-restraint-or-not",
"subtraction-kmt2a-lost-from-protein-not-from-cell"
],
"title": "Multiple interactions recruit MLL1 and MLL1 fusion proteins to the HOXA9 locus in leukemogenesis.",
"authors": "Milne TA, Kim J, Wang GG, et al.",
"venue": "Mol Cell",
"year": 2010,
"title_provenance": "Title/authors/venue backfilled from PubMed esummary on 2026-09-20 because this citation stored a bare PMID with no title, which made it unscoreable by verify-citations.py. BACKFILLING IS NOT VERIFICATION: it confirms the PMID resolves to a real paper, not that this is the right paper for the claim. That still requires reading."
},
{
"pmid": "39121437",
"citations": 23,
"distinct_findings": 15,
"restatements": 8,
"largest_cluster": 5,
"example": "Revumenib was studied and approved in KMT2A-translocated relapsed/refractory acute leukemia without restriction by partner gene, achieving CR+CRh 22.8% and ORR 63.2%.",
"entries": [
"ana-duration-adjusted-value-menin-sequencing",
"ana-menin-dependence-untested-in-septin-and-cytoskeletal-kmt2a-fusions",
"ana-menin-resistance-systematic-enumeration",
"ana-phase2-model-response-prediction",
"analogy-finance-duration-and-reinvestment-risk",
"collab-syndax-pharmaceuticals",
"concept-endocrine-endpoint-absence-menin-trial-registries",
"disease-kmt2a-r-all",
"drug-revumenib",
"exp-menin-pulse-chase-hysteresis",
"fusion-kmt2a-afdn",
"fusion-kmt2a-aff1"
],
"title": "Menin Inhibition With Revumenib for KMT2A-Rearranged Relapsed or Refractory Acute Leukemia (AUGMENT-101).",
"authors": "Issa GC, Aldoss I, Thirman MJ, et al.",
"venue": "J Clin Oncol",
"year": 2025,
"title_provenance": "Title/authors/venue backfilled from PubMed esummary on 2026-09-20 because this citation stored a bare PMID with no title, which made it unscoreable by verify-citations.py. BACKFILLING IS NOT VERIFICATION: it confirms the PMID resolves to a real paper, not that this is the right paper for the claim. That still requires reading."
},
{
"pmid": "15199122",
"citations": 12,
"distinct_findings": 5,
"restatements": 7,
"largest_cluster": 4,
"example": "Biochemical purification of MLL identified a SET1-like complex containing ASH2L, WDR5, RBBP5, HCF-1, HCF-2 and menin. Oncogenic mutant (fusion) forms of MLL retain the ability to interact with menin b",
"entries": [
"ana-kmt2a-interactome-systematic-enumeration",
"ana-kmt2a-lost-module-ledger",
"exp-hbm-reinsertion-kmt2a-aff1",
"exp-turboid-map-of-kmt2a-aff1",
"gene-ash2l",
"gene-dpy30",
"gene-hcfc1",
"gene-rbbp5",
"gene-wdr5",
"hyp-menin-primacy-is-geometry-not-biology",
"subtraction-kmt2a-hcf-binding-motif",
"subtraction-kmt2a-win-wrad-interface"
],
"title": "Leukemia proto-oncoprotein MLL forms a SET1-like histone methyltransferase complex with menin to regulate Hox gene expression.",
"authors": "Yokoyama A, Wang Z, Wysocka J, et al.",
"venue": "Mol Cell Biol",
"year": 2004,
"title_provenance": "Title/authors/venue backfilled from PubMed esummary on 2026-09-20 because this citation stored a bare PMID with no title, which made it unscoreable by verify-citations.py. BACKFILLING IS NOT VERIFICATION: it confirms the PMID resolves to a real paper, not that this is the right paper for the claim. That still requires reading."
},
{
"pmid": "20159561",
"citations": 15,
"distinct_findings": 9,
"restatements": 6,
"largest_cluster": 4,
"example": "AFF4 is a component of the ELL/P-TEFb elongation complex and a shared subunit of MLL chimeras; AFF4 is required for the stability of that complex, and AFF4 knockdown reduces expression of MLL-chimera ",
"entries": [
"concept-transcription-elongation",
"exp-aff1-idr-condensate-separation-of-function",
"fusion-kmt2a-aff1",
"gene-aff1",
"gene-aff4",
"gene-ccnt1",
"gene-ell",
"gene-ell2",
"hyp-aff1-condensates-dispensable-for-kmt2a-fusion-function",
"hyp-oligomerisation-interfaces-as-mutation-resistant-targets",
"mechanism-chromatin-elongation-hijack",
"mechanism-condensate-transcriptional-rewiring"
],
"title": "AFF4, a component of the ELL/P-TEFb elongation complex and a shared subunit of MLL chimeras, can link transcription elongation to leukemia.",
"authors": "Lin C, Smith ER, Takahashi H, et al.",
"venue": "Mol Cell",
"year": 2010,
"title_provenance": "Title/authors/venue backfilled from PubMed esummary on 2026-09-20 because this citation stored a bare PMID with no title, which made it unscoreable by verify-citations.py. BACKFILLING IS NOT VERIFICATION: it confirms the PMID resolves to a real paper, not that this is the right paper for the claim. That still requires reading."
},
{
"pmid": "24389101",
"citations": 7,
"distinct_findings": 1,
"restatements": 6,
"largest_cluster": 7,
"example": "MM-401 inhibits MLL1 activity by blocking the MLL1-WDR5 interaction and preventing complex assembly, without affecting other MLL-family methyltransferases, and specifically blocks proliferation of MLL",
"entries": [
"gene-ash2l",
"gene-dpy30",
"gene-rbbp5",
"gene-wdr5",
"subtraction-kmt2a-lost-from-protein-not-from-cell",
"subtraction-kmt2a-set-domain",
"subtraction-kmt2a-win-wrad-interface"
],
"title": "Targeting MLL1 H3K4 methyltransferase activity in mixed-lineage leukemia.",
"authors": "Cao F, Townsend EC, Karatas H, et al.",
"venue": "Mol Cell",
"year": 2014,
"title_provenance": "Title/authors/venue backfilled from PubMed esummary on 2026-09-20 because this citation stored a bare PMID with no title, which made it unscoreable by verify-citations.py. BACKFILLING IS NOT VERIFICATION: it confirms the PMID resolves to a real paper, not that this is the right paper for the claim. That still requires reading."
},
{
"pmid": "37099340",
"citations": 17,
"distinct_findings": 12,
"restatements": 5,
"largest_cluster": 4,
"example": "In KMT2A-rearranged infant ALL, most relapses occur during treatment, with two thirds within one year and 90% within two years of diagnosis, and 3-year event-free survival is below 40%.",
"entries": [
"ana-duration-adjusted-value-menin-sequencing",
"ana-hazard-term-structure-relapse-kmt2a",
"ana-infant-trial-feasibility-arithmetic",
"analogy-finance-hazard-term-structure",
"collab-pieters-interfant",
"comut-low-mutation-burden-infant-kmt2a-r",
"comut-ras-pathway-kmt2a-r",
"concept-fever-and-infection-at-presentation",
"disease-infant-all",
"drug-blinatumomab",
"epi-infant-leukemia-incidence",
"epi-infant-leukemia-uk-and-australia"
],
"title": "Blinatumomab Added to Chemotherapy in Infant Lymphoblastic Leukemia.",
"authors": "van der Sluis IM, de Lorenzo P, Kotecha RS, et al.",
"venue": "N Engl J Med",
"year": 2023,
"title_provenance": "Title/authors/venue backfilled from PubMed esummary on 2026-09-20 because this citation stored a bare PMID with no title, which made it unscoreable by verify-citations.py. BACKFILLING IS NOT VERIFICATION: it confirms the PMID resolves to a real paper, not that this is the right paper for the claim. That still requires reading."
},
{
"pmid": "41027490",
"citations": 17,
"distinct_findings": 12,
"restatements": 5,
"largest_cluster": 4,
"example": "Heavily pretreated KMT2A::AFF1 ALL samples were markedly less menin-inhibitor-sensitive than earlier samples from the same patients with no prior class exposure, with transcriptomics showing sustained",
"entries": [
"ana-menin-resistance-cell-state-signature",
"ana-menin-resistance-systematic-enumeration",
"ana-resistance-route-cooccurrence",
"analogy-finance-duration-and-reinvestment-risk",
"analogy-finance-portfolio-covariance-of-resistance",
"analogy-finance-pre-existing-impairment-diligence",
"exp-serial-sample-menin-sensitivity-ladder",
"hyp-cell-state-plasticity-limits-interface-disruption",
"hyp-menin-inhibitors-belong-frontline-in-infants",
"hyp-resistance-routes-negatively-correlated-genetic-vs-plastic",
"hyp-therapy-installed-impairment-index",
"mechanism-fusion-dependency-reversibility-kinetics"
],
"title": "Upfront menin-inhibitor resistance in multiply pretreated leukemias.",
"authors": "Mahdavi L, Alikarami F, Goodrow H, et al.",
"venue": "Exp Hematol",
"year": 2026,
"title_provenance": "Title/authors/venue backfilled from PubMed esummary on 2026-09-20 because this citation stored a bare PMID with no title, which made it unscoreable by verify-citations.py. BACKFILLING IS NOT VERIFICATION: it confirms the PMID resolves to a real paper, not that this is the right paper for the claim. That still requires reading."
},
{
"pmid": "41351513",
"citations": 12,
"distinct_findings": 7,
"restatements": 5,
"largest_cluster": 3,
"example": "After menin-inhibitor failure, median overall survival from the start of next therapy was 4.4 months in 84 adults, with 15.4 months for those achieving CR/CRi versus 3.4 months otherwise and 40% recei",
"entries": [
"ana-duration-adjusted-value-menin-sequencing",
"ana-resistance-route-cooccurrence",
"analogy-finance-duration-and-reinvestment-risk",
"analogy-finance-pre-existing-impairment-diligence",
"hyp-common-cause-failure-defeats-orthogonal-combination",
"hyp-menin-plus-engager-orthogonal-pre-emption",
"hyp-menin-value-dominated-by-bridge-rate",
"hyp-therapy-installed-impairment-index",
"resistance-cross-resistance-between-menin-inhibitors",
"resistance-men1-interface-mutations",
"resistance-ras-flt3-wt1-comutation-menin",
"resistance-tp53-apoptotic-priming-menin"
],
"title": "Outcomes of relapsed or refractory acute myeloid leukemia after menin inhibition failure.",
"authors": "Chin KK, Ball BJ, Abaza Y, et al.",
"venue": "Blood Adv",
"year": 2026,
"title_provenance": "Title/authors/venue backfilled from PubMed esummary on 2026-09-20 because this citation stored a bare PMID with no title, which made it unscoreable by verify-citations.py. BACKFILLING IS NOT VERIFICATION: it confirms the PMID resolves to a real paper, not that this is the right paper for the claim. That still requires reading."
},
{
"pmid": "20541477",
"citations": 10,
"distinct_findings": 5,
"restatements": 5,
"largest_cluster": 6,
"example": "MLL interacts directly with PAFc through sequences flanking the CxxC domain, PAFc knockdown disrupts MLL-fusion-mediated transcriptional activation and MLL recruitment to target loci, and MLL deletion",
"entries": [
"exp-rtf1-h2bub-separation-of-function-kmt2a",
"gene-ctr9",
"gene-leo1",
"gene-paf1",
"gene-rtf1",
"gene-skic8",
"hyp-h2bub-required-but-not-selective",
"hyp-menin-primacy-is-geometry-not-biology",
"mechanism-paf1c-couples-elongation-to-h2bub",
"mechanism-set-loss-neomorph-not-haploinsufficiency"
],
"title": "The PAF complex synergizes with MLL fusion proteins at HOX loci to promote leukemogenesis.",
"authors": "Muntean AG, Tan J, Sitwala K, et al.",
"venue": "Cancer Cell",
"year": 2010,
"title_provenance": "Title/authors/venue backfilled from PubMed esummary on 2026-09-20 because this citation stored a bare PMID with no title, which made it unscoreable by verify-citations.py. BACKFILLING IS NOT VERIFICATION: it confirms the PMID resolves to a real paper, not that this is the right paper for the claim. That still requires reading."
}
]How this could bite us back
- The honest headline for this project is not the evidence-item count but the count of distinct findings and distinct papers, and every past cycle log has quoted the former.
- This analysis measures its own subject badly: it excludes 671 of 2906 evidence items for lacking a PMID, so the one number it exists to produce is computed on a convenience sample.
Still open
- Should entries for subunits of one complex share a single canonical evidence item by reference rather than by copy? That would keep one-entry-per-thing while making the evidence count honest, but no current schema field supports evidence-by-reference.
- Is 0.6 the right Jaccard threshold? Nobody has built a labelled set of true restatements to calibrate against, so the 18% carries unquantified error.
- 671 of 2906 evidence items carry no PMID and were excluded. What is the redundancy among registry- and database-backed claims, where the same NCT record is plausibly cited far more often?
Every source behind this entry (1)
- E1moderate confidence 3 of 5: One strong study, or several smaller consistent ones.analysis
Clustering every PMID-backed evidence claim in the graph by paper and then by token-set Jaccard similarity (>= 0.6) reduces 2235 evidence items across 926 papers to 1822 distinct (paper, finding) pairs — 413 restatements, 18.5%. The restatement concentrates on protein-complex subunit entries: PMID 24389101 is cited 7 times carrying 1 finding, and PMID 20159607 is cited 14 times with a single 12-member identical cluster, in both cases across the WRAD entries gene-ash2l, gene-dpy30, gene-rbbp5 and gene-wdr5.
Evidence redundancy audit of the Oncofusion knowledge graph · Oncofusion knowledge graph (internal analysis) 2026What this evidence cannot show
Run by the coordinator on 2026-09-19 over 653 entries. Textual similarity only, with error in BOTH directions: two genuinely distinct findings of one paper written in similar words merge and are undercounted as independent, while two restatements written differently do not merge and are counted as independent. The 18.5% is an estimate, not a measurement, and the Jaccard threshold of 0.6 was chosen by inspection rather than calibrated against a labelled set. Evidence items with no PMID (671 of 2906) are excluded entirely, so this says nothing about registry- or database-backed claims.
Evidence that argues against this (2)
- The framing 'the graph overstates its evidence' is the wrong reading, and the data argue against it. Restatement is concentrated exactly where it SHOULD be under this project's own rules: a complex-level result genuinely is evidence for each subunit entry, and the alternative is a single WRAD summary entry, which the PI's brief forbids by name. A reader wanting independent support has always been able to count distinct PMIDs, which this analysis did not change. The number is a useful denominator, not a defect that was concealed. Oncofusion knowledge graph (internal analysis) 2026
- The threshold does real work and is unvalidated. At Jaccard 0.6 two sentences differing only in 'MLL' versus 'KMT2A' merge, which is correct, but so would two claims about different cell lines phrased identically, which is not. No labelled set was built, so the 18% could move materially with a different threshold and nobody has checked how far. Oncofusion knowledge graph (internal analysis) 2026
Connected to
Both measure the project against its own reporting; this one shows the evidence count overstates independent support by 18%.
- related to →WDR5 — the partner that is not a partner: it binds the half of KMT2A the fusion throws away
One of the four WRAD subunit entries where complex-level evidence is copied per subunit.
- related to →ASH2L — the WRAD subunit that has DPY30's essentiality profile and none of its co-dependency signal
One of the four WRAD subunit entries carrying identical copies of complex-level evidence.
One of the four WRAD subunit entries carrying identical copies of complex-level evidence.
- related to →DPY30 — the smallest COMPASS subunit, and the only one that behaves like a selective dependency
One of the four WRAD subunit entries carrying identical copies of complex-level evidence.
Both measure how much independent support the graph actually has; this one ranks where the gaps cost most.