Auditing the graph against the PI's brief, not against its own ledger (run)
also called charter audit · requirements coverage · patient stories gap
For thirty cycles this project has measured itself against a list of research beats it wrote for itself, and nobody had checked that list against what the PI actually asked for. Most of the brief is covered well, but one thing the PI asked for by name — real patients and their stories — has no entries at all.
The same thing, for a scientist
First audit of the graph against the founding prompt rather than `docs/coverage.md`. Sixteen requirements traced. Twelve are well covered, three are partially covered, and **one has zero entries: the patient-stories arm**, despite `patient_story` being a defined SCHEMA type and the prompt asking for it in two separate sentences. The beat ledger could not surface this because it tracks Q1-Q16 research beats, and patient stories were never made a beat.
Why the ledger could not see this
docs/coverage.md tracks sixteen research beats, and every cycle picks the coldest. That is a good way to spread effort evenly and a bad way to notice something that was never made a beat at all. A beat at 0 entries gets dispatched; a requirement with no beat is invisible.
The type census makes it obvious once looked at. Across 509 entries: 88 mechanism, 88 protein, 53 analysis, 50 concept, 44 hypothesis, 29 experiment, 22 target, 22 collaborator, 20 fusion — and 0 patient_story, a type the schema defines 1.
The clinical beat looks like the patient arm and is not. Its seventeen entries are the signs (pallor, bruising, fever, hepatosplenomegaly, leukaemia cutis, CNS involvement, hyperleukocytosis), the tests (blood film, flow cytometry, karyotype, FISH, molecular confirmation), two disease entries and two late-effect concepts. Every one is a category. None is a person.
What the prompt asked for, verbatim
Two separate sentences, quoted from prompts.md:
"...patients, you should be able to find real patients"
"patient stories: which patients actually were affected by this, and what kinds of symptoms they [had]"
This was not a passing remark. It sits alongside the collaboration arm and the impact arm as one of three named arms, and the other two are at 26/26 and 23/23 counter-evidence coverage with dedicated beats. The patient arm has nothing 1.
The scorecard
| Requirement | State |
|---|---|
| Dashboard, nested drop-downs, evidence + confidence + caveats + counter-evidence | covered — 509 entries, 81% carrying counter-evidence |
| King → queens → workers hierarchy | covered |
| Cross-domain analogies (finance, art, law) in a hypotheses section | covered — 37 entries, fully swept |
| An experiment and a runnable analysis per hypothesis | covered — 29 experiments, 53 analyses, 22 actually executed |
| Check CPU/RAM before launching anything | covered — every cycle |
| Which cell type, cell state, where the cells are | covered — cells beat, fully swept |
| Endothelial or epithelial? | covered, and answered: neither, it is haematopoietic |
| What kinds of breaks, and do different breaks differ | covered — 9 breakpoint entries |
| 3D visualisation of breaks and conserved domains | covered — 31 viewer lanes, 29 with real 3' coordinates |
| Proteins beyond MEN1 that bind the fusion | covered — 62 interactome entries, systematically enumerated |
| Why this fusion causes this phenotype, against other fusions | covered — 43 comparative entries |
| Why fusions arise; are they temporary; can we intervene at fusion level | covered — genesis beats, fully swept |
| Bite-back: compensation, toxicity, non-essential targets | covered — bite_back on interventions, plus a dedicated endocrine-liability beat |
| Genome-wide and non-coding sweep | covered — 23 entries |
| Collaboration arm | covered — 26/26 |
| Impact arm (~3 per million, platform argument) | covered — 23/23 |
| Phase 2: per-hypothesis druggability, datasets, models, go/no-go | covered — 25 entries and a register |
GitHub repo and prompts.md | covered |
| Patient stories: real patients, real symptoms | ZERO ENTRIES |
Partially covered, worth naming rather than claiming:
- "drop-down-able to arbitrary depth" — 342 entries nest two levels, 168 one level, and exactly one reaches three. Two levels is useful and is not arbitrary depth. Whether that matters is the PI's call, but the graph should not claim more than it has.
- "comparative side-by-side graphs" — comparison exists as entries and as the 3D viewer's lanes, but there is no side-by-side view of two fusions as such.
- "real clinical data" — trial registry records and label data are in, but no patient-level clinical dataset has been touched.
What the patient arm should contain, and the constraint on it
The obvious sources are published case reports and case series, which for infant KMT2A-rearranged leukaemia are numerous and are proper citable literature. A patient entry should carry the presentation, the fusion partner and breakpoint where reported, the treatment and what happened — and should connect to the sign and test entries that already exist, which would finally give those seventeen clinical concepts something concrete to attach to.
The constraint, which matters more than the gap. These are identifiable infants in published case reports. Anything written here must stay inside what the publication itself discloses, must not aggregate details across sources in a way that could re-identify a family, and must not be written as narrative about a named child. The PI asked for patient stories to make the disease legible to a newcomer, not to build a dossier. Published case reports are the right source precisely because the disclosure decision has already been made by the authors and their ethics committee.
Patient-advocacy accounts and published first-person narratives are a second legitimate source and carry the same rule: cite what is published, link to it, and do not extend it.
| audit date | 2026-09-19 |
|---|---|
| n entries at audit | 509 |
| requirements traced | 19 |
| covered | 15 |
| partial | 3 |
| zero | 1 |
| why missed | the ledger tracks research beats; a requirement that was never made a beat is invisible to it |
zero coverage
[ "patient stories / real patients" ]
partial coverage
[ "arbitrary-depth nesting (max depth 3, one entry)", "comparative side-by-side graphs (no two-fusion view)", "patient-level clinical data (registry and label only)" ]
type census — the numbers behind the chart
{
"mechanism": 88,
"protein": 88,
"analysis": 53,
"concept": 50,
"hypothesis": 44,
"experiment": 29,
"target": 22,
"collaborator": 22,
"fusion": 20,
"patient_story": 0
}section depth census — the numbers behind the chart
{
"1": 168,
"2": 342,
"3": 1
}- This audit is the coordinator marking its own homework, and it graded most of it 'covered'. A reading by the PI would likely find requirements this counts as covered that do not feel covered to the person who wrote them.
- The patient arm is the one gap, and it is the one arm where doing the work badly causes harm rather than waste. Real infants, real families, published case reports. The rule has to be set before an agent is dispatched, not after.
- Thirty cycles of disciplined beat-selection produced a systematic blind spot, because the selection mechanism and the requirements list were never the same document. Any future measure of 'what is cold' should be checked against the charter, not only against itself.
- How many published case reports and case series of infant KMT2A-rearranged leukaemia are there, and how many report the fusion partner and breakpoint alongside the presentation?
- Does the PI want patient entries anchored to published cases, to advocacy-published first-person accounts, or both? The privacy constraint differs between them.
- Should `docs/coverage.md` carry the founding requirements alongside the research beats, so a requirement without a beat cannot go unnoticed for thirty cycles again?
- Is two-level nesting enough for 'drop-down-able to arbitrary depth', or should deep entries be restructured?
- Is a genuine side-by-side two-fusion comparison view worth building, given the 3D viewer already puts break variants in adjacent lanes?
- E1established confidence 5 of 5: Multiple independent lines of evidence; treated as settled in the field.dataset
Across 509 entries the graph contains 0 of type `patient_story`, a type defined in data/graph/SCHEMA.md, while containing 88 mechanism, 88 protein, 53 analysis, 50 concept, 44 hypothesis, 29 experiment, 22 target, 22 collaborator and 20 fusion entries. The clinical beat's 17 entries are 14 concepts, 2 diseases and 1 risk, none describing an individual patient. The founding prompt asks for 'real patients' and for 'patient stories: which patients actually were affected by this, and what kinds of symptoms', in two separate sentences. Section nesting reaches depth 2 in 342 entries and depth 3 in 1.
Type census and section-depth census over data/graph/nodes/, against the founding prompt recorded verbatim in prompts.md · audit run by this project · Repository audit 2026What this evidence cannot show
A census of the repository as of 2026-09-19, not a judgement about whether every requirement was interpreted as the PI intended. 'Covered' here means entries exist and carry sourced claims; it does not mean the PI would agree the requirement is satisfied. Two agents were mid-run when this was taken, so counts will have moved. The prompt is paraphrased in the scorecard and quoted verbatim only where marked.
Evidence that argues against this (1)
- The absence of a `patient_story` type is not the same as the absence of patient content. The clinical beat describes exactly what an affected infant looks like at presentation — pallor, bruising, fever, an enlarged abdomen, skin nodules — and the epidemiology beat carries incidence and outcome. A reader could learn a great deal about what happens to these children without a single patient entry. The gap is that nothing is anchored to an actual documented case, which is what the prompt asked for and is a weaker claim than 'the patient arm is missing'. this project 2026
the patient arm the prompt asked for would attach documented cases to this disease entry
the clinical beat describes signs and tests as categories; no entry anchors them to a documented case
the collaboration and impact arms were made beats and are fully swept; the patient arm never was
- related to →How many children this affects
incidence answers how many children, not which children, which is what the prompt asked for
Both measure the project against its own reporting; this one shows the evidence count overstates independent support by 18%.
The coverage ledger reports counter-argument percentage; this says which of the gaps to close first.
Same failure mode at a different level: a rule satisfied by a pointer rather than by the thing it names.