Oncofusion

start / graph / ana-q203-of-three-msa-free-co-folding-predictors-only-the-single-sequence-one-recovers-menin-mbm1-and-af9-aff1-and-its-peptides-sit-in-the-deposited-groove

Analysismoderate confidence 3 of 5: One strong study, or several smaller consistent ones.king

Of three co-folding predictors run without alignments, only the single-sequence one (ESMFold2) recovers menin-MBM1 above shuffled-peptide controls, and its real peptides sit within 0.6 A (core) to 2 A of the deposited structures while shuffled peptides do not - on interfaces that were in its training data

also called co-folding positive control · ESMFold2 menin MBM1 pose check · shuffled-peptide negative control co-folding

Before trusting a computer model that predicts how two proteins fit together, we tested it on two contacts whose answer is already known, and against scrambled versions of the same protein pieces that should not fit. Of the three models, only one passed on both known contacts, and it placed the real pieces almost exactly where the lab structures put them while the scrambled pieces landed far away. Because those two contacts were part of what the model learned from, this shows the setup works, not that it can find a new contact.

The same thing, for a scientist

Seven jobs (menin + KMT2A 1-43; AF9 + AFF1 738-779; a menin-KMT2A 1-153-LEDGF ternary; two seeded shuffles of each peptide) were run MSA-free and template-free with AlphaFold 3 (25 samples per job), OpenDDE (5) and ESMFold2 (5 seeds), scored by median ipTM under a pre-registered rule: the real interface must beat the highest of its own two shuffles. ESMFold2 passed both (menin-MBM1 0.770 vs 0.383; AF9-AFF1 0.620 vs 0.501) on median and max; AF3 and OpenDDE failed the menin test and passed AF9-AFF1 narrowly. A geometric pose check against PDB 4GQ6 (menin superposed, whole-chain CA fit about 2.4 A) gives a peptide RMSD of 0.58 A over RWRFP (1.86 A over MBM1 6-15) for the real peptide and about 19.6 A for both shuffles; for AF9-AFF1 against the best of the 10 models of PDB 2LM0, 2.09 A for AFF1 contact residues versus 12.5 and 13.7 A for the shuffles. The high ipTM of AFF1 shuffle 1 (0.50 ESMFold2, 0.59 AF3, 0.85 OpenDDE) is therefore a confident wrong pose. Both positives were deposited before ESMFold2's stated data cutoff (Sept 2021).

Go deeper
Design fixed before any run

Identical sequences went to every predictor; the shuffles keep composition and length and destroy order; the aggregator (median ipTM over samples, max reported alongside) and the rule were fixed before the per-job numbers were read 1. A control that can fail did: AF3 and OpenDDE did not recover menin-MBM1 in this mode 1.

What the pose check adds

ipTM is a confidence score and is length-sensitive, so a low value does not prove a wrong pose and a high value does not prove a right one. The pose check measures where the peptide is. The real peptides land at 0.58 A (core) and 2.09 A from the deposited structures, the shuffled peptides at 12-20 A 2. That separates a confident wrong pose from a right one in a way the confidence number could not.

What it does not show

Both interfaces were deposited in 2012-2013, before the model's stated Sept 2021 data cutoff 3. Recovering them shows that the pipeline and controls work and that the model can place these peptides. It is not evidence that the model predicts a new interface. A held-out test needs an interface deposited after the cutoff with no binary precursors in the training data; none has been identified.

Facts and figures
jobs7
torch pin noteESMFold2 first run failed at the CUDA check (default torch build is CUDA 13, node driver 12.8); the cu126 build ran.
esmfold2 median iptm
menin_best_shuffle0.383menin_best_shuffle: 0.383af9_aff1_best_shuffle0.501af9_aff1_best_shuffle: 0.501af9_aff1_real0.620af9_aff1_real: 0.62menin_real0.770menin_real: 0.77
scripts
[
  "analyses/q203-cofolding-the-retained-interfaces/score.py",
  "analyses/q203-cofolding-the-retained-interfaces/pose_check.py"
]
esmfold2 median iptm — the numbers behind the chart
{
  "menin_real": 0.77,
  "menin_best_shuffle": 0.383,
  "af9_aff1_real": 0.62,
  "af9_aff1_best_shuffle": 0.501
}
pose rmsd A
{
  "menin_rwrfp_real": 0.58,
  "menin_6_15_real": 1.86,
  "menin_shuffles": [
    19.58,
    19.56
  ],
  "af9_aff1_real_best_model": 2.09,
  "af9_aff1_shuffles": [
    12.46,
    13.7
  ]
}
predictors
[
  "AlphaFold 3 (25 samples)",
  "OpenDDE (5 samples)",
  "ESMFold2 (5 seeds)"
]
How this could bite us back
  • A predictor that passes a control on training-set interfaces can be mistaken for one that works on new ones; used to nominate a novel interface or rank a drug, its output would carry a false authority. Every use must keep its shuffled-peptide controls and a geometry check.
  • Reading the AF3 and OpenDDE failure as a verdict on them would discard tools that may work with alignments; the result is specific to the alignment-free mode.
Still open
  1. Is there an interface deposited after September 2021, with no binary precursors in training data, that can serve as a held-out positive control for ESMFold2?
  2. Do AF3 and OpenDDE recover menin-MBM1 when run with alignments (the AF3 databases are not installed on the cluster)?
  3. Do the same predictors place the menin-KMT2B MBM peptide in the groove, and with what confidence relative to KMT2A (the atlas FM-020 question)?
Every source behind this entry (3)
  1. E1moderate confidence 3 of 5: One strong study, or several smaller consistent ones.database

    Under a pre-registered rule, ESMFold2 run single-sequence gives a median ipTM of 0.770 for menin + KMT2A 1-43 against a best shuffle of 0.383, and 0.620 for AF9 + AFF1 against 0.501; AlphaFold 3 and OpenDDE run without alignments failed the menin test and passed AF9-AFF1 narrowly.

    What this evidence cannot show

    Two shuffles per interface cannot estimate the spread of shuffle scores (AFF1 shuffle 1 scores far above shuffle 2 for every predictor). ESMFold2 has one diffusion sample per seed, AF3 has 25 samples per job, OpenDDE has 5; sampling depths differ. AF3 and OpenDDE were run in the mode they are least suited to, so their failure here is not a statement about them with alignments.

    Evidence that argues against this (1)
    • AF3 and OpenDDE are designed to use multiple sequence alignments; run with alignments they may recover menin-MBM1. This analysis tested only the alignment-free mode that all three share.
  2. E2moderate confidence 3 of 5: One strong study, or several smaller consistent ones.database

    The ESMFold2 peptide for menin + KMT2A lies at a median CA RMSD of 0.58 A over RWRFP (1.86 A over residues 6-15) from the peptide in PDB 4GQ6 after superposing menin, and the AF9 + AFF1 prediction lies 2.09 A from the best of the 10 models of PDB 2LM0 over the AFF1 contact residues, whereas shuffled peptides lie 19.6 A and 12.5-13.7 A away.

    What this evidence cannot show

    The 2LM0 reference is an NMR ensemble of an engineered single-chain construct, so the best of 10 models is quoted and the median (2.53 A) is also given. The 4GQ6 peptide is a 12-mer. Menin superposes at about 2.4 A over the whole chain, so a small part of the peptide RMSD is the receptor fit.

    Evidence that argues against this (1)
    • Both references were in the predictor's training data, so a close pose may reflect memorisation of these entries and not an ability to predict a new interface.
  3. E3moderate confidence 3 of 5: One strong study, or several smaller consistent ones.web

    ESMFold2 is described by its developers as able to predict structures from a single sequence, with optional MSA conditioning for the larger variant, and the model card gives a data cutoff of September 2021.

    What this evidence cannot show

    Model card read 2026-10-14. It states benchmark claims that were not re-run here and are the developers' own.

    Evidence that argues against this (1)
    • A data cutoff of September 2021 means the 2012-2013 positives were available for training; the card does not say which individual entries were included, so memorisation is likely but not shown.
Connected to