See what every mutation does to a protein, and how well models predict it.
Predicted effects of every single amino-acid substitution in 217 ProteinGym assays, tested against real measurements.
Start with an example
What the benchmark shows
- SaProt 650M vs ESM-2 650M, zero-shot
- +0.046Spearman
- Adding AF2 structure tokens to the same architecture. Paired 95% CI [+0.028, +0.062], 213 assays with a structure.
- Supervised gain on unseen positions
- +0.092Spearman
- Ridge, sequence + structure − Zero-shot SaProt 650M, contiguous split (whole blocks of positions held out); paired 95% CI [+0.065, +0.115]. Modulo split: +0.135. 213 assays.
- Coverage of 90% intervals
- 90.8%mean
- Share of held-out measurements inside their interval, on test folds the intervals never saw (contiguous split, split-matched calibration). 0 of 213 assays more than 10 points below target.
Read from the bundle's benchmarks.json. Every chart, split and caveat.
Try your own sequence
Paste any protein sequence (up to 1,022 residues) and get a zero-shot landscape from ESM-2, computed live. The server sleeps when idle, so the first request can take a moment.