Survey of ESM Protein Language Models (2026)

Abstract

The ESM series applies large language models to protein science, trained on vast unlabeled sequence data to capture mutation/conservation patterns shaped by evolution. This survey reviews ~100 papers, categorizing usage techniques and downstream applications, and critically assesses ESM's strengths, limits, and potential.

Scope & Findings

• Covers ESM-1b → ESM-3; 6 major variants, parameter ranges 8M → 98B.
• ESM-1b trained on 250M sequences (UniParc).
• Downstream apps surveyed: structure prediction, function prediction (EC/GO), interaction (PPI/PLI/DTI), variant-effect, directed evolution, de novo + conditional protein design.
• ESMFold much faster than AlphaFold2 with minor accuracy loss; ESM vs ProtT5 each win on different metrics.

Limitations Identified

• Data bias: UniProt species imbalance (human/model organisms over-represented); de novo designed proteins absent from pretraining.
• Compute: O(n²d) attention bottleneck; memory scales hard with size (≥16GB for 1B params).
• Task gaps: weak on antibody thermostability, out-of-domain generalization; CNNs still beat ESM in some structure tasks.
• Interpretability: "black box"; attention/gradient methods far from mechanistic understanding.

Metadata

Quantitative Biology 14(1):e70013; DOI 10.1002/qub2.70013; PMID 41676321; online 2025-09-21, collection Mar 2026; CC BY 4.0.

Source: Survey of ESM applications (PMC12806033) — Yang, Yu & Zheng, ShanghaiTech, Quantitative Biology. Full-text fetched.