---

name: model-embeddings description: "Load ESM/ESM-2/MSA Transformer models, tokenize protein inputs, compute representations or contacts, and build bulk FASTA embedding extraction commands." disable-model-invocation: true metadata: disco-role: operating license: MIT

Model Embeddings

Use this sub-skill when an agent needs embeddings, logits, or unsupervised contact maps from ESM, ESM-2, ESM-1/1b/1v, or MSA Transformer language models. It covers Python APIs for model loading and tokenization plus the esm-extract bulk FASTA CLI.

Do not use this sub-skill for structure prediction, inverse folding, or mutation-effect scoring. Route those tasks to sibling sub-skills instead:

• Structure prediction / ESMFold PDB output: ../structure-prediction/SKILL.md
• Inverse folding / ESM-IF1 design and structure-conditioned scoring: ../inverse-folding/SKILL.md
• DMS or zero-shot mutation scoring: ../variant-effect-prediction/SKILL.md

Start Here

1. For Python model loading, tokenization, layer selection, output tensors, and MSA shapes, read references/api-reference.md.
2. For common embedding/contact workflows and safe model choices, read references/workflows.md.
3. For bulk FASTA extraction, .pt contents, truncation, CPU/GPU flags, and command construction, read references/cli-reference.md.
4. For large ESM-2 15B inference with Fairscale FSDP CPU offload, read references/fsdp-offloading.md.
5. For failures such as bad model names, download issues, MSA misuse, duplicate FASTA labels, layer errors, and OOM, read references/troubleshooting.md.

Bundled Helper

Use the safe command builder when constructing a reproducible esm-extract invocation without triggering model downloads:

python sub-skills/model-embeddings/scripts/esm_extract_command_builder.py \
  esm2_t33_650M_UR50D input.fasta embeddings_out \
  --repr-layers -1 --include mean contacts --nogpu --print-only

The helper validates that the FASTA exists, checks extraction options, warns about unsupported MSA Transformer model names, and prints a shell-quoted command by default. It does not import esm or load model weights for --help or --print-only.

Quick Decisions

• Use esm.pretrained.load_model_and_alphabet(name) or a specific esm.pretrained.<model>() function for Python embeddings.
• Use alphabet.get_batch_converter(truncation_seq_length=...) for single-sequence models and MSA models; MSA alphabets return MSABatchConverter automatically.
• Use model(tokens, repr_layers=[...], return_contacts=True) for hidden representations plus contact maps.
• Use esm-extract only for single-sequence models; MSA Transformer requires Python API input shaped as a batch of MSAs.
• Force CPU with --nogpu in esm-extract or by keeping tensors/model on CPU in Python; expect slow inference for large models.