# Prott5

> Skill for protein embeddings using ProtTrans models (ProtT5, ProtBERT) from Rostlab. Use this skill whenever a user wants protein embeddings or representations for downstream ML tasks, protein classification (subcellular localization, membrane prediction, secondary structure annotation), similarity search, or regression tasks (stability, fitness) using Rostlab's ProtTrans family. ProtT5-XL outper…

- **Type:** Skill
- **Install:** `agentstack add skill-naity-fm4life-prott5`
- **Verified:** Pending review
- **Seller:** [naity](https://agentstack.voostack.com/s/naity)
- **Installs:** 0
- **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [naity](https://github.com/naity)
- **Source:** https://github.com/naity/FM4Life/tree/main/skills/prott5

## Install

```sh
agentstack add skill-naity-fm4life-prott5
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# ProtT5: Protein Language Model Embeddings

## Overview

ProtT5 is a family of T5-based protein language models from Rostlab, trained on millions of UniRef protein sequences. The recommended model — **ProtT5-XL-U50** — is an encoder-decoder transformer with 3B parameters whose encoder produces 1024-dimensional embeddings that outperform earlier BERT-based protein models and match or exceed ESM2 on most tasks.

**Primary use cases:**
- Per-residue and per-protein embeddings for downstream ML
- Protein classification (localization, membrane, function)
- Regression (stability, fitness, thermostability)
- Sequence similarity search

## Installation

```bash
pip install transformers torch sentencepiece
```

## Model Selection

| Model | Type | Params | HF ID | Best for |
|---|---|---|---|---|
| **ProtT5-XL-U50** | T5 encoder | 3B | `Rostlab/prot_t5_xl_half_uniref50-enc` | **Best default** — half-precision encoder only |
| ProtT5-XL-U50 (full) | T5 enc-dec | 3B | `Rostlab/prot_t5_xl_uniref50` | When you also need the decoder |
| ProtT5-XXL-U50 | T5 encoder | 11B | `Rostlab/prot_t5_xxl_uniref50` | Maximum accuracy, multi-GPU |
| ProtBERT-BFD | BERT | ~420M | `Rostlab/prot_bert_bfd` | Faster/lighter, lower quality |
| ProtBERT | BERT | ~420M | `Rostlab/prot_bert` | UniRef100 trained BERT |

Use `prot_t5_xl_half_uniref50-enc` as the default: it's encoder-only (no decoder weights), loads in half-precision, and is what all published benchmarks use.

## Critical Preprocessing

**This is the most important thing to get right.** ProtT5 requires two preprocessing steps that ESM2 does not:

```python
import re

def preprocess(sequence: str) -> str:
    # 1. Map rare/ambiguous amino acids to X
    sequence = re.sub(r"[UZOB]", "X", sequence)
    # 2. Space-separate every amino acid (ProtT5 is character-level)
    return " ".join(list(sequence))

sequence = "MKTAYIAKQRQISFVK"
processed = preprocess(sequence)
# → "M K T A Y I A K Q R Q I S F V K"
```

Skip either step and you will get garbage embeddings — the model was trained on this exact format.

## Core Usage

### Single Sequence

```python
import re
import torch
from transformers import T5Tokenizer, T5EncoderModel

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

tokenizer = T5Tokenizer.from_pretrained("Rostlab/prot_t5_xl_half_uniref50-enc", do_lower_case=False)
model = T5EncoderModel.from_pretrained("Rostlab/prot_t5_xl_half_uniref50-enc").to(device)
if device.type == "cpu":
    model = model.float()   # half-precision not supported on CPU
model.eval()

sequence = "MKTAYIAKQRQISFVK"
processed = " ".join(list(re.sub(r"[UZOB]", "X", sequence)))

ids = tokenizer(processed, return_tensors="pt", add_special_tokens=True).to(device)

with torch.no_grad():
    output = model(**ids)

# output.last_hidden_state: (1, seq_len + 1, 1024)
# T5 encoder adds an EOS token at the end — slice to remove it
per_residue = output.last_hidden_state[0, :len(sequence)]   # (L, 1024)
per_sequence = per_residue.mean(dim=0)                       # (1024,)

print(per_residue.shape)   # (16, 1024)
print(per_sequence.shape)  # (1024,)
```

**Note:** T5 encoder output has shape `(seq_len + 1, 1024)` — the `+1` is the trailing EOS token. Slice `[:len(sequence)]` to get only residue embeddings. Unlike BERT/ESM2, there is **no leading [CLS] token** in the T5 encoder output.

### Batch Processing

```python
def get_embeddings(model, tokenizer, sequences, device):
    """Mean-pooled per-sequence embeddings for a list of sequences."""
    # Preprocess all sequences
    processed = [" ".join(list(re.sub(r"[UZOB]", "X", s))) for s in sequences]
    lengths = [len(s) for s in sequences]

    ids = tokenizer(
        processed,
        return_tensors="pt",
        padding="longest",
        add_special_tokens=True,
    ).to(device)

    with torch.no_grad():
        output = model(**ids)

    embeddings = []
    for i, seq_len in enumerate(lengths):
        # Slice per-residue embeddings (no leading special token, EOS at end)
        per_residue = output.last_hidden_state[i, :seq_len]  # (L, 1024)
        per_sequence = per_residue.mean(dim=0)               # (1024,)
        embeddings.append(per_sequence.cpu().float())

    return torch.stack(embeddings)  # (N, 1024)
```

See `scripts/embed.py` for a CLI that reads FASTA and saves embeddings.

## Scripts

```bash
# Batch embed a FASTA file → .npy
python scripts/embed.py sequences.fasta --output embeddings.npy

# Save as HDF5, L2-normalized
python scripts/embed.py sequences.fasta --output embeddings.h5 --normalize

# Use ProtBERT-BFD for faster/lighter inference
python scripts/embed.py sequences.fasta --model Rostlab/prot_bert_bfd --output embeddings.npy
```

## Fine-Tuned Checkpoints

Rostlab publishes ready-to-use fine-tuned models:

| Task | HF ID |
|---|---|
| Secondary structure (3-class) | `Rostlab/prot_bert_bfd_ss3` |
| Membrane prediction | `Rostlab/prot_bert_bfd_membrane` |
| Subcellular localization | `Rostlab/prot_bert_bfd_localization` |

Load with the appropriate HuggingFace pipeline for quick prediction without training.

## Comparison with ESM2 and ESM-C

| | ProtT5-XL-U50 | ESM2-650M | ESM-C 600M |
|---|---|---|---|
| Architecture | T5 encoder | BERT encoder | Custom encoder |
| Package | `transformers` | `transformers` | `esm` SDK |
| Hidden dim | 1024 | 1280 | 1152 |
| Preprocessing | space-sep + X mapping | none | none |
| Special token handling | slice `[:seq_len]` | slice `[1:-1]` | none |
| Subcell. Loc. (DeepLoc) | **86** | 83 | — |
| Variant effect (DMS) | **0.53** | — | — |

ProtT5 is a strong choice for localization, function prediction, and variant effect tasks. ESM-C is faster for pure embedding throughput.

## Best Practices

- **Always use half-precision on GPU** (`prot_t5_xl_half_uniref50-enc` is already half-precision by default)
- **Max sequence length:** 1022 residues (1024 minus 2 for special tokens) — truncate or chunk longer sequences
- **L2-normalize** before cosine similarity or FAISS indexing
- For comparison with published benchmarks, always use `prot_t5_xl_half_uniref50-enc`

## Resources

- **GitHub:** https://github.com/agemagician/ProtTrans
- **HuggingFace:** https://huggingface.co/Rostlab
- **Paper:** Elnaggar et al., IEEE TPAMI 2021 — https://doi.org/10.1109/TPAMI.2021.3095381
- **Pre-computed embeddings:** available via UniProt for all reviewed sequences
- **LambdaPP webservice:** https://embed.predictprotein.org/

## References

- `references/prott5-api.md` — batch processing, layer extraction, ProtBERT usage, downstream tasks, FAISS, fine-tuning patterns

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [naity](https://github.com/naity)
- **Source:** [naity/FM4Life](https://github.com/naity/FM4Life)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** yes

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: flagged — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-naity-fm4life-prott5
- Seller: https://agentstack.voostack.com/s/naity
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.