Protein Qc

Protein Design Quality Control

Critical Limitation

Individual metrics have weak predictive power for binding. Research shows:

• Individual metric ROC AUC: 0.64-0.66 (slightly better than random)
• Metrics are pre-screening filters, not affinity predictors
• Composite scoring is essential for meaningful ranking

These thresholds filter out poor designs but do NOT predict binding affinity.

QC Organization

QC is organized by purpose and level:

| Purpose | What it assesses | Key metrics | |---------|------------------|-------------| | Binding | Interface quality, binding geometry | ipTM, PAE, SC, dG, dSASA | | Expression | Manufacturability, solubility | Instability, GRAVY, pI, cysteines | | Structural | Fold confidence, consistency | pLDDT, pTM, scRMSD |

Each category has two levels:

• Metric-level: Calculated values with thresholds (pLDDT > 0.85)
• Design-level: Pattern/motif detection (odd cysteines, NG sites)

---

Quick Reference: All Thresholds

| Category | Metric | Standard | Stringent | Source | |----------|--------|----------|-----------|--------| | Structural | pLDDT | > 0.85 | > 0.90 | AF2/Chai/Boltz | | | pTM | > 0.70 | > 0.80 | AF2/Chai/Boltz | | | scRMSD | 0.61 | > 0.70 | AF3/Boltz (see ipsae) | | | ipTM | > 0.50 | > 0.60 | AF2/Chai/Boltz | | | PAEinteraction | 0.50 | > 0.62 | PyRosetta | | | interfacedG | 0.0 | > 0.2 | ESM2 | | | Folding ΔG | = 3 consecutive | Proteolysis | Flag | | >= 6 hydrophobic run | Aggregation | Redesign |

See: references/binding-qc.md, references/expression-qc.md, references/structural-qc.md

---

Interface metrics (PyRosetta)

Beyond shape complementarity and interface_dG, two interface metrics catch common de novo failure modes:

• Buried unsatisfied H-bonds (BUNS): buried polar atoms making no hydrogen bond.

This is a dominant energetic failure mode and is orthogonal to dG and dSASA. Keep interface BUNS at or below 4 (standard) or 2 (stringent).

• ContactMolecularSurface: shape-complementarity-weighted contact area that, unlike

dSASA, is not fooled by gappy or holey interfaces. Higher is better.

Both are in the Cao 2022, AlphaProteo, and BindCraft filter sets.

For structure-quality ranking, biomodals also provides modal_af2rank.py (AF2Rank), which scores how well a design re-predicts from its own structure as a template.

Binder ranking (benchmark-backed)

A meta-analysis of 3,766 experimentally tested binders across 15 targets (Overath et al., bioRxiv 2025, doi:10.1101/2025.08.14.670059) found that AF3 ipSAE_min is the single best in-silico predictor of binding, and that a simple linear model of three features generalizes best across targets. Complexity did not help: gradient-boosted and many-feature models did not beat the linear one.

Recommended filtering strategies from that work:

1. Threshold on one of: AF3 ipSAE_min > 0.61, or ipSAE_min × interface_ΔG/ΔSASA 0.42.
2. Pre-filter on shape_complementarity > 0.62 and `RMSD_binder 0.85]

Stage 2: Self-consistency

designs = designs[designs['scRMSD'] 0.5) & (designs['PAE_interaction'] 0.0]

Stage 5: Expression checks (design-level)

designs = designs[designs['cysteinecount'] % 2 == 0] # Even cysteines designs = designs[designs['instabilityindex'] 15% | Excellent | Above average, proceed | | 10-15% | Good | Normal, proceed | | 5-10% | Marginal | Below average, review issues | | 0.85)

Low pLDDT across campaign
├── Check scRMSD distribution
│   ├── High scRMSD (>2.5Å): Backbone issue
│   │   └── Fix: Regenerate backbones with lower noise_scale (0.5-0.8)
│   └── Low scRMSD but low pLDDT: Disordered regions
│       └── Fix: Check design length, simplify topology
├── Try more sequences per backbone
│   └── modal run modal_ligandmpnn.py --input-pdb bb.pdb --params-str "--number_of_batches 32 --temperature 0.1"
├── Use SolubleMPNN instead of ProteinMPNN
│   └── Better for expression-optimized sequences
└── Consider different design tool
    └── BindCraft (integrated design) may work better

Too Few Pass ipTM Filter ( 0.5)

Low ipTM across campaign
├── Review hotspot selection
│   ├── Are hotspots surface-exposed? (SASA > 20Ų)
│   ├── Are hotspots conserved? (check MSA)
│   └── Try 3-6 different hotspot combinations
├── Increase binder length (more contact area)
│   └── Try 80-100 AA instead of 60-80 AA
├── Check interface geometry
│   ├── Is target flat? → Try helical binders
│   └── Is target concave? → Try smaller binders
└── Try all-atom design tool
    └── BoltzGen (all-atom, better packing)

High scRMSD (> 50% with scRMSD > 2.0Å)

Sequences don't specify intended structure
├── ProteinMPNN issue
│   ├── Lower temperature: --sampling_temp "0.1"
│   ├── Increase sequences: --num_seq_per_target 32
│   └── Check fixed_positions aren't over-constraining
├── Backbone geometry issue
│   ├── Backbones may be unusual/strained
│   ├── Regenerate with lower noise_scale (0.5-0.8)
│   └── Reduce diffuser.T to 30-40
└── Try different sequence design
    └── ColabDesign (AF2 gradient-based) may work better

Everything Passes But No Experimental Hits

In silico metrics don't predict affinity
├── Generate MORE designs (10x current)
│   └── Computational metrics have high false positive rate
├── Increase diversity
│   ├── Higher ProteinMPNN temperature (0.2-0.3)
│   ├── Different backbone topologies
│   └── Different hotspot combinations
├── Try different design approach
│   ├── BindCraft (different algorithm)
│   ├── ColabDesign (AF2 hallucination)
│   └── BoltzGen (all-atom diffusion)
└── Check if target is druggable
    └── Some targets are inherently difficult

Too Many Designs Pass (> 50%)

Suspiciously high pass rate
├── Check if thresholds are too lenient
│   └── Use stringent thresholds: pLDDT > 0.90, ipTM > 0.60
├── Verify prediction quality
│   ├── Are predictions actually running? Check output files
│   └── Are complexes being predicted, not just monomers?
├── Check for data issues
│   ├── Same sequence being predicted multiple times?
│   └── Wrong FASTA format (missing chain separator)?
└── Apply diversity filter
    └── Cluster at 70% identity, take top per cluster

---

Diagnostic Commands

Quick Campaign Assessment

import pandas as pd

df = pd.read_csv('designs.csv')

# Pass rates at each stage
print(f"Total designs: {len(df)}")
print(f"pLDDT > 0.85: {(df['pLDDT'] > 0.85).mean():.1%}")
print(f"ipTM > 0.50: {(df['ipTM'] > 0.50).mean():.1%}")
print(f"scRMSD  0.85) & (df['ipTM'] > 0.5) & (df['scRMSD']  0.85).mean()  0.50).mean() < 0.1:
    print("ISSUE: Low ipTM - check hotspots or interface geometry")
elif (df['scRMSD'] < 2.0).mean() < 0.5:
    print("ISSUE: High scRMSD - sequences don't specify backbone")

---

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: adaptyvbio
• Source: adaptyvbio/protein-design-skills
• License: MIT

Install and usage instructions live in the source repository linked above.