02-architecture.md

SignalP 6.0 — Architecture & Training

SignalP 6.0 is a protein language model (encoder) + conditional random field (decoder). The language model reads the sequence and produces a rich per-residue representation; the CRF turns those representations into a coherent left-to-right labeling of the sequence (which residues are n/h/c-region, where the cleavage site is, which SP type).

FASTA sequence
      │
      ▼
[ organism-group embedding ]  ──►  prepended as extra token
      │
      ▼
30-layer BERT protein LM (ProtBERT, pretrained on UniRef100)   ← encoder
      │  per-residue hidden states
      ▼
linear projection → CRF emissions
      │
      ▼
linear-chain CRF with learnable transition matrix              ← decoder
      │
      ▼
per-residue labels → SP type, cleavage site, n/h/c regions

Encoder — the protein language model

Backbone is the 30-layer BERT model from ProtTrans (ProtBERT-BFD), pretrained on UniRef100 (~200M+ protein sequences) with masked-language-model self-supervision. This pretraining is where the model learns evolutionary and structural regularities without any signal-peptide labels.
SignalP 6.0 removes the final pretraining head and fine-tunes the encoder on the SP task.
Organism conditioning: four randomly-initialized embedding vectors represent the organism groups — Archaea, Eukarya, Gram-positive bacteria, Gram-negative bacteria — and are fed in so the model can adjust its expectations by kingdom. (At inference you only need to say eukarya vs other; Eukarya restricts output to Sec/SPI.)

Decoder — conditional random field

A linear-chain CRF predicts a label at every position, with a learnable transition matrix (C × C, where C = number of states) enforcing that labels follow biologically valid orderings (e.g., n → h → c → mature, cleavage after c).
A "multi-tag" CRF formulation lets a single position carry overlapping region labels during training, which is how it learns region boundaries jointly with SP type and cleavage site.
Region prediction via weak supervision: there are no ground-truth n/h/c labels, so the training uses rule-based weak labels + regularization —
n-region: ≥2 residues, enriched in basic residues;
h-region: located at the peak hydrophobicity over a sliding 7-residue (Kyte–Doolittle) window;
c-region: ≥3 residues for Sec/SPI (absent for lipoprotein SPs). A region regularizer pushes the three regions toward divergent amino-acid compositions (cosine similarity of normalized score vectors), keeping predictions biologically plausible.

Training data

Homology-reduced set of experimentally annotated proteins (the SignalP 5.0 dataset, extended with the rare types):

| Class | Sequences | |-------|-----------| | Sec/SPI | 3,352 | | Sec/SPII | 2,261 | | Tat/SPI | 595 | | Sec/SPIII (pilin) | 113 | | Tat/SPII | 36 | | Intracellular (negative) | 16,421 | | Transmembrane (negative) | 2,615 |

The two new types (Tat/SPII: 36, Sec/SPIII: 113) are strikingly data-poor — this is exactly why a pretrained language model was needed to make them learnable.

Training procedure

Threefold nested cross-validation with homology partitioning at 30% sequence identity — homologous sequences are forced into the same partition to prevent train/test leakage and give honest generalization estimates.
15 epochs, Adamax optimizer, slanted triangular learning rate, dropout on the BERT hidden states.
Hyperparameters tuned with SigOpt.
The released predictor is an ensemble of the cross-validation models.

Fast vs. slow mode

fast (default): a distilled / lighter version of the model for throughput.
slow: the full model (~6× slower); use it when you need accurate region borders. There is also slow-sequential for lower memory.
Runs on CPU or GPU (PyTorch); GPU mainly helps large batches.

Next: 03-usage.md for running it.