Edit · 02 Architecture ← View page Saving needs a token. Log in or register to get one. Edits are public and licensed CC-BY-SA-4.0. # SignalP 6.0 — Architecture & Training SignalP 6.0 is a **protein language model (encoder) + conditional random field (decoder)**. The language model reads the sequence and produces a rich per-residue representation; the CRF turns those representations into a coherent left-to-right labeling of the sequence (which residues are n/h/c-region, where the cleavage site is, which SP type). ``` FASTA sequence │ ▼ [ organism-group embedding ] ──► prepended as extra token │ ▼ 30-layer BERT protein LM (ProtBERT, pretrained on UniRef100) ← encoder │ per-residue hidden states ▼ linear projection → CRF emissions │ ▼ linear-chain CRF with learnable transition matrix ← decoder │ ▼ per-residue labels → SP type, cleavage site, n/h/c regions ``` ## Encoder — the protein language model - Backbone is the **30-layer BERT** model from **ProtTrans (ProtBERT-BFD)**, **pretrained on UniRef100** (~200M+ protein sequences) with masked-language-model self-supervision. This pretraining is where the model learns evolutionary and structural regularities **without any signal-peptide labels**. - SignalP 6.0 **removes the final pretraining head** and fine-tunes the encoder on the SP task. - **Organism conditioning:** four randomly-initialized embedding vectors represent the organism groups — **Archaea, Eukarya, Gram-positive bacteria, Gram-negative bacteria** — and are fed in so the model can adjust its expectations by kingdom. (At inference you only need to say `eukarya` vs `other`; Eukarya restricts output to Sec/SPI.) ## Decoder — conditional random field - A **linear-chain CRF** predicts a label at every position, with a **learnable transition matrix** (C × C, where C = number of states) enforcing that labels follow biologically valid orderings (e.g., n → h → c → mature, cleavage after c). - A **"multi-tag" CRF** formulation lets a single position carry overlapping region labels during training, which is how it learns region boundaries jointly with SP type and cleavage site. - **Region prediction via weak supervision:** there are no ground-truth n/h/c labels, so the training uses rule-based weak labels + regularization — - n-region: ≥2 residues, enriched in basic residues; - h-region: located at the peak hydrophobicity over a sliding 7-residue (Kyte–Doolittle) window; - c-region: ≥3 residues for Sec/SPI (absent for lipoprotein SPs). A **region regularizer** pushes the three regions toward divergent amino-acid compositions (cosine similarity of normalized score vectors), keeping predictions biologically plausible. ## Training data Homology-reduced set of experimentally annotated proteins (the SignalP 5.0 dataset, extended with the rare types): | Class | Sequences | |-------|-----------| | Sec/SPI | 3,352 | | Sec/SPII | 2,261 | | Tat/SPI | 595 | | Sec/SPIII (pilin) | 113 | | Tat/SPII | 36 | | Intracellular (negative) | 16,421 | | Transmembrane (negative) | 2,615 | The two new types (**Tat/SPII: 36**, **Sec/SPIII: 113**) are strikingly data-poor — this is exactly why a pretrained language model was needed to make them learnable. ## Training procedure - **Threefold nested cross-validation** with **homology partitioning at 30% sequence identity** — homologous sequences are forced into the same partition to prevent train/test leakage and give honest generalization estimates. - **15 epochs**, **Adamax** optimizer, **slanted triangular learning rate**, dropout on the BERT hidden states. - Hyperparameters tuned with **SigOpt**. - The released predictor is an **ensemble of the cross-validation models**. ## Fast vs. slow mode - **`fast`** (default): a **distilled / lighter** version of the model for throughput. - **`slow`**: the **full model** (~**6× slower**); use it when you need **accurate region borders**. There is also **`slow-sequential`** for lower memory. - Runs on CPU or GPU (PyTorch); GPU mainly helps large batches. Next: [03-usage.md](03-usage.md) for running it. Save Cancel Preview · live