---
title: "Paraformer — Non-Autoregressive Parallel Transformer for ASR"
created: 2026-05-01
updated: 2026-05-01
type: concept
tags: [model, architecture, speech-model, end-to-end, open-source]
sources: [raw/papers/2022/06/2206.08317.md, raw/papers/2023/05/2305.11013.md]
---
# Paraformer — Non-Autoregressive Parallel Transformer for ASR

> arXiv 2206.08317 | Interspeech 2022 | Alibaba DAMO Speech Lab
> First NAR model to match AR transformer accuracy on large-scale ASR

## Core Problem

Autoregressive (AR) ASR models (Transformer, Conformer) generate tokens sequentially:

```
P(Y|X) = ∏ P(yₗ | y<ₗ, X)    ← linear in output length L
```

This creates a fundamental latency bottleneck — decoding time scales linearly with utterance length.

Non-autoregressive (NAR) models solve this by parallel generation:

```
P(Y|X) = ∏ P(yₗ | X)    ← all tokens simultaneously
```

But vanilla NAR suffers from two critical problems:
1. **Length prediction**: Hard to predict output token count from variable-rate speech
2. **Independence assumption**: Tokens generated without inter-dependency → high substitution errors

Paraformer is the first NAR model to achieve **comparable accuracy to AR transformers** while delivering **10x+ speedup**.

## Architecture

```
                    Paraformer Architecture
═══════════════════════════════════════════════════════

     f-banks → ┌─────────────┐
               │  Encoder    │  (Conformer/SAN-M blocks)
               │  H₁:ₜ       │
               └──────┬──────┘
                      │
              ┌───────┼───────────────┐
              ▼       ▼               │
     ┌────────────┐  ┌────────────┐   │
     │ Predictor  │  │  Sampler   │   │
     │ (CIF)      │  │  (GLM)     │   │
     │            │  │            │   │
     │ α weights  │  │ Eₐ → Eₛ    │   │
     │ Eₐ embeds  │  │ (training) │   │
     └─────┬──────┘  └─────┬──────┘   │
           │               │          │
           ▼               ▼          │
     ┌──────────────────────────┐     │
     │  Parallel Bi-Decoder     │     │
     │  Pass 1: Eₐ + H (no grad)│─────┘
     │  Pass 2: Eₛ + H (w/ grad)│
     │  Infer: Eₐ + H (single)  │
     └──────────┬───────────────┘
                ▼
           Y'' predictions
     ┌──────────────────────┐
     │  Loss:               │
     │  L = γL_CE + L_MAE   │
     │      + L_MWER        │
     └──────────────────────┘
```

## Key Mechanism 1: CIF Predictor

**Continuous Integrate-and-Fire** solves the length prediction problem.

The predictor produces frame-level weights αₜ ∈ [0,1]:

```
α₁:ₜ = Sigmoid(Linear(Conv(H₁:ₜ)))
```

CIF accumulates weights and integrates encoder outputs until reaching threshold β:

```
# CIF process (β = 1.0):
α = (0.4, 0.8, 0.3, 0.5, 0.2)

E₁ = 0.4 × e₁ + 0.6 × e₂     ← sum crosses 1.0, fire
E₂ = 0.2 × e₂ + 0.3 × e₃ + 0.5 × e₄  ← next boundary
```

General formulation:
```
Eᵤ = Σₜ wᵤₜ · hₜ    where Σₜ wᵤₜ = 1 for each output token u
```

**Dynamic threshold** β eliminates train/inference mismatch:

```
β = Σ αₜ / ⌈Σ αₜ⌉
```

MAE loss guides convergence:
```
L_MAE = |N - Σₜ αₜ|
```

## Key Mechanism 2: GLM Sampler

**Glancing Language Model** fixes the independence assumption.

During training, the sampler replaces some acoustic embeddings with ground-truth token embeddings:

```
Eₛ = Sampler(Eₐ, E_c, ⌈λ · d(Y, Y')⌉)
```

Where:
- `λ` = sampling factor (optimal ≈ 0.75)
- `d(Y, Y')` = Hamming distance between prediction and ground truth
- The number of replacements decreases as training progresses

The GLM loss:
```
L_GLM = Σ log p(yₙ | GLM(Y, Y'), X; θ)
         yₙ ∈ GLM(Y, Y')
```

At inference, the sampler is **inactive** — pure single-pass parallel decoding.

## Key Mechanism 3: MWER Training

**Minimum Word Error Rate** training with negative samples:

```
L_Nwerr(x, y*) = Σ [P_b(yᵢ|x) · W(yᵢ, y*) - W̃]
                  yᵢ ∈ Sample(x, N)
```

Negative candidates generated by randomly masking the top-1 score token during training.

Total loss:
```
L_total = γ·L_CE + L_MAE + L_Nwerr(x, y*)
```

Note: In FunASR (2023), MWER was removed as it contributed little to performance gains; a CE loss on first-pass decoder was added instead to reduce train/inference gap.

## Benchmarks

### AISHELL-1 (178 hrs Mandarin)

| Model | Type | Dev CER | Test CER | RTF (bs=1) |
|-------|------|---------|----------|------------|
| A-FMLM | NAR | 6.2 | 6.7 | 0.2800 |
| Mask-CTC | NAR | 6.9 | 7.8 | 0.0500 |
| CASS-NAT | NAR | 4.8 | 5.2 | 0.0037 |
| AR Transformer | AR | 4.7 | 5.4 | 0.0230 |
| **Paraformer** | **NAR** | **4.6** | **5.2** | **0.0037** |

### AISHELL-2 (1000 hrs Mandarin)

| Model | Type | Test iOS CER | RTF |
|-------|------|-------------|-----|
| AR Transformer | AR | 6.18 | 0.0168 |
| Vanilla-NAR | NAR | 6.23 | 0.0168 |
| **Paraformer** | **NAR** | **6.19** | **0.0026** |

### Industrial 20,000 hrs

| Model | Far-field CER | Common CER |
|-------|--------------|------------|
| CTC baseline | 17.71 | 9.93 |
| **Paraformer** | **13.94** | **7.97** |
| AR Transformer | 14.72 | 8.66 |

### Error Type Analysis (Industrial 20k hrs)

Paraformer vs vanilla NAR:
- **Substitution errors**: Paraformer ~0.02 vs vanilla NAR ~0.07 (GLM reduces this ~3x)
- **Insertion errors**: Comparable
- **Deletion errors**: Comparable

The GLM sampler dramatically reduces substitution errors — the main weakness of vanilla NAR — by learning inter-token dependencies during training.

## Sampling Ratio Study

| λ | Far-field CER | Common CER |
|---|--------------|------------|
| 0.2 | 14.64 | 8.22 |
| 0.5 | 14.37 | 8.13 |
| **0.75** | **14.17** | **7.98** |
| 1.0 | 14.24 | 8.09 |
| 1.5 | 14.47 | 8.13 |

Optimal λ = 0.75 balances learning signal and acoustic fidelity.

## Evolution & Variants

- [[seaco-paraformer]] — Adds hotword customization via CIF-based contextual module
- [[sa-paraformer]] — Adds speaker attribution for multi-speaker meetings
- [[paraformer-v2]] — Replaces CIF with CTC predictor for noise robustness & multilingual
- [[funasr]] — Production toolkit packaging all variants with FSMN-VAD, CT-Transformer

## Comparison with Other Approaches

| Aspect | AR Transformer | Vanilla NAR | Paraformer |
|--------|---------------|-------------|------------|
| Decoding | Sequential | Parallel | Parallel |
| Token count | Implicit | Predictor | CIF |
| Context modeling | Built-in | None | GLM sampler |
| Speed | 1x | 10x+ | 10x+ |
| Accuracy | Baseline | -15% | ≈AR |
| Train passes | 1 | 1 | 2 (inference: 1) |

## Related

- [[funasr]] — Production toolkit
- [[paraformer-v2]] — CTC-based successor
- [[nim4-asr]] — LLM-based ASR alternative (2026)
- [[qwen3-asr]] — Alibaba's LLM-ASR (2026)