概念导读

Commercial ASR systems need hotword customization — users should be able to bias recognition toward names, places, domain-specific terms. Prior approaches:

1. Core Problem
2. Architecture
3. Key Mechanism 1: CIF-Based Contextual Module
4. Key Mechanism 2: Attention Score Filtering (ASF)
5. Key Mechanism 3: Explicit Hotword Prediction
6. Benchmarks
7. Hotword Customization Results
8. ASF Impact

SeACo-Paraformer — Semantic-Augmented Contextual Paraformer

arXiv 2308.03266 | Alibaba DAMO Speech Lab | GitHub: R1ckShi/SeACo-Paraformer NAR ASR + explicit hotword customization without sacrificing parallel decoding

Core Problem

Commercial ASR systems need hotword customization — users should be able to bias recognition toward names, places, domain-specific terms. Prior approaches:

Approach	Type	Issue
CLAS (Contextual LAS)	Implicit	Training instability, invisible activation, hard to debug
CPP Network	Implicit	Adds CTC head; couples general ASR with contextual modeling
CIF ColDec	Explicit	Good decoupling, but backbone model accuracy lags AED

SeACo-Paraformer combines: 1. Paraformer's AED-level accuracy + NAR efficiency 2. CIF ColDec's explicit, decoupled hotword prediction 3. Attention Score Filtering (ASF) for large-scale hotword lists

Architecture

              SeACo-Paraformer Architecture
═══════════════════════════════════════════════════════════

  Speech ──→ Encoder ──→ Predictor (CIF) ──→ Eₐ (acoustic embeds)
                                      │
                    ┌───────────────────┤
                    ▼                   ▼
          ┌─────────────────┐  ┌──────────────────┐
          │  ASR Decoder    │  │ Hotword Embedder │
          │  (Paraformer)   │  │  (LSTM)          │
          └────────┬────────┘  └────────┬─────────┘
                   │                    │
                   │              E_h (hotword embeds)
                   │                    │
                   ▼                    ▼
          ┌─────────────────────────────────────┐
          │   CIF-based Contextual Module       │
          │                                     │
          │   MHA(Eₐ, E_h) → g (hotword sem.)   │
          │   FF(g) → P_hotword (per-token)     │
          │                                     │
          │   E'_a = Eₐ + α · MHA_output        │
          └──────────────────┬──────────────────┘
                             ▼
                   Final ASR Prediction

Key Mechanism 1: CIF-Based Contextual Module

The contextual module runs synchronously with ASR output, leveraging CIF's monotonic alignment:

gₜ = MHA(Eₐₜ, E_h, E_h)          # Attend hotwords from each token position
P_hotwordₜ = FF(gₜ)               # Feed-forward → hotword probability

The hotword embedding E_h is produced by an LSTM over the hotword list:

E_h = LSTM(w₁, w₂, ..., wₙ)      # Last hidden state

At each CIF output step, the model produces an external hotword probability independent of the main ASR prediction. These two probability streams are merged at inference.

Key Mechanism 2: Attention Score Filtering (ASF)

When the hotword list grows large (thousands of entities), performance degrades. ASF filters irrelevant hotwords:

# Compute attention scores between acoustic context and each hotword
scoreⱼ = attention_score(Eₐ, E_hⱼ)

# Keep only hotwords above threshold τ
E_h_filtered = {E_hⱼ | scoreⱼ > τ}

This prevents attention dilution — when too many hotwords compete for attention weight, none receives sufficient signal. ASF maintains high F1 even with large hotword inventories.

Key Mechanism 3: Explicit Hotword Prediction

Unlike implicit methods (CLAS) that bias the ASR decoder internally, SeACo-Paraformer's contextual module produces a separate hotword probability stream:

# ASR branch: P(yₜ | X)
# Hotword branch: P(hotword | Eₐₜ, E_h)

# Merge at inference:
P_final = P(yₜ | X) + γ · P(hotword) · indicator(hotword matches yₜ)

The hotword training target uses position-aware labels:

# ASR target: "A B C D E F"
# Hotword target: "A B # # # #"  (when "A B" is a hotword)

This decoupling means: - Hotword module parameters are independent of ASR parameters - Bad cases are easier to debug (you can see whether the hotword detector failed or the ASR failed) - The hotword detector works as an explicit hotword presence signal

Benchmarks

Hotword Customization Results

Dataset	Hotwords	w/o Hotword CER	w/ Hotword CER	F1 (w/o)	F1 (w/)
AISHELL NE subtest	187 entities	10.01	4.55	27	85
Industrial AI domain	70	7.96	6.31	82	93
Industrial common domain	67	9.47	8.75	80	88

Key result: +58% F1 improvement on AISHELL-1 named entity subtest.

ASF Impact

Hotword Scale	w/o ASF CER	w/ ASF CER
Small (< 50)	Baseline	Comparable
Large (500+)	Degrades	Maintained

ASF prevents the performance drop that occurs when large hotword lists dilute attention.

50,000 hrs Industrial Data

Full model trained on 50k hours of industrial data, outperforming strong baselines (CLAS, CPP Network variants) in customization tasks while maintaining NAR parallel decoding efficiency.

Comparison: Explicit vs Implicit Contextual ASR

Aspect	CLAS (Implicit)	CPP (Implicit)	CIF ColDec	SeACo-Paraformer
Backbone	LAS (AR)	CTC/Trans/Trans	CIF-AED	Paraformer (NAR)
Hotword signal	Implicit bias	Implicit CTC	Explicit	Explicit
Decoupled	No	No	Yes	Yes
Parallel decoding	No	Varies	No	Yes
Large hotword support	Poor	Moderate	Poor	Yes (ASF)
Train stability	Issues	OK	Good	Good

Related

• paraformer — Base NAR architecture
• funasr — Production toolkit (includes Contextual Paraformer)
• fun-codec — Neural codec toolkit
• nim4-asr — LLM-ASR with phoneme-level RAG hotwords (2026), alternative approach to customization