1|# Speech LLM 2| 3|Speech Large Language Models (Speech LLMs) are end-to-end (E2E) multimodal models that process continuous speech signals directly, replacing traditional cascaded architectures (ASR + LLM + TTS). 4| 5|## Key Properties 6|- Direct speech modeling: Operates in continuous speech signal space rather than discrete text tokens 7|- Reduced latency: Eliminates intermediate ASR/TTS conversion steps 8|- Paralinguistic capture: Models intonation, emotion, and environmental context 9|- Performance gap: Exhibit significant degradation in complex instruction following, logical reasoning, and knowledge-intensive queries compared to text-based counterparts 10| 11|## Current Challenges 12|- Scarcity of high-quality paired speech-reasoning data 13|- Misalignment between continuous acoustic representations and discrete logical space of text LLMs 14|- Inability to transfer high-quality SFT and RL data from text LLM training 15|- Exposure bias in offline distillation methods 16| 17|## Notable Systems 18|- GPT-4o, Gemini 2.5, Qwen3-Omni, Voxtral 19| 20|## Training Paradigms 21|- Standard SFT + RL (insufficient for gap closure) 22|- Offline distillation (exposure bias issues) 23|- X-OPD: Cross-Modal On-Policy Distillation for capability alignment 24| 25|[src: raw/ingested/2026/03/2603.24596.md]

Related

• seeduplex
• moshi
• minmo
• llama-omni
• freeze-omni
• full-duplex-speech-model
• stream-omni — streaming speech model for real-time interaction