总览 / 概念 / Mixture of Experts
↗ 源 Markdown
概念
architecture model
创建2026-04-15
更新2026-04-15
阅读量级1 分钟
概念导读

Mixture of Experts (MoE) is a neural network architecture that routes tokens to different "expert" sub-networks, enabling larger model capacity without proportionally increasing compute per forward pass. MoE architectures are used in models like qwen3 and composer2, where they provide efficient scaling for large language models by activating only a subset of...

  1. Related
  2. References

Mixture of Experts (MoE)

Mixture of Experts (MoE) is a neural network architecture that routes tokens to different "expert" sub-networks, enabling larger model capacity without proportionally increasing compute per forward pass. MoE architectures are used in models like qwen3 and composer2, where they provide efficient scaling for large language models by activating only a subset of parameters per token.

  • qwen3 — Qwen3 uses MoE architecture in its larger model variants
  • composer2 — Composer 2 is built on a MoE architecture (1.04T/32B)

References