总览 / 概念 / Reinforcement Learning from Human Feedba
↗ 源 Markdown
概念
rlhf rl alignment
创建2026-04-15
更新2026-04-15
阅读量级1 分钟
概念导读

Reinforcement Learning from Human Feedback (RLHF) is a post-training technique that aligns language models with human preferences by training a reward model on human preference data and then optimizing the language model policy against this reward using reinforcement learning. RLHF is a key component of modern LLM training pipelines, including qwen3, and is ...

  1. Related
  2. References

Reinforcement Learning from Human Feedback (RLHF)

Reinforcement Learning from Human Feedback (RLHF) is a post-training technique that aligns language models with human preferences by training a reward model on human preference data and then optimizing the language model policy against this reward using reinforcement learning. RLHF is a key component of modern LLM training pipelines, including qwen3, and is related to methods like grp-o which provide alternatives to traditional RLHF.

  • qwen3 — Qwen3 uses RLHF in its post-training pipeline
  • grp-o — GRPO provides a group-relative alternative to standard RLHF
  • free-process-rewards — Process-level reward signals as an alternative to human feedback

References

  • Ouyang et al. (2022). "Training language models to follow instructions with human feedback." NeurIPS.