Reasoning distillation is the process of transferring reasoning abilities — particularly chain-of-thought capabilities — from a teacher model to a student model. This is a specialized form of knowledge-distillation that focuses on preserving the quality and structure of reasoning traces rather than just final answer accuracy. The on-policy-distillation-surve...
- Related
- References
Reasoning Distillation
Reasoning distillation is the process of transferring reasoning abilities — particularly chain-of-thought capabilities — from a teacher model to a student model. This is a specialized form of knowledge-distillation that focuses on preserving the quality and structure of reasoning traces rather than just final answer accuracy. The on-policy-distillation-survey discusses reasoning distillation methods, including approaches like deepseek-r1-distillation.
Related
- on-policy-distillation-survey — Surveys reasoning distillation approaches
- chain-of-thought — CoT traces are the primary target of reasoning distillation
- knowledge-distillation — Broader knowledge distillation framework
- deepseek-r1-distillation — A prominent example of reasoning distillation in practice