Free Process Rewards refers to the idea of obtaining process-level reinforcement learning reward signals without requiring manual annotation. This concept is explored in generalized-on-policy-distillation as a way to provide richer training signals for on-policy distillation. By leveraging existing model outputs or automated verification, free process reward...
- Related
- References
Free Process Rewards
Free Process Rewards refers to the idea of obtaining process-level reinforcement learning reward signals without requiring manual annotation. This concept is explored in generalized-on-policy-distillation as a way to provide richer training signals for on-policy distillation. By leveraging existing model outputs or automated verification, free process rewards aim to reduce the cost of RL training while maintaining process-level granularity.
Related
- generalized-on-policy-distillation — Introduces free process rewards in the G-OPD framework
- grpo-rl-training — GRPO training methodology that can leverage process rewards
- reinforcement-learning-from-human-feedback — RLHF as the broader reward framework