概念导读

Video-OPD (Video On-Policy Distillation) is an efficient post-training framework for Temporal Video Grounding (TVG) that combines on-policy optimization with dense token-level supervision from a fixed frontier teacher model.

1. Core Idea
2. Training Pipeline (4 Steps)
3. Key Advantages over GRPO
4. Integration with TVDF
5. Experimental Results
6. Relation to Existing Concepts

Video-OPD

Video-OPD (Video On-Policy Distillation) is an efficient post-training framework for Temporal Video Grounding (TVG) that combines on-policy optimization with dense token-level supervision from a fixed frontier teacher model.

Core Idea

Video-OPD addresses two key limitations of GRPO-based post-training for TVG:

1. Sparse reward signals: GRPO provides only sequence-level rewards, causing poor credit assignment over long temporal horizons.
2. Prohibitive computational overhead: GRPO requires multiple on-policy rollouts per training sample for variance reduction, which is expensive for video understanding with long visual contexts.

Video-OPD preserves strict on-policy optimization (trajectories sampled from current student policy) while replacing sparse episode-level rewards with dense token-level supervision via reverse KL divergence against a teacher model.

Training Pipeline (4 Steps)

Step	Description
1. On-Policy Trajectory Sampling	Student samples trajectory τ=(a_1,...,a_T) ~ π_θ(·
2. Teacher Evaluation	Fixed teacher π_tea computes log-probabilities for each student token
3. Dense Token-Level Supervision	Per-token reward r_t = -(log π_θ(a_t
4. Policy Update	Standard policy-gradient with token-level rewards, single rollout per sample

Key Advantages over GRPO

Property	GRPO	Video-OPD
Optimization	On-policy	On-policy
Supervision	Sparse (sequence-level)	Dense (token-level)
Rollouts per sample	Multiple (e.g., 8)	Single
Credit assignment	Poor for long horizons	Fine-grained per token
Convergence	Slow	Substantially faster
Computational cost	High	~20% of GRPO

Integration with TVDF

Video-OPD is enhanced by Teacher-Validated Disagreement Focusing, a training curriculum that uses ground-truth annotations as validation signals to prioritize teacher-reliable, high-disagreement trajectories.

Experimental Results

• Outperforms GRPO by average 17% on Charades-TimeLens, ActivityNet-TimeLens, QVHighlights-TimeLens
• Achieves SOTA among open-source models, approaching Gemini-2.5-Flash performance
• Strong generalization to TempCompass, MVBench, Video-MME
• Student consistently matches or surpasses teacher after multi-round training

Relation to Existing Concepts

• Extends on-policy-distillation to temporal video grounding with unique challenges of long-horizon temporal reasoning
• Contrasts with OPSD: uses external teacher rather than self-teaching
• Contrasts with vold: focuses on TVG specifically rather than modality transfer
• Uses reverse KL (mode-seeking) rather than forward KL (mode-covering) for supervision
• Related to PPD but uses simpler policy-gradient without PPO clipping

[src: raw/ingested/2026/02/2602.02994.md]