vespo-variational-sequence-rl

【效率与协作】Stabilize off-policy RL training for LLMs by deriving principled importance weight reshaping from variational optimization. Instead of heuristic clamping, VESPO uses closed-form exponential weighting W^c1 * exp(c2*(1-W)) to suppress overweighted samples while maintaining smooth gradients. Enables stable training at 64× policy staleness and under full…

独立 GitHub 快照 · 自动分类 · 以原作者文档为准