rlp-rl-pretraining-objective

【开发与工程】Integrate reinforcement learning into the pretraining phase by measuring the utility of intermediate reasoning for predicting subsequent tokens. This approach generates dense reward signals during standard pretraining, enabling models to develop reasoning abilities earlier and with significant performance gains (19% improvement on 1.7B, 45% lift on 1…

独立 GitHub 快照 · 自动分类 · 以原作者文档为准