rlp-rl-pretraining-objective

Integrate reinforcement learning into the pretraining phase by measuring the utility of intermediate reasoning for predicting subsequent tokens. This approach generates dense reward signals during standard pretraining, enabling models to develop reasoning abilities earlier and with significant performance gains (19% improvement on 1.7B, 45% lift on 12B mode…

Independent GitHub snapshot · Automated classification · Original documentation governs