let-late-early-training

Accelerate LLM pretraining by leveraging small pretrained models as teachers, enabling large models to explicitly learn late-layer knowledge in early layers via alignment loss, achieving 1.6x convergence speedup and 5% downstream improvement even with 10x smaller teachers.

Independent GitHub snapshot · Automated classification · Original documentation governs