let-late-early-training

【效率与协作】Accelerate LLM pretraining by leveraging small pretrained models as teachers, enabling large models to explicitly learn late-layer knowledge in early layers via alignment loss, achieving 1.6x convergence speedup and 5% downstream improvement even with 10x smaller teachers.

独立 GitHub 快照 · 自动分类 · 以原作者文档为准