progressive-residual-warmup

【效率与协作】Improves LLM convergence and downstream task performance by introducing time-dependent scaling to residual connections, enabling shallow layers to learn first before deeper layers activate. Apply during model pretraining to achieve 0.4-4.86 perplexity reduction.

独立 GitHub 快照 · 自动分类 · 以原作者文档为准