progressive-residual-warmup

Improves LLM convergence and downstream task performance by introducing time-dependent scaling to residual connections, enabling shallow layers to learn first before deeper layers activate. Apply during model pretraining to achieve 0.4-4.86 perplexity reduction.

Independent GitHub snapshot · Automated classification · Original documentation governs