mu-parametrization-moe

Apply μ-parametrization to Mixture-of-Experts architectures to enable reliable hyperparameter transfer across model sizes, eliminating costly retuning when scaling to trillion-parameter systems.

Independent GitHub snapshot · Automated classification · Original documentation governs