muon-optimizer-tail-memory-learning

Improve LLM training efficiency by selectively applying Muon optimizer to Value-Output attention weights and FFN layers, which function as associative memories. Use when training data exhibits heavy-tailed distributions requiring robust rare-fact learning.

Independent GitHub snapshot · Automated classification · Original documentation governs