noloco-low-communication-training

Scale distributed LLM training without all-reduce synchronization using dynamic pipeline routing and modified Nesterov momentum, achieving 4% faster convergence than DiLoCo with exponentially lower communication.

Independent GitHub snapshot · Automated classification · Original documentation governs