sonicmoe-training-optimization

Optimize MoE training through memory-efficient backward pass, IO-aware kernel design overlapping memory operations, and token rounding routing. Avoid caching large-scale activations, fuse operations with GEMM, implement ping-pong scheduling. Achieve 1.86× compute throughput on Hopper GPUs and 16% higher TFLOPS in sparse configurations.

Independent GitHub snapshot · Automated classification · Original documentation governs