minicpm-sala

Combine sparse attention (25% of layers) and linear attention (75% of layers) via strategic layer placement to handle 1M-token contexts with 75% training cost reduction. Hybrid positional encoding preserves long-range information while maintaining position awareness.

Independent GitHub snapshot · Automated classification · Original documentation governs