minicpm-sala

【效率与协作】Combine sparse attention (25% of layers) and linear attention (75% of layers) via strategic layer placement to handle 1M-token contexts with 75% training cost reduction. Hybrid positional encoding preserves long-range information while maintaining position awareness.

独立 GitHub 快照 · 自动分类 · 以原作者文档为准