infllm-v2-dense-sparse-switchable-attention

【效率与协作】Implement dense-sparse switchable attention enabling LLMs to scale from short to long sequences with 4× speedup and 98-99.7% performance retention, requiring no extra parameters by reusing pretrained attention weights through trainable sparse pattern selection.

独立 GitHub 快照 · 自动分类 · 以原作者文档为准