infllm-v2-dense-sparse-switchable-attention

Implement dense-sparse switchable attention enabling LLMs to scale from short to long sequences with 4× speedup and 98-99.7% performance retention, requiring no extra parameters by reusing pretrained attention weights through trainable sparse pattern selection.

Independent GitHub snapshot · Automated classification · Original documentation governs