hybrid-linear-attention

【设计与创意】Design hybrid architectures mixing linear and full attention at optimal ratios. Superior standalone linear models don't necessarily excel in hybrids; recall improves significantly with more full attention layers at ratios below 3:1, enabling efficient long-sequence processing.

独立 GitHub 快照 · 自动分类 · 以原作者文档为准