sparse-sparse-attention

【效率与协作】Overcome the attention gap in sparse transformers by training with both full and sparse attention simultaneously, aligned through bidirectional losses that encourage naturally sparser distributions while maintaining learning capability, enabling efficient inference without capability degradation.

独立 GitHub 快照 · 自动分类 · 以原作者文档为准