mixture-of-depths-attention

【效率与协作】Allow attention heads to reference features from multiple depths by accessing both current-layer and depth key-value pairs. Prevent signal degradation in deep models while maintaining computational efficiency.

独立 GitHub 快照 · 自动分类 · 以原作者文档为准