mixture-of-depths-attention

Allow attention heads to reference features from multiple depths by accessing both current-layer and depth key-value pairs. Prevent signal degradation in deep models while maintaining computational efficiency.

Independent GitHub snapshot · Automated classification · Original documentation governs