h-net-dynamic-chunking-hierarchical-sequence

Eliminate fixed tokenization by learning data-dependent segmentation jointly with the model through dynamic chunking, matching BPE-tokenized Transformers at equivalent compute while showing improved robustness and better downstream task performance without vocabulary constraints.

Independent GitHub snapshot · Automated classification · Original documentation governs