specontext-speculative-caching

Uses distilled language model (DLM) for KV cache token retrieval, reducing layer-wise retrieval parameters by 90% via head-level attention weights, with asynchronous prefetching and elastic CPU offloading. Deploy for ultra-long-context inference with memory constraints.

Independent GitHub snapshot · Automated classification · Original documentation governs