specontext-speculative-caching

【部署与运维】Uses distilled language model (DLM) for KV cache token retrieval, reducing layer-wise retrieval parameters by 90% via head-level attention weights, with asynchronous prefetching and elastic CPU offloading. Deploy for ultra-long-context inference with memory constraints.

独立 GitHub 快照 · 自动分类 · 以原作者文档为准