dualpath-storage-bandwidth-optimization

【效率与协作】Optimize disaggregated prefill-decoding LLM serving for multi-turn (agentic) workloads by introducing dual-path KV-cache loading. Traditional approach loads all KV-cache to prefill engines, saturating their storage network. DualPath loads to decoding engines first, then transfers via compute network (lower contention). Adaptive routing selects path b…

独立 GitHub 快照 · 自动分类 · 以原作者文档为准