Llumnix-KV#
Llumnix-KV is a general, flexible, and high-performance KV cache transfer and storage framework for distributed LLM inference consisting of two core components: Hybrid Connector and Blade-KVT.
Contents
Llumnix-KV is a general, flexible, and high-performance KV cache transfer and storage framework for distributed LLM inference consisting of two core components: Hybrid Connector and Blade-KVT.
Contents