Buoyant Releases Kinference: An Open Source Inference Control Plane for Kubernetes and vLLM

Buoyant has launched Kinference, an open-source control plane designed to manage vLLM inference workloads running on Kubernetes. The tool introduces declarative routing and cache-aware load balancing to improve performance, with internal testing showing a significant reduction in time to first token. By providing per-tenant metrics and refined traffic management, Kinference aims to simplify the infrastructure requirements for deploying large language models at scale.