AlloyDB Ships Proxy Models That Replace LLM Calls with Local Inference Inside the Database

Google has released AlloyDB AI functions that allow databases to perform inference locally using lightweight proxy models instead of external large language model calls. By training these models on previous LLM outputs, the system increases throughput by up to 2,400 times and removes the latency associated with network-based model requests.