AWS Lambda Now Runs GPU Inference with 48 Concurrent Invocations Sharing One Model per GPU
AWS Lambda now supports GPU-accelerated inference by allowing up to 48 concurrent function invocations to share a single model on an EC2 G-series instance. This update utilizes vLLM and SGLang base images, enabling developers to deploy machine learning models without writing custom handler code. By allowing asynchronous timeouts of up to 90 minutes, this change simplifies the deployment of resource-intensive AI workloads within the serverless environment.
Covered by 1 source
- IInfoQ AI↗Steef-Jan Wiggers6h ago