Cloud Run Now Supports NVIDIA GPUs, Making AI Workloads Easier and More Cost-Efficient

June 2, 2025

Google Cloud has announced that NVIDIA GPU support for Cloud Run is now generally available (GA). This update brings significant improvements to the serverless runtime, making it a powerful option for running AI workloads. Here’s what changed technically and why it matters to practitioners:

Key Technical Changes

• Pay-per-second Billing: You're only charged for the GPU resources you use, down to the second. This granular billing model ensures you’re not paying for idle time.
• Scale to Zero: Cloud Run automatically scales your GPU instances down to zero when no requests are received. This is particularly useful for sporadic or unpredictable workloads, as it eliminates idle costs.
• Rapid Startup and Scaling: Instances can go from zero to a fully configured state with a GPU and drivers installed in under 5 seconds. For example, during cold starts (scaling from zero), the Time-to-First-Token for the gemma3:4b model was approximately 19 seconds, which includes startup time, model loading time, and inference execution.
• Full Streaming Support: Cloud Run now supports HTTP and WebSocket streaming out of the box. This is ideal for building interactive applications where you need to provide real-time responses.

Implementation Details

• GPU Configuration: To use GPUs in Cloud Run, simply add --gpu 1 to your command-line deployment or enable the "GPU" checkbox in the console. No quota requests are required, which removes a significant barrier to entry.
• Model Inference: The rapid startup and scaling capabilities make it ideal for running large language models (LLMs) and other AI inference tasks. For instance, the gemma3:4b model can be up and running quickly, providing fast responses to user queries.

Benefits for Practitioners

• Cost Efficiency: Pay-per-second billing and scale-to-zero functionality mean you only pay for what you use, reducing costs significantly.
• Flexibility: The ability to handle both CPUs and GPUs in the same serverless runtime provides a versatile environment for various workloads.
• Simplicity: No need to manage infrastructure or request quotas. You can focus on your application logic rather than operational overhead.

Use Cases

• AI Inference: Ideal for running large language models, image generation, and other AI tasks that benefit from GPU acceleration.
• Interactive Applications: Build real-time applications with support for HTTP and WebSocket streaming, ensuring smooth user experiences.
• Data Processing: Handle data-intensive tasks efficiently with the power of GPUs.

Production-Ready

With general availability, Cloud Run with GPU support is now covered by Google Cloud's Service Level Agreement (SLA), providing reliability and uptime guarantees. This makes it a robust choice for production environments.

Industry Reaction

"Serverless GPU acceleration represents a major advancement in making cutting-edge AI computing more accessible. With seamless access to NVIDIA L4 GPUs, developers can now bring AI applications to production faster and more cost-effectively than ever before.", Dave Salvator, Director of Accelerated Computing Products, NVIDIA

Getting Started

To start using GPUs with Cloud Run, simply use the --gpu 1 flag in your deployment command or enable the "GPU" checkbox in the Google Cloud Console. No quota requests are necessary, making it easy to get up and running.