Cursor’s AI-Powered IDE: Scaling to 1M+ QPS and Billions of Code Completions Daily

Cursor, the AI-powered IDE that has become a favorite among software engineers, has seen an astounding 100x increase in load over just one year. The platform now handles over 1 million queries per second (QPS) for its data layer and serves billions of code completions daily. This rapid growth is a testament to both the product's utility and the engineering prowess behind it. I had the opportunity to sit down with Sualeh Asif, cofounder of Anysphere (the company behind Cursor), to dive into the technical details of how they've scaled such an ambitious project.

Key Technical Changes and Why They Matter

Tech Stack

• Languages: TypeScript for the frontend and Rust for backend services.
• Cloud Providers: AWS (Amazon EC2, AWS Firecracker), with monitoring tools like Datadog and alerting via PagerDuty.
• Orchestrator Service: Anyrun, a custom service built in Rust to manage cloud agents securely.

Autocomplete Engine

The autocomplete feature is crucial for Cursor's real-time code suggestions. Here’s how it works:

• Low-Latency Sync Engine: Encrypted context from the user's IDE is sent to the server.
• Server-Side Inference: The server runs inference on this context to generate code completions, ensuring low latency and high accuracy.

Chat Without Storing Code

One of the standout features of Cursor is its ability to provide AI-powered chat without storing any source code on the server. This is achieved using:

• Merkle Trees: These data structures allow the server to verify and search through user code without actually storing it.
• Embeddings: The system uses embeddings to enable semantic search, ensuring that users can find relevant information quickly.

Anyrun: Secure Agent Management

Anyrun, a Rust-based service, is responsible for launching and managing agents in the cloud. Key points:

• Security: Ensures secure process isolation using Amazon EC2 and AWS Firecracker.
• Scalability: Manages the dynamic scaling of agents to handle varying loads.

Engineering Challenges

Usage Patterns

The team had to adapt their technology stack based on how users interacted with Cursor. For example:

• Cold Start Problem: Ensuring that new users get accurate code completions without a large initial dataset.
• Sharding Challenges: Managing data sharding to distribute load effectively across multiple servers.

Database Migrations

Despite starting with Yugabyte, a database designed for infinite scalability, the team faced issues and had to migrate:

• From Yugabyte to PostgreSQL: This move was driven by performance bottlenecks and operational challenges.
• Turbopuffer Migration: A significant effort to integrate Turbopuffer, which improved data handling and query performance.

Notable Additions in Cursor 1.0

The latest major release, Cursor 1.0, includes several new features:

• AI Code Review (BugBot): An automated tool that reviews code for bugs and suggests improvements.
• Background Agents: These agents run tasks in the background, enhancing productivity without interrupting the user's workflow.
• Memory Support: The system now remembers details from past chats, providing a more personalized experience.

Conclusion

Cursor’s rapid growth and innovative features are a result of thoughtful engineering decisions and a deep understanding of user needs. By leveraging advanced technologies like Merkle trees, secure cloud management, and efficient database migrations, the team has built a robust and scalable platform that continues to set new standards in AI-powered development tools.