Google's Genie 4 world model adds audio, longer scenes, and real-time editing

Google DeepMind put Genie 4 into research preview on Tuesday, and the headline upgrade over its predecessor isn't subtle: this thing generates sound now. Music, ambient noise, interactive dialogue, all synced to whatever photorealistic world the model is rendering in real time. If you've been tracking world models as a category, that's the piece that's been conspicuously missing for a while.

For context, a world model is a system trained to simulate an environment, predicting how a scene evolves frame by frame based on user input, rather than just generating a fixed video clip from a prompt. Genie 3 launched as a research preview a little over a year ago and could already do photorealistic, playable environments. Genie 4 pushes further on nearly every axis that matters for this category:

• Scene consistency now holds for up to two minutes, double Genie 3's limit, meaning fewer jarring discontinuities as the simulation runs longer
• Free-form mid-simulation edits: change the weather, swap the camera angle, drop in new people or objects, all without restarting the scene
• Better physics and more humanlike character behavior
• Real-time synced audio generation across music, ambience, and dialogue
• Tighter integration with Gemini, which now acts as a planning layer on top of Genie's visual generation

That last point is probably the most structurally interesting change. Gemini isn't just a chat interface bolted onto Genie, it's functioning as a director: planning steps, checking frames, and feeding instructions to Genie in real time. Tejas Kulkarni, research director at Google, described the broader pattern during a briefing: "What we'll increasingly see in the future are these hybrid world models, where you have a combination of Gemini writing code and text, and Genie ingesting that in real time and simulating reality at different levels of abstraction."

That hybrid approach got a concrete test run in a research experiment tied to Gemini Robotics. A Gemini 4 Argon model broke a task down into discrete steps, and Genie 4 simulated each one in real time. It's a small-scale demo, but it points toward a real workflow: testing robot behaviors in simulation before anyone risks hardware in the physical world. That's a meaningfully cheaper and safer iteration loop for robotics teams, assuming the sim-to-real gap holds up under scrutiny.

Where this fits in a crowding field

Genie 4 isn't landing in a vacuum. Runway shipped its second world model, GWM Worlds 2, just a couple of months ago, also playable, also responsive to user input, also with audio generation baked in. The timing isn't a coincidence. Every major lab working on video and simulation has converged on roughly the same thesis: interactive, audio-complete world models are the next big surface area, whether the target market is gaming, robotics, or film.

And the category is consolidating fast. Fei-Fei Li's World Labs is being acquired by AMD for a reported $8.2 billion. Amazon is reportedly in talks to buy Decart for around $7 billion. Rivian just scooped up Third Dimension to bring self-driving simulation capabilities in-house. For a field that barely existed as a distinct product category two years ago, that's a lot of M&A velocity, and it signals that the big players think world models are close enough to commercial utility to justify writing nine-figure checks.

Google's pitch for Genie 4 covers the usual suspects: gaming, interactive education, robotics. One of the more fun speculative examples floated during the briefing involves stepping into a historical painting and chatting with the figures depicted in it, which sounds gimmicky until you think about it as an education tool rather than a toy.

But the use case that trusted testers keep gravitating toward is different: film prototyping. Diego Rivas, a Google DeepMind engineer, framed it in terms of preserving creative momentum. "The real time aspect becomes most valuable in the areas where keeping a state of flow in the creativity space becomes incredibly useful," he said. "For example, in the ideation phase, in the storyboarding phase, anytime you're kind of in need of rapid ideation."

That's a sensible niche. Storyboarding and previsualization are iterative, fast-moving processes where latency kills the creative flow, so a model that lets you reshape a scene on the fly without re-rendering from scratch is a genuine productivity unlock, not just a demo trick.

It's worth noting Google already has Veo for high-definition video generation with synced audio from text, image, or video prompts, a tool that's been gaining traction among indie filmmakers. Rivas suggested that down the line, Gemini Omni, the company's multimodal model that generates video from images, audio, and text, could turn Genie-generated prototypes into finished creative assets. He was careful to flag that those capabilities are still a ways off, though, and the team was upfront that Genie 4's end-to-end product experience isn't settled yet.

Under the Hood

The gaps are real and worth naming. Details and objects can still drift the longer a simulation runs, a known failure mode for autoregressive video generation where small per-frame errors compound over time. The team also hasn't cracked reliable audio accuracy or legible in-world text, two problems that sound minor until you try to build a product around them. Garbled signage or mismatched dialogue timing is the kind of thing that breaks immersion fast, and it's a reminder that "photorealistic" and "production-ready" are still different bars.

Genie 4 is rolling out now as a research preview for trusted testers. In the coming weeks it'll extend to Google AI Ultra subscribers through Project Genie, plus early access partners on the Gemini API and Google AI Studio. Given how fast the competitive and M&A landscape is moving around world models right now, Google doesn't have the luxury of taking its time getting this right.