Google DeepMind released its latest world model, Genie 4, in a research preview on Tuesday. The new system generates photorealistic and interactive environments, allowing users to modify scenes while the simulation is running. It also produces real-time audio that aligns with the visual content, including ambient sounds, music, and dialogue.
Real-Time Editing and Audio Integration
Genie 4 represents a significant step forward from its predecessor, Genie 3, which launched as a research preview slightly over a year ago. The new model maintains consistent scenes for up to two minutes, doubling the previous limit. It also features improved physics and more human-like character movements.
Users can now issue freeform edits during a simulation. This includes changing weather conditions, adjusting camera angles, or adding new objects and people to the scene. The model integrates closely with Google’s Gemini AI, which acts as a director to plan steps and verify frames while Genie 4 handles the visual generation.
In a related experiment involving Gemini Robotics, a Gemini 4 Argon model broke down tasks into steps that Genie 4 simulated in real time. This setup offers a potential method for testing robot scenarios in a virtual environment before deploying them in the physical world.
Tejas Kulkarni, research director at Google, described the future direction of these tools during a briefing. He noted that hybrid world models will likely combine Gemini writing code and text with Genie ingesting that data to simulate reality at various levels of abstraction.
Commercial Applications and Current Limitations
The technology targets several commercial sectors, including gaming, interactive learning, and robotics. One speculative application for education involves users entering a digital recreation of a historical painting and conversing with the subjects depicted in the artwork.
Early access users, referred to as trusted testers, have identified potential uses in film prototyping. Diego Rivas, a Google DeepMind engineer, highlighted the value of real-time interaction for creative workflows. He explained that the ability to maintain a state of flow is particularly useful during ideation and storyboarding phases.
Google also offers Veo, a separate AI video generation model that creates high-definition clips with synchronized audio. While Veo is gaining popularity among filmmakers, Rivas suggested that future versions of Gemini Omni could eventually turn prototypes into full creative assets. He noted that those capabilities remain distant.
Despite the advancements, the team acknowledged several gaps in the current version. Details and objects can drift as simulations run longer. The team has not yet solved issues regarding audio accuracy or ensuring that text within the world remains legible. Rivas emphasized that the end-to-end product experience is still being defined.
Genie 4 is currently available to trusted testers. The model will roll out to Google AI Ultra subscribers in Project Genie in the coming weeks. Early access partners of the Gemini API and Google AI Studio will also receive access. This release follows similar moves by competitors, including Runway’s GWM Worlds 2 and acquisitions by AMD and Amazon in the simulation space.
Source: TechCrunch
