General Intuition’s $2.3B bet that video games can train AI agents for the real world

Upon entering the R&D floor of General Intuition’s New York office, the company’s 31-year-old co-founder and CEO, Pim de Witte, pointed to a monitor on a standing desk. It seemed like someone was playing Fortnite, but it wasn’t a person.

“Our agent has been playing for 100 hours straight,” said Kent Rollins, the company’s chief product officer, clearly proud.

Before I could fully take in the sight of an AI navigating the game’s virtual world, I heard the electronic footsteps of a large quadrupedal robot making its way toward us.

“The same brain powering the agent playing the game is powering the robot,” de Witte explained.

Josh Duplantis, a data analyst with a laptop streaming a live feed from the robot’s single camera, explained that the bot’s default mode was “exploration.”

Using its camera as a single eye, the giant, bug-like robot approached me, circled around, and continued through the office. It occasionally bumped into chairs or a stray trash bin, resembling a toddler learning to navigate its surroundings. Duplantis noted that only eight minutes of real-world robotics data were needed to fine-tune an AI model for the quadruped. Interestingly, this data was gathered on the street, not inside the office where the bot was currently moving.

The ability to generalize from gameplay to simulation to physical embodiment is central to General Intuition’s mission. This model’s capability to understand its environment has attracted significant backing.

Recently, General Intuition announced it raised $320 million at a $2.3 billion valuation, confirming earlier reports by JS. This round increases the startup’s total disclosed funding to $454 million, following a $134 million round when it launched last October.

General Intuition emerged from de Witte’s other company, Medal, which enables gamers to upload and share video game clips. The vast hours of gameplay uploaded provided the initial dataset for training General Intuition’s model in spatial-temporal reasoning, understanding movement through space and time.

However, the crucial element wasn’t the gameplay footage itself, but the action labels embedded in those clips: precise records of what buttons a player pressed and when. De Witte argues that many competitors try to infer actions solely from video, which he believes is inadequate.

“We view this as just the next stage of future pre-training,” de Witte said. “We have a single model that can respond to Fortnite information on the screen and take action, but also to real-world dynamics in ways an LLM could never achieve.”

De Witte demonstrated General Intuition’s world model on a laptop, a simulated environment generated frame-by-frame rather than through a traditional game engine. Unlike other demos where agents might pass through walls, this model recognized walls, knew ladders were for climbing, and understood shadows lengthen as the sun moves.

For General Intuition, this world model is a training environment, internally called “the gym.” The company aims to sell the agentic model itself, arguing that action data from gameplay helps the model distinguish the “self” from the “environment,” enriching its understanding of causality.

Despite impressive demonstrations, the challenge of making such a model viable in the physical world on a large scale remains unresolved. Most similar approaches need vast amounts of real-world data, gathered slowly and at considerable cost. General Intuition believes gameplay offers a scalable shortcut.

Investors seem confident in this approach. The latest funding round, led by Khosla Ventures, included participation from General Catalyst, Jeff Bezos, Eric Schmidt, Nico Rosberg, and researchers from Google DeepMind and MIT.

Most of the new funds will enhance computing capacity. General Intuition has partnered with CoreWeave and plans to focus on pre-training the next version of its model. Part of the funds will also help make its API more widely available by the end of summer.

Vinod Khosla, whose firm led the investment round, was drawn to de Witte’s vision and the company’s proprietary data advantage.

“If you look at LLMs, when reasoning emerged, it was a quantum leap,” Khosla remarked during a phone interview. “In world models, I think the quantum leap is the emergence of intuition in AI, a human intuition-like capability. The human action data and reaction data in games is crucial to this emergence of intuition.”

The vision is a generational company

General Intuition is not alone in recognizing the value of Medal’s human action data for constructing dynamic world models and general agents. Brianna Martin, the startup’s chief of staff, mentioned that the company was partly founded after Medal declined an acquisition offer from a major lab. There have been additional offers since then.

De Witte and his co-founders, Eloi Alonso, Adam Jelley, and Vincent Micheli, have no interest in being acquired, nor are the startup’s investors seeking an exit yet. The proprietary data General Intuition holds through Medal is a key reason Khosla believes the startup represents a generational opportunity, rather than just an acquisition target. This data could serve as the backbone for generalized agents and world models, both in simulations and the real world.

“At this point, it would be a data acquisition, which is sort of uninteresting,” Khosla commented.

Part of this investment also involves faith in de Witte’s values.

Having spent three years in the humanitarian sector, including with Doctors Without Borders, de Witte has established a clear boundary on the use of General Intuition’s technology: it will not be used to harm humans.

“We don’t want to be an escalatory part of the system,” de Witte stated. “If I were to announce, ‘We’re doing lethal autonomy,’ what do you think would happen in other countries?”

This restriction on military applications comes as Silicon Valley increasingly focuses on military tech, though de Witte supports using his models for search and rescue missions.

Being Dutch, with a largely European team, influences the company’s identity. De Witte hired Martin partly due to her decision to publicly leave Palantir over its work with the United States Immigration and Customs Enforcement.

“I don’t know why Silicon Valley does what it does,” he remarked. “There’s a reason I’m not there.”

De Witte’s ethics extend beyond what the models won’t do. A gamer who made $1.5 million by running a private RuneScape server as a teenager, de Witte is also concerned about those who might be left behind by advancements in AI.

General Intuition recently launched Nerve, a job marketplace where gamers can earn money using their current setups. Participants begin with data labeling and can progress to tasks like robot teleoperation. De Witte noted that Medal’s user base is precisely the generation most vulnerable to AI-driven job displacement, and he wants them to be part of the future.

A data flywheel

De Witte envisions General Intuition as an ecosystem enabler, akin to Anthropic or OpenAI, providing a model that others can build upon. Currently, the startup has a few customers in gaming, simulation, and robotics.

“We’re not gonna build a self-driving car company,” de Witte said. “We’re gonna make it 10 times easier for the next person to build a self-driving car company.”

Once its API reaches more customers, General Intuition aims to test its capabilities across various applications — from testing a robot on a digital twin of a factory floor, powering a human-like bot in a gaming studio, to sending a quadruped into hazardous environments.

While a quadruped is the first physical form General Intuition has tested in the real world, they have also experimented with drones and other devices, even testing the model in driving games.

“It works on anything that you can control using a game controller or a keyboard mouse,” de Witte mentioned.

Building a data flywheel is among the goals.

“We’ll choose customers where we can diversify the embodiments that this generalized foundation model supports,” de Witte explained. “So we prioritize customers who can provide real-world data that is interesting and useful to advance research, and who have agile internal teams so we can be embedded partners and learn from each other.”

Khosla emphasized that General Intuition’s proprietary data has been crucial to its success so far, and continuing to collect unique data will be vital. Despite impressive demos, whether the simulation-to-real-world transfer can be achieved at scale remains an open question.

Correction: The headline previously misstated how much General Intuition raised in this round. The error has been fixed.