GameNGen, the first game engine powered entirely by a neural model that enables real-time interaction with a complex environment over long trajectorie

[The axe murderer]

 

[bnew]

​

Explaining GameNGen in Layman's Terms​

What is GameNGen?
GameNGen is a new kind of game engine that uses artificial intelligence (AI) to simulate and generate game environments. Unlike traditional game engines that rely on pre-programmed rules and graphics, GameNGen uses neural models—essentially sophisticated AI algorithms—to create the game world. Key Points:

1. Real-Time Interaction: GameNGen allows players to interact with complex game environments in real-time. This means you can play the game as if it were running on a normal computer, but it's actually being generated by an AI.
2. High Quality: The simulation quality is very high, comparable to what you'd see in a regular game. For example, it can run the classic game DOOM at over 20 frames per second (which is smooth enough for most players) using just one Tensor Processing Unit (TPU), which is a specialized computer chip designed for AI tasks.
3. Next Frame Prediction: The AI predicts what the next frame of the game should look like based on what happened in previous frames and any actions taken by the player. This prediction is so good that it's hard for humans to tell whether they're watching real gameplay or simulated gameplay.
4. Training Process:To achieve this, GameNGen goes through two training phases:
   • Phase 1: An AI agent learns how to play the game by playing it many times and recording these sessions.
   • Phase 2: Another AI model called a diffusion model is trained using these recorded sessions to predict what the next frame should look like based on past frames and actions.

Simplified Example​

Imagine you're watching a video of someone playing DOOM. Now imagine an AI system that can predict what happens next in that video just by looking at previous frames and knowing what actions were taken (like moving left or shooting). This prediction happens so quickly and accurately that it looks almost indistinguishable from real gameplay.

Technical Terms Explained​

• Neural Model: A type of AI algorithm inspired by how human brains work.
• RL-Agent (Reinforcement Learning Agent): An AI that learns by trial and error through rewards or penalties.
• Diffusion Model: An advanced type of neural network used here for generating images based on past data.
• PSNR (Peak Signal-to-Noise Ratio): A measure of image quality; higher values mean better quality. In this case, GameNGen achieves a PSNR of 29.4, which is comparable to lossy JPEG compression—a common method used for compressing images without losing too much detail.
• TPU (Tensor Processing Unit): A specialized computer chip designed specifically for running neural networks efficiently.

Summary​

GameNGen represents a significant advancement in using AI for generating interactive game environments in real-time with high quality. It uses sophisticated neural models to predict future frames based on past actions, allowing for seamless gameplay experiences that are hard to distinguish from actual human-played games.

 

[Gizmo_Duck]

Google DeepMind] We present GameNGen, the first game engine powered entirely by a neural model that enables real-time interaction with a complex environment over long trajectories at high quality. GameNGen can interactively simulate the classic game DOOM

 

[daze23]

this is crazy

AI is basically drawing the game, frame by frame, in real time. that may seem the same as how a computer normally renders a game, but really it's very different

-they have an AI that runs through DOOM, probably millions of times, doing basically everything possible.

-another AI watches the recordings of the first AI playing the game, and learns to predict what the next frame would look like, based on previous actions

-now this AI basically draws the game frame by frame, from what it learned watching the game a lot. the player inputs a control, and the AI predicts what the next frame should look like



I, for one, welcome our new neural model enabled overlords

 

[The axe murderer]

this is crazy

AI is basically drawing the game, frame by frame, in real time. that may seem the same as how a computer normally renders a game, but really it's very different

-they have an AI that runs through DOOM, probably millions of times, doing basically everything possible.

-another AI watches the recordings of the first AI playing the game, and learns to predict what the next frame would look like, based on previous actions

-now this AI basically draws the game frame by frame, from what it learned watching the game a lot. the player inputs a control, and the AI predicts what the next frame should look like



I, for one, welcome our new neural model enabled overlords

Yeah I'm skeptical of all the promises that tech bro grifters make with tech AI and all. But this here is legit impressive anyway you look at it

 

[Diunx]

So what exactly can be done with this? Gameplay videos? Or is the plant to feed it so much game data that when you ask it to shyt out an RPG it will do so?

 

[ORDER_66]

I knew this day was coming where NPC's themselves become self aware and the games are gonna start getting harder...

GTA 7 gonna be fukking lit...

 

[JNew]

At some point the games will become broken if the difficulty becomes infinite.

First thing that comes to mind is spawning enemies out of nowhere or do things game developers never intended.

 

[TDT]

​

Explaining GameNGen in Layman's Terms​

What is GameNGen?
GameNGen is a new kind of game engine that uses artificial intelligence (AI) to simulate and generate game environments. Unlike traditional game engines that rely on pre-programmed rules and graphics, GameNGen uses neural models—essentially sophisticated AI algorithms—to create the game world. Key Points:

1. Real-Time Interaction: GameNGen allows players to interact with complex game environments in real-time. This means you can play the game as if it were running on a normal computer, but it's actually being generated by an AI.
2. High Quality: The simulation quality is very high, comparable to what you'd see in a regular game. For example, it can run the classic game DOOM at over 20 frames per second (which is smooth enough for most players) using just one Tensor Processing Unit (TPU), which is a specialized computer chip designed for AI tasks.
3. Next Frame Prediction: The AI predicts what the next frame of the game should look like based on what happened in previous frames and any actions taken by the player. This prediction is so good that it's hard for humans to tell whether they're watching real gameplay or simulated gameplay.
4. Training Process:To achieve this, GameNGen goes through two training phases:
   • Phase 1: An AI agent learns how to play the game by playing it many times and recording these sessions.
   • Phase 2: Another AI model called a diffusion model is trained using these recorded sessions to predict what the next frame should look like based on past frames and actions.

Simplified Example​

Imagine you're watching a video of someone playing DOOM. Now imagine an AI system that can predict what happens next in that video just by looking at previous frames and knowing what actions were taken (like moving left or shooting). This prediction happens so quickly and accurately that it looks almost indistinguishable from real gameplay.

Technical Terms Explained​

• Neural Model: A type of AI algorithm inspired by how human brains work.
• RL-Agent (Reinforcement Learning Agent): An AI that learns by trial and error through rewards or penalties.
• Diffusion Model: An advanced type of neural network used here for generating images based on past data.
• PSNR (Peak Signal-to-Noise Ratio): A measure of image quality; higher values mean better quality. In this case, GameNGen achieves a PSNR of 29.4, which is comparable to lossy JPEG compression—a common method used for compressing images without losing too much detail.
• TPU (Tensor Processing Unit): A specialized computer chip designed specifically for running neural networks efficiently.

Summary​

GameNGen represents a significant advancement in using AI for generating interactive game environments in real-time with high quality. It uses sophisticated neural models to predict future frames based on past actions, allowing for seamless gameplay experiences that are hard to distinguish from actual human-played games.

Wonder if it cam simulate the latest DOOM game..

 

[Artificial Intelligence]

Show me something like gta 6 and I’ll be actually impressed



This is cool
Tho
I’m
Trollin

 

[bnew]

this is crazy

AI is basically drawing the game, frame by frame, in real time. that may seem the same as how a computer normally renders a game, but really it's very different

-they have an AI that runs through DOOM, probably millions of times, doing basically everything possible.

-another AI watches the recordings of the first AI playing the game, and learns to predict what the next frame would look like, based on previous actions

-now this AI basically draws the game frame by frame, from what it learned watching the game a lot. the player inputs a control, and the AI predicts what the next frame should look like



I, for one, welcome our new neural model enabled overlords

this is how you get games truly rated E for everyone when you can prompt it at the beginning to exclude certain images like blood, foul language or anything that can be triggering or not approved by a parent.

 

[bnew]

 

[Mowgli]

Need a neural final fantasy engine so I can play infinite epic final fantasies with 400 special attacks. 800 epic bosses, 200 summons and 300 magic spells each game

 

[bnew]

Spoiler: thread continued

Spoiler: full text & large images

1/14
@AngryTomtweets
Gaming will never be the same.

This is Mirage, a real-time interactive world model for video games!

Build your gameplay in real-time using natural language, a keyboard, or a controller.

More examples below + link to try:



https://video.twimg.com/amplify_video/1940542746890326016/vid/avc1/720x720/uJqkh9A8MTQKyCR6.mp4

2/14
@AngryTomtweets
The future of gaming isn't built level by level by expert designers, it's imagined, generated, and played by anyone, in real time.



https://video.twimg.com/amplify_video/1940542810962513920/vid/avc1/1280x720/ffixX0QRktIWaRVZ.mp4

3/14
@AngryTomtweets
User-Generated Content (UGC)

Players don’t just replay a pre-fit game, they can edit, extend, and create entirely new worlds.



https://video.twimg.com/amplify_video/1940542878113320960/vid/avc1/1280x720/quEt30AgO_yCN9sQ.mp4

4/14
@AngryTomtweets
Rich Visuals & Controls

Supports complex, immersive experiences in the style of GTA or Forza, beyond the blocky simplicity of Minecraft or DOOM.



https://video.twimg.com/amplify_video/1940542942462271488/vid/avc1/1280x720/F1iugsbeOL29QVKF.mp4

5/14
@AngryTomtweets
Long-Form Gameplay

Enables sustained play sessions of 5–10 minutes or more, rather than just short bursts of a few seconds.



https://video.twimg.com/amplify_video/1940543007918641154/vid/avc1/854x480/PUSJ0FtNz3nv16U5.mp4

6/14
@AngryTomtweets
You can try two real-time demos!

GTA-style urban chaos. Link: Mirage Urban Chaos Demo
Forza Horizon-style coastal drift. Link: Mirage Coastal Drift Demo



7/14
@AngryTomtweets
GTA-style urban chaos.



https://video.twimg.com/amplify_video/1940543083940401155/vid/avc1/1280x720/-6t7KyqC6zR6LZyE.mp4

8/14
@AngryTomtweets
Forza Horizon-style coastal drift.



https://video.twimg.com/amplify_video/1940543148209643520/vid/avc1/1280x720/AsR39B0H0k4SNbMU.mp4

9/14
@AngryTomtweets
More here: Mirage: AI UGC game engine



10/14
@AngryTomtweets
That's a wrap!

If you enjoyed this thread:

1. Follow me @AngryTomtweets for more of these
2. RT the tweet below to share this thread with your audience

[Quoted tweet]
Gaming will never be the same.

This is Mirage, a real-time interactive world model for video games!

Build your gameplay in real-time using natural language, a keyboard, or a controller.

More examples below + link to try:


https://video.twimg.com/amplify_video/1940542746890326016/vid/avc1/720x720/uJqkh9A8MTQKyCR6.mp4

11/14
@codewithimanshu
Perhaps the real game changer is how we interact, not just what we play.



12/14
@heygurisingh
Wow, this sounds like a game changer! Can’t wait to see how my keyboard and I become best buddies in this new world!



13/14
@t0biasFlow
Curious to see how natural language boosts game creation complexity and player agency.



14/14
@broskiplays_
I have to try this, this allows for unlimited content




To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

 

[Barney Rubble]

This is some wild shyt

https://blog.dynamicslab.ai/assets/videos/mp4/720p/Mirage-multiple.mp4