Mirage 2 – Generative World Engine
Notes on my experience:
- Infra/systems: I was able to connect to a server within a minute or two. Once connected, the displayed RTT (roundtrip time?) was around 70ms but actual control-to-action latency was still around ~600-700ms vs the ~30ms I'd expect from an on-device model or game streaming service.
- Image-conditioning & rendering: The system did a reasonable job animating the initial (landscape photo) image I provided and extending it past the edges. However, the video rendering style drifted back to "constrast-boosted video game" within ~10s. This style drift shows up in their official examples as well (https://x.com/DynamicsLab_AI/status/1958592749378445319).
- Controls: Apart from the latency, control-following was relatively faithful once I started holding down Shift. I didn't notice any camera/character drift or spurious control issues, so I guess they are probably using fairly high-quality control labels.
- Memory: I did a bit of memory testing (basically - swinging view side to side and seeing which details got regenerated) and it looks like the model can retain maybe ~3-5s of visual memory + the prompt (but not the initial image).
I was mildly amused (but not especially surprised) to see that the "Hunter's Vale" initial image includes what's pretty clearly a partial Skyrim HUD compass at the top.
The styles of Cyberpunk 2077 and Red Dead Redemption 2 are also dead givaways about their training data. There might also be a whiff of the Witcher 4 demo in one sequence.
The interesting possibility is that all you may need for the setting of a future AAA game is just a small bit of the environment to nail down the art direction. Then you can dispense with the army of workers to place 3D models on the map in just the right arrangment to create a level. The AI model can extrapolate it all for you.
Clearly the days of fiddly level creation with a million inscrutable options and checkboxes in something like Unreal, or Unity, or Godot editors are numbered. You just say what you want and how you want to tweak it, and all those checkboxes and menus are disposable. As a bonus that's a huge barrier to entry torn down for amateur game makers.
Brightvale's protagonist looks 99% like Link
Starlight Village just has Scar the lion from Lion King right there at the bottom lol
Super fun to try a playable world model for the first time! I picked a random picture and got ChatGPT to write a game description, then could move within that world. Very laggy and buggy, but very fun to try!
Hackers screenshot + file system context = Ideal navigation
while I couldn't test it out (wait time 45min), I'm so insanely jealous. This would make one hell of an Inception style game.
The tech alone of being able to take prefabs and just prompt your way to a world is amazing. Now to get that in blender...
It says 45 min, but you might get lucky. I only waited a couple of minutes before I got to give it a try.
That was terrible. Controls are slow and at times unresponsive. I rode towards a hill that kept getting further away, and tried following a trail that would disappear and reappear. A horse appeared out of nowhere. The terrain detail was hallucinogenic as grass would mutate into different shapes.
Will be really cool when they are ready to handle traffic.
Actually it said 45mins wait but only took a couple of minutes. Very cool, enormous input lag etc but definitely awesome.
Uploaded my own sci-fi art and was amazed by the result
It's fun, but in the initial demo no matter how hard I am trying to make the horse go into the water, it stays on the trail.
Create a cuberpunk animated world
Create a cuberpunk animated world
I picked a Morrowind screenshot of Vivec city and after a few (laggy) frames, it teleported me to a lotr-looking forest and then quickly to a fallout landscape.
Might have potential, but I wasn't terribly impressed by the lack of consistency.
That's a common issue with all video (and audio, but at longer durations, like a few minutes) models. The context is simply too long for current LLMs.
genrate a post punk cyber world
That is impressive. The controls are essentially unresponsive, but the fact that it starts with an image and goes from there bodes well for generative game building.
For those wanting to see it in action, the wait times are wildly inaccurate. Wait five or six minutes and you'll probably get through.