The first game engine for robotics (opens in new tab)

Hey I work @ Lucky Robots.

The differences are pretty big, but the simplest way to illustrate is to try to use gazebo, isaac...etc, and then try to build a whole physically interactive kitchen.

First off, it's gonna take you 3 months to author that thing, if you don't ragequit along the way.

Then, when you go to run, your "50 million steps per second" sim becomes 500 steps per second.

The reason we have robots doing backflips and acrobatics instead of actually useful stuff like picking up your house is making the scenes and getting the data is tough. It requires sensors like cameras and rendering, vs purely proprioceptive-only envs with a flat ground plane and no other physics interactions.

Right now, the industry is doing manual teleop to collect data because it's straight up easier than trying to build these sorts of things in simulators.

This is why we're building Lucky.

nothing

Probably the name is

Vulkan is a rendering technology, Unreal is an engine, that can render using Vulkan under the hood. You can absolutely do this in Unreal. I implemented this at Cruise using UE4 for integration testing, and it worked great for inference (we weren't doing training on sim at that point, but I was pushing for it! There was a paper out in 2018 or so that showed mixing in a bit of simulated data had an outside positive impact on the outputted model). There are companies out there right now doing this with even more modern renderers. I can't comment on how much the rendering realism gap matters here. I think there's some people out there using a variant of lower quality rendering + some kind of diffusion to get "better" images without having to do detailed modeling/lighting for their sims (fuzzy memory, I don't have a source on this).

Unreal will definitely get better results out of the box, but it's also possible to do photorealism with significantly less overhead (particularly UE shader compilation overhead) - useful for single purpose platforms. If you don't need to support lots of specific editor or game features, it may be a valuable investment.

UE is definitely used to obtain simulation data in other domains (this is coming from first hand experience in big tech), but usually through scripting UE handmade levels in python which also needed convoluted server systems at the time (hopefully this has gotten better now).

Their claim is actually: "the world's first game engine purpose-built for robotics."

Idk if that's true or not, but it does exclude all the engines you mentioned.

This is an active area of research (look up papers that mention the "sim2real gap"). It is indeed a problem, but there are known strategies to deal with it.

You add variance to the sim parameters with the hope that the model learns to adapt to changes in environment

Is robotics going to be the next industry to jump on the We'Re sO DaNgErOuS!1!1! marketing bandwagon?