undefined | Better HN

0 pointsforrestthewoods4y ago0 comments

Pretty much no video game engine scales with cores. There are too many dependent systems. There's no game engine that I'm aware of that effectively uses more than a couple of cores for gameplay simulation.

Distributed physics is a well understood problem with no "solution". Just different trade-offs. This solution is cool, but it's not novel or anything. Minecraft is particularly challenging because the entire world is highly mutable.

There are no general solutions to any of this. Just a bunch of custom one-offs. SpatialOS is trying. My opinion on it is extremely, extremely negative. And I'll leave it at that.

Unity and Unreal both have primarily single-threaded gameplay. Writing multi-threaded gameplay code is extraordinarily difficult. Unity has been working on their DOTS/ECS system for years, but it does not appear to be close to ready for the mainstream.

So I think the short answer is "it's really really hard. Like, radically harder than you are imagining. Even if you think it's hard. The value of that work is likely not worth the cost. If most players WANT to play on small servers with close friends then the mountain range of work to effectively support a 1000 player server is not worth it".

0 comments

10 comments · 4 top-level

8note4y ago· 3 in thread

Couldn't Minecraft chunks run fairly independently, with some message passing when something needs to move between chunks?

kevingadd4y ago

Yes*, but players build complex machinery fairly often that can span chunk boundaries, so that requires synchronous coordination between the chunks for those machines to work right.

devoutsalsa4y ago

When will players learn to parallelize their simulated DOGE coin miners?!

Kiro4y ago

Imagine two players fighting on a seam between chunks.

Jweb_Guru4y ago· 2 in thread

Video games can scale just fine with many cores, including MMOs and games like Minecraft. Veloren, for example, uses multiple cores everywhere for just about everything--most of the gameplay loop, physics, chunk meshing, world generation, realtime simulation, rendering (can be parallelized a lot further now that we've switched to wgpu), networking, and background tasks like snapshotting persistence--and it is certainly not hard to find places where parallelization yields real speedup (quite the opposite--every time we further parallelize physics it yields significant performance improvements!). There are many hard problems in video game performance, but "games just can't make good use of multiple cores" is not one of them.

If anything, I often wish we could rely on many more cores being available than actually are! It is certainly far far easier to see real performance wins with multicore than by using multiple servers, which introduce very heavy coordination costs.

forrestthewoodsOP4y ago

> There are many hard problems in video game performance, but "games just can't make good use of multiple cores" is not one of them.

That's not quite what I said.

I'm happy you're using specs to work on an open source game. Specs and bevy and all the ECS work being done is super exciting and fun. I <3 Rust.

In the meantime there are no major games that effectively scale to, let's say, 64 cores. I don't know of anything shipped that can saturate a 12-core/24-thread Ryzen. ECS alone will not get us to that level of scaling.

Yes it's trivial to to throw audio, networking, and a few other subsystems onto separate threads. Modern games definitely leverage 4 cores. Although several of those cores will be severely under utilized.

Modern ECS designs are rapidly evolving and rapidly improving our ability to better leverage multiple cores. But we're not yet to a point where games can easily and efficiently saturate 10+ cores.

Personally I'd love to see a game like Eve Online that can effectively simulate a universe with tens of thousands of players either spread across the universe or all in one places during one giant battle.

> It is certainly far far easier to see real performance wins with multicore than by using multiple servers, which introduce very heavy coordination costs.

This is extremely true.

Jweb_Guru4y ago

If you're just using ECS to parallelize disjoint subsystems then no, it won't get you there. But if you're using it (as Veloren is, and as I hope more people do) in a more deliberate way, to further parallelize within a system, you can indeed scale quite well to large numbers of cores. I've done some theoretical bottleneck calculations and we will still have tons of work to do with 64 cores available, if the game is written properly. We can already get decent utilization out of 32 threads at busy times, and the server was not really close to peak load in terms of player count (and we are very far from done optimizing): https://media.discordapp.net/attachments/539518074106413056/....

That's just for our server. Our clients can make use of cores in even more ways, although they have less work to do and generally have fewer cores available, and you can see Veloren taking advantage of 16 client threads with similar utilization here: https://twitter.com/sahajsarup/status/1431837669391142916.

The most important thing I want to note is that in both cases, you are not seeing tremendous imbalance between the cores most of the time. While there are definitely single-threaded bottlenecks in games, you have to be working pretty hard before they start bottlenecking the workload! Instead, we are just suffering from a combination of general inefficiency and lack of work to do.

So no, I'm going to push back against this notion that multicore scaling for games is some sort of crazy intractable problem. It's not. Like any other kind of parallel scaling, it's trivial in some places, more challenging in others, and depends a lot on your workload (including you actually having enough work to saturate the cores in the first place!). But there's nothing special about games here.

1 more reply

Animats4y ago· 1 in thread

There are no general solutions to any of this. Just a bunch of custom one-offs. SpatialOS is trying. My opinion on it is extremely, extremely negative. And I'll leave it at that.

I'd like to hear more about that. nagle at animats.com, if you don't want to say much in public.

This ties into the "metaverse" business. Lots of metaverse talk about big seamless worlds full of user-created content, but not much is running. So far, nothing with user created content really scales. There are lots of little shared worlds, like Breakroom/Sinespace, Facebook Horizon, IMVU, etc. There are big-space voxel worlds, such as Dual Universe and Roblox. There are general purpose region-oriented big "seamless" worlds such as Second Life, which have trouble at the seams and can't handle crowds in one place, the same problem these Minecraft improvers hit in round 1.

Spatial OS was going to fix all this. Their system is basically objects which can be accessed remotely and which migrate to where the most accesses are coming from. It cost over $100 million to develop, they had to do the hosting, and the first four games all went broke due to the high cost of hosting. So, since they had too much venture capital, they set up an in-house game studio and created Scavengers, which is reportedly a so-so shooter.

Roblox has plans to solve this, somehow, by sheer money power. When you have a few billion dollars to spare, that might work.

If we're headed for the "metaverse", this has to be cracked. Somehow.

(I've been writing a multi-threaded Rust/Vulkan client for Second Life / Open Simulator so I'm painfully aware of these problems.)

evgen4y ago

Forgive the late follow-up, but my first startup in the valley did this back in the mid-90s (Electric Communities, everal EC founders and key hires are mentioned in the Snow Crash acknowledgements) and it is amusing to see people continue to make the same mistakes and step on the same landmines every few years. EC created an entire language (E) to try to hide the complexity and remain secure, had a PERT chart where an actual color on the chart (purple I think) was used to signify that this task would be a good PhD thesis for someone, and really only managed to get a feel for just how large this elephant was before eventually pivoting in a direction that had the possibility to lead to profit in the pre-dotCom era. At best, I think they managed to stumble in the right direction for close to a decade by collecting a large group of really smart people and somehow keeping them happy and engaged for several years and through the ability of the president of the company to get Hollywood people to keep throwing money at them year after year. If you want an interesting read for the evening you should find a copy of Chip Morningstar's 'Cyberspace Protocols Requirements' paper; it holds up incredibly well for something written almost 25 years ago when most people were still on dial-up and we thought 64MB of RAM and a Pentium Pro was too much to require on the client side...

imtringued4y ago

I agree it's hard. It's only worth doing if your business depends on it.

j / k navigate · click thread line to collapse