HN Theater @HNTheaterMonth

This Rust conference talk, of all things, is still my favorite ECS explanation, because she builds the concept piece by piece starting from megaclass-hell to full entities-as-IDs with reasons for each step. Also she's a pretty entertaining speaker IMO. https://youtu.be/aKLntZcp27M

⬐ dkbrk

She also wrote a long-form essay which covers the same material but goes into much more detail: https://kyren.github.io/2018/09/14/rustconf-talk.html

⬐ fartsucker69

This is also one of my favorite talks, because it showed me that Rust is mostly recommended by people who don't know what they're talking about made me not waste any time on it (besides the horrendous compile times in real projects even compared to cpp).

For example, she spends a significant amount of time there explaining how to implement her component placement strategy while working around the borrow checker and at the end proclaiming how great it now is compared to a cpp implementation...

...not realizing that what she's just done is implement a custom allocator (not literally allocating memory, but the operations are the same: give me space for a component, free it, memory is latter accessed by an address instead of being some opaque thing hidden behind language semantics, etc) that can fail in most of the typical ways and that she's gained nothing over the cpp implementation while wasting effort to force Rust into allowing her to do something really basic.

⬐ spoiler

Wow... Okay. I see you didn't even bother watching the video in full.

You somehow managed to ignore all the points of the talk. Like literally every one of them, and I'm not even exaggerating. And then you somehow both managed to focused on irrelevant details and to talk out of your ass (like, at least be right about the details?), all to make a tangential point about how "sUpErIoRlY sMaRt CpP dEvs ArE" because they know this really basic shit.

But, I'll respond in case your misinformed comment manages to deter people from watching it. Let's start.

First you talk about "borrow checker issues" she's demonstrating in the OOO design. The point here was to illustrate the amount of coupling and "explosion" in size with this approach. The point was "OOO is probably not a good design paradigm for game dev". Then she continues how Rust surfaces this bad design paradigm much earlier than it would've happened in other languages/ecosystems. That's all.

More, what you call a custom allocator with tombstoning/generations are just things that exist with a lot of ECS implementations (regardless of language), and it's just become "tribal wisdom" at this point that this table-esque storage backend work really well with ECS (I dunno which one came first; older game devs might know more about ECS history/evolution than I do).[1]

Also not sure what you mean by "that can fail in most of the typical ways and that she's gained nothing over the cpp implementation while wasting effort to force Rust into allowing her to do something really basic"... Maybe I misunderstand what you mean, but it sounds to me like you're comparing working with `Option` as being the same failure mode as working with `void*`. That's just utterly ludicrous nonsense.

Her ECS implementation is... Well, it was made to fit inside a few slides for a 40-minute talk. So, yeah it's not great; there's plenty of small issues with it, but none of them are with Rust, or with ECS itself... Did you expect a production ready ECS library you can copy off of slides and use them in your next game, lol? If anything, her implementation has too much passing resemblance with what a C++ implementation would look like (which is something she's very familiar with and her starting point) than a Rust one. So, I'm just bewildered why this is the bone you decided to pick.

If you're interested in what a production-ish ECS implementation in Rust looks like, check out Amethyst's Legion or Bevy's ECS. Although, Bevy takes takes ECS a few... staircases (rather than steps) further with its own ECS. I really encourage everyone to read up on Bevy's ECS and check out the unofficial Bevy cheatsheet book; I'm certain at leat UE game devs will know how to appreciate how beautiful and ergonomic the design is, but others should as well). Aside: her ECS implementation is still safer than most C/C++ implementations used in published games, but that's a bit besides the point since that's the borrow checker doing its job.

Anyway, the irony is that she much better explains the pitfalls of her own implementation details that you do and she also explains why they're "fine" sometimes. And it was kinda implied that it's fine (from what I remember) because it's no worse than what you end up doing in C++ or Java when implementing ECS, but at least it's safe in Rust. She acknowledges that there's probably better ways to do it.

Here's a few links on some of the stuff I mentioned:

- https://github.com/amethyst/legion

- https://bevyengine.org/learn/book/getting-started/ecs/

Edit: formatting

[1] Edit 2: I kinda phrased this poorly. I'm not saying the best way to implement tables or just ECS is this approach, but it's common enough approach used to avoid memory bandwidth/allocations in games.

⬐ andrewflnr

Even assuming you're right about the content of this talk,

> showed me that Rust is mostly recommended by people who don't know what they're talking about

You do realize that this is, like, colossally, stereotypically bad reasoning, right? It's ad hominem and improper generalization simultaneously, which is really quite impressive and indicates a singular degree of motivated reasoning.

Yes, lots of Rust evangelists are idiots (not Catherine West, u/spoiler did a decent job showing that). No, that doesn't significantly affect the technical properties of the language that make it useful or worth learning about. There are good reasons to ignore Rust, yours is not one of them.

It is a very different and seemingly weird approach. It's almost like writing SQL queries for your game world (e.g. "find all entities with a Flying component and attach them a WooshSound component). A clever memory layout makes these queries super fast.

It makes separation of concerns very natural. Different parts/layers of the game, like inputs, AI, sounds, graphics, effects and animations can be coded separately. You don't end up with "god class" for player's object that does a bit of everything. You have player entity that has components attached, and each system operates on its own set of components.

I can't really do it justice in a comment. It's a quite different approach, so it's not directly comparable. It's like OOP vs Rust's traits. ECS happens to match Rust's memory model and traits incredibly well.

Here's a more in-depth talk about this: https://www.youtube.com/watch?v=aKLntZcp27M

⬐ wdroz

Even the Data-oriented design[0] is eyes opening (which is part of the ECS).

The classic example is[1]: Instead of having a list of structs, where each struct get some types (int, float, ...), you use one struct, where the fields are list of types (array of int, array of floats, ...)

[0] -- https://en.wikipedia.org/wiki/Data-oriented_design [1] -- https://en.wikipedia.org/wiki/AoS_and_SoA

⬐ kramerger

Unity uses ECS, just in a more visual way.

⬐ wdroz

I did some Unity3d back in 2013, I don't remember using ECS (intentionally?). After a quick search, it's seem that the MonoBehaviour are the old way and ECS the new ways? That's nice.

⬐ kramerger

When you drag and drop something into an object, you are actually adding a component to an entity.

It's kinda behind the scene ECS.

This is basically what an Entity Component System is, right? I always find myself linking back to this talk: https://youtu.be/aKLntZcp27M

I just recently came across these two videos.

Using Rust For Game Development by Catherine West: https://www.youtube.com/watch?v=aKLntZcp27M

Counter-rant from Jonathan Blow: https://www.youtube.com/watch?v=4t1K66dMhWk

⬐ badsectoracula

I'm not sure if it was that first video or i saw/read about this approach somewhere else (i'm almost certain that i heard about it years before that video though), but my reaction was pretty much the same as Jon Blow's (though not so long winded :-P): aren't you just working around the borrow checker and making your own allocator with its own faux pointers via indices? Sure, it wont crash the game if your index is invalid, but you'll still access the wrong data - which can end up with hard or weird bugs and corrupted state (e.g. in savegames and/or editor).

⬐ andrewflnr

Yeah, that's pretty much working as intended. The first priority of Rust's safety is crashes and security. Any other bugs that get squashed are a bonus, and you can't expect it to get all of them (whatever any overly enthusiastic evangelists might say).

⬐ adwn

> Counter-rant from Jonathan Blow

I've just watched the first 20 minutes of that video. So far, it's all "yeah, she's basically right". When does he get to the point? I.e., what does he actually object to?

⬐ indy

An issue that Jonathan Blow had was that one of the touted benefits of Rust is it's ownership semantics, yet the ECS (Entity Component System) that the talk demonstrated was effectively bypassing that.

⬐ adwn

> ownership semantics, yet the ECS (Entity Component System) that the talk demonstrated was effectively bypassing that.

I don't see how that's the case [1]. The entities are owned by the ECS (or the game state, or whatever), not by each other. Entities can conceptually reference each other, but not own each other. The only time this is problematic, is when one entity is destroyed while another one is still holding a reference to it. At the beginning of the video, he discussed basically all approaches to solve that problem:

1) Raw pointers. Bad for obvious reasons.

2) Smart pointers which keep the referenced object alive. No good, he says, because one entity should not keep another one alive, if the game logic says it should be removed.

3) Weak pointers which are safely invalidated when the referenced entity is removed. No good, he says, because keeping track of back-references is inefficient.

4) Weak pointers which check whether the referenced entity is still alive before accessing it. Which is exactly what the ECS does. Rust's ownership semantics still help here, because entities are unambiguously owned by the ECS, which returns a type-safe None value when you try to access a deleted entity.

[1] I'm not arguing against you, indy; I understand that you paraphrased Blow's argument.

⬐ meheleventyone

Yeah the confusion comes because entities are no longer a concrete element of the program but a concept tying components together. It’s just as valid to look at a subset of the components that are referenced by an entity id as a view as it is to look at them all. Like a relational database.

And by breaking things into components the granularity of ownership is increased compared with the equivalent concrete representation of the same data. So whilst you can’t reason about ownership of an entity as it primarily exists conceptually the ownership of its constituent parts is well defined.

⬐ gameswithgo

tldnr is that Rust guides the programmer only halfway to the solid engineering solution, the programmer is on their own to make the arena solution safe.

its a pedantic nitpick, as is his way. also keep in mind that of late Blows focus has been single player video games made by very small very talented teams. A domain where the benefits of Rust are very small, possibly even a net loss compared to the fast compiling language they use now.

⬐ pmarin

Keep watching.

⬐ neutronicus

Her talk is basically "Introduction to Entity Component Systems for Rust Programmers," and it has an implicit secondary thesis that implementing the ECS in Rust adds value vs implementing it in e.g. C++.

He seems to disagree with that secondary thesis. He thinks she did a good job implementing a toy ECS, but that Rust itself wasn't particularly helpful to her.

⬐ beigeoak

Idk who that guy is in the counter-rant portion, but if he can make a 1-hour long video against a language, why doesn't he make his own language instead or write his own game engine in his own language or something like that? I highly doubt he would get sub 1-second compile times, like he claims at 1:03:52.

Talk is cheap, SAD!

⬐ Thaxll

The sutdio Catherine West worked for stopped using Rust and went back to C++ fyi.

https://www.reddit.com/r/programming/comments/atyzz4/halley_...

⬐ ImprobableTruth

... because Catherine West left for personal reasons and so it was easier to shift the project over to their existing and already working C++ engine (that they were continuously developing and using, so they didn't really 'go back').

Chucklefish, the developer of Starbound, uses Rust too. The lead dev on the project gave a great presentation [1] at RustConf.

[1] https://youtu.be/aKLntZcp27M

⬐ steveklabnik

(They did but have since stopped.)

⬐ Etzos

Do you have any articles or any info on why they decided to stop using Rust? I remember seeing things about them using Rust but never saw anything about them not using it and I'm curious.

⬐ steveklabnik

They made a few offhand comments on Reddit, but that's it.

They decided that having a unified tech stack for the games themselves was worth more than Rust's advantages. Basically, they decided to re-build Witchbrook on top of Wargroove's engine.

That being said, I guess I should have put an asterisk on there; they aren't using Rust in game anymore, but they are still (apparently, last we heard) using Rust for server-side components. Wargroove's servers are implemented in Rust.

⬐ Impossible

I'm aware of Chucklefish's usage of Rust, but they aren't a good AAA example even if they've shipped some of the most successful indie titles of all time. Also, as mentioned in a sibling comment they have abandoned Rust because the lead programmer that was driving it moved on.

But do they use linked lists for them? I would have assumed you would use an arena for something like that, precisely because you already need to iterate over all of them and can represent any graph as keys into a generational index, so that reading stale keys is trivially handled.

Again, I'm not a game dev and this is just my understanding after reading up on these topics after watching https://youtu.be/aKLntZcp27M.

⬐ chongli

They certainly used linked lists all over the place in StarCraft, WarCraft, and Diablo [1]. I don't know that many game developers would use complicated data structures like the one you've described here. Linked lists are fantastic for insertion/removal in arbitrary places.

Game development is not really about showing off with cutting-edge CS research, it's about getting things done. Maybe your arenas with generational indices would be better, but they could take a long time to figure out and lead to a big mess that doesn't go anywhere.

[1] https://www.codeofhonor.com/blog/tough-times-on-the-road-to-...

⬐ jfkebwjsbx

You are wrong in several fronts:

+ Linked lists are a thing of the past.

+ Game engines use complex data structures and algorithms. Some are cutting-edge research implemented from papers, specially in graphics.

+ A generational index is not complex.

Yes, game development (as opposed to game engine development or video/audio rendering) is a mess. That does not mean the actual technical fields involved are simple.

⬐ renox

While I don't disagree on your first two points, I disagree with you third: there's nothing complex about generational index.

⬐ Narishma

Those are pretty old games written when caches were small and the CPU-memory speed gap wasn't as large as it is today.

⬐ ssalazar

Arenas aren't "cutting edge CS research," theyre used widely in latency-sensitive applications like gaming. This article [1] describes how to implement one for a game engine. Basic versions are easy to understand and provide good results.

Starcraft/etc. are 20+ year old games and software development patterns as well as gaming hardware have changed in fundamental ways since then. Conventional wisdom nowadays is that cache misses on every list iteration are a lot worse than shuffling a few contiguous bytes around or marking things inactive when removing items.

[1] https://www.gamasutra.com/blogs/MichaelKissner/20151104/2582...

"RustConf 2018 - Closing Keynote - Using Rust For Game Development by Catherine West"

https://www.youtube.com/watch?v=aKLntZcp27M

Catherine is relatively known in the Rust community, hence why I didn't mention anything else.

Here is her keynote talk at Rust Conf 2018.

https://youtu.be/aKLntZcp27M

⬐ holy_city

Ah ok had to skim through the slides a bit, I think you meant to use the phrase "generational indexes" which is why googling didn't help me. This project was more helpful for my understanding:

https://github.com/fitzgen/generational-arena

I'm not sure what your ultimate point was though? You don't have to use anything like that in normal Rust code

⬐ pjmlp

Only if you understand by normal Rust code, CLI apps.

If we are talking about graphical applications, that would be a common usage.

+1.

Like you, I've seen both sides. Unreal lineage evolved from inheritance-based into ECS-like. Frostbyte is heavily ECS.

Googling... the author is ex-amazon, ex-microsoft - with xbone and 360 title experience. Perhaps the author was exposed to that pattern at one of those institutions, or perhaps the author saw the 2018 Rust Conf Closing Keynote[0] and was inspired to continue down that path.

I think people tend to either go with what they've seen work in the past, or heard works well. Maybe it's cargo culting, or maybe in the case of rust (which I have no current experience with), it helps? Either way, the most important point is to focus on the fun and not let technologies get in the way of making a good game.

[0] - https://www.youtube.com/watch?v=aKLntZcp27M

⬐ eigenloss

Especially ironic given that Rust's default package/project management tool is named 'cargo.'

⬐ jorgeavaldez

For some people (myself included) the technology is part of the fun. The cargo cult popularity can definitely influence the decision to play with the technology, and by no means does it indicate that it is entirely appropriate for the task at hand. I personally use it as an exercise to familiarize myself with the patterns in a toy environment, which leaves me better equipped to apply them for a problem that lends itself to the technology. It also helps in preparing me to find those applications with a better understanding of the costs and semantics. While the knowledge may not be used for a practical purpose, at the end of the day, I learn something new, and I have a new way of looking at problems and understanding how to solve them with fresh perspective.

⬐ visualphoenix

I agree.

It's definitely fun to play with new programming languages and technologies. Using games as a medium to explore a new language is a great way to learn and have fun at the same time. That said, when a game development luminary like jblow takes the time to make a comment, it's good to think about why.

It seems to bother jblow that people are buying into the use of this system without thinking about the costs as well.

Its the same as the debate about when it is the right time to move to a distributed microservice architecture from a monolith. In this case, the use of an Entity Component Systems in a small game could be compared to the choice to use a Distributed Microservice Architecture for a small application.

Monolithic applications are easy to build and maintain by a small solo developer/shop, but they don't scale as well as you increase the number of engineers and expand the scope of the product. So a monolith is split into domain specific distributed microservices to allow for an increased number of developers working in parallel. When you pick to employ this tactic is up for debate.

Building a distributed system is hard. Debugging a distributed system without solid tooling/tracing is even harder. In the microservice world we have a lot of tooling to help us. Games, usually much less so. Where you might use GRPC in microservices and have stuff like OpenTracing to lean on, in games with Entity Component Systems you'll usually have some form of a message bus/event system without the same level of tooling and tracing.

Entity Component Systems/Distributed Systems are a useful pattern to implement when your desire is to build a reusable generic game engine/application to be implemented by large or scaling engineering teams. That's why you see these patterns employed and used by large game companies/startups and game engine/SaaS developers.

In summary, perhaps jblow's point is that small/solo developers can build a good game/application without employing the ECS/distributed microservice architecture pattern. And like you said, it's also useful to try building/using these patterns in toy applications to gain experience.

⬐ ehsanu1

I'd say the gulf between monolithic and microservice architectures is much wider than between ECS and non-ECS in games. The cost of microservices is much higher than that of ECS.

While building a game using some inheritance based approach, one will quickly hit limits even with a single engineer, whereas a monolithic web app can scale to a team of a couple dozen engineers easily.

⬐ drainyard

You don't have to use an inheritance based approach if you don't make it an ECS. The point is that using any "something-based" approach is not necessary for most small and even medium sized games. Just make what seems obvious until it doesn't work and then change it. If you start thinking about architecture from the start you will have an issue when you find out it won't work for your project no matter the architecture you are using.

⬐ jblow

The idea that you would “quickly hit limits” on a 1-programmer project is completely baseless. In fact a 1-programmer project is easier without an ECS as it’s one less system whose constraints you would have to comply with at all times.

Please stop saying this stuff as it just contributes to the general confusion.

It depends, I am over a year in Rust and it doesn't take me that much longer to write something in it than say in Python. The confidence I have that the thing I wrote is way higher in Rust and it is usually much faster.

And: it is incredible easy to build and deploy.

I think whether Rust is useful or not depends entirely on the application. If you need high confidence in what the thing is doing, it should run parallel and fast and you are familiar with the concepts Rust is using – it isn't a bad choice. For me it replaced Python in nearly every domain except one-use scripts and scientific stuff.

It can be hard for advanced programmers to abandon certain patterns they bring from other languages though. In the first months I tried too much to use OOP, which doesn't make any sense and leads to convoluted code. If you work more in a compositional and data oriented way while making use of Types and Traits, you will end up with much simpler solutions that work incredibly well.

Katherine West's RustConf 2018 Talk on ECS Systems describes this incredibly well and might be even good to watch if you are never intending to use Rust at all, because the patterns discussed are quite universal: https://www.youtube.com/watch?v=aKLntZcp27M

⬐ ndesaulniers

> I tried too much to use OOP, which doesn't make any sense and leads to convoluted code.

Oh, well, you don't need Rust for that! :P

⬐ Impossible

Yes the same is true with C++ or C# for example, which is were the move out away from OOP and towards DOD/ECS started.

⬐ pjmlp

Which is ironic, given that people tend to forget that it was C++ which made OOP mainstream, there wasn't any Java or C# back then, two fully OOP based languages.

The other part is that component oriented programming is actually a branch of OOP, from CS point of view, with books published on the subject at the beginning of the century.

"Component Software: Beyond Object-Oriented Programming"

https://www.amazon.com/Component-Software-Beyond-Object-Orie...

First edition uses Component Pascal, Java and C++, with the 2nd edition replacing Component Pascal for C#.

I rather enjoyed RustConf 2018's Closing Keynote, "Using Rust For Game Development" by Catherine West: https://www.youtube.com/watch?v=aKLntZcp27M

I learned about the ECS pattern and got to see some Rust refactoring in action.

Previous HN discussion here: https://news.ycombinator.com/item?id=17977906

⬐ None

None

⬐ brodo

This one is one of my favorites of the year too. ECS seems like a pattern which can also be used in other contexts then games. I‘m currently writing a simulation tool and I am considering switching to ECS.

⬐ seventhtiger

Simulations and games are highly integrated and iterative. Having every process be done in frames or steps makes the preservation of information for the next step/frame really important. Systems are also tightly coupled and not easily separated.

That's why I like ECS. I also like the EC part to just be a table I can query, and the S system should be very tight and the dependencies should be used explicitly. That way you go around some thing like dynamic ordering of the update function based on object pools and other nonsense.

⬐ sn9

Jonathon Blow had a really interesting but subtle response to this talk: https://www.youtube.com/watch?v=4t1K66dMhWk&feature=youtu.be

⬐ brodo

Thanks, very interesting. He is right, the Rust borrow checker can not deal with arbitrary graphs and she added a memory management layer on top of the Rust one. However, you would have to do the same in C++. It is an interesting obervation. I spoke to Benedikt Meurer from the Google V8 team recently and he pointed out the same for Javascript Engines. If you do not have DAGs, Rust can‘t help you with safety. (at least it can not prevent use after free) I still like the whole ECS idea and it makes sense for me that using ECS in Rust will provide a performant and clean architecture.

Entity-Component-System, amusingly explained in this great talk from RustConf.

https://www.youtube.com/watch?v=aKLntZcp27M

⬐ jcelerier

but... all ECS systems use classes or equivalent at some point

⬐ coldtea

If the classes are shallow that's fine. The problem is using classes to model the whole behavior, not using them to hold some values together.

⬐ adnzzzzZ

That solution is more burdensome than OOP

⬐ coldtea

Not at all. It also has excellent visibility, extensibility, and debugabillity.

⬐ Epskampie

Diagree. Unity works like this, and it’s great for attaching multiple behaviours to an object (i.e. a Player can be Hit, but an Enemy as well). It saves you having to make up a inheritence hierarchy that never seems to work out (Player extends Hittable or something? But it also needa these 10 other behaviours...)

There was a great talk recently about how to properly build an entity component framework in Rust if you haven't seen it yet.

https://youtu.be/aKLntZcp27M

⬐ Svip

Thank you for that link. Very useful. It already highlighted a solution in Rust for a problem that I've run into with my Go prototype. And I appreciate there also was a blog post to read.[0] I personally prefer articles over videos.

[0] https://kyren.github.io/2018/09/14/rustconf-talk.html

Easily solved with generational indexes as described in https://www.youtube.com/watch?v=aKLntZcp27M

⬐ pjmlp

I have seen that talk.

To me it feels a bit like a workaround for something that cannot be validated by the borrow checker, because it is something that developers have to go the extra mile to implement, or get a third party library, which someone has to validate that actually works as expected.

In a way, it isn't much different than expecting C and C++ developers to use static analysis for lifetime validations.

Meaning, using a tool outside of the core language for added safety.

⬐ nicoburns

Requiring a library is only really a big deal in the C/C++ world though. Everywhere else it's trivial, and most projects will depend on some foundational libraries.

That's pretty different to static analysis tools which most likely won't always work.

⬐ pcwalton

Nobody is asking for generational indices to be added to the core language. There would be zero benefit. We have a package manager for a reason.

Maybe we could uplift them to the nursery, but again, generational indices are nowhere near the top 10 crates on crates.io.

⬐ kibwen

> To me it feels a bit like a workaround for something that cannot be validated by the borrow checker

Is it so hard to believe that references are not the correct abstraction for 100% of use cases? Reaching for something other than references when references are the wrong tool for the job is not working around the borrow checker; it's choosing the right abstraction for the right task. Being hung up on the borrow checker is missing the forest for the trees.

> get a third party library, which someone has to validate that actually works as expected.

How is this different from using any third-party library, ever?

> Meaning, using a tool outside of the core language for added safety.

Rust explicitly supports users defining their own smart pointer types to provide pointer-like abstractions with custom semantics; using tools outside of the core language for added safety (for whatever definition of "safety" one wants) is completely expected and encouraged.

⬐ pjmlp

The point being, that for developers focused on managed languages, C++ takes the role of the unsafe layer.

Meaning Java/C++, .NET + (C++/CLI | C++/WinRT), Node.js + C++, Swift / Objective-C++.

So with C++ improving its safety history, usually with ideas taken from Rust, Rust ergonomics and tooling need to have a better story than C++'s to replace it on the above stacks.

⬐ pcwalton

I can confidently predict that the lifetime profile for C++ will get almost no real-world use.

⬐ kibwen

Interesting, because in my experience people use Java/C, Node/C, Swift/Objective-C, Python/C, Ruby/C... rarely do I encounter anyone using C++ as an extension language (I encounter Rust being used more often than C++, in fact, but perhaps that's an artifact of the circles I'm in).

As for C++ adding more static analysis, its lifetime analysis is a nice-to-have, but doesn't compare to Rust's borrow checker. You simply can't tack on a sound borrow checker to C++, because the language wasn't designed to accommodate one, and trying to impose the concomitant rules regarding mutability, aliasing, and single-ownership would break every C++ program ever written. For anyone who prioritizes sound static analysis WRT lifetime verification, C++ isn't a competitor to Rust. And there are plenty of people for whom that isn't the case, and they will continue to use C++, and that's not a problem. Rust exists to provide an alternative systems language for people who favor memory safety, and it's pretty good at that. :)

⬐ pjmlp

I guess you spend more time in UNIX platforms?

As for being an alternative systems language for people who favor memory safety, I fully agree, my point is that it still needs to improve its productivity and eco-system.

At CppCon 2018 Embedded Development panel, one theme was that only now companies are slowly willing to migrate from C to C++11(!), with a language that allows for a progressive rewrite from C while keeping the existing toolchains.

Another productivity example, with .NET I can get the safety I advocate, while C++/CLI/CX/WinRT allow for a seamless interoperability story with native code.

So even if the lifetime analysis is a subset of what Rust is capable of, mixed debugging and seamless CLR/COM/UWP integration are more attractive than rewriting that code in Rust, without having VS integration and WIP integration with Windows APIs.

I think Rust on its current state, is more indicated for GC free scenarios with either CLI or headless execution.

⬐ pcwalton

> I think Rust on its current state, is more indicated for GC free scenarios with either CLI or headless execution.

That's funny, because the largest deployment of Rust is in Firefox, which has a UI.

⬐ pjmlp

So can you point us to an example in Firefox's source how to create an UI widget in pure Rust code?

As far as I am aware, Rust is only being used for low level rendering, not widgets.

⬐ donmcc

And here’s the follow-up blog post: https://kyren.github.io/2018/09/14/rustconf-talk.html

⬐ nightcracker

I'm the author of slotmap (https://github.com/orlp/slotmap) briefly mentioned at the end. I'm working hard on the requested 'independent allocation' (or secondary maps as I'm currently leaning on), and will have it ready soon.

⬐ rapsey

How is it different from slab crate?

⬐ nightcracker

https://docs.rs/slotmap/0.2.0/slotmap/#why-not-slab

⬐ gamegoblin

Not Rust specific, but in the same train of thought, here is an excellent series of articles going into other issues with OOP and game development, and touches on a few similar topics of cross-cutting concerns and their interaction with interface/class design:

https://ericlippert.com/2015/04/27/wizards-and-warriors-part...

https://ericlippert.com/2015/04/30/wizards-and-warriors-part...

https://ericlippert.com/2015/05/04/wizards-and-warriors-part...

https://ericlippert.com/2015/05/07/wizards-and-warriors-part...

https://ericlippert.com/2015/05/11/wizards-and-warriors-part...

⬐ lackbeard

I'll have to re-read this series, as it's been a while but I remember feeling dissastified after finishing it the first time. I think the conclusion was basically: yup, programming is hard, no matter what you do you're going to wind up with a big mess!

⬐ phantarch

That's an easy sentiment to come away with from every article in the series except the very last.

The final one does offer (imo) a pretty clean solution to the problem he's been exploring, which boils down to viewing the rules of the game as the class definitions and using the things in the game world as data to be plugged in to the rules, rather than the other way around.

⬐ mrec

Just watched this yesterday. I really like the pragmatic tone ("you don't have to go all the way, you can stop at any of these points and be fine"), and I love the "egoless" style of presentation. Illustrating your point using your own mistakes is much more compelling than picking on some artificial strawman.

⬐ arconis987

The generational index idea was exactly what I needed for a search index project I’ve been working on. Makes it much easier to safely insert and delete from a memory arena.

Also, fascinating that Rust’s borrow checker sort of encourages you over time to adopt an entity component system architecture instead of a naive OO architecture.

⬐ faitswulff

> Also, fascinating that Rust’s borrow checker sort of encourages you over time to adopt an entity component system architecture instead of a naive OO architecture.

Is this true outside of gamedev contexts?

⬐ unrealhoang

Yes, for example UI programming. Since Rust makes none-linear data flow very inconvenient (Rc and RefCell), people often try to find data-oriented solution like ECS instead.

⬐ bluejekyll

I'm not familiar with the parent comment, but after seeing this talk at RustConf, I've been strongly considering this pattern as a better way of storing associated data to an entity than OO design, especially in regards to high-scale systems where portions of an Entity might be stored across different datastores.

The idea of operating on individual components of an entity is very compelling in those cases, it's not even novel. We often have done this by pulling data out of datastores in parallel array like requests. The difference with this model is that it creates an elegant interface over the data for an entity that isn't a bastardized version of OO, instead just the data and structures you need present in the context in which your working.

As a non-gamedev, I found this really compelling.

⬐ anderspitman

I had a very interesting experience as a Rust newbie developing a genetic algorithm simulation[0]. I started with my mental model of how to represent things, and I definitely felt like the borrow checker pushed me to adopt more ECS principles.

[0] https://github.com/anderspitman/battle_beetles

⬐ erlend_sh

I encourage anyone interested in game development with Rust to check out the yet-to-be-formalised working group:

https://internals.rust-lang.org/t/a-working-group-for-rust-g...

At the moment the main focus is just to get organised around some common tooling that everyone can get behind. At this point I'd say the most impactful project to contribute to is gfx-rs.

⬐ Jare

This is a fantastic talk about game architecture, regardless if you are interested in Rust or not.

⬐ agumonkey

I read some 2015 review of rust from viva64 where they used benchmarkgame to show rust is still 3x C. Nowadays benchmarkgame has Rust and C in the same numbers. That's quite a feat.

⬐ steveklabnik

Discussed three years ago: https://news.ycombinator.com/item?id=9531822

and two years ago: https://news.ycombinator.com/item?id=12232385

⬐ adwhit

Rust hasn't really got 'faster' in that time, barring the odd optimizer improvement as noted in sibling. Much more likely that whatever benchmark you refer to has been rewritten for better performance.

⬐ raphlinus

Another major evolution is that SIMD is becoming part of stable Rust. It is available in C and C++ as GCC and Clang extensions, but not part of the standard language. Careful use of SIMD can result in fairly massive speedups across a wide range of problem domains.

⬐ steveklabnik

Don't discount multiple years of "odd optimizer improvements". They add up.

That said, yes, re-writes are probably a bigger thing here. Especially lately: https://llogiq.github.io/2018/09/06/fast.html

⬐ agumonkey

maybe stable semantics helped people writing better libs too ?

⬐ steveklabnik

That's true, though the benchmark game doesn't use many libraries, and the post was written just before 1.0, as things were settling down a bit.

⬐ gameswithgo

Rust just recently got SIMD support in stable and they have been working on applying it to benchmarkgame very recently, so it should be improving more soon.

⬐ vvanders

I've mentioned it before but there's a chance for Rust to get better than most C code with the ability to implicitly mark mut& as restrict.

Restrict is one of those very, very sharp tools that's usually only reserved for cases where you've profiled and done the extensive work to guarantee non-aliasing pointers. The fact that it's just implicit as part of the ownership design of Rust is pretty bad-ass.

⬐ bluejekyll

For those of us unfamiliar with restrict, I found this article: https://en.cppreference.com/w/c/language/restrict

"The intended use of the restrict qualifier (like the register storage class) is to promote optimization, and deleting all instances of the qualifier from all preprocessing translation units composing a conforming program does not change its meaning (i.e., observable behavior).

The compiler is free to ignore any or all aliasing implications of uses of restrict.

To avoid undefined behavior, the programmer must ensure that the aliasing assertions made by the restrict-qualified pointers are not violated. ..."

⬐ kibwen

AIUI it seems that one of the issues that Rust is having right now is that it actually has more precise information than LLVM is really capable of taking advantage of, which makes sense given that LLVM was originally intended to take advantage only of C's coarser restrict semantics. It's sort of a funny situation where the fact that Rust could improve on C's optimization capabilities is the same reason that it can't yet do so: it's using an optimizer designed for C!

That said, I'm hardly an expert here and the situation on the ground may have changed since last I looked (or I may be completely misunderstanding the discussions that I've peeked into). An example of the problem in this GitHub issue: https://github.com/rust-lang/rust/issues/53105

⬐ Manishearth

Yeah, this is true.

We do the equivalent of telling the compiler that things don't alias at function boundaries (i.e. restrict) and not much more.

Nobody writes C code with restrict all over the place, which means that not only is this the finest grained level we can give this info to LLVM, it doesn't even necessarily support it well! We had to turn it off for &mut for a long time because of a bunch of bugs.

The weird thing is, to be able to do most optimizations LLVM does its own alias analysis anyway! (I don't know the details).

A lot of program analysis research is focused on improving alias analyses which are usually slow, imperfect, and global. Rust's aliasing info is "free" (you don't need to compute anything extra to get it), local, and perfect (ish). We really could get a lot of wins from doing our own aliasing-aware analyses, or by reordering the code in such a way so as to make it easier for LLVM (e.g., hoisting reads and writes to function boundaries so llvm just sees a bunch of locals)

⬐ epage

For me, this was the introduction to ECS that made me "get it" and I'm slowly morphing my Rust implementation of the Liquid template language over to an ECS architecture.

⬐ pcwalton

This is a really good talk. The name undersells it: it's really "how to stop fighting the borrow checker" in general and not specific to games.

⬐ Koshkin

Rust: too little, too late.

⬐ steveklabnik

Why?

⬐ zerr

So far interesting, but it is very annoying that cynicism became the standard way to drive talks.

⬐ raphlinus

I found this excellent talk to be complementary to my talk[1] on data-oriented GUI in Rust, not to plug my own work too much.

I found a lot of common ground:

* Trying to write object-oriented code in Rust doesn't work well. Writing data-oriented code does.

* "Fighting the borrow checker" is a sign you might be doing it wrong. Try data-oriented approaches instead.

* Use a structure-of-arrays rather than an array-of-structures.

* Use what I call "state splitting" to borrow mutable references to just the state you need; this works well with the structure-of-arrays approach and is a powerful motivator to use it.

* To build graph-like structures, reach for a Vec of components, and indexes into the Vec for relationships, as your first choice. Self-references, arena allocators, and Rc are viable alternatives, but you should have a good reason to use them instead of Vec.

* Some form of dynamic typing is useful to keep your system loosely coupled (though we ended up with very different forms of dynamic typing, see below).

* Data-oriented approaches have taken root in the C++ gaming community, mostly motivated by performance, but adapt well to Rust, and some of the ideas may be useful in domains beyond gaming.

There were some other points that went beyond what I talked about, but could fit in well:

* A "generational index" is a good way to avoid use-after-free style errors that result from the use of a stale index. The slotmap crate can help.

And now for the things that are different.

* The exact nature of dynamic typing is different. An ECS usually uses a registry of the different component types (anymap is a useful crate) and is quite open-ended in adding new types of components. My xi-win-ui, by contrast, has two main component types, Widget and listener, and does dynamic dispatch on a Widget trait.

This naturally raises the question, which is better? From my perspective, both are valid. The pure ECS approach definitely makes sense when the components are interacting with each other in diverse ways (collisions, damage, etc). In a GUI, the components are mostly interacting with the framework, and have diverse behaviors within standardized interfaces (input, layout, paint).

Thus, one point of my talk is that you don't have to reject all vestiges of object oriented programming, it's perfectly reasonable to hybridize, using data-oriented approaches for state splitting and graph structure, and dynamic dispatch for the behavior variation.

My talk also went deeper into the ideas of "data flow rather than control flow" and the use of an event or command data structure rather than method calls. I'm not sure how deeply these ideas apply to games, but wouldn't be surprised if they do.

[1] video: https://www.youtube.com/watch?v=4YTfxresvS8 , slides: https://docs.google.com/presentation/d/1aDTRl5R-icAF38Di-qJ4...

⬐ MichaelMoser123

# Use a structure-of-arrays rather than an array-of-structures.

Could you please explain this in more detail?

# To build graph-like structures, reach for a Vec of components, and indexes into the Vec for relationships

And that also allows you to reference deleted nodes?

⬐ steveklabnik

> Use a structure-of-arrays rather than an array-of-structures.

instead of this:

  struct World {
      players: Vec<Player>
  }

  struct Player {
      name: String,
      health: i64,
  }

which is a "array of structures", see Vec<Player>, you do this:

  struct World {
      player_names: Vec<String>,
      player_health: Vec<i64>,
  }

"A structure of arrays".

"Player zero" is no longer an index into a players array, but an index into many arrays, all of which hold certain kinds of data about a player.

> And that also allows you to reference deleted nodes?

... which is why the talk then references generational indices as a way of dealing with it.

⬐ philipov

This sounds like it is just going to trade one set of problems for another. It makes it impossible to write generic container types. What if elements of the same collection need to have different structure? What benefits justify this extremely tight coupling?

⬐ Tuna-Fish

> What if elements of the same collection need to have different structure?

They don't. If you ever end up in a situation where you feel they do, the correct solution is typically instead to split the component into multiple, different components, only some of which will be used for any given entity. This is basically the same as defining a schema for an SQL table. A component is just a set of state, there is no requirement for it to map 1-to-1 to a specific functionality.

> What benefits justify this extremely tight coupling?

The most commonly stated one is speed. ECS was adopted first in game design because it is just so much faster. On modern OoO cpus it's typically something like 5-10x faster than traversing an object graph. On the previous generation consoles (PS3, XB360) with their in-order CPUs and crappy load/store subsystems, it could easily be 20x-50x faster. I don't know how relevant this is to your typical GUI, though; the speedup in an ECS comes from linear memory access, which means that the prefetchers make sure every memory access is an L1 access, which is great when you have a game that has thousands of entities, which don't fit into any cache. But just how many GUIs have enough state to overflow the L1 anyway?

However, speed is not the only benefit. This is somewhat subjective, but having implemented similar logic for ECS and OO based games, I feel that the logic is almost always much clearer, more understandable and less buggy in the ECS versions. Basically, in OO doing things that have cross-cutting concerns tends to get split into many small parts done in multiple places, and it's hard to understand the whole system at once. In an ECS, the logic for one system is implemented in one place, it is always just a transformation that reads in some data, does some computation on it, and writes out some data, without complex control flow. It's so much easier to understand and test.

> It makes it impossible to write generic container types.

An example of a Generic container type is AnyMap, which holds one value of each type (and each value will typically be either a straight Vec for small/common components, or some kind of more complex set for components that hold a lot of data.)

(edit: looked up old numbers and found that 100x was pushing it, even on Xenon. 50x ought to be realistic.)

⬐ philipov

Thanks, i'm starting to see how this does naturally encourage organizing type extension around composable traits rather than inheritance

⬐ steveklabnik

I mean, that's sorta the point of the talk. Did you watch it?

> This sounds like it is just going to trade one set of problems for another.

Sure, that's exactly what a tradeoff is.

> what substantiates the claim that it is generally superior?

I don't think the claim is that it is always superior.

Oh, and I would see this refactor of a decoupling. The issue is that, if you’re trying to process Players in certain ways, the fact that the name and health are coupled together in a single struct is an issue. This pulls them apart.

⬐ raphlinus

As steveklabnik says, it absolutely is about tradeoffs. The anymap crate may provide enough of "generic container types" to be useful, and can avoid a lot of repetition of per-type code. A graph with heterogeneous node types is definitely possible, Box<Any> is one solution and there are others.

⬐ s73v3r_

"What if elements of the same collection need to have different structure?"

The answer is, they usually don't.

⬐ meheleventyone

And even if you did with Rust you could theoretically use enums to achieve it (or a union in C).

⬐ tdsamardzhiev

Everything we ever write as programmers is trading one set of problems for another. The key thing is to identify which problems matter the most.

⬐ db48x

It's frequently done that way for performance. Imagine a game of Starcraft, with 1000 zerglings rushing your base. The game has to repeatedly loop over all the zerglings to move them. Since there are lots of other fields tracking all of the other data about each zergling, the normal AOS approach has poor data locality; you load a cache line and then you only touch a few bytes of it. With the SOA approach you're looping over an array of positions, so every byte that you fetch from memory ends up being used.

⬐ LBarret

I agree with you but this I am not sure it is the right example. In this case, the position of the zergs would be indirectly a component. You would have arrays of struct Zergs, each would only have a ref (or an index) to an arrays of positions. this array would be updated efficiently.

⬐ db48x

Yes, you could do that, but then you don't have SOA or AOS, you have SORTA (struct of references to arrays).

⬐ axilmar

Doen't using indexes into arrays introduce, more or less, the same problems with C pointers? after all, a C pointer is an index into a huge array, the current process' memory space.

⬐ steveklabnik

Not quite; you can't cause memory unsafety with the indexing version.

⬐ anderspitman

I highly recommend watching Raph's talk for anyone writing Rust apps that manage non-trivial state. Ah heck just watch it no matter what it's great.

⬐ raphlinus

Aww, thanks :)