HN Theater @HNTheaterMonth

I really can't agree with this premise after seeing how Zig implements color-less async/await:

https://kristoff.it/blog/zig-colorblind-async-await/

I highly recommend watching Andrew Kelley's video (linked in that article) on this topic:

https://youtu.be/zeLToGnjIUM

⬐ whizzter

Zig seems to be doing a lot of things right (just like Rust did).

Sadly I think that their disregard for safety (the manual states that dangling pointers is the developers problem) kinda makes it a no-starter for many people. I personally consider possible information breach far worse than a crash or runtime stall.

⬐ dnautics

> Sadly I think that their disregard for safety

I wouldn't call it a disregard. Zig's approach is to give you easy and automatic tools to detect and handle most of those memory safety problems in its language-first-class test system, which you can think of as a carrot to get you to write tests.

⬐ whizzter

Automatic tools is definitevly a plus, I guess the only thing is that adversaries can possibly see what tests missed (UAF often involves code behavior differences between modules).

⬐ kbd

> disregard for safety

Your opinion seems to be that any future systems language that doesn't implement a heavy Rust-style borrow checker and explicit safe/unsafe modes "disregards safety"?

Zig does have a lot of checks, such as bounds checking[1]. There are also test modes that help catch errors. I don't know what you're referring to about "information breach".

> The manual states that dangling pointers is the developers problem...

In a systems language where you can cast an int to a pointer:

    const ptr = @intToPtr(*i32, 0xdeadbee0);

or where you have to manually manage memory, what else do you expect?

Zig made the design choice to not split the world into safe vs unsafe. It seems a bit unwarranted to say that because they didn't make the same design choices Rust did that they have a "disregard for safety".

[1] https://ziglang.org/documentation/master/#Index-out-of-Bound...

⬐ whizzter

Out of bounds check is definitely a responsible start.

As for the exact methods I'm not really partial to anything (Be it borrow-checker,gc,compiler-time-analysis,etc) but we've seen time and time out of bounds(handled by Zig atleast), UAF and other issues be exploited so having "safe" defaults for the majority of code isn't something I think we should skip on, especially if it's a "systems language" since the exploitation surface will end up everywhere.

The Rust borrow checker can probably feel cludgy at times, and in many ways it's a result of creating a scheme that is verifiasble. Since Zig already has comptime I guess having expand compile time capabilties to analyze for common UAF conditions,etc shouldn't be unfeasible?

⬐ valenterry

Zig also uses colored functions here, but introduces a mode to globally(?) block on every async call. While you can certainly do that, I'm not sure if that is a great language feature to have, to be honest - at least if I don't misunderstand their idea.

⬐ kbd

I think you're confusing the io_mode with async/await.

⬐ cygx

In Rust, you color functions at declaration time through special syntax, and, depending on the color of the caller, you have to use either block_on() or .await to get a value out of a call to an async function.

That's not the case for Zig. There's no special decorator, no block_on/await distinction, and regular calls will return the expected value by implicitly awaiting (in case evented mode is enabled). A callsite may decide not to implicitly await the result by using the async keyword, which then requires an explicit await on the returned promise.

edit: fix caller/callee typo

⬐ valenterry

Well yeah - you can do exactly the same in other languages, such as Scala. It just isn't a good idea. They explain it themselves:

> there’s plenty of applications that need evented I/O to behave correctly. Switching to blocking mode, without introducing any change, might cause them to deadlock

⬐ dnautics

zig doesn't switch. It's statically typed, and I think (not 100% sure) it detects if you have called a function with the async keyword and when that happens it triggers compiling a second version of the function with async support, so there are actually two versions of the function floating around in your binary. Since zig is itself a parsimonious compiler, if that's never in a codepath (say it's provided by a library), it doesn't get compiled. The reverse is true, too, if you ONLY call a function with the async keyword, it doesn't compile the non-async version. Again, with the caveat that I think that's what's going on.

⬐ dnautics

Correction: my understanding is incorrect, zig does not compile two versions of the functions. Nonetheless, it does not switch.

⬐ valenterry

Well, I'm quoting their own explanation...

I recently wrote an IO abstraction over io_uring using Zig's async/await.

Here's how you would do a write()/fsync()/read() with this (https://github.com/coilhq/tigerbeetle/blob/beta/src/io.zig#L...):

  const offset: u64 = 0;
  const bytes_written = try io.write(fd, buffer_write[0..], offset);
  try io.fsync(fd);
  const bytes_read = try io.read(fd, buffer_read[0..], offset);

Other sync functions can use this asynchronous IO completion code in a synchronous style (as this snippet shows) and still get all the zero-syscall and asynchronous performance of io_uring. What this is actually doing under the hood is filling SQEs into io_uring's submission queue ring buffer and then later reading completion events off io_uring's completion queue ring buffer, so it's fully asynchronous in the I/O sense but this hasn't spilled out and leaked over into the control flow. The control flow is as it should be, nice and simple and synchronous.

Beyond this, Zig still allows you to explicitly indicate concurrency with the `async` keyword, for example if you wanted to run multiple async code paths concurrently.

But the crucial part is that Zig's async/await does not force function coloring on you to do all of this: https://youtu.be/zeLToGnjIUM

Pretty incredible on Zig's part to be able to pull this off. Huge kudos to Andrew Kelley. Also, thanks to Jens Axboe and io_uring, what you saw above was first-class single-threaded or thread-per-core, there's no threadpool doing that for you, it's pure ring buffer communication to the kernel and back, no context switches, no expensive coordination. Pure performance. There's never been a better time for Zig's colorless async/await. The combination with io_uring in the kernel is going to be explosive. It's a perfect storm.

You can also watch the linked video which demos the strace.

https://youtu.be/zeLToGnjIUM?t=546 (9:06 to 15:24)

Direct link: https://www.youtube.com/watch?v=zeLToGnjIUM

There is an interesting approach in Zig to explicitly kick call frames to achieve colorless asyncness.

https://www.youtube.com/watch?v=zeLToGnjIUM

> Code that's meant to be pinned (and may behave incorrectly if not pinned) looks no different from any other code.

Same goes for async/await. The decision to keep you running on the same native thread is up to the scheduler.

> Java isn't positioned as a high-level, high-abstraction language and that's not, IME, the user community it has.

I beg to differ. It aims to be a good compromise between productivity, observability and performance. Every choice, from JIT to GC, is about improving performance for the common case while helping productivity, not improving performance by adding fine-grained control. There are a few cases where this is not possible. Primitives is one of them, and, indeed, 25 years later, we're "expanding" primitives rather than find some automatic way for optimal memory layout.

> It's always been advertised as a safer alternative to C++ - rather like Rust.

I think you're mistaken, and in any event, this is certainly not our position. Java is designed for a good blend of productivity, observability and performance, and unless the circumstances are unusual, it opts for improving common-case performance with high abstractions rather than worst-case performance with low abstractions like C++/Rust. Roughly speaking, the stance on performance is how do we get to 95% with the least amount of programmer effort.

Anyway, regardless of what I said above, the constraints on the design of usermode threads other than philosophy are also very different for Java than for C++/Rust for reasons I mentioned. Still, Zig does it more like Java (despite still using the words async and await, but they mean something more like Loom's submit and join than what they mean in C#): https://youtu.be/zeLToGnjIUM

⬐ lmm

> Same goes for async/await. The decision to keep you running on the same native thread is up to the scheduler.

The scheduler decides what happens at each yield point, but code that doesn't yield is guaranteed to stay pinned to a single native thread. A non-async function is somewhat analogous to a critical section; the difference between async and not is a visible distinction between must-run-on-a-pinned-native-thread functions and may-be-shuffled-between-native-threads functions.

⬐ pron

Can you give an example where this matters -- i.e. it's useful and allowed to move between native threads but not between well-known points -- given that the identity of the carrier thread cannot leak to the virtual thread?