HN Theater @HNTheaterMonth

(I guess it's time to post this video again.)

Saša Jurić - The Soul of Erlang and Elixir: https://youtu.be/JvBT4XBdoUE

This is in my opinion the best demo of Elixir, the BEAM, OTP and any programming language, ever. If this doesn't pique your interest, nothing will.

⬐ srikz

I was already fascinated by Elixir, but this video drove the nail home. It's a great presentation and I don't think I have watched any conference talks (on YouTube) without taking a break except this one.

I hadn't seen those videos before, thanks.

Let me add one to the list: "The Soul of Erlang and Elixir • Saša Jurić" — https://www.youtube.com/watch?v=JvBT4XBdoUE

I'm positive this video can turn any software engineer into a BEAM/OTP convert. It's one of my favourite demos.

⬐ el_oni

I love this demo, it what got me to pick up Sasa's book.

I was attending a show and tell following seeing this where an engineer was trying to isolate failure of certain functions from the rest of the system and i sent this to them.

No idea if they watched it but i became an OTP evangelist in that moment

F# does have the `MailboxProcessor`. It is quite simple and capable.

https://fsharp.github.io/fsharp-core-docs/reference/fsharp-c...

https://fsharpforfunandprofit.com/posts/concurrency-actor-mo...

I haven't looked at Orleans in a while. Last time I did, I came away with the viewpoint that it was very much designed to have a C# interface, which while manageable in F#, doesn't really provide for an idiomatic F# experience. I'm also not too sure, but I think it basically requires a cloud or distributed environment. Can it just be run on a local computer?

But even the `MailboxProcessor` doesn't do what Elixir and Erlang do on top of the BEAM VM. An instance of a BEAM VM is a single OS process, but the BEAM can handle millions of Erlang processes, which are not OS processes or even threads. They are their own thing that are extremely lightweight. The BEAM scheduler is very nice as well, such that it switches among all the running processes.

Even though I develop in Elixir, I am not yet a BEAM expert. So before I say something incorrect, I recommend taking a look at the talk The Soul of Erlang and Elixir by Sasa Juric.

https://www.youtube.com/watch?v=JvBT4XBdoUE

It fully encapsulates and describes what makes processes so special in Elixir and Erlang.

There is the Gleam language, which is a statically typed language on the BEAM VM. However, they moved away from the ML-dialect syntax, which is unfortunate.

⬐ pharmakom

Maybe one day Fable will add an Erlang backend.

⬐ cassepipe

... and also Caramel which is a Ocaml/Reason dialect so you get to keep the Ocaml syntax

I have never tried it though

https://caramel.run/manual/introduction.html

If you want to see a really nice demo of this, check out https://www.youtube.com/watch?v=JvBT4XBdoUE -- The Soul of Erlang and Elixir by Saša Jurić.

There's a really neat demo of a machine at 100 CPU% but that's still responsive because of Erlang's preemptive scheduling. There's a ton of other goodies in that video if you're interested.

It really shows off the Erlang BEAM VM.

Love this! Bouncing between python and elixir projects, I have been wanting this for some time.

I would love to see stress/performance testing results for this. I may take a stab at it myself; something like this simple, venerable demo from Saša Jurić[0] (starting around 8 mins)

[0] https://youtu.be/JvBT4XBdoUE

⬐ linkdd

Thanks for the feedback!

For now it's a toy project, so I would not be confident over its performance. I might try to come up with a good scenario for a benchmark and post the results in the future. Contributions are welcome!

I don't claim that Elixir is the language to end all languages. My comment was more about the holistic ecosystem that keeps improving. There's one common development and project management tool (Mix). There's one package manager (Hex). There's OTP, Ecto, Phoenix, Liveview, etc. There's the Elixir Forum for questions.

I'd use F# for writing compilers.

Interactive programming is probably better in Common Lisp and also Smalltalk descendants like Pharo. But Elixir has some great interactive tooling. I think this is an excellent presentation that summarizes the capability (not necessarily saying other languages don't) in terms of real world applications: https://www.youtube.com/watch?v=JvBT4XBdoUE

⬐ medo-bear

> I don't claim that Elixir is the language to end all languages.

i don't know much elixir. i was asking because those things interest me and they are what drove me to common lisp. for my purposes i need an industrial fast interactive language that is self hosted and well tested in the wild. i want a fast language that is well suited to make custom symbolic computer algebra procedures, custom differential algorithms, custom probabilistic programs, and custom data processing solutions. i think that common lisp best fits the criteria for my use case

I found this video really good introduction to what BEAM is and allows for programmers: https://www.youtube.com/watch?v=JvBT4XBdoUE&pp=sAQA

I feel like the post brushes past the best parts to line the learnings up with container concepts and k8s.

The Erlang application/service should not be crashing. A crash happens in a process in a tree of supervisors and each supervisor can have strategies to recover if its child supervisor/process fails.

Supervision in Erlang/BEAM is not just microservice infrastructure with restarts. It handles many types of failures knside the application and only if all mitigations repeatedly fail would we crash all the way up to the trunk of the tree and crash the application. If that happens the Erlang heart should be starting it back up or otherwise invoking some last-ditch mitigation. Or if you run Erlang in a container, that's when you'd have your infrastructural stuff restart it.

Supervision in Erlang is much more granular and not about restarting the entire service.

I think this gives a good view of Erlang resiliency: https://youtu.be/JvBT4XBdoUE

> Pre emp scheduling

Userspace pre-emptive schedulers do not incur the same level of overhead that kernel threads would normally have. To address another comment you made in another chain, Erlang accomplishes pre-emption through reductions which is roughly equivalent to ~4000 function calls, if I remember correctly and if I have read the correct documentation, before it yields the Erlang process.

Other forms of async to my knowledge essentially queue a task while the main thread of execution continues before awaiting; and the queued task is run in some form of thread pool (or other forms of executors). Without going too deep into this subject (and assuming I have made no logical errors) it is possible to have misbehaving tasks in async tank performance and latency, while the same misbehaving process in Erlang wouldn't. See Saša Jurić's demonstration [1] for an example. Do keep in mind that async is not forte, what I have stated is merely my observations. I write primarily with kernel threads.

This is why it is possible to spin up millions of, in Erlang terms, processes with minimal overhead. See this article by the Phoenix Framework team regarding an experiment on having two million active websocket connections on a single (albeit beefy) host [2].

> This and everything else is very nice but do we need a full VM for that?

See [3] regarding an effort to port the Erlang process and supervision strategy to Rust. In my naive view, it is possible to write something similar to BEAM in another language, but the more troubling portion of the work is to jury-rig the original language into a concurrency model it isn't designed for whereas Erlang by contrast was created purely for this task.

[1] https://www.youtube.com/watch?v=JvBT4XBdoUE

[2] https://www.phoenixframework.org/blog/the-road-to-2-million-...

[3] https://github.com/bastion-rs/bastion

⬐ jimsimmons

Appreciate the answer. I just saw the Sasa talk the other day incidentally and pre emption value is pretty clear now.

It’s so clever and well thought out, I think I start too see value of Erlang now

Almost no business application is written for performance at raw request speed. What Elixir brings is far more the that. Watch this if you want to understand why so many people fall in love with it. https://youtu.be/JvBT4XBdoUE

This video by Saša Jurić -- https://www.youtube.com/watch?v=JvBT4XBdoUE -- is really interesting.

It moves kind of fast and you might need to rewatch a few parts, but it demonstrates some of the principals that make Elixir / Erlang so compelling.

If you watch this, imagine what you would need to do as a developer to make this work in your language of choice.

Sorry only have a video for you, still worth watching IMHO https://youtu.be/JvBT4XBdoUE

⬐ konart

Thank you!

This talk by Saša Jurić (author of Elixir in Action) should answer most of your questions regarding the benefits of the BEAM (Erlang VM).

https://youtu.be/JvBT4XBdoUE

Seriously, if you're even the slightest bit curious you should watch the talk, I'm very confident that you'll find impressive what the BEAM brings to the table.

⬐ einpoklum

I will (even though I have no idea what Elixir in Action is). Thanks!

⬐ auraham

Great talk! Wonder if the source code of the demo is available.

⬐ sfusato

Here you go: https://github.com/sasa1977/demo_system

⬐ auraham

Thanks!

⬐ Bluestein

Oh! Nice :)

  And, the talk is, indeed, great ...

⬐ Bluestein

"Whether you're developing a small web site or a large-scale distributed system, Erlang and Elixir are great tools for building server-side systems. Both languages can help you start quickly and deal with highly complex problems at a large scale. This talk aims to explain what makes Erlang and Elixir suitable for such tasks. We'll look past the syntax and the ecosystem, focusing instead on the concurrency model of these languages.

Combining a bit of high-level theory and a couple of demos, we'll examine some important properties of Erlang/Elixir concurrency, with a special emphasis on why these properties are very relevant when building highly available systems. The talk targets experienced backend developers, but no knowledge of Erlang or Elixir is required. You will walk away from [...]"

For completeness, you should check out Elixir and Phoenix (channels and presence) for the server. Easy websockets, isolated player processes, non-blocking VM, plus deep introspection for debugging issues. https://youtu.be/JvBT4XBdoUE. We see more and more indie games being built with Phoenix LiveView.

I can recommend watching this[0] to understand BEAM design better.

[0] - https://www.youtube.com/watch?v=JvBT4XBdoUE

⬐ whiskeytuesday

+1, probably my favourite talk on the BEAM too, even allowing for the fact that listening to Joe Armstrong himself was always a distinct pleasure.

⬐ mrlucax

Really great talk! He talks about some problems about the BEAM Distribution, but didn't get into details about it. Do you have any idea about those problems?

That sounds like one of Saša Jurić's talks:

GOTO 2019 • The Soul of Erlang and Elixir • Saša Jurić

https://www.youtube.com/watch?v=JvBT4XBdoUE

https://elixirforum.com/uploads/default/original/2X/a/a2524c... (also from Saša Jurić's book)

⬐ jamil7

Thats it! Thank you!

⬐ vram22

Thanks.

⬐ stevbov

To be perfectly clear, in his talk he says that image is an ideal endpoint that is possible - but that ideal is not really there yet.

https://www.youtube.com/watch?v=JvBT4XBdoUE

⬐ s3cur3

This talk is what persuaded me to choose Elixir for our MMO game server. Best technical decision I’ve ever made. Saša Jurić’s book on Elixir is also great (although not strictly necessary considering how amazing the official Elixir intro tutorials are).

⬐ iforgottherest

Clicked it because the hyperlink wasn't clear from the description.....

GOTO 2019 • The Soul of Erlang and Elixir • Saša Jurić

This talk feels, as a coding analogy, a lot more like throwing ingredients into a recipe than architecting an abstract machine.

⬐ probotect0r

This is what sold me on Elixir. I promptly purchased his book which was also amazing.

⬐ realusername

My favorite talk is the Live View twitter demo on my case: https://www.youtube.com/watch?v=MZvmYaFkNJI.

Never before I've been so bluffed by a new technology, he's building a small twitter clone in 20min with live reload, browser sync and everything, this would take days with any other traditional tech I know.

If you're interested in Erlang/Elixir, you might want to watch this video: https://www.youtube.com/watch?v=JvBT4XBdoUE (GOTO 2019 • The Soul of Erlang and Elixir • Saša Jurić)

It shows off the power of the BEAM VM -- and kind of makes you stand back and go "whoa.. how do they do that?"

I would find a list of good talks much more useful. For example, 'The Soul of Erlang and Elixir' talk by Sasa Juric [1] really sold me on Elixir.

[1] https://www.youtube.com/watch?v=JvBT4XBdoUE&t=1s

⬐ itsAWindowsSys

+1 for using YouTube for conference talks more than "watch me program" type videos. I love browsing through all of the Pycon [1] talks for example. It's annoying that each year is a different channel though. Oh well.

There are also a lot of other great videos in the channel [2] where the talk you mentioned came from.

[1] https://www.youtube.com/channel/UCMjMBMGt0WJQLeluw6qNJuA

[2] https://www.youtube.com/channel/UCs_tLP3AiwYKwdUHpltJPuA

One major thing would probably be OTP, which would be very difficult to implement as-is on top of Go. A recent comment thread [1] over at Lobste.rs explored this in more depth.

Saša Jurić made an absolutely wonderful hands-on presentation [2] of what (oftentimes rather unique) value propositions Erlang/Elixir/BEAM bring to the table. It's a very tightly-packed presentation, but I strongly recommend having a look if you're curious.

1. https://lobste.rs/s/ntati1/even_go_concurrency_is_still_not_...

2. https://www.youtube.com/watch?v=JvBT4XBdoUE

There's this talk, The Sould of Erlang and Elixir, that opened my eyes to what makes this platform great:

https://youtu.be/JvBT4XBdoUE

It's not so much about what Erlang, as a language, "offers that Go does not". It's about the BEAM VM that runs it that makes all the difference.

I very much recommend watching the talk if you're interested in the topic :)

⬐ coder543

I watched that talk just now, and it was basically a retelling of everything I really like about Go. Except, Go is statically typed, and a lot faster than BEAM in benchmarks that I've seen. Go is also a very "normal" language, so it's usually easy to onboard developers into it, no matter what their background is. JetBrains GoLand makes the IDE experience essentially impeccable. I can refactor code with a few clicks, and GoLand precisely updates all references thanks to the static type system.

Go can run massive numbers of Goroutines on a single system, and it preemptively timeslices individual Goroutines, so I'm never worried about any single task preventing other tasks from making progress, which was like 75% of that talk.

Go has a fantastic production runtime debuggability story, which was the other 25% of that talk.

Go's pprof support feels magical: https://golang.org/pkg/net/http/pprof/

You can connect to running systems and see flamegraphs of where CPU time is being spent, you can take traces of the goroutines to understand why the system is idle when you think it shouldn't be, you can see how much memory different parts of the system are using, and so much more. It's not mentioned as prominently as I think it should be, but `go tool pprof` has this `-http=:` option that will open a web browser to let you just see everything graphically and click around, and it's an amazing experience.

There were really only three things in there that Go doesn't offer:

1. Forcefully terminating processes

Go doesn't allow anyone to forcefully terminate a Goroutine except the Goroutine itself. Personally, if I identified a buggy Goroutine, I would just focus on writing a patch and deploying it. Manually terminating a process is cool, but then the user types "-1" again and again, and you're just sitting there playing whack-a-mole instead of fixing the problem and deploying the fix.

2. Patching the running system without restarting it

The advantages of this are difficult to enumerate. It sounds nice and clean conceptually, but I would worry about existing processes / goroutines being in unexpected states because of erroneous code that they had previously run. Having a way to gracefully shut down a system and restart it without abruptly terminating responsive client connections is much more predictable to me.

3. Seamlessly distributed execution

He briefly talks about distributed Erlang towards the end of the talk. but mentions that it has serious problems and that a number of people don't recommend using it in production. Personally, even if it were nearly perfect, I think the benefits of it are limited. It's not difficult to have nodes communicate with each other using any number of other things, including proper RPC libraries all the way down to just raw TCP connections. The hard thing is service discovery, and I don't think distributed erlang meaningfully solves the problem different than other solutions. If everything is running homogeneous code, why bother your nodes with knowing about the existence of other nodes? Just use a load balancer, and then do everything internally on each node. If your systems are heterogeneous, you need a way to send the right messages to the right nodes, and that's where a named load balancer for each type of node can do just fine.

⬐ tokenrove

I've worked a bunch in both, and Go really doesn't compare; you can't judge based on watching a talk. I find it much, much easier to write safe, robust code in Erlang, and operationally, you have so much more power to inspect the state of the BEAM. Another key BEAM thing is the process heap, which amounts to region allocation when used carefully. The reduction mechanism used for scheduling processes, though crude, is also still much lighter-weight than Go's approach.

On the other hand, Erlang is a weird language and a weird environment, that takes a lot of time to really understand, so I understand why it's never going to "win", and golang tools and the runtime will continue to improve. YMMV.

⬐ sodapopcan

I only started with Elixir in recent months and it's the first language that has ever made me comfortable about writing concurrent code. I didn't spend a lot of time with Go, but the idea of just calling a function that is now running in parallel but was always disconcerting to me. Of course, I could have spent more time with it and gotten more comfortable learning the ins-and-outs, but Erlang/Elixir's addressable processes running with their own stack/heap/gc and passing messages between each other is something that clicked very quickly with me. It's such an incredibly simple idea. For being a "weird" language, I think there is a lot of power the simplicity of its design, especially around learning. You just have to get over the weird syntax (which is a hot topic).

For transparency, I've never written any production code in either Elixir or Go.

⬐ rightbyte

The funny thing is that Erlang is the Ericsson language. Go is the Google language.

Ericsson never had the Google "street cred" or K& not R. Ericsson is boring phone interchanges and boring radio. Google was hipster latte with software freedom before it turned out to be worse than MS, IBM and Oracle combined.

One might wonder what could have been without all the smoke and mirrors.

⬐ coder543

> The reduction mechanism used for scheduling processes, though crude, is also still much lighter-weight than Go's approach.

I'm not sure how it could be lighter weight than what Go has implemented natively at the machine level, instead of in what amounts to a bytecode interpreter. If you have any links, I would be very interested to read more.

> I've worked a bunch in both, and Go really doesn't compare

I respect your experience, so it is probably a failure of imagination on my part that I simply can't imagine how the BEAM tools could be that much better. I haven't had a problem with a rogue task (or really anything else) in production with Go where I was unable to figure out what was going on extremely quickly. The tooling has been amazing for me, unlike basically every other language I've worked seriously with. The YouTube talk certainly didn't do the BEAM tools justice if they are that much better.

With Go, I actually get really nice GUIs to look around the running Go process and analyze the situation, instead of just an interactive CLI session where I have to craft my own commands to find the top tasks and such. The Go pprof tools also work as an interactive shell (but for analysis, not for remote arbitrary code execution), but I would rather just have the flamegraph in front of me 99% of the time. I fully admit that I've never had a chance to use Erlang/Elixir/BEAM in any meaningful way, but I have tried to understand what they offer, and I haven't seen the compelling magic that some people talk about.

Now, if someone is running a Go service without the HTTP pprof server running on some port that they can access, then yes... it wouldn't even come close to comparing to what BEAM offers when you have the option to connect to a running BEAM instance.

⬐ tokenrove

And, note, I'm not saying Go is bad, here, just that there is a lot that is underrated and misunderstood about Erlang.

On the Erlang side, check out the BEAM Book chapter on scheduling: https://blog.stenmans.org/theBeamBook/#CH-Scheduling and the core scheduling loop in the BEAM: https://github.com/erlang/otp/blob/master/erts/emulator/beam...

On the Go side, check out https://github.com/golang/go/tree/master/src/runtime/proc.go and the asm* implementations.

It's been a little while since I looked at it, but I recall that much less state had to be saved in an Erlang process switch in the usual case; I seem to recall it can be done in a handful of instructions in many cases. Go of course has to save a bunch of registers much as you'd have to do in any native context switch.

Edited to add: it can be useful to look at that part of the BEAM disassembled in objdump or gdb, to appreciate it, since it's hard to tell how much work is happening with all those macros.

Maybe https://ferd.ca/the-zen-of-erlang.html

Or this kind of fun video https://youtu.be/JvBT4XBdoUE

⬐ codemonkey-zeta

Thanks for sharing. I found the second video to be very entertaining as someone who hasn't seen a lot of Erlang in action. At 32:38 in the video he mentions some issues that plague BEAM which cause some engineers to eschew its use in production. Do you know what issues he could be referring to?

⬐ conradfr

Actually yes he detailed it a bit here https://www.reddit.com/r/elixir/comments/bronlx/discover_wha...

> [and particularly, videos] go out of their way to discuss how process management and IPC communication works

After watching a bunch of videos on BEAM/erlang/elixir I came to the conclusion that it isn't a platform for computation, it's a platform for communication. The best video (by far) was The Soul of Erlang and Elixir • Saša Jurić https://www.youtube.com/watch?v=JvBT4XBdoUE

Two shallow benchmarks:

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

https://www.techempower.com/benchmarks/ (phoenix is at 175)

I replaced a python/uwsgi stateless stack with an elixir stateful system. Flipped it over and it worked flawlessly, and hasn't died once in several months of operation. Latencies are way down, and operation of it is simpler. Full deploy to the head node is one command, not an ansible script.

I guess part of it is that I'm lazy and haven't learned supervisor strategies like kubernetes (the place I work doesn't deploy kubernetes anyways)

This video is instrumental in giving me the confidence to do what I did: https://youtu.be/JvBT4XBdoUE

Elixir (really, the Erlang VM it's built on) is designed for highly concurrent, reliable systems. For my org, building a distributed webapp, we chose it over the languages you listed because of how much you get "out of the box" with Elixir.

Saša Jurić's talk "The Soul of Erlang and Elixir" (https://www.youtube.com/watch?v=JvBT4XBdoUE) is a good introduction to why someone might choose Elixir. The payoff is the slide (https://imgur.com/gallery/G6lfUW6) where he compares the stack of a "traditional" web app (Nginx + Ruby on Rails + Go + Redis + Mongo + Cron + Upstart) to the Erlang rewrite of the same app, which is... just Erlang.

This isn't to say that Erlang ships a complete replacement for Redis, Mongo, Upstart, etc.—but it provides most of the functionality that most apps will need, such that your dependency tree can be much simpler. It's really nice to not have to become an expert in all the infrastructure surrounding a normal web app and instead just learn Erlang/Elixir.

⬐ jamil7

I hadn't seen that slide, it's pretty compelling. Is there a story for typechecking or a typed language that runs on BEAM? This is one aspect that holds me back a little.

⬐ s3cur3

Erlang's type checker Dialyzer has an Elixir wrapper called Dialyxir (https://github.com/jeremyjh/dialyxir). As someone coming from C++, Dialyzer leaves a lot to be desired—I still miss C++ type checking—but it's better than nothing.

The Norm library (https://github.com/keathley/norm) is an attempt at higher-level, more expressive type checking which I think shows a lot of promise, but I've not adopted it in a real project yet.

⬐ rkangel

> It's really nice to not have to become an expert in all the infrastructure surrounding a normal web app and instead just learn Erlang/Elixir.

It is also so much easier to test when all of your architecture is in one form. You don't have to spin up whole deployments on real infrastructure to see if the parts still talk - your normal local unit testing architecture gets you much more confidence.

The Soul of Erlang and Elixir, by Sasa Juric

video: https://www.youtube.com/watch?v=JvBT4XBdoUE

HN discussion: https://news.ycombinator.com/item?id=20942767

Preventing the Collapse of Civilization, by Jonathan Blow

video: https://www.youtube.com/watch?v=pW-SOdj4Kkk

HN discussion: https://news.ycombinator.com/item?id=19945452

⬐ srik

While on that topic, I also enjoyed this talk by Fred Herbert recently — Operable Erlang and Elixir. https://youtu.be/OR2Gc6_Le2U

⬐ anderspitman

Jonathan Blow's talk was the first one I thought of.

⬐ tyzerdak

Shit talk. I find it's funny to look at dude that say 1 hour 1 sentence but can't say clear about alternative / solving this.

⬐ anderspitman

Ok

⬐ muhic

+1 for both talks

Saša Jurić is fantastic at condensing lots of information in a 1 hour talk without losing the audience, he gave another great talk this year called Parsing from first principles (https://www.youtube.com/watch?v=xNzoerDljjo).

> explore Go or Elixir

I have never seen a good argument for using golang for business logic. If you are writing the actual server then sure, use golang. If you are writing some high-speed network interconnect, use golang. Some crazy caching system, sure use golang. The public WS endpoint, use golang.

But if you need to access a DB with golang for anything more than, like, a session token, then you made the wrong choice and you need to go back and re-assess.

Elixir is in the "germination phase" and I predict massive adoption in the next 5 years. It is a truly excellent platform, every fintech company I know at least has their toe in the water. Everyone I show this video to [1] just says "well, shit."

[1] https://www.youtube.com/watch?v=JvBT4XBdoUE

⬐ yurish

What is wrong with accessing DB from golang?

⬐ spamizbad

Nothing. But I imagine with “business logic” you’d favor expressiveness over speed and type safety.

I feel like this talk does a great job of answering that question - The Soul of Erlang and Elixir by Saša Jurić (https://www.youtube.com/watch?v=JvBT4XBdoUE)

⬐ preommr

lol, there's the hot reloading.

But that's an interesting talk. I didn't have time to go through the whole thing but the core of it seems to be the pros of erlang like BEAM's high concurrency advantages and uniformity of tools running on top of otp.

It's an interesting argument.

⬐ albydarned

BEAM is amazing. The scaling is insane. I built a business texting platform using it (https://pigeonsms.com).

As we brought on our first few customers I kept expecting we would need to increase our hosting capacity. Here we are 2 years later and we are still running the whole thing with our initial hosting setup.

⬐ abledon

Neat business, do you use twillio elixir api?

It seems it definitely hits a niche market in the current time but say in 2030, I’d imagine most people will have migrated over their phone setups to a modern text-supported by default one ?

⬐ ak47surve

Interesting. Mind sharing a little bit about your production hosting setup?

⬐ brianpgordon

In the talk he glosses over what he sees as serious flaws in Erlang distribution. I did some digging and it turns out that he elaborated on Reddit:

https://www.reddit.com/r/elixir/comments/bronlx/discover_wha...

⬐ olah_1

>First, in my opinion distributed BEAM is mostly intended to run on a network which is fast and more reliable (such as local network). While in theory it can also work on a less reliable/slower network (e.g. geographically dispersed machines connected via Internet), in practice you might experience more frequent netsplits which could cause various problems, such as worse performance or less consistency.

This is exactly why I wasn't excited about LiveView[1]. It felt like a step backwards in terms of human-centric design. Another tool that makes us consider our network bars first and our life second.

In general, I'm kind of disappointed that Elixir isn't leading the way on decentralized and offline-first technology, but I guess it's a limitation of BEAM running on small/low-powered devices?

[1]: https://github.com/phoenixframework/phoenix_live_view

⬐ rozap

LiveView is not distributed erlang.

⬐ di4na

This work tend to happen at a lower level, ie in erlang. Check Lasp https://lasp-lang.readme.io/ and its Partisan distribution networking strategies.

⬐ delta1

How does live view have anything to do with network connectivity?

If you need a live server to provide you with updates what difference does it make whether it's live view or some other json/websocket setup?

⬐ olah_1

LiveView makes basic functionality of websites only work if you have a constant, high quality connection to the server.

Even things like UI changes require the server connection.

⬐ elcritch

Have you actually tried it on a cellphone though? LiveView works decently enough for me over my cell phone connection even when using a vpn. I wouldn’t want say a todo app made using LiveView, but for dashboards and other sites which require server data anyways it does fine. Oddly I find LiveView can even feel subjectively faster than an SPA making a discrete request and then rendering the result.

⬐ olah_1

"but for dashboards and other sites which require server data anyways it does fine."

I understand that apps require data anyway, but I find offline-first design to be much more compelling for most apps.

Things shouldn't halt or break if your connection goes out or is spotty.

Maybe I'm wrong and the future will be everyone cooking in a giant microwave of internet waves :)

⬐ brightball

I guess the question is how low powered and small you want to go? Nerves seems to be flourishing.

⬐ olah_1

That's a good point. But does Nerves advance any kind of offline-first design? It still assumes a constant and high quality network connection, right?

⬐ brightball

Nerves doesn’t make any assumptions about connectivity.

https://nerves-project.org/

⬐ kungfooguru

Yes, check out http://partisan.cloud/. Erlang distribution was not designed for the cloud.

⬐ davidw

Note also the response from José Valim.

⬐ toast0

From that thread:

> Finally, I've seen various reports that the practical size limit of a BEAM cluster is in the range of 50-100 nodes. The reason for this is that BEAM cluster establishes a fully connected mesh (each node maintains a TCP connection to all other nodes), so at some size this starts to cause problems. As far as I know, the OTP team is working to improve this, but as of OTP 22 it is still not done.

I've run clusters of 1-2k machines at my last job (maybe it was bigger, but I can't remember for sure). Holding a TCP connection to each other node is not a problem --- we certainly had a lot more connected clients than connected servers, tuning memory for buffers can be an issue on low ram systems. Global can get to be a problem, I'm not sure of the state in open source OTP, but if you have multiple nodes contending on the pg2 global lock for a group, it can get really slow; there's ways to make that better, but you do need to be careful not to introduce new deadlocks. If it's still using the simple method of try to lock everyone, if unsuccessful unlock and wait a bit and try again doesn't work well under significant contention or if one (or more) nodes is unhealthy and running slowly, but staying online.

The quality of network needed really depends on your tick timeouts, and the amount of data you're transmitting. Dist will work with slow and lossy networks as long as it can get a ping transmitted often enough. I think the default tick time is 30 seconds, and four failed ticks disconnects, so you really just need pings coming through once every two minutes, and for your OS not to give up on the TCP connection.

It wouldn't work well for mobile, but between two reasonably connected datacenters, it should be fine. Anyway, dist should only be used between nodes at the same trust level --- anything you can do on one node can be done from the other node; consider it a bidirectional shell. I've debugged plenty of cases where an intermediate link was congested resulting in very low throughput, and tens of minute message delays on dist; it was still working ok --- just anything synchronous would take forever.

⬐ rramadass

See also: Erlang for bare-metal embedded systems - https://www.grisp.org/

Has anybody here played/worked with this ?

⬐ elcritch

Not with the GRiSP board, but I’m working with Nerves and embedded Elixir. Even using Erlang distribution to communicate between devices over an internal Ethernet lan. Works pretty well actually.

⬐ bmalehorn

The speaker mentions that Redis, MongoDB and background jobs were replaced by Erlang.

What does he mean by that exactly? Erlang provides some persistent state storage? Or is he just saying he used Erlang database drivers to access Redis / MongoDB?

⬐ bennyp101

You can use ETS and Mnesia, and overall it is built to be distributed so you can pass messages between processes/nodes without needing something like RabbitMQ.

Most of the time Erlang and OTP provide what you need already without having to reach for an external tool. (Obviously depending on your use case)

⬐ namelosw

Furthermore, job processing is much easier with Erlang because the processes and the scheduling mechanism - there could be no job, only processes doing their things.

⬐ lostjohnny

> Erlang provides some persistent state storage?

Yes, it's called Mnesia.

If you need non blocking, concurrency aware memory storage that can eventually be serialized to disk, look at ETS/DETS. They are part of the stdlib.

But you could also use Riak[1], which is entirely written in Erlang.

[1] https://en.wikipedia.org/wiki/Riak

⬐ davidw

Looks interesting; wish there were a transcript. Example code appears to be here: https://github.com/sasa1977/demo_system/tree/master/example_...

⬐ kuzee

This is magic

⬐ foobar_

How does this compare to Go?

⬐ lostjohnny

Immutable, share nothing, completely separated architecture.

Go has no such guarantees.

⬐ mikece

BEAM is such an awesome and beautiful VM. Not only is concurrency at a scale unimaginable in other languages the norm for BEAM, but the isolation of running processes didn't hit me until I asked an Elixir dev about running BEAM in Docker. The confused look told me I was asking an odd question: processes in BEAM are as isolated from each other as console apps running in different VMs in VMWare server. These processes can send messages to each other but they cannot access each other's memory. It was truly a mind-blowing revelation!

⬐ kerkeslager

> The confused look told me I was asking an odd question: processes in BEAM are as isolated from each other as console apps running in different VMs in VMWare server. These processes can send messages to each other but they cannot access each other's memory. It was truly a mind-blowing revelation!

Not quite: the messages sent are kept in shared memory (which is reference-counted). Cycles can't occur due to immutability. But with everything else, you're correct. One reason for this, is that every thread has its own garbage collector! This allows multithreaded garbage collection without stop-the-world pauses.

Source: Did a bunch of research into BEAM for implementing my own multithreaded GC.

BEAM really is beautiful!

⬐ ramchip

> the messages sent are kept in shared memory (which is reference-counted)

Only large binaries and slices of them use reference counting, other things are copied. And of course, if the process is remote, everything is copied :)

⬐ kerkeslager

Interesting. I'm glad to be wrong because then I learn something!

I'm not sure if you have answers, but I have questions:

1. How do they determine what size to start sharing/refcounting at? Now that I'm actually considering it, it's obvious that they'd copy small stuff--an obvious lower limit would be sizeof(<reference_counter_type>) + sizeof(<pointer_type>) because smaller than that, it actually is faster/less memory to copy. But the numbers you're mentioning are higher than that--is there some metadata being stored beyond the reference count and pointer?

2. Is copied data stored in thread storage and garbage collected by that thread's garbage collector? That would be my guess.

Obviously the source of truth for all these questions is the code[1], but it's been a while since I dug in there. Maybe it's time to go again.

[1] https://github.com/erlang/otp

⬐ elcritch

Regarding question 1, my understanding is that all non-binaries messages are copied regardless of size. Only binaries ever use shared memory. Early in BEAM development they debated optimizing shared memory for local messages but ended up not using it, partly due to the extra code complexity. Always copying messages means there’s little difference between local or remote messaging code paths and removes ref counting. Haven’t read the source myself regarding that, but it would be an interesting price to study.

I’d bet optimizing local message passing would be a good feature of a BEAM implementation in Rust since the ref counting and other details would be less worrisome. Another research topic would be how BEAM would compare to say Java/Dotnet runtimes on cpu cores with lessened hardware shared memory guarantees and relying on message copying between cores instead.

⬐ ramchip

1. See: https://blog.stenmans.org/theBeamBook/#_the_tagging_scheme (search for "64 bytes") and code: https://github.com/erlang/otp/blob/OTP-22.0.7/erts/emulator/...

2. See: https://www.erlang-solutions.com/blog/erlang-garbage-collect...

There's no thread-local GC, it's really per process. For instance a scheduler thread that's running a process and hits the need to do an expensive GC may delegate the task to a dirty scheduler (another type of thread, dedicated to long blocking tasks), and go back to running other processes in the meanwhile.

It's frequent for processes to move around between schedulers, they steal work from each other.

⬐ toast0

Not all refcounted binaries are large btw. It's pretty easy to construct a refc binary [1]

   Bin0 = <<0>>,                    %% 1
   Bin1 = <<Bin0/binary,1,2,3>>,    %% 2

Bin0 was on the process heap, but Bin1 is a refcounted binary with 256 bytes of storage. If you're putting Bin1 into ets/mnesia, use binary:copy/1 to get a clean copy that doesn't reference a lot of extra storage.

[1] http://www1.erlang.org/doc/efficiency_guide/binaryhandling.h...

⬐ vvanders

> This allows multithreaded garbage collection without stop-the-world pauses.

This was one of the things that really floored me when I learned about how BEAM was designed. It's incredible that how all the seemingly small changes come together in a way that works really well for the problem domains Erlang was used for.

⬐ macintux

This is exactly the theme of my favorite talk. Erlang has some cool features, but it's the way they enhance each other that's amazing.

https://youtu.be/E18shi1qIHU

⬐ arthurcolle

Anyone interested in this subject would be extremely well-served by picking up a copy of the late, great Joe Armstrong's "Programming Erlang." Putting together a minimal "server" and then slowly evolving it into a GenServer is incredibly fun and rewarding. And then the kicker is you get hot-swapping for (almost) free! Learning about Erlang hot-swapping was probably one of my most favorite moments in my brief time using Erlang before moving onto Elixir, where you get it all right out of the box.

BEAM is truly a modern engineering marvel.

⬐ onezerozeroone

Wouldn't the point of running in Docker still be to guarantee version, environment, and config compatibility? Just because you run an Elixir app on your local machine doesn't mean it will work worry-free on Prod. Without Docker, you might have a different version of BEAM installed (or none at all), etc.

⬐ tidepod12

This is a question that I see hotly debated a lot in Elixir circles. BEAM and the Elixir toolset do have a lot of tools that overlap with Docker, so there's a lot of people that advocate for "not bothering" with Docker, while there's other people that are familiar with Docker (and perhaps not as familiar with BEAM) that will tend to use Docker. They both have more or less the same capability to deliver the same functionality.

IMO, in most use cases it makes sense to just use whichever you're more familiar with and will be more productive with. After all, that is the entire point of said tools...

⬐ kungfooguru

They really do not overlap at all. I don't understand why the comparison with docker and Kubernetes continues.

Running BEAM on docker in kubernetes is great, a lot is added and operations smoothed by doing so.

⬐ tidepod12

Elixir Releases/Distillery definitely do overlap with Docker deployment techniques. The BEAM also has scaling capabilities that are tantamount to K8s horizontal scaling.

⬐ kungfooguru

Deployment doesn't overlap. Erlang releases bundle up your code but you still need libraries like openssl, to have a nice bundle that can actually just run anywhere Docker is the way to go.

And horizontal scaling between the two do not overlap either, not sure what you mean. Packing containers (processes) efficiently into a cluster of nodes is not something Erlang does.

Edit: Erlang/OTP does offer fail over for what it calls "distributed applications" but based on a static set of nodes -- not horizontal scaling, anymore than other languages/frameworks do by letting you spawn new instances...

⬐ tidepod12

>Deployment doesn't overlap. Erlang releases bundle up your code but you still need libraries like openssl, to have a nice bundle that can actually just run anywhere Docker is the way to go.

And with Docker you still need libraries like, y'know Docker for your code to run.

>Packing containers (processes) efficiently into a cluster of nodes is not something Erlang does.

How much have you used BEAM? It definitely does do that. Scalability is one of Erlang/Elixir's primary benefits, and node clustering is exactly how it does that.

⬐ kungfooguru

> How much have you used BEAM?

Over a decade professionally.

The closest thing Erlang has to it is pool http://erlang.org/doc/man/pool.html

⬐ ZephyrP

BEAM doesn't do everything containers or a container orchestration system can, but in view of my relatively short time as a regular Erlang programmer (definitely less than 10 years), the spirit of tidepod's point accords with my experience (with a weaker interpretation of his term "node clustering"). BEAM and OTP don't do automatic load distribution between nodes, but can be fashioned to do so easily with application support.

We never used pool. The nodes were mapped onto heterogenous machines sharing the host with a 3rd-party daemon. It's configuration changes even took place through a module update hook written in Erlang itself. We both deployed new code and distributed work "manually" across them entirely on OTP.

[NOTE] It it surprising, or was to me, that there are problems with having a fairly small number of nodes fully connected. I'm lucky enough to have avoided learning this the hard way, but imagine this could serve as a painful backbone to an "Erlang deployment war story".

⬐ kungfooguru

Sure, you can write it yourself, and probably it is even easier to write in Erlang -- up to a limit due to the issues with distributed Erlang discussed above.

I worry about, and have seen this both in the first hype phase of Erlang a number of years ago, the misconception about what Erlang offers and the resulting frustration, blaming it on the tool and quitting.

Hoping in the chapters I'm working on for https://adoptingerlang.org/docs/production/ I can better explain the benefits of running on k8s (or similar), while also making clear it certainly isn't the right choice in all cases.

- Tristan

⬐ elcritch

Nice synopsis in the docs. I’d suggest that instead of saying k8s and OTP don’t overlap that it’s more that "they don’t completely overlap" Fenn diagram style. That’d be a bit more nuanced answer and help people judge which category they fall into.

There are many cases where having an Erlang cluster and doing ‘naive pooling’ at the application level would get you 80% of what k8s + routing layer would get you. I’m assuming more of an an all Erlang/Elixir environment which many small startups could get by with. Even then Docker + Fargate or whatnot would still simplify deployment. Personally I harbor a secret desire to see if I could replicate part of the k8s interface using Nerves images + some otp tooling. Probably better things to do with my life though. ;)

Also by ‘heart’ are you referring to how ‘epmd’ works or Erlang distribution?

⬐ kungfooguru

For 'heart' here I'm referring to the heart program that'll start if you run erl with `-heart` http://erlang.org/doc/man/heart.html -- it will restart a down node.

I think part of the confusion is you in general don't need k8s, but when your size and requirements get to a point that k8s makes sense (which arguably lowers as hosted k8s becomes better) it is not in conflict with your also use of Erlang/Elixir.

I find distributed Erlang much nicer in k8s env (I don't have to maintain it obviously) where I get an IP per node, can add a k8s service making it possible to use DNS SRV queries to find all other nodes and letting k8s worry about where pods run and keeping them up.

Plus there is configuration management and consistent storage (resources in etcd).

⬐ lostjohnny

BEAM tools are kubernetes done right for a single environment.

I say done right because kubernetes doesn't allow hot code reload without restarts.

⬐ marci

Isn't that the purpose of tools like https://erlang.mk/guide/getting_started.html#_getting_starte... ?

Edit: and for cross-system dev (e.g. debian to windows) -> https://erlang.mk/guide/cross_compiling.html

⬐ lostjohnny

you still need to compile your system on the same combination of OS/libc

libc builds don't work on Alpine

⬐ walterstucco

Correct, but you still need to compile the release on a similar system.

For example if your dev machine is Debian it won't work if the deploy machine is Alpine (similarly Windows builds won't work on Linux or Mac).

⬐ onezerozeroone

I think OTP Releases would still depend on the target host being configured correctly and having all dependencies installed that your application needs to run.

Containers are all about consistency of the environment. For example, maybe your Elixir app wants to log out to something like Splunk or DataDog...if those agents aren't installed on the host, it doesn't matter what facilities BEAM provides. Or maybe your app depends on certain configs, libraries, or files being available at specific paths on the system. Things like that would be captured in the image your containers are running and guarantee consistency across machines and environments.

⬐ kungfooguru

Right, they do, that is why containers are still useful to releases for deployment.

⬐ marci

My bad, I sould have specified, I was only addressing "Without Docker, you might have a different version of BEAM installed (or none at all)"

⬐ onezerozeroone

Ah, makes sense!

⬐ didibus

I'm not a BEAM user, so I may be wrong, but as I understand it, that's not really true. Two VMs are isolated at a much more fundamental level which matters for security and containment. For example, two BEAM process, while they don't have direct access to each other's memory in the sense of say global shared variables, still have access to the same physical memory, the same disk partition, the same file descriptors, the same sockets, etc..

So I'd imagine there's potential utility in still running inside docker, as a means to deploy the expected OS, the correct assets, the version of BEAM itself, etc. And there are reasons to isolate all that with VMs, to prevent sharing of machine resources like disk, file descriptors, sockets etc.

⬐ toast0

> The confused look told me I was asking an odd question: processes in BEAM are as isolated from each other as console apps running in different VMs in VMWare server. These processes can send messages to each other but they cannot access each other's memory.

The isolation is really not that high. erlang:process_info[1] allows one process to observe an awful lot about another. sys:get_status [2] often works (depending on software design choices) and provides even more data. I would not run two applications that needed isolation for security from each other in the same BEAM or as part of the dist network.

[1] http://erlang.org/documentation/doc-5.7.4/erts-5.7.4/doc/htm...

[2] http://erlang.org/doc/man/sys.html#get_status-1

You also have to consider that the whole BEAM philosophy is "let it crash".

On a "normal" language you have to deal with all error paths or your app goes down. On the BEAM you handle the happy case and common and unexceptional errors. Otherwise, let it crash.

Example: if your app depends on a database that once in a blue moon is unreachable, don't test for connection errors. Just assume the connection was successful, and if it wasn't, most of the time once the process restarts everything is fine.

I have an Elixir app in production, I get Sentry errors once in a while about some weird state outside my control (we work with external APIs) that caused a process to crash, I just ignore them and go on with my life.

EDIT: I highly recommend this video from Saša Jurić: https://www.youtube.com/watch?v=JvBT4XBdoUE

⬐ mercer

I'm a big fan of the 'let it crash' approach. A lot of code I write is hacker-ish stuff where I just want to get something done and I don't need it to be pretty. I've been amazed at how many of my projects in Elixir just kind of keep chugging along despite the fact that I mostly bothered coding only the 'happy path'.

A while ago I wrote some code that retrieves data from various online sources (cryptocurrency exchanges) at regular intervals. A friend of mine wrote essentially the same code in Python. His code was littered with try/catch statements and even with those the app would often crash because these API's sucked ass. As a result, there'd be gaps in the data, or he'd have to restart the app or set up whatever restart mechanism one uses in the python ecosystem.

Meanwhile my elixir project just kind of kept running and retrieving data, and barring permanent changes in the various API's, it would keep collecting data even in every once in a while the endpoints would misbehave.

My typical approach to projects is to tinker and cobble things together but then also overly worry about all the things that can go wrong. I know I'm not alone in being this way; I've met plenty of developers who seem to be a combination of just wanting to get results but then worrying excessively about everything that could go wrong.

Elixir is singular in how it allows me to approach projects in this manner while still feeling confident that it'll work well enough. And while that might not be the most common use-case, it's really helped reinvigorate my enjoyment of programming.

If anyone is on the fence on learning Elixir / Phoenix (Elixir's most popular web framework library) please do yourself a favor and give it a shot. If you're on a time crunch and want the best bang for your buck on "why bother learning yet another language to write web apps in?" then watch this talk called "The Soul of Erlang and Elixir": https://www.youtube.com/watch?v=JvBT4XBdoUE

I'm not affiliated with the language but I just want to throw out that I've been around since the Geocities era as a web developer and I haven't seen a tech stack or platform this well thought out and so in tune with what makes developing (and testing!) modern web applications an absolute joy. There's really something special here. I see myself sticking with it for 10+ years with 0 regrets.

The best part about it is everything is ready today (and was before 1.9 too btw).

⬐ freehunter

My preferred way of learning a new language/framework is watching someone walk through creating a real app in that language/framework so I can see all the folders/files I need to touch and the thought process moving step-by-step. Unfortunately I haven't found much in the way of that for Phoenix, the stuff on Udemy is pretty outdated last time I checked. Any recommendations?

⬐ AlchemistCamp

That's exactly what I do at https://alchemist.camp. Most the videos and all the intro-level ones are free.

I've also put up about a hundred screencasts at http://youtube.com/alchemistcamp.

⬐ PopeDotNinja

Looks like a cool site. I'll check it out. My Elixir is Good. My Phoenix is not.

⬐ AlchemistCamp

I'd say you're in good shape, then. Phoenix is really well-designed and doesn't have a lot of "magic" in it.

⬐ aczerepinski

I’m building a side project in Phoenix right now. For me the learning curve is wondering whether Phoenix has a built-in or “everybody does it this way” solution to the problem I’m facing or whether it’s something I should implement myself.

In Rails the answer is always “use this popular gem” and in Go the answer is always “roll your own”. Phoenix is somewhere in the middle.

⬐ prodiguy89

The good thing about Phoenix is you are not forced to go in any particular convention for everything, you can craft them yourselves from scratch. The bright side of this is you will only every little thing in the implementation and you'll know how to optimize it.

⬐ PopeDotNinja

The one time I really want to override something in a Phoenix app was deleting a response header bolted on by Cowboy (I think it was the server header). I couldn't find a work around, so I did something lame like intercept the formatted response and modify it. Some flexibility in that area would have been nice, but it was a pretty niche corner case on a not too important app, so a time crunch hack was my best option.

⬐ relativeadv

https://pragmaticstudio.com/elixir

https://pragmaticstudio.com/courses/unpacked-full-stack-grap...

These have been my go-to that seem to fit what you're describing. I hope this helps.

⬐ tarunkotia

For event sourcing in Elixir this is a very good walk-through: https://blog.nootch.net/post/event-sourcing-with-elixir/

Another example is: https://github.com/AltTracker/alttracker

It's an implementation of fully functional site. The beauty of Elixir is that the code is very readable.

⬐ nickjj

The way I've learned so far was building my own app while skimming the docs and looking at successful open source projects like https://github.com/thechangelog/changelog.com for inspiration and guidance. Basically error driven development and looking up things as needed. Of course also taking advantage of IRC / Slack / Forums when I get really stuck.

But recently I read the Programming Phoenix 1.4 book (written by the authors of Elixir and Phoenix) and it really filled in a ton of gaps and made so many things click together. I can't speak highly enough about it. In fact, finishing that book today is what provoked me to write my comment here. The book might be exactly what you're looking for as like 5% of the book focuses on the super basics and a toy example, but 95% of the book focuses on building up a pretty real app and layering on features as you go. It's also up to date and feature complete even though the book is labeled as a beta release.

By the time you finish the book, you exercise a ton of interesting things along the way (including how to take advantage of Erlang's VM to build features that would be 100x more effort and probably 10x worse in other tech stacks). You definitely don't just build a simple todo app. I don't even want to say what we build in the end because it feels like a spoiler. I only say that because when I hit the last section of the book I had no idea what was coming and was blown away by what it covers. It literally made me smile for hours just thinking about how badass everything is when it all comes together.

⬐ freehunter

Glowing recommendation, I'll have to check it out!

⬐ m45t3r

I concur. Programming Phoenix (I read the old version, 1.2 or 1.3, don't remember) and it is simply so well thought and helps you understand why Phoenix is not as magic as its appears in the start.

I also recommend you to try to play with Plug. Things make much more sense in Phoenix after you understand how Plug works.

⬐ zerr

The thing is, nowadays, statically typed languages became so ergonomic, there is really no reason to go with dynamic languages.

⬐ enraged_camel

Most people who point to Elixir's lack of static typing as a weakness, or believe their productivity would be adversely affected by it, do so because they haven't actually used the language.

Anyways, this subject occasionally comes up for discussion in the community. This post sums up my thoughts fairly well: https://elixirforum.com/t/static-vs-dynamic-typing/9824/3

⬐ chessturk

I agree. I've sincerely asked for explanations what is missing when pattern matching is so similar to type inference. Combined with guard clauses and "let it fail" philosophy, what is the problem other than boilerplate? Is it lack of mechanical refactoring? Totally honest question for type enthusiasts, my other posts have been met with silence.

⬐ 59nadir

It's a near certainty that I've written an order of magnitude more elixir code than you have and I can say without a shadow of a doubt that elixir's lack of static typing definitely makes it less useful.

Even if you disregard the modeling power that you can get from this there is an upper bound on any elixir project after which any work on it becomes less and less easy to do, as with all languages that lack a static type system. We write assertive elixir code as much as we can, but that doesn't mean that you can actually guarantee anything about a code path that is less traveled.

dialyzer is also not the answer. Oftentimes we'll find ourselves in a situation where dialyzer complains because someone who made a major library simply doesn't use dialyzer. You might wonder why they don't in that case, because it would pick this particular issue up very easily, but the next time you run into some garbage error that dialyzer spat out that you are instantly reminded that you can neither trust it to be correct or safe, so people turn it off.

Elixir has upsides: I think it's solidly the best language for creating servers of different kinds, but it's near useless after you pass a fairly short distance with it. I would absolutely never, in my personal endeavors (as opposed to my working contract) write an elixir server that actually tries to do anything meaningful itself other than just route messages to other servers.

⬐ enraged_camel

> It's a near certainty that I've written an order of magnitude more elixir code than you have and I can say without a shadow of a doubt that elixir's lack of static typing definitely makes it less useful.

You should really read that post I linked, and do a bit more research on this topic. The consensus in the Elixir community (where many members have written an order of magnitude more Elixir than you) is that static typing would be "nice to have", but is not critical by any means. Your post seems to be an attempt to spread FUD.

⬐ 59nadir

I've done all the research I need to, thank you very much, and I was there when this thread was started and much earlier than that. The consensus in the elixir community is hardly relevant when most of the users aren't experienced enough with type systems to know what they're talking about.

If you asked a community of people who primarily have never used languages that don't have garbage collection, they'll likely not understand at all why it could be a bad thing.

Someone who has used a language without garbage collection and is comfortable with basic allocation strategies is very likely to remark that having control of memory allocation and deallocation is very often something that you end up wanting instead of having to re-architect your solution in indirect ways to influence the garbage collector. This comes from having a wider perspective and the elixir community at large does not have this.

Given that the elixir community is also very cultlike it's hardly a productive thing to take what they say as the objective truth.

(I'm not saying this is an outsider at all, I've been a part of the elixir community since 2016. It's not really despite that I am saying these things, it's because of that.)

⬐ enraged_camel

Friend, you're making a lot of assumptions and claims about people you don't know (including me), and have a certain streak of arrogance that is difficult to get past. This makes it impossible to have a productive conversation, if having a productive conversation is your goal.

I advise some humility, as well as some effort to recognize that your opinions sound very dogmatic. You are of course free to like or dislike any language you want, but it is important to understand that going from one's own preferences to grandiose claims about a language's merits and usefulness is a rather big leap.

⬐ 59nadir

> Friend, you're making a lot of assumptions and claims about people you don't know (including me), and have a certain streak of arrogance that is difficult to get past.

This is rich, coming from someone who claimed that anyone who wants a type system in elixir hasn't actually used elixir.

> You are of course free to like or dislike any language you want, ...

This is the most common reaction within the elixir community, to assume that anyone who doesn't put the language on a pedestal dislikes it or hates it. You might want to consider what that actually says about the community and the atmosphere of near unconditional admiration it displays. It's an extremely distasteful attribute for a community to have.

Elixir is useful for shuffling data from point A to point B... That usefulness has its limits, that's it.

⬐ pdimitar

> This is the most common reaction within the elixir community, to assume that anyone who doesn't put the language on a pedestal dislikes it or hates it.

Not sure you and I read the same forum. My reaction to Elixir criticism usually is -- "well, if you dislike it for this or that reason, or simply don't find it useful, then don't use it, there are a lot of other languages out there". Many others do the same.

I haven't noticed cargo-culting on ElixirForum in a long time. You might simply be too cynical to differentiate people whose work life has been genuinely improved (and they express that sentiment on the forum) and people who overhype something you dislike (for valid reasons for your use-cases).

⬐ enraged_camel

> This is rich, coming from someone who claimed that anyone who wants a type system in elixir hasn't actually used elixir.

You may want to read my original post more carefully. I said most people who complain about the lack of static typing haven't actually used it. Obviously there are some, like you, who have experience with the language and dislike the lack of static typing. And that is fine.

I mean, you only need to casually browse HN threads about new languages to find posts along the lines of "this looks interesting, but the lack of static typing means I probably won't use it." Instead of taking a holistic view of the language and its merits and researching its design choices and trade-offs, they adopt a dogmatic mindset and dismiss it outright.

>>This is the most common reaction within the elixir community, to assume that anyone who doesn't put the language on a pedestal dislikes it or hates it.

No, that's just me, and it was a figure of speech. Elixir community is very mature and welcoming, and most people in it come from other languages, and are able to objectively discuss the language, its merits and shortcomings. The post I linked in my OP is a great example.

>>Elixir is useful for shuffling data from point A to point B... That usefulness has its limits, that's it.

I mean, again, that's just your opinion, and it's a simplistic one. I obviously don't know the depth and breadth of your experience (and won't make assumptions about it, like you did about mine), but Elixir is useful for, and excels at, more things than simply "shuffling data". That is why there is so much excitement for it: the use cases it enables, and the elegance and simplicity with which it enables them, are nearly unparalleled. The lack of static typing does not change this fact, even if it can make certain things a bit more cumbersome towards the extreme end of the project size and complexity spectrum.

⬐ AlchemistCamp

What language would you use to write a server "that actually tries to do anything meaningful"?

⬐ 59nadir

Any language with the possibility to express variant types that all can have different structure and have that structure be checked at compile time, properly, will be a better choice for describing simple, medium and complex domains.

Whether or not this is based on nominal or structural typing is less important, because the feature itself only depends on differentiating between what is essentially different cases and having that be done safely and properly at compile time.

Elixir has no satisfying solution to this problem and it likely will never be able to. It is a huge penalty in domain modeling, which is why I personally don't want to use it for anything that actually has to deal with the flow of things.

To some extent I expected and wanted the VM to feel like it makes up for this by being so great that what it does, but there's no band-aid you can use to fix lack of modeling power.

⬐ ashton314

Erlang’s Dialyzer (and the related Elixir project Dialyxer) provide compile-time type safety analysis. It’s optional and only at compile time, but immensely helpful in verifying correctness.

⬐ strmpnk

I also highly recommend the dialyzex [0] project as well. While it's not going to match every feature of some type systems, it does a much better job than one might expect.

It's always a little frustrating that folks pass on Erlang or Elixir as entirely dynamic. It's a lot closer to what you find in gradually typed languages if you take advantage of the tools available.

[0]: I know there are a few of these elixir projects, though I am not familiar with the parent's. Perhaps it's a typo?

⬐ 59nadir

dialyzer is terrible and doesn't actually work in practice, as is evidenced by the many dialyzer errors that slip through to current releases on hex of libraries even the size of Phoenix, StreamData, etc.... Even the major community members don't use it. It's not a substitute for a type system and especially not one that allows you to properly model things in it.

⬐ dnautics

have you used it with elixir_ls? I find that in practice it catches about 80% of my errors before I send them to test.

⬐ 59nadir

Yeah, I managed to get our team members to use ElixirLS for this very reason, but in the end I found that it doesn't work when it matters. The good thing about ElixirLS is that at least you have to configure something to disable dialyzer, so by default to at least get some coverage…

In the end you need to some kind of compiler that does proper, rigorous and unapologetic type checking. If Alpaca could reach critical mass I could convince our project leader to use it, which would solve the problem entirely.

Sure, the ecosystem would likely remain the same in that the majority of people simply don't actually have any familiarity with strong, static typing, but at the very least you can now set up these guards yourself and you can have code that is much more likely to be safe and good.

⬐ enraged_camel

Worth noting that Elixir Language Server will also hook into your text editor/IDE and provide warnings every time you save a file (it compiles the code behind the scenes).

https://github.com/JakeBecker/elixir-ls

⬐ pmarreck

Imagine typechecking a cluster of nodes all sending messages to each other.

⬐ panopticon

Statically typed languages deal with data un/marshaling all the time. Maybe I’m missing something, but how would this be an issue?

⬐ pdimitar

I partially agree, but none have the preemptive green threads of the BEAM VM and its amazing tooling around supervisors and worker hierarchies.

Trust me, if somebody puts that in Go, Rust or OCaml -- I'd ditch Elixir tomorrow.

The BEAM VM is just too good. You can have thousands of smaller tasks running in parallel and nothing lags until you hit the physical limits of the hardware -- which 99% of the apps never do.

⬐ 59nadir

I think this is right on point. The BEAM is finely tuned for a specific problem and it solves it beautifully in a way that is extremely hard to replicate unless you actually are willing to invest in some kind of run time (which instantly makes you ineligible for consideration in a lot of peoples' minds). Every language that has tried, as far as I know, has done so as an afterthought and obviously does not end up replicating the important parts.

Knowing that, it's perhaps more pertinent to talk about what you can actually bring to that VM. Alpaca [0] could potentially be the solution. Interoperation with the rest of the VM languages and static typing on top of that. I have no doubt that most of the community would simply not use it, because they don't care about static typing, but I have zero reservations about saying it would be a better way to write code for the BEAM in the future.

Without something like that, it's unlikely that I will actually put any code that does more than something like routing messages to other services or the like on the BEAM in the future. Beyond a fairly low number of lines I simply don't trust anyone to write code that does exactly what they think it will do and no more in a dynamically typed language.

[0] - https://github.com/alpaca-lang/alpaca

⬐ pdimitar

> The BEAM is finely tuned for a specific problem and it solves it beautifully...

Not disagreeing with you per se but that was true 10 or 20 years ago. Nowadays the BEAM is very solid general-purpose runtime with the best parallelism on the planet attached.

> Knowing that, it's perhaps more pertinent to talk about what you can actually bring to that VM. Alpaca [0] could potentially be the solution.

Realistically, I'd say that ship has sailed. Elixir has a lot of inertia for that to happen at this point. People focus really hard on improvements over Dialyzer (like Dialyxir + improving warning messages), linting, standard formatting, higher-level code generators and what-have-you.

At this point I'd think an OCaml-to-Elixir transpiler that enforces the compile-time guarantees of OCaml and translates them to Elixir code is the much likelier road to strong static typing on the BEAM. I might be wrong though, it's just a speculation.

⬐ giancarlostoro

One thing I love about Elixir is I had read the Erlang book from Joe Armstrong ages back, and a lot of ideas and concepts still apply. I just didn't get into Phoenix cause PGSQL is a PITA on Windows, so I have to go back and try with MySQL instead.

⬐ nickjj

You could run PG in either WSL or a Docker container.

I currently use Windows (where I spend 99% of my time in WSL) and everything runs great.

⬐ wwweston

Maybe somebody else knows where to wave the dead chicken to make postgres under WSL work well, but my last two tries (Feb 2018, Nov 2018) suggested that trying to do so invites lots of yak-shaving, pain, and poor performance. And that's just using the package manager to install, if you want to build from source, it may not work at all.

Stories to the contrary and hints how to make it work are welcome.

⬐ anarazel

Might be worth retesting, we did some changes to PG to handle the unsupported syscalls more gracefully (warn once, and then not anymore) in a recent set of minor releases.

⬐ RexM

What issues are you having getting Postgres setup in Windows?

I've used the Windows installer[1] and have had no issues.

[1]: https://www.postgresql.org/download/windows/

⬐ giancarlostoro

I was unable to figure out how to setup accounts, and I am not used to PostgreSQL, I couldnt figure out my root password, which I swear was just "developer" but that didn't seem to work. I'm used to other database engines, despite my love of the idea of using PGSQL.

⬐ RexM

Ah ok. Maybe it's because my install was basic without additional accounts. I just have my phoenix apps setup to use the default postgres/postgres login.

⬐ giancarlostoro

For some reason I remember having this PGSQL issue a couple of times on Windows, so I may either go with MySQL or what others suggested with WSL / Vagrant. Also I always forget database passwords after a week if it's not saved somewhere.

⬐ sosborn

That’s not a windows issue, it’s an unfamiliar software issue. Take a weekend to learn how Postgres handles accounts. It takes a little getting used to but it is very much worth it.

⬐ prodiguy89

Screw windows and move to Linux! Opens up a ton of possibilities including docker

⬐ RussianCow

Use Vagrant! Then you don't have to install/run PostgreSQL natively at all, and can run it inside a Linux VM.

https://www.vagrantup.com/

⬐ mars4rp

I gave it a try, but for me after years of programming in C like languages the syntax is really hard.

I know it is very similar to Ruby.

I wish there was a ElixirC syntax that would also compile to Earlang.

⬐ rehemiau

It seems similar to Ruby, but in the end it's more like Prolog

⬐ walterstucco

I think you might find one that suits your needs in this list

https://github.com/llaisdy/beam_languages

⬐ lliamander

After years of programming with C-like langauges, I actually came to really appreciate the syntax of Erlang. It's actually fairly simple and straightforward. I think the biggest issues people have with it are:

* Capitalized variables names

* use of comma, semi-colon, and period as clause delimiters

It's interesting, but I think that Prolog-style syntax is probably the easiest to read if you don't have syntax highlighting available (though I don't think there is as much of a difference with syntax highlighting).

⬐ dnautics

IMO, the use of header files is a bit baroque and I find it makes it harder to reason about your code. Erlang structs are also no fun at all.

⬐ ncfausti

Fantastic talk. I have been meaning to learn more about Elixr for a while now, and this video was perfect. Thank you!

⬐ hinkley

I sometimes joke that microservices are half of Erlang, badly implemented (I guess Greenspun’s Curse is broken?).

Elixir has been second or third on my todo list for a long time. With the scary news from Vue, it might have just jumped the queue.

⬐ sondh

What is the scary news from Vue?

⬐ quells

> Scary news from Vue

Do you have a link?

⬐ ineedtosleep

He's most likely referring to: https://news.ycombinator.com/item?id=20237568

Which is a bunch of FUD (see top comment by EvanYou).

⬐ quells

Thanks

⬐ di4na

Thanks Robert Virding

http://rvirding.blogspot.com/2008/01/virdings-first-rule-of-...

⬐ hinkley

See also, Stigler's Law.

Or Poe's law, thus named in 2005, but based on a phenomenon that was well known by the time of Godwin's law.

⬐ neduma

That was exactly my thought when I went thru release notes.

⬐ sansnomme

The Authn/Authz story of Phoenix is a joke though compared to Rails/Devise or Django. The framework authors refuse to ship a first party implementation and instead leave it to the community. Problem is, the ones available are mostly extremely low level (ueberauth, guardian etc.) and you have to spend ages customizing it. On Rails it's literally two commands to set up user registration, email confirmation, password reset. Despite its supposed Rails roots, Phoenix has a long way to go (the greater irony is that the company behind Devise, Platformatec, is also the company behind Elixir).

If you only need basic functionality and absolutely have to use Phoenix, Pow is a good choice:

https://github.com/danschultzer/pow

⬐ bitscrapper

As someone who used Devise multiple times and always ended up regretting said choice in the long term, I am actually glad there is no Devise for Phoenix.

In one day I can quickly get something up and running with Guardian and have full control over how my app does authentication. Password reset and confirmation are easily done with Phoenix.Token without a need to touch the database.

Also, authorisation is a breeze on Phoenix. Take a look at Canada: https://github.com/jarednorman/canada - and it is only 10 LOC.

⬐ pbowyer

> The Authn/Authz story of Phoenix is a joke though compared to Rails/Devise or Django. The framework authors refuse to ship a first party implementation and instead leave it to the community.

I agree, but it's not unique to Phoenix. In the PHP world the otherwise excellent Symfony framework is only starting to add generation of authentication/authorization, and email confirmation and password reset remains the developer's responsibility.

I see two reasons for this.

1. Frameworks see themselves as building blocks, not RAD systems. When the core framework developers are (mostly) working on APIs they don't see the need for approval/password reset. After all "you'll issue the user with a JWT so who cares about the rest (shrug)"

2. Flexibility to auth with anything. Apparently not bundling a system makes it more "flexible".

I believe (1) to (a) be shortsighted and (b) we need RAD frameworks, of higher-level flexible components that can be bolted together.

I believe (2) is wrong, and bundling standard auth flows does not reduce flexibility: you implement your own if you need something else.

⬐ conradfr

That's partially true because you have FosUserBundle as kind of defacto standard for user facing auth (but yes it's a community package) and many built-in things for APi auth.

⬐ pbowyer

FosUserBundle is no longer recommended, and is superseded by Guard. Guard is considerably nicer to work with, and doesn't take long to build the extra bits (password reset etc) but (a) it's not built-in, and (b) if done as a bundle, it doesn't get the same level of support.

⬐ mmartinson

>> The framework authors refuse to ship a first party implementation and instead leave it to the community

Maybe for AuthN, but AuthZ is pretty far from the business of a web framework. The framework authors seem, for the most part, to be avoiding first-party endorsement of secondary tooling that doesn't deeply benefit from being first-party. I find it super refreshing coming from Rails where it is either the golden path or good luck.

A lot of the early AuthN libraries seem to target the controller layer, but I've found, having worked on a couple of sizeable Phoenix production codebases, that AuthN at the app domain layer works better. Regular use of channels as an alternate entrypoint to web request controllers means that controller based helper libraries are less valuable. Once you push this logic inside the app domain, it really isn't the business of Phoenix to know or care about it, since you're just working in plain Elixir, probably with some sort of Ecto struct to represent identity.

⬐ di4na

I would like to point out that it is not only the company but the person behind Devise (ie José Valim). It may means, possibly, that they are not really a fan of what was created with Devise and think the experiment should not be replicated. You know.

⬐ nickjj

I can't speak for the Phoenix dev team but the fact that some of the Devise authors are now heavily invested in Elixir / Phoenix and they didn't port Devise to Phoenix by now makes you wonder if they still think the idea of Devise is a good one.

Personally I haven't found auth to be a problem in Phoenix and I have lots of experience with Rails (and using Devise).

You can put together an iron clad user registration + auth system in about 150 lines of app-level code (not including lines of code for the template forms, but with Devise you would end up customizing your own forms anyways). This includes registering new users, authenticating by email + encrypted password, logging in, logging out, session management and having the idea of a current_user in your system, along with a way to restrict routes, controllers or actions to only logged in users. The essentials basically.

I would much rather manage ~150 lines of code in my own project that I fully know and can easily customize than pull in something like Devise which is 6,000+ lines of Ruby and have to configure / override a number of things.

Everyone has their own opinions but I would much rather spend my time developing features for my app than trying to figure out how to customize a massive third party dependency.

Personally I'm rolling with a magic link authentication system in my Phoenix app and it was also around 200 lines of code to implement everything, including configuring and sending the emails out. There's no external libs beyond Phoenix to get it all working (except for sending the emails out, in which case I use Bamboo which is a lib dedicated to sending emails).