HN Theater @HNTheaterMonth

The best talks and videos of Hacker News.

Hacker News Comments on
The Tragedy of systemd

linux.conf.au · Youtube · 93 HN points · 21 HN comments
HN Theater has aggregated all Hacker News stories and comments that mention linux.conf.au's video "The Tragedy of systemd".
Youtube Summary
Benno Rice

https://2019.linux.conf.au/schedule/presentation/156/

systemd is, to put it mildly, controversial. As a FreeBSD developer I decided I wanted to know why.

I delved into the history of bootstrap systems, and even the history of UNIX and other contemporary operating systems, to try and work out why something like systemd was seem as necessary, if not desirable. I also tried to work out why so many people found it so upsetting, annoying, or otherwise rage-inducing.

Join me on a journey through the bootstrap process, the history of init, the reasons why change can be scary, and the discovery of a part of your OS you may not even know existed.

linux.conf.au is a conference about the Linux operating system, and all aspects of the thriving ecosystem of Free and Open Source Software that has grown up around it. Run since 1999, in a different Australian or New Zealand city each year, by a team of local volunteers, LCA invites more than 500 people to learn from the people who shape the future of Open Source. For more information on the conference see https://linux.conf.au/

#linux.conf.au #linux #foss #opensource
HN Theater Rankings

Hacker News Stories and Comments

All the comments and stories posted to Hacker News that reference this video.
Nov 04, 2022 · 11 points, 0 comments · submitted by popcalc
If you have the time for it, this might interest you then:

https://youtu.be/o_AIw9bGogo

It's 2022 and if you still can't see the good in systemd then it's you choosing ignorance.

Related: https://www.youtube.com/watch?v=o_AIw9bGogo -- The tragedy of systemd.

Where Benno Rice (FreeBSD Committer / FreeBSD Core member) explains the value of something like systemd.

There was a great talk by a FreeBSD developer about systemd, and specifically the (in his eyes, somewhat unwarranted) hatred it has attracted and why

https://www.youtube.com/watch?v=o_AIw9bGogo

Whenever these discussions of systemd come up, I am reminded of this talk[0].

It will be interesting to see if one day a replacement for systemd comes along and people who once championed systemd will begin to use the arguments the people who do not prefer systemd use to defend their choices for not wanting to use the next init system manager.

[0]: https://youtu.be/o_AIw9bGogo

einpoklum
> see if one day a replacement for systemd comes along

Part of the critique of systemd is the basic architectural choice of having this monolithic layer between regular user apps and the kernel. So, in a sense, the idea is _not_ to replace systemd with a better-written systemd, but to do things differently.

kemotep
Yes but in doing things differently with the new one people will defend systemd the same way people defend sysvinit was my point. They will claim the new way is too complicated or tries to replace too much.

I might not be getting the point of the talk but I really appreciate the argument that Benno Rice presents.

You have probably already seen this, but I found this an interesting explanation to the why question: https://www.youtube.com/watch?v=o_AIw9bGogo
Jun 21, 2021 · 1 points, 0 comments · submitted by belter
May 18, 2021 · 4 points, 0 comments · submitted by botayhard
You can get it in audiobook form here: https://www.youtube.com/watch?v=o_AIw9bGogo
Wouldn’t have to be systemd, macOS has launchd for init, FreeBSD could come up with its own thing. This is a good talk I saw recently (by a FreeBSD contributor) on why these sorts of systems have become more common, and why they seem so sweeping in scope. https://youtu.be/o_AIw9bGogo

(In fact systemd and launchd are so tailored to their respective operating systems that FreeBSD likely would have to do their own init replacement if they cared to.)

petre
Have you even used launchd? It's an XML based init system with inconsistent launchdctl directives. I'd even use systemd any day in place of it.
tal8d
> This is a good talk

That is not a good talk unless you already advocate for abandoning standards, and celebrate Linux's growing power to dictate how the shrinking number of remaining OSes design things. You might easily miss it if you're already onboard with systemd, but in his point by point address of systemd's shortcomings - he repeatedly handwaves and redirects.

Systemd is a mess, which is an amazing accomplishment considering the fact that it is supposed to be this great unifying/simplifying layer. So awful that the DoD held out for the longest time in formalizing any kind of baseline security audit procedures that address it. Compare that to something like Solaris SMF. I've never had a problem with the rc system that made me blame the underlying design, but if I did - systemd would be at the bottom of a very long list of my potential solutions, right under 'Set the machine on fire and pickup a copy of "Industrial Society and Its Future"'.

I really like the guy running the OpenZFS project, but it is painfully obvious that it is only going to increasingly cater to Linux - and coincidentally a lot of the really annoying aspects of the project are a result of that. For example: that Rube Goldberg build system, with massive amounts of code duplication... that wouldn't be there if not for autotools and gnu export symbol games.

oblio
Sys V init was not a standard. Is it in POSIX? I can't find it.
tal8d
One of these init systems is run by a guy that is openly hostile to POSIX, the other can be found on several different operating systems and has been around forever. hmmm...
mapgrep
Honestly, what do we need POSIX for in 2020? As discussed in the talk, it traces to the age of the Unix wars, when there was a panoply of processors and Unix variants. POSIX is great if you’re concerned with recompiling some C so it works on SPARC and Alpha and POWER, on Solaris, Tru64, and AIX. That world is gone.

Yes, I’m for abandoning pointless standards. There are concrete benefits to systemd - faster boot, comprehensive service management, power savings, memory savings, a consistent interface to changing system state. What is the benefit to sticking with an rc script architecture designed to run a handful of processes on a pdp?

tal8d
> Honestly, what do we need POSIX for in 2020?

You've never written portable software, have you? You can't write portable software without a common interface. Have you ever looked at what autotools vomits out? You'll see shell scripts containing the likes of "echo $1 | sed 's/^x//'" in order to just get things to where they have a chance of sharing enough commonality to compile. Imagine how much worse it would be without POSIX. Nothing would run on anything that the developers didn't account for, this includes OS (plus version), environment, and hardware architecture. That is the world you are asking for, not the one we presently live in.

> What is the benefit to sticking with an rc script architecture designed to run a handful of processes on a pdp?

Freedom, for not only the end user but all the way up to distro packager. Systemd is designed to be non-portable, and force a network-effect pressure. I was similarly suspicious about WSL designing toward a linux API instead of POSIX... an effort to reduce the potential for alternatives emerging in the future, where your choices are windows or linux.

oblio
> That is the world you are asking for, not the one we presently live in.

So, like mobile? Which is 70%+ of computing these days? :-)

> Freedom, for not only the end user but all the way up to distro packager. Systemd is designed to be non-portable, and force a network-effect pressure. I was similarly suspicious about WSL designing toward a linux API instead of POSIX... an effort to reduce the potential for alternatives emerging in the future, where your choices are windows or linux.

Freedom to do what? Init systems should be a solved problem, done and done. Innovation should happen higher up the stack, the init system should be uniform, hopefully flexible and universally adopted, and ideally standardized fully (as in ECMA & co.).

tal8d
> So, like mobile? Which is 70%+ of computing these days? :-)

Media consumption spy devices + smiley emoticon. Yuck.

> ...and ideally standardized fully...

uh, you know that you are advocating for the direct opposite of that - right? Systemd is designed to be impossible to use outside of linux. So what, everyone should adopt the perpetually changing linux ABI as their point of commonality? You know that would be insanely stupid, right?

oblio
My point is that they should turn systemd into a real standard.
tal8d
My point is that systemd is intentionally designed to make that impossible to do without also pulling linux (ABI, environment, whatever) within the same standard.
oblio
Which "different operating systems"?

Only some Linux distros use SysV init and they're not any of the big players (Fedora/RH was the distro creating systemd), Ubuntu adopted systemd as the default 5 years ago, SUSE did the same 6 years ago, Debian 5 years ago. Almost everything else is a hobbyist OS (I've never seen Gentoo or Slackware in production anywhere else than at small companies where the server was on the critical path for maybe several tens of thousands of dollars, at best, which is peanuts on a global scale) or a niche OS (Alpine is used 95% as a container distro, and who freaking cares about the init system in a container? :-) ). Basically anyone who "puts there money where their mouth is" uses systemd.

SysV init was never adopted by BSD.

I think Solaris used to use it but it's been using Service Management Facility for the past 16 years.

HP/UX and AIX use SysV init, I think, but they're so vanishgly rare that 99% of developers and sysadmins out there could have the most successful careers in history and both HP/UX and AIX could be nuked from history for all they care.

And my point was that nobody even bothered to standardize SysV officially, though it was supposed to be a standard. That's how little everyone cared.

Poettering might be annoying, but he cares, he's opinionated and is actually knowledgeable. I've been using his software for a while and it's quite solid and his vision makes sense. POSIX isn't enough for a modern operating system.

tal8d
So what are you arguing? Because I see a list of disparate platforms sharing the same init system, and I see the homogeneity that is systemd/linux. What is that a counterpoint to? Or is this a celebration of network effect?

> POSIX isn't enough for a modern operating system.

So there should be no standard? I can think of only one time I ran into a shortcoming that can be blamed on POSIX, and a few more where the OS just did a bad job in implementing the standard. Can you imagine how awful things would be right now if the DoD hadn't forced the concepts of interface commonality and multi-source procurement? You'd likely be reading this on a Honeywell glass TTY hooked up to an IBM timeshare. Because without standards the network effect acts as an insurmountable barrier to entry, and incumbents fully control the height of that barrier with needless complexity and arbitrary breaking changes. Hell, not that long ago there was talk about wielding GPL export symbols as a weapon to punish nvidia... I'm no fan of nvidia, but I like the idea of an API inspired by spite a lot less. Also, in a world where your vision prevailed, there would be no AMD - because Intel already set the industry "standard", so why worry about second-source availability?

oblio
> Because I see a list of disparate platforms sharing the same init system

Which one? I just listed the others and they all use different init systems...

> So there should be no standard

No, they should turn systemd into a real standard. ECMAinitd :-p

tal8d
Are you intentionally missing that the point is the fact that any one of them can use the portable init system that isn't systemd?

> No, they should turn systemd into a real standard. ECMAinitd :-p

Well systemd can only work on linux, by design. So every standard compliant init system would require that it package a linux kernel. That is not only moronic, but with history as our guide: would directly result in never patched, network active, embedded software.

znpy
Standards can become outdated you know?

And by the way standards aren't really the word of some kind of god that you should follow blindly and never question.

Many standards are ratified just to settle the status quo regarding various solutions to a common problem.

At this point, one might even argue that the next "standard" for init systems should just be based off systemd.

tal8d
lol, the Internet Explorer school of thought. Sure, form a work group for an open init standard with systemd as the basis - chaired by people both inside and outside the systemd project. But before that, rent a home in Malibu and wire it for reality TV, because that would be the most entertaining WG ever.
"Linux has systemd" and that's why the author didn't select it supposedly. I hear this line of thought from people that use FreeBSD often, although I suspect if you ask many of those people why systemd is so horrible you won't actually hear a legitimate (and still relevant) reason from them. I suggest anyone that thinks systemd is terrible and doesn't spend lots of time in Linux to watch this presentation by Benno Rice.

https://youtu.be/o_AIw9bGogo

_lffv
> Linux has systemd, not my favorite thing out there, Windows is privacy nightmare. That left me with 2 major options: Linuxes without systemd (Gentoo, in my case) or BSDs.

Even if you don't like systemd (I don't but I live with it because I never have to interact with it anyway), this is such a weird way to phrase it. Imagine saying "Linux has `apt`, not my favorite thing out there. That left me with two major options: Linuxes without apt (Arch, in my case) or BSDs." Just... change the program. Apparently Arch can switch over to sysvinit just by installing two packages from the AUR (see https://wiki.archlinux.org/index.php/SysVinit), Devuan exists, et cetera.

Of course, the author was sold on BSD before making this argument - and fair enough, at least it's a free system. But in my opinion it's ridiculous to discount a family of systems because of one common program.

toast0
My basic dislike for systemd is it a large change to the overall system, and doesn't provide me any benefits that I can tell. Beyond that, I've experienced or read about many negative things.

a) I ran into an issue with changes in startup scripts in Debian which meant I could no longer hit ctrl-c to stop network initialization on a laptop when it couldn't get a dhcp lease (it was either not connecting well to wifi , or trying to get a lease on a disconnected wired NIC; it was a while ago, I don't quite remember)

b) there have been many secuirty issues in systemd and systemd-* utilities. Quite a few of which were repeats of issues existing daemons had been through, that shouldn't have been repeated.

c) I have read that in default configurations a user's programs will be terminated after the user logs out; that's not acceptable for me, and a large change in default behavior

For me, systemd is yet another churny subsystem that drives aggrivation, so since I was already exposed to FreeBSD through work, it made sense to me to go in that direction at home, instead of sticking with Debian and accepting systemd.

gorgoiler
Charitably, this feels like a good presentation if you are already sold on the idea.

Realistically, it’s pretty snarky and unconvincing to all but the choir to whom he preaches.

ysleepy
Ever used systemd-timesyncd or the systemd-dhcp thing, oh and systemd-resolved ? All these inferior implementations are barely configurable and lack features for use-cases outside of desktop users.

The init system is nice, but why the hell does it re-implement svchost.exe

1_player
systemd-resolved is inferior to what? It's the only DNS system on Linux that can handle LLMNS + split-DNS (i.e. a VPN and your LAN running alongside each other) + DNS-over-HTTPS correctly. Sure it's a pain in the arse to configure correctly, but it's more featureful than NetworkManager.

systemd-timesyncd is inferior to what? a fully fledged ntpd? go ahead, use that if you really need to, but on 99% of Linux machines systemd-timesyncd is enough.

chmln
All of these are optional, just disable and use an alternative?

I use resolved and timesyncd however and they work amazing for my basic needs on the laptop.

d21d3q
I did. I've been maintaining python+ncurses tool for configuring system: network, ntp, etc. If you don't have dhcp server running on your network, then ifup will block until it resolves address (ui freeze). If not, then you have to kill it and also dhcp reolver. After switching to systemd-networkd it just works in background.

All configuration is done with .ini style files so that no need to use special parser for /etc/network/interfaces.

Disclaimer: I jumped on linux few years ago when systemd already had some momentum, so that I wasn't used to either init or systemd. I found the later easier to pick up.

nix23
>I hear this line of thought from people that use FreeBSD often

That's BS and you know it.

antranigv
author here!

I'm not against the "concept" of systemd, I think the BSDs need a "system" layer as well, just not systemd. The ideas are amazing, the implementation is the problem.

Nov 27, 2020 · 2 points, 0 comments · submitted by mariuz
- Hardware support out of the box

I can be given a work laptop, slap some Debian or derivative along with XFCE, and I know that most hardware will work out of the box without much fuss (including but not limited to: webcams, video cards, wireless/bluetooth, thunderbolt docking station).

- Containers

They're just very good. FreeBSD looks like it has most of the pieces to get containers going but it's not their priority to get that going. The people at Joyend did that, reimplementing the docker server APIs to manage underlying zones. I wonder why don't FreeBSD do something similar.

- Reference documentation

I like to use RHEL and derivatives (CentOS) when I can, and the reference documentation its so, so, so good, ample and complete. The FreeBSD reference guide is good, but not as thorough and/or up to date as the RHEL documentation.

- systemd & NetworkManager

They're rock solid, battle tested and well-thought. systemd is much more than init and does a lot of system management, stuff that previously was simply not being done by anything else. BSDs lack that completely -- see https://www.youtube.com/watch?v=o_AIw9bGogo

pmiller2
Don’t forget the Arch wiki for documentation! Sure, only a minority of users run Arch, and probably a minority of those users are running it in production. But, it has some amazing documentation, especially when it comes to weird combinations of things people don’t usually see.
eb0la
Funny, I choose OpenBSD (back in '94) instead of Linux as a desktop (text-only) because it had very good documentation available for kernel and userland stuff.

I am typing this from Ubuntu because I need some basic Wine emulation, Docker, and K8s. Not for work, just for tinkering.

I still have an OpenBSD vm around, but I don't use it much :-(

runjake
FreeBSD arguably did containers before Linux — except they’re called Jails in the BSD world.

But it looks like FreeBSD is joining the Linux container party with their recent release which implements initial support for native Linux containers.

Jan 18, 2020 · jammygit on What Unix Cost Us [video]
He gave a good talk about SystemD once too: https://www.youtube.com/watch?v=o_AIw9bGogo

edit: found a version with better camera angles

sprash
I didn't like that talk at all. I disagree with almost everything he says[1].

This talk is much better but some of his criticism are rather pointless because what he wants is a complete new OS design vastly different from UNIX. If he likes Windows/MacOS so much he should just use that.

1.: https://news.ycombinator.com/item?id=19966321

Koshkin
I agree that criticizing UNIX for being UNIX is pointless.
oooooo, the grandparents video is very different in tone than the version I watched, which is the same talk: https://www.youtube.com/watch?v=o_AIw9bGogo

I'm sorry, I had assumed I had seen the exact video, but I had just seen the talk from another venue.

Aug 16, 2019 · Sammi on Cool but obscure X11 tools
Relevant talk:

The Tragedy of systemd: https://www.youtube.com/watch?v=o_AIw9bGogo

"systemd is, to put it mildly, controversial. As a FreeBSD developer I decided I wanted to know why.

I delved into the history of bootstrap systems, and even the history of UNIX and other contemporary operating systems, to try and work out why something like systemd was seem as necessary, if not desirable. I also tried to work out why so many people found it so upsetting, annoying, or otherwise rage-inducing.

Join me on a journey through the bootstrap process, the history of init, the reasons why change can be scary, and the discovery of a part of your OS you may not even know existed."

liability
I've been using SystemD with little complaint since around 2011 or so when Fedora switched. I don't count myself among the systems haters, but that talk is hot garbage. It's just a big ad hominem that boils down to "people just hate change."

But that lacks explanatory power. Lots of things change and don't receive backlash, even in the desktop Linux domain. Plenty of people are skeptical of Wayland for instance, but the conversations tend to stay relatively cool-headed.

I think there are two things going on; the first and most significant is that most Linux desktop users (not sys admins or packagers, just desktop users) never had to know what a sysv init system was and never fucked with it, but then their distro switched, something broke (maybe their packagers' fault, maybe not!), they googled the problem and at the end of that Google dive they found out the answer involved a component they were previously unaware of, making itself known to them in a way that caused them hassle. Any upsides were washed out by this initial first bad experience. For me, I was using Fedora at the time so I knew I was getting bleeding edge shit and might get cut. That was exciting for me. It was my hobby, so I liked systemd. But that didn't blind me to the wants and desires of more typical desktop users who wanted the "just werks" experience.

(The other factor is that Lennart has bad social skills and it seems to rub off on the people he works with. Some might argue that the critics started it, but I don't care who started it. Stop jabbing your little sister or I'm turning this car around!)

Additionally, no alternative project is even providing close to the set of features that i.e. systemd or pulseaudio does, not even via modules. sysvinit for example is only an alternative to systemd the same way a SUV is an alternative to F1.

(watch https://www.youtube.com/watch?v=o_AIw9bGogo if you don't know why)

_emacsomancer_
> sysvinit for example is only an alternative to systemd the same way a SUV is an alternative to F1

Perhaps. But this is the same tired old argument that ignores the existence of any other init/daemon-manager aside from systemd.

And I would use the same car metaphor for comparisons with some of the other options, esp. runit, except with systemd in the SUV slot (e.g. my runit systems all behave themselves; my systemd systems frequently hang on reboot, often needing a REISUB).

AsyncAwait
> Perhaps. But this is the same tired old argument that ignores the existence of any other init/daemon-manager aside from systemd.

Do you mean openRC, s6 etc? They still all use bash scripts, (which as a result depend on the dev for quality and can vary quite a bit) vs systemd's clear, uniform service definitions. They don't provide much beyond starting services and are more like wrappers around sysvinit than anything else.

> my runit systems all behave themselves; my systemd systems frequently hang on reboot

Every time I've seen this in practice it turned out the system was actually not configured properly, (usually crypttab or something related), and systemd has a default 90s timer to try to wait for everything to dismount, shut down properly vs other init systems just yank the cord, but it seems faster, so I guess it's 'better', (in which case you can just shorten the systemd timer to like 1s and it would be even faster doing that - look for 'DefaultTimeoutStopSec' in /etc/systemd/system.conf).

PlutoIsAPlanet
if your systemd isn't able to "shut things down" quickly, that's the fault of an application not responding to a shutdown signal and systemd waiting before it kills it.
_emacsomancer_
For the timer-issues, perhaps. But that doesn't cover the completely black screen hangs.
CameronNemo
s6 is often used with execline, a stripped down scripting language that is highly performant.
_emacsomancer_
> Do you mean openRC, s6 etc? They still all use bash scripts, (which as a result depend on the dev for quality and can vary quite a bit) vs systemd's clear, uniform service definitions. They don't provide much beyond starting services and are more like wrappers around sysvinit than anything else.

I'm not that familiar with openRC or s6 (I've played with them, but don't use them on any 'real work' systems), though my understanding is that there is a proper openrc-init being developed.

I was actually thinking more of runit and shepherd, which provide their own init and daemon-management.

> > my runit systems all behave themselves; my systemd systems frequently hang on reboot

> Every time I've seen this in practice it turned out the system was actually not configured properly, (usually crypttab or something related),

This is across multiple systems, from Arch boxes, which I've hand-configured to vanilla Ubuntu boxes, and ranges in behaviour from multiple 90s timers to just hanging on completely black screen until forced to reboot. There are certainly no obvious configuration issues on any these, definitely not crypttab or the like.

> shut down properly vs other init systems just yank the cord, but it seems faster, so I guess it's 'better', (in which case you can just shorten the systemd timer to like 1s and it would be even faster doing that)

But, even timers aside, runit is faster than systemd on both boot and shutdown, for similarly-configured systems. And provides (me at least) a best user-experience than systemd.

runit shuts things down properly; with systemd I often have to literally 'yank the cord'.

AsyncAwait
I guess I did not explain myself properly, s6 is not actually a wrapper over sysvinit in practice, it's its own thing, like runit. But like runit it relies on bash scripts, which are not declarative and open to a wide variety in quality, just like with sysvinit.

Also, runit does i.e rely on logind, which everybody forgets systemd took upon themselves as consolekit was unmaintained. Why does runit rely on systemd's work here?

> runit shuts things down properly; with systemd I often have to literally 'yank the cord'

I've looked at how systemd handles shutting things down vs other inits and it's just a lot more paranoid, i.e. it waits for confirmation of every single service being shut down etc. In other init systems it tends to be a case of 'send shutdown signal, assume process shuts down'. You could fault systemd for this, but it's more of an application-level issue.

_emacsomancer_
> But like runit it relies on bash scripts, which are not declarative and open to a wide variety in quality, just like with sysvinit.

In theory, this makes sense, with potential issues of poorly written shell scripts. In practice (I assume arising from its increasingly baroque growing complexity) systemd ends up with stability issues which aren't pleasant from an end-user perspective.

Setting shutdown aside, I had a horribly difficult-to-debug boot issue with systemd: a daemon failing in an unexpected way caused a failure to bring up any services, but without systemd failure messages. The problem was a systemd emacs service. On one boot I had an issue in init.el which caused emacs initialisation to hang, so the root cause wasn't systemd. But systemd didn't fail gracefully (or informatively) in this case. And it took me a while to figure out why systemd wouldn't bring up any services, since it wasn't reporting to me that the emacs service (or any service) had failed.

I've since reverted to launching emacs daemons via .xinitrc or the like on my systemd boxes, so that any emacs init issues don't pull the rest of systemd down.

acdha
> On one boot I had an issue in init.el which caused emacs initialisation to hang, so the root cause wasn't systemd. But systemd didn't fail gracefully (or informatively) in this case. And it took me a while to figure out why systemd wouldn't bring up any services, since it wasn't reporting to me that the emacs service (or any service) had failed

If it’s hanging it hasn’t failed. If that daemon is configured as a blocking dependency, waiting is the correct thing to do — this is why timeouts and health-checks are as important as making sure that your dependencies are as flat as possible.

_emacsomancer_
But as far as I can tell, the Emacs daemon shouldn't be a blocking dependency. Here's the systemd unit definition I would have been using at the time: https://wiki.archlinux.org/index.php?title=Emacs&oldid=46958... . And here's a more recent one: https://wiki.archlinux.org/index.php/Emacs#As_a_systemd_unit . Neither of these should be blocking dependencies, should they? `WantedBy=default.target` doesn't make it blocking, does it?

Is there something more that should be added to these sorts of unit definitions to improve graceful failure scenarios in systemd?

PlutoIsAPlanet
I think it might, this really just sounds like a misconfigured unit.

in which case systemd is doing exactly as it's told.

Edit: That linked unit is for user level systemd, so it shouldn't be involved in the system boot at all.

_emacsomancer_
It definitely is/was a user-level unit, which makes it unclear to me why it should have interfered with bringing up other services in general.
It's worth watching this[1] talk about systemd by a FreeBSD person, of all people.

1 - https://www.youtube.com/watch?v=o_AIw9bGogo

Systemd is a bunch of loosely coupled tools, though; it's not a monolith at all. PulseAudio was written by Lennart but isn't part of systemd. You don't have to use systemd's resolver. systemd-nspawn is a completely separate process and isn't required for systemd as a whole.

It's worth watching Benno Rice (a BSD developer, actually) discuss systemd and some of the misconceptions, some of which you're expressing, therein: https://www.youtube.com/watch?v=o_AIw9bGogo

JdeBP
> Journald is also not part of systemd core--at least, I think so; this one I've never really looked at myself.

You really should. journald is not optional.

eropple
Yup, seems you're right - journald appears to be the only mandatory non-PID1 process for systemd. Good to know. It also will forward happily to rsyslogd if that's your bag and then you never have to look at it, so I'm gonna just give a big ol' emoji shrug on this one.
JdeBP
> journald appears to be the only mandatory non-PID1 process for systemd.

You need to read a lot more. The Desktop Bus is mandatory too. Lennart Poettering reminded everyone of this back in 2015.

eropple
D-Bus, as discussed elsewhere in this thread (where I learned about this, too), is a specification rather than a concrete implementation. Are you asserting that systemd does not work with alternative D-Bus implmentations? That would be weird, seeing as how they wrote an alternative one...

Also, since I've been "reading more", and I can check after I get back from lunch--at least at some point Debian shipped systemd without D-Bus on minimal installs. Maybe that's no longer the case, but the stuff I'm reading seems to be post-2015.

JdeBP
> Are you asserting

You need to stop constructing straw men, too. I asserted what I actually wrote. And Lennart Poettering asserted that

> D-Bus has been a core component of Linux systems since more than 10 years.

and

> D-Bus refers to both a specification, and a reference implementation.

back in 2015. I suggest that instead of arguing fallaciously about something that you tell us you have only just learned about, that you go and do that reading.

eropple
Asking for clarification isn't a strawman, sorry. And it sure seemed like you were correcting me, which is why I asked. But I said that journald was the only mandatory component of systemd outside of PID1. You said this was wrong. I'm waiting for you to extrapolate? Does systemd only work with the "canonical" dbus-broker? Or does it want ("want" as I'm reading stuff that suggests it isn't absolutely required, but I'm not sure and I'm happy to caveat where I'm not sure) a D-Bus broker? Because if it's the latter, and as it's specified and they seem big on meeting specs I'm betting that it's the case, then what I said is correct and you're just being kind of a jerk about it.

Or you can keep trying to big-time me instead of engage on the merits of your position. That's cool people stuff, I guess.

effie
This is out of the line. You were given civil arguments for why systemd requires Dbus but in your response, you accuse the other person of being a jerk and "big-timeing". Please use the strongest interpretation of other people's claims on the topic, before using them as a means to accuse them of wrongdoing.
zapzupnz
That person is big-timing. Their tone ("you should read more" and so on) is in violation of the HN guidelines, filled with needless snark and condescension. The offending comment has been flagged by multiple users who agree that the comment doesn't meet community standards.
effie
Perhaps the tone of JdeBP wasn't great, but they had a point. On the other hand, eropple's comment tone wasn't ideal either and they consistently miss the original point of systemd requiring dbus. Unfortunately, I am unable to check this since JdeBP post's have been censored.
eropple
I am inviting the strengthening of his position in order to better understand it. I asked him to elaborate because as near as I can tell there is no hard dependency on any D-Bus broker. There is possibly--I genuinely can't tell from the documentation I've been browsing through--a dependency on a D-Bus broker. Requiring a D-Bus broker does not make any one D-Bus broker mandatory...and so, if that is the case, then my assertion that journald is the only non-PID1 part of systemd that is required would hold.

I'm happy to be corrected if I'm wrong. I invited it, even; I'm wrong all the time and being wrong is how one learns. The big-timing I referred to is the refusal to materially engage. Instead he quotes Lennart about D-Bus--out of context? who knows! It isn't an indication of the state of the software in question, just his opinion at a point in time--and to tell me to go read (read what, exactly? what generosity in citation!). Oddly enough, though, I don't see you tagging in to lecture him about that sort of sneering. Why's that?

dang
Please don't be personally abrasive in arguments here. No matter how right you or feel you are, it breaks the site guidelines: https://news.ycombinator.com/newsguidelines.html.
tyingq
"Systemd is a bunch of loosely coupled tools"

As just one example, the nspawn owning package requires a libnss package which pulls in systemd-resolve. I believe it also requires dbus.

yarrel
It's good that systemd apologists have moved on from arguing that it isn't a single binary. But it is not a bunch of loosely coupled tools any more than it is "just" an init replacement.
eropple
Hmm. "Apologists." Couldn't be that it's been a lot better for me where I regularly interact with it and has been no worse where I don't. It's gotta be apologism. We have to be out to getcha.

I think you owe me an apology for that one, dude.

More productively: what parts of systemd's ecosystem--aside, apparently, from journald, but the above no-harm-no-foul reply to the sibling still applies as far as I'm concerned--have you had trouble pulling out when you wanted to, out of curiosity?

ben509
> It's gotta be apologism.

Apologists are engaging in apologia, which means a defense of one's ideas. It's not suggesting a hidden agenda, to the contrary, it suggests the agenda is fully on display since it's plain what is being defended.

luke0016
There are lots of variations on the UNIX philosophy [1], but two core points are to:

1. Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new "features".

2. Expect the output of every program to become the input to another, as yet unknown, program. Don't clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don't insist on interactive input.

systemd has binary format data all over the place, and it is quite difficult to separate out an individual component and have it work properly on its own. Not to mention trying to replace a subcomponent with something else.

systemd is, in general, a tightly coupled system with lots of interdependencies. It is, in many respects, the antithesis of the UNIX design philosophy.

Hell, even modern Software Engineering practices still talk about decreasing coupling and increasing cohesion.

[1] https://en.wikipedia.org/wiki/Unix_philosophy

AsyncAwait
Regarding the UNIX philosophy thing, it's worth noting that it is not automatically The Right Thing to stick by it at any cost. You could argue that UNIX itself does not use the philosophy when it's inconvenient, for example many UNIX-(like) kernels were/are monolithic, despite micro kernels with message passing akin to GNU Hurd embody the UNIX philosophy much better than the original UNIX kernel did(!)

Also, how do you define doing "one thing" exactly? I can see it for `ls` or `cat`, but for anything more complicated than that...not so sure. Is Vi following the UNIX philosophy? If you say; yes, it's 'just an editor', despite 'an editor' being composed of several components, then I can say that systemd is just 'a service manager'. It's a semantics game that does not make sense in today's world.

The Unix philosophy has its uses for sure, but it's far from an universal truth in today's world.

luke0016
> for example many UNIX-(like) kernels were/are monolithic, despite micro kernels with message passing akin to GNU Hurd embody the UNIX philosophy much better than the original UNIX kernel did(!)

Sure. The UNIX Philosophy applies more to userland, I think, than the kernel.

Edit: I'll add with regard to the kernel, the internal structure of the kernel still has fairly well-defined subsystems that interact in clear, mostly consistent ways. So while from an execution standpoint it is monolithic, the philosophy is still something that is taken to heart to some extent.

> Also, how do you define doing "one thing" exactly? I can see it for `ls` or `cat`, but for anything more complicated than that...not so sure. Is Vi following the UNIX philosophy? If you say; yes, it's 'just an editor', despite 'an editor' being composed of several components, then I can say that systemd is just 'a service manager'. It's a semantics game that does not make sense in today's world.

I think dismissing it as a semantics game is taking it pretty far. It's clearly not a black and white rule, but more a guiding philosophy. When it works, it's beautiful and elegant.

The ability to pipe basic UNIX utilities and programs together is something that has always left me a little awestruck. The flexibility and simplicity there is almost profound. SysVinit was far from perfect, but you could at least trace the execution flow from basically start to finish and, if you understood basic *NIX utilities, have a good idea of exactly what was happening.

> The Unix philosophy has its uses for sure, but it's far from an universal truth in today's world.

Again, it's a philosophy, not a universal truth. A guiding light, if you will.

With systemd, that light is pretty dim, if it's on at all.

Ah, you may be right, that's what I get for trying to remember a talk without double-checking. I thought I remembered that systemd was a port of launchd here [1]. My mistake!

[1] https://www.youtube.com/watch?v=o_AIw9bGogo&t=2178s

I recommend watching this to give you some perspective on the SystemD hate: https://www.youtube.com/watch?v=o_AIw9bGogo

I too was furious about the existence of systemd, but now I realize that was mostly because I don't want new things and consider many new things to be bad because they are new and there have been a few 'bad apples' in my personal experience

> SystemD definitely spurred some controversies years ago but is it that bad in 2019?

The reasons for the controversies have not really changed (lennart, the idea that it's not "unixy", a takeover of many subsystems by various systemd components, …) so people who didn't like systemd years ago probably still don't.

As somebody who's not really involved, I found Benno Rice's The Tragedy of Systemd[0] interesting and enlightening.

[0] https://www.youtube.com/watch?v=o_AIw9bGogo

dijit
I found that talk incredibly condescending. He speaks about launchd as if it’s a good thing but it’s quite universally accepted as a bad thing and apples attempts to upstream the problem to the bsd developers has been met with open hostility.

It’s also to the point where Benno states that people don’t like change, but that’s really not true. Everyone jumped on virtualisation (except Theo De Raat), everyone jumped on containers, Kubernetes, GO, Rust, eBPF et al.

There are new technologies which don’t have to eat the universe to function.

Most people who have a problem with systemd don’t actually hate it, they’re upset because the environment that allowed systemd to grow no longer exists for its replacement. Coupled with the fact that we should really consider systemd a proof-of-concept for something better, means that we’ll be stuck with buggy, weird, non-deterministic C++ for decades.

That’s the root of the problem, not that we nerds dislike change, we tend to love change.

JdeBP
systemd is not written in C++.
acdha
> He speaks about launchd as if it’s a good thing but it’s quite universally accepted as a bad thing

This is your personal opinion being presented as fact. Speaking from some experience in the Mac admin world, I’d really want to see the survey you did to support that claim.

Feb 20, 2019 · 43 points, 91 comments · submitted by DyslexicAtheist
timw4mail
The tragedy of having to watch a youtube video instead of reading a transcript...
loeg
https://news.ycombinator.com/item?id=19023232

LWN does a reasonably faithful summary.

ardy42
Previously:

LWN article:

https://news.ycombinator.com/item?id=19023232 (1/29/2019)

This talk:

https://news.ycombinator.com/item?id=19157293 (2/13/2019)

Another youtube version of it:

https://news.ycombinator.com/item?id=17778202 (8/16/2018)

syn0byte
I ask this in every systemd thread but I genuinely would love a real answer. Can anyone give me exact real-world examples of where systemd is objectively better suited than any number of other init systems?

The fact that I still have yet to see any such examples in the last half a decade seems pretty telling. Wouldn't and shouldn't there be tons of such examples given the rhetoric?

zaarn
If you want to container your services better, then systemd is a good choice; it has plenty of isolation options to secure yourself against misbehaving code. By default systemd will not leave any orphans lying around in your system if the service dies unexpectedly.

You can also manage much more complex dependency situations (like two services being dependent on each other and requiring a restart of both if one crashes).

Systemd Timers are also much more robust and predictable than piling scripts on top of cron, I use them extensively for managing backups on both servers and desktops with automatic reporting for failures.

foldr
It makes it a lot easier to set up a service than System V init. Writing a service file is much easier and less error prone than writing shell scripts.
JdeBP
syn0byte explicitly said "any number of other init systems", and was clearly trying to avoid people trotting out, as you and several others have, the tired old fallacy, called out by the Uselessd Guy long since, of assuming that only systemd and van Smoorenburg init+rc exist ...

* http://uselessd.darknedgy.net/ProSystemdAntiSystemd/ (https://news.ycombinator.com/item?id=8488235)

* https://blog.darknedgy.net/technology/2015/09/05/0/

* http://jdebp.eu./FGA/run-scripts-and-service-units-side-by-s...

... and then calling the latter "System V init", when AT&T System V had actually adopted a new system with separate system and service management several years before Linux was even invented.

* https://news.ycombinator.com/item?id=18788974

foldr
I took syn0byte to be asking for an example of an init system that systemd is better than. I can't really see any other interpretation of their comment.

>and then calling the latter "System V init",

This is what everyone calls it in the context of Linux init systems. You may not like that particular piece of terminology, but please don't give me a hard time for following the standard usage.

JdeBP
> I can't really see any other interpretation of their comment.

Then you are unable to distinguish singulars from plurals.

foldr
No, the comment asks whether it's better than any number of other init systems. In other words, whether there are any other init systems that it's better than. So a counterexample could be a single init system. I see no other sensible interpretation of the sentence. Could you paraphrase what you take it to mean?
syn0byte
"any other init system..." originally included "...besides an ancient SysV strawman." but I removed it because it seemed overly snarky.

Having written and dealt with both; It's about equal in my time and effort to write either in sh or windows ini/'unit' files.

foldr
If you claim that System V init is just as good as systemd, then it is a relevant comparison rather than a straw man, no? In many cases there were a substantial number of people who wanted to keep System V init rather than move to systemd (e.g. https://wiki.debian.org/Debate/initsystem/sysvinit).

System V init wasn't an ancient straw man for Debian users. It was the init system that systemd replaced.

altmind
Systemd is great if you/the company are developing your own software that need to be daemonized. Its easy to write/copy one .service file for all the distributions than to create some shell scripts/service descriptiors for a bunch of different distros.

The alternative approach is to use supervisord - it also have service descriptors and does what systemd does. The last time we tried to use it(rhed6 had no systemd) we found it lacking some options important to us.

loeg
> The fact that I still have yet to see any such examples in the last half a decade seems pretty telling.

Yes, it's telling; it suggests you've had your fingers deliberately in your ears for five years.

aequitas
I would like to ask the question the other way around. Now I've been using systemd for a while and gotten used to it I can't image wanting to replace systemd with a collection of scripts.

Maybe systemd is not objectively better or worse. But I'm at least glad there is a standard among Linux systems so I only have to learn the quirks of one init system.

jpdb
It's much easier to manage an application's life-cycle in a systemd unit file than it is in sysvinit bash script.

Unit files are much more declarative and self-contained compared to a bash script. And the bonus is that if you need/still want to use bash, you can do so in systemd.

rkangel
To echo the sibling, but with some more detail:

I worked on a system that (among other things) packaged up a Linux system as a network appliance. The device ran multiple daemons. I have built similar things before in a SysV world, and systemd made the following things easy:

* Starting each daemon only when the system resources it needed (e.g. network up) were ready

* Starting the daemons in the right order relative to each other

* Putting daemon log output in syslog

* Restarting when crashed (for those where it was appropriate)

These all required only a few lines of systemd config, and no special coding in the application. For instance, logging was just printing to stdout/stderr and systemd took care of the rest.

ilovecaching
What problems do people have with systemd? I actually think it’s pretty great. They also made some tools like busctl that are really nice for Linux developers.
npongratz
A litany of arguments against systemd are listed here: http://without-systemd.org/wiki/index.php/Arguments_against_...
nickik
Or other wise called a list of dependable design choice that the author doesn't like and a list of incompatibility with the tools the author used before.
evol262
The actual speaker is great, but the vast majority of people who have a problem with systemd get there philosophically.

In a "I never actually had to write any code which had significant interaction with existing solutions (openrc, upstart, sysv, bsd, whatever), but I know they exist, so I'll talk about them like they solve the same problems as systemd" way. Which they do not. Want socket activation, double forking, namespaced children, sane scripts/unit files that can build a good dependency tree? You have one option.

Additionally, people see other tools like systemd-network and say "but systemd is doing everything!" Which it isn't. Almost no distros package those as part of the default.

Even if they did, many of the anti-systemd crowd would rather live with a different, slower system (NetworkManager, for example) which is vastly most complicated than what they need. Ditto for systemd-resolved and local caching dnsmasq.

Systemd is not trying to replace NM or BIND or anything. It's basically saying "it's 2019. If you want to be able to work with a system without knowing the ins and outs of 30 different configuration files, and speed up boot time/auditability of your system significantly, you can use these services"

Some things are bad for the community in general. The dependence on dbus (and kdbus), systemd-logind, and other tools aren't very friendly to BSDs to anti-systemd distros. Frankly, from my POV, this isn't systemd's problem. The anti-distros are free to do what they want, knowing the limitations. The BSD guys have written compatibility layers, which you can do, because they're open standards.

Broadly, I'd say that the anti-systemd crowd holds to an idea of "UNIX" purity, as if HP-UX, Tru64, AIX, Solaris, IRIX, and others were uniform. The reality is that they were more different than they were alike, in terms of administrative tools, filesystem layout (is this /sbin/init.d, /etc/rc.d, /sbin/rc.d, etc?) and even software portability. A large reason that GNU autoconf exists is to detect all of the quirks on all of these platforms.

Additionally, anti-systemd people either never used X11R6/OSS/lilo and other old tools, or don't remember how terrible they were. Any large change in the community (X.org fork, ALSA->PulseAudio, grub) draws ire. From the same people who are trying to use syslinux and efibootmgr instead of just using grub2.

Some people would rather debate philosophy than get actual work done, and that's great for open source, even if it's frustrating. But those people never dug into the guts of upstart to make something work, or had 250 lines of sysvinit boilerplate+supervisord saved off somewhere to speed up a process that you can do in 4 lines in a systemd unit file.

nickik
Solaris and other Unix had things like systemd already.
evol262
SMF wasn't added until later in Solaris 10. Other unixes (Tru64, HP-UX, Ultrix, AIX, IRIX, etc) did not have anything like it. They were either BSD init or sysvinit
JdeBP
Rubbish. AIX had the SRC doing service management since 1992. And the SAF, also in Solaris, had been doing management of several classes of services since 1988.

* http://jdebp.eu./FGA/inittab-getty-is-history.html

* http://jdebp.eu./FGA/run-levels-are-history.html

* http://jdebp.eu./FGA/unix-service-access-facility.html

evol262
This is _exactly_ what I'm talking about. Neither SAF or srcmstr are equivalent to systemd, and you know it. Making the argument that they're even in the same ballpark is fatuous, except insofar as they have some abstract concept of service groups and replacing sysvinit with something better.
JdeBP
What is you are really talking about is self-contradictory drivel, claiming that Unices were diverse and in the next breath stating that they only had "either BSD init or sysvinit". Clearly, you are presenting ahistoric rubbish. Trying to introduce strawmen, as you now are, when shown that is at best diversionary.

The simple, but sadly not well known and in your case outright denied, truth is that the Unix world took stuff out of the old inittab+runlevels system that you erroneously claim was one of the the only two things going, and had service managers with many of the features that one would recognize today, from auto-restart through requirements not to "daemonize" and specialized configuration languages for setting processes up that replace shell scripts and dedicated control/administration tools to per-socket-connection instantiation with environment variables describing the endpoints, two decades ago.

evol262
I honestly have no idea what you're talking about. The idea that UNIXes diverge from either BSD or SysV is not controversial. The fact that virtually every commercial UNIX had a completely different set of administration tools (other than vxfs as a add-on product to many) for any and every administrative task and init scripts in different locations are not orthogonal. "This system derives from SysV and that system derives from SysV, so I can administer them the same way" is complete bollocks and always was, even down to the location of the init scripts. That's the point.

At no point did I make any claim that inittab+runlevels was the "only thing going". I claimed that many systemd detractors hold to some abstract idea of a "pure" UNIX which never existed in the first place. As mentioned in my last comment (or my first comment) recognizing that there are fundamental limitations to traditional init and replacing or supplementing it with whatever (SAF, srcmstr, openrc, initng, you name it) and "being like systemd" are not even remotely the same thing except that they both manage service initialization.

This kind of false dichotomy is the point of the original post, which you missed in your fervor to turn it into a strawman. launchd, upstart, smf, runit, and other modern replacements are also not "like systemd" except that they manage services. SRC didn't replace init _at all_. It was a system run in parallel, as you well know. It was not a replacement for sysvinit, and AIX still has sysvinit. Again, as you well know, but it doesn't fit your narrative.

seba_dos1
busctl looks simply like mdbus made 10 years later
isostatic
Something that annoyed me 10 minutes ago

Old fashioned way, when I restart apache having made a typo:

  Output of config test was:
  AH00526: Syntax error on line 2 of /etc/apache2/sites-enabled/mysite.conf:
  Invalid command 'blah', perhaps misspelled or defined by a module not included in the server configuration
  Action 'configtest' failed.

Now

   [....] Restarting apache2 (via systemctl): apache2.serviceJob for apache2.service failed because the control process exited with error code.
   See "systemctl status apache2.service" and "journalctl -xe" for details.
    failed!
The former is far more informative
gh02t
I mean I agree that it should just show you the output of "systemctl status" in the error message, but your comparison isn't entirely fair. The commands systemd error message give will show you the same thing as the generic sysv output.
isostatic
But it's that type of unhelpful behaviour that leads to dislike and accusations of arrogance (how dare you type a command that doesn't have the word "systemd" in it)
kbenson
Is it unhelpful, or just different? In the case of systemd, isn't it logging all the errors in the journal so you can see prior occasions it had a problem and errored? What is the error output in the jouirnal? Is it better, worse, or the same?

Systemd makes the process two steps, which you think is worse, but you've failed to address whether the possible benefits of those two steps (error retention, possibly more or different levels of error reporting) have been taken advantage of, and how that looks.

There's a difference between something being worse and just different. So far all I've seen is different, and possibly worse, but you've left out the information that would be required to determine that.

sliken
Systemd is pretty good at hiding error messages. Even if you run the commands it claims will show you error message it makes stupid assumption that your window is infinitely wide or that you want lines truncated instead of wrapped. Even with an infinitely wide window you still don't get nearly as much debugging info as previous systems. For anything complex I end up reverse engineering the startup script and running it from the command line. Then if I update the service systemd notices the file changed, but instead of loading it it runs the old version and complains.

It really seems rather sysadmin hostile and replaces tried and true daemons like ISC's named or unbound with a rather poor systemd replacement.

isostatic
When you're using an interactive shell to restart a process, and it fails, why wouldn't you want it to tell you why it failed.

  sudo systemctl restart apache2.service || systemctl status apache2.service
does the job, it's just ridiculous to seperate a system that used to be helpful.

Of course if you run sudo systemctl restart apache2.service You get no visual indication it's worked

  sudo /etc/init.d/apache2 restart
  [ ok ] Restarting apache2 (via systemctl): apache2.service.
tells you it worked, at least on this ubuntu 1804 machine.

This regressive change in behaviour (taking away pertinent information for no obvious benefit) leads to assumptions being made that things are getting worse for normal people. This view is then backed up by the volume of "defending systemd" statements that seem to stem from "scripting is hard".

People get turned off before they even begin to understand what systemd is trying to do, not because of core reasons, but because of one of the first interactions the average person has with systemd is to make their life harder.

gh02t
I mean, I still think it's just a matter of being different. What systemd does is actually more consistent with the old Unix philosophy (in this case at least, systemd is hardly a paragon of Unixy-ness), where a command succeeding without a problem should just return silently. If it fails, you get a short message that says why and reminds you how to check the logs.

I agree that it should just output the log messages, but I don't think there is any serious UI problem here apart from just being different. It's also worth noting that the detailed error message you get from Apache isn't universal and depends on the app and how the init script is written. Plenty of sysv services just give you a cryptic error message on failure, e.g. when I restart postgres on my FreeBSD server with invalid configuration it just says "failed" and I have to then enter another command to see why.

kbenson
> This regressive change in behaviour (taking away pertinent information for no obvious benefit)

There is a benefit. The benefit is that it's logged, always, and through a mechanism that is the same for every process. It's a trade off. We can argue whether it's a worthwhile trade off, or one that even really needs to exist (is there a reason they don't immediately show an error? I don't know), but to say there is no obvious benefit even after I pointed out this specific benefit in my prior comment is odd.

> People get turned off before they even begin to understand what systemd is trying to do, not because of core reasons, but because of one of the first interactions the average person has with systemd is to make their life harder.

Any change from what existed previously that requires a different way of doing something, whether ultimately better or worse, makes the user's life harder in the short run since they need to learn something new.

That's not to say that's all that's happening here. There appears to be a regression in usability in at least one aspect. That said, I sort of understand why they didn't make it output that info. There have been plenty of times in the past for where sysV has reported a service started, but it actually failed very shortly after start. Generally, figuring out why can be painful, because STDOUT/STDERR have been redirected (or likely sent to /dev/null). Keeping a log of the stdout so you can see what happened on that last service action (and not hope it was logged to a log file, which it sometimes isn't...) is very useful. Outputting that the service started successfully is nice. Outputting that it started successfully when it didn't is misleading and more of a problem. Again, that's a trade-off, where it appears they've decided to favor accuracy over convenience in this case.

loeg
> The benefit is that it's logged, always, and through a mechanism that is the same for every process. It's a trade off. We can argue whether it's a worthwhile trade off, or one that even really needs to exist (is there a reason they don't immediately show an error? I don't know)

Yeah, it seems feasible (and maybe a good default) for systemctl to dump out the same last few lines it would for 'status' if an administrative request fails. I don't know why this has to be a tradeoff; systemctl actions could easily both log and print (some limited subset of) the log on error.

isostatic
Logging it is great, it's the fact the interface says "look in the log" rather than doing it for you.

It's tiny annoying things like that which leave a bad first impression. Rather than going "hey this system is great, it logs what it tells me", instead I'm left thinking "this system isn't for people like me, it doesn't think I need to know why it's broken"

First impressions are so important.

lightedman
"Logging it is great, it's the fact the interface says "look in the log" rather than doing it for you."

So now I have to go digging instead of being told upfront what the problem is. No thanks. Time is money, and having to dig through a logfile when simply putting the error message after the failure to execute would have sufficed, just wastes time and money.

That's why we removed every Linux machine from our production lines at the solar plant I manage, and went back to (ugh) embedded Windows. SystemD is a waste of time and money in every single one of our experiences with it.

cr0sh
What I have come to believe is the "Tragedy of Systemd" - and if I am wrong on this, please let me know - is that it was a virtually unilateral decision to make the change, by one person, without any input from others.

It was like all at one shot - here's this huge and largely non-transparent init system created by seemingly one person who it felt like was shoving it down everyone's throat. My way or the highway style.

That, on top of what felt/feels like a lack of transparency about how it works (I could look at init scripts and understand them fairly easily and how they worked together - systemd seems almost a mystery to me; I just have to trust it works - so far so good I guess).

Maybe I am wrong on all of this - again, if I am (and there's a good chance I am I feel) - please let me know.

JdeBP
You are wrong. Read https://news.ycombinator.com/item?id=14735100 .
fridsun
I just must let this one thing out.

> 34:02 binary logs are not a bad thing as long > 34:06 as you have the tools to pull them apart

Really? Did he just "as long as" the biggest downside of binary log compared to text log, that is way fewer tools work on a binary log due to its special encoding?

With the same logic, flu viruses are not a bad thing as long as you are immune to them.

Gby56
Great talk imo, my TLDR would be : > systemd is not the best it could be, but it's there. > distros patting themselves on the back because they don't use it means they haven't listened to actual needs and stayed behind. > death threats over software that nobody else tried doing are ridiculous
episodeiv
> death threats are ridiculous

Fixed that for you...

mondoshawan
Don't forget the gaslighting with "software has bugs" and "you just dont like change", while glossing over legit complaints about the outright narrow minded design of the system that is actively harmful in some cases (systemd-timesyncd, I'm looking at you).

Personally, the video made me rather angry because all he does is set up straw men and knock them down using the same terrible pro-systemd arguments that get trotted out every time someone has a legit problem with its design.Some of the distros that use systems like runit and openrc are using those sustems because systemd doesn't fit their technicals needs, and likely won't ever because of designs that are made without community input.

Oh, sure, he mentioned the Debian developer resignations, but outright failed to point out the reason why they resigned: systemd was being railroaded into the system as a political move rather than being judged on its technical merits.

theamk
> Oh, sure, he mentioned the Debian developer resignations, but outright failed to point out the reason why they resigned: systemd was being railroaded into the system as a political move rather than being judged on its technical merits.

No. Ian resigned because he tried to ignore systemd's technical merits and tried to used political moves (GR process) to to kill it. As a result, most of Debian's community told him to top messing around with politics. Ian's resignation letter [1] explicitly says that:

> I should step aside to try to reduce the extent to which conversations about the project's governance are personalised. [...]

> The majority of the project have voted to say that it was wrong of me to bring this GR at this time.

[1] https://lwn.net/Articles/621895/

Please stop saying systemd was being railroaded. I have no idea what was going on in Redhat (I do not follow their mailing lists), but at least in Debian, it got on its own technical merits, and Poettering had no special influence there.

(That is not to say it is without problems; there are plenty of them. Still, the lesser of evils...)

loeg
> Please stop saying systemd was being railroaded. I have no idea what was going on in Redhat (I do not follow their mailing lists), but at least in Debian, it got on its own technical merits, and Poettering had no special influence there.

Systemd got in on its merits in Fedora as well. (It had to — no amount of railroading is going to overcome something totally dysfunctional, and Fedora was the first adopter.) As I recall, the transition was fairly smooth.

isostatic
I think that the section on change at is great. The first 1500 seconds are about systemd any why it is and what it's trying to do, but that doesn't really get to the core of why I complain about it.

https://youtu.be/o_AIw9bGogo?t=1526

Systemd is change. Massive change that's being forced on us by a twat. Unlike Linux, which was a kernel delivered by a twat but was take-it or leave-it, this is being forced on us. It's untrustworthy and threatening and unknown. In some cases it's an attack on our identity -- "the way you did it in the past is rubbish, do it this way". Clearly we are going to be defensive about it.

This isn't unique to people who've used linux for the last 3 decades. Douglas Adams knew this 20 years ago:

  1) everything that’s already in the world when you’re born is just normal;
  2) anything that gets invented between then and before you turn thirty is incredibly exciting and creative and with any luck you can make a career out of it;
  3) anything that gets invented after you’re thirty is against the natural order of things and the beginning of the end of civilization as we know it until it’s been around for about ten years when it gradually turns out to be alright really.” 
His original article (in a newspaper - remember them?) is at http://www.douglasadams.com/dna/19990901-00-a.html

He mentions John Humphrys and his sneering view of this new fangled internet. Humphrys was (and still is) the bbc radio 4 breakfast presenter.

At the same time Terry Wogan was the radio 2 breakfast presenter, a similar age (about 60 at the time). His attitude was far more opening, energizing a whole generation of middle-aged Brits to technology and the internet. Change was something to be embraced, not feared.

This talk has made me realise that my attitude to systemd is far more like Humphreys than Wogan, and that scares me.

mmastrac
"forced on us by a twat" is not constructive language.
isostatic
Pretty much a quote from the video

"ability to have sympathy with the people who's he's asking to change doesn't appear to be the greatest"

creshal
The real tragedy is that so many complain about systemd and insist it's awful, yet nobody ever bothers to actually make a better init system. It's either 1980s crap that's totally unsuited to modern environments, or systemd with all its bugs and design flaws.
onli
There is runit, used in void. And OpenRC in Funtoo/Gentoo. And the systems used in alternative Operating Systems, Solaris just got mentioned.

There are a bunch of better alternatives out there.

Accacin
I believe in the talk he goes into why this (in his opinion) isn’t the case.
onli
I don't think so? I skipped only the questions, maybe there? I didn't hear him mention those other systems, might have missed that.

You can argue that he does that indirectly: He likes that systemd provides something like a service layer, which the tools I mentioned don't really do. Maybe you mean that. That would work, I just happen to think he is wrong there. That's not the job of init, making it a plus point of an init system is some kind of mental gymnastics I don't agree with (though I liked that talk).

Bluecobra
I thought that Sun did a pretty good job with SMF in Solaris 10, it’s too bad that Oracle did everything they could to ruin Solaris.
sgt
They say Larry Ellison is still burning Solaris and other Sun books over the weekends. At one point his Japanese bamboo house almost caught fire.
isostatic
What modern environments?

The problem with systemd is that it's replaced systems that met peoples needs with a single system that many people consider to be worse.

Perhaps there are some niche environments that systemd is needed, and that's great -- but why break things for everyone else?

loeg
To flip this around, if you replaced systemd with the old rc init in e.g. Fedora today, you could make the exact same argument:

> The problem with [old init] is that it's replaced systems that met people's needs [systemd] with a single system [old init] that many people consider to be worse. > > Perhaps there are some niche environments that [old init] is needed, and that's great -- but why break things for everyone else?

So the argument is kind of meaningless without a tie-breaker of some kind; perhaps popularity vote.

I think you're under the impression that your views of systemd's relative suitability are held by the majority of users, but I believe that is incorrect.

isostatic
Change is something many people are naturally averse to, so to change something there should be a good reason for it.

If you sell something and people choose to use it to solve a problem, that's great. If you force change on someone, and not only not deliver any benefit to them, but actually make things worse, expect pushback.

Remember too that most people's systemd impact wasn't just replacing init, it was a whole raft of changes from log formats to dns.

Imagine you had a working systemd setup and someone came along and replaced everything. They claim that there are use cases it's better for, but you don't see yourself with those use cases. You would be averse to that change. You then find out that pretty much every distribution going has got rid of systemd. I suspect you wouldn't be happy.

keypusher
No, the tragedy is that you didn't watch the video, because the author isn't complaining or insisting that it's awful. Instead, he (a well-known FreeBSD developer) goes through the history of init systems and offers some justifications for why systemd is the way that it is, and what challenges might need to be overcome to make a "better" init.
colpabar
zing!
antirez
The tragedy of systemd is that it's the wrong solution to a real problem. Because most people against systemd will keep saying that the old init system was ok, they don't move forward trying to provide a true alternative. So systemd became a standard despite the issues, because it solves certain real problems that people have, even if not great.
upofadown
>Because most people against systemd will keep saying that the old init system was ok, ...

There isn't and wasn't hardly anyone saying that. It is and was well known that there were better things out there than the hacked up sysV style init systems produced by the various Linux distributions.

This is a common strawman that seems to get rolled out any time any discussion involves systemd.

joepie91_
Nearly every systemd-disliking person I've personally spoken to, has trotted out that exact line about sysV init being fine.

It's not just a strawman, unfortunately. I wish it were.

nickik
And yet basically no alternative to systemd was produced by any of those people.
bluejekyll
Does upstart not count as an alternative?
loeg
Is upstart better in any way?
sliken
Upstart doesn't include implementations on NTP, DNS, DHCP, or anything that opens a network port. It does however allow parallel boots with dependency analysis for quick boots.

Systemd on the other hand seems to be trying to take over syslog, ntpd, dhcpd, named, and related with generally poorer implementations. Feature creep seems constant with systemd, for no particularly good reason.

nickik
Upstart is older then systemd and it was mainly developed by Ubuntu who dropped it. After that is continued to exist as a project but its basically not used anywhere and id didn't continue to develop all that much.
bluejekyll
I didn't mean to imply that it was built as a competitor to systemd, more that it exists as a different alternative. I'm aware of what happened with Ubuntu dropping it, and I think it's a shame, as I like it as a simple response to the problem.
antirez
A few of the authors of one of the Debian based distributions dropping systemd are friends of mine and all of them see things in that way, that the good old init is fine and there is no need to replace it. In the Facebook group where they use to stay the others I don't know first-hand seem to share this POV.
isostatic
But it's not just replacing init - it's replacing a dozen other services including things like ntp or dns, and often it's replacing them with inferior solutions.
antirez
Yes, that's a design problem due to what the systemd main author believes to be a good thing.
nickik
systemd can be used without most of these things. systemd as a project has other services in but those you can use or not use.

And the claim that they are 'inferior solutions' is just absolutely false in most cases. It often replaced horrible old tools that haven't been updated in 30 years. It often provides far better interfaces and tooling then the existing solutions.

None
None
isostatic
The DNS resolver seems to break DNS, the time system no longer acts as a server

This isn't necessarily a problem with the systemd project, it's more on the distros that bundle these service, but when you have the same branding across multiple systems, you have to take the bad with the good.

Had they called it "multidnsd" or "ntpclientd" then people wouldn't think "systemd sucks because when MyDistro 9 came along", they've blame multidnsd and let the systemd init system stand on it's own.

zaarn
I don't recall the DNS resolver actually breaking DNS without it being fixed. And I'm not sure why most systems need NTP to run as a server.

You can always disable them at runtime or during compile and use your own though.

kbenson
My understanding is that the reason it started replacing other services is that there's a dependency graph that ends up pulling them in, and very early in the boot process depending on configuration[1]. That, and in some cases the available solution did not support their needs[2] (which might have been solved better through upstream work, or maybe not, if the targeted project was slow or unwilling, I'm not sure).

1: https://lists.freedesktop.org/archives/systemd-devel/2014-Ma...

2: https://www.reddit.com/r/linux/comments/6kdcme/why_does_syst...

altmind
>Because most people against systemd will keep saying that the old init system was ok, they don't move forward trying to provide a true alternative.

The problem with systemd it is not just an init system. Its network-state-aware, device-state-aware container launcher(nspawn), cron, logind and syslog implementation.

The scope creep of systemd is enormous, nobody asked for that when they wanted reliable daemons.

w323898
More or less this. Gummiboot is nearly pointless, and ntpd and resolved have both caused problems for me. As an init system, systemd is fine, but shipping these bad components while giving them the systemd "seal of approval" is dumb.

On the other hand, I've had no issues with logind and gdm. I've been interested in trying S6 but given it's quite easy to substitute out the bad parts of systemd, so I just haven't bothered.

Probably the biggest problem with systemd is that it's hurting BSD. Then again, gnome devs are having enough trouble getting things working on Linux... also binary format journals are so dumb. How have they not changed that?

dvfjsdhgfv
The tragedy of systemd is tight coupling. What makes Linux great is that I used to always have a choice - regarding what kernel I use, what editor, what shell, even what Init I use... Systemd changed all that. Distro makers gave up supporting alternatives because it would mean too much work. (I almost flipped out when I read somewhere they had to do it because of some Gnome-related component... I mean I don't care about Gnome, I don't even use X, but I have hundreds of servers to manage - now let's change everything because one guy said it's the only way...)

Nobody would make what Lennart did if we had a choice. Those who liked Systemd would use it, and the rest would use the init of their choice. As simple as that.

kfwhp
systemd became a standard because the guy works at Red Hat so he can make GNOME require it (there is no way to use GNOME without systemd; distros that allow that such as Gentoo do so by patching GNOME, which is costly.)

I agree that a more modern init system is necessary, but the idea of systemd is meh and the implementation is subpar.

tapoxi
systemd became a standard because Debian adopted it after a lengthy discussion, partially because big Debian users like Spotify expressed an interest in systemd.

https://lists.debian.org/debian-ctte/2014/01/msg00287.html

kevin_b_er
The intentional and willful breaking of screen and tmux was to fix a GNOME bug of GNOME not closing up as it should when the user logs out, so systemd was changed to mass kill processes. The interplay between GNOME and systemd in backroom dealings is a major pain point.
kbenson
I think that's an oversimplification. There are definitely cases where some thing started as a daemon under a user session, graphical or not, should be killed to be safe. For example, ssh-agent.

There are other things that are meant to be left around, as that's their purpose, such as screen and tmux.

I would say the safe solution would be to kill everything in the user session at multiple levels (GNOME, and user session management demon like logind, etc), and to provide a defined safe way for the very few programs that want to stick around to do so. That seems like what systemd came up with as well.

nickik
Conspiracy nonsense. It became standard because the existing solutions were crappy and nobody else bothered to do anything about it.
kfwhp
I don't believe this to be a "conspiracy". It's nothing shadowy, really. What they did is completely natural. That doesn't mean I support the idea though.
taffer
You make it sound like the systemd author was something the CEO of all Linux Distros.

The maintainers of Arch Linux have given an explicit list[1] of the reasons why they decided to introduce systemd:

    0) it is hotplug capable
    1) we can know the state of the system
    2) it is modular
    3) it allows dbus/udev to go back to doing the task they are meant to do
    4) we can reduce the number of explicit ordering dependencies between daemons
    5) we get a lot of security/sandboxing features for free
    6) systemd service files can be written and distributed upstream
    7) systemd is a cross-distro project
    8) logind will finally deliver on what consolekit was supposed to do
    9) systemd is fast
[1]: https://bbs.archlinux.org/viewtopic.php?pid=1149530#p1149530
sametmax
I concur. Before systemd, I used supervisor because I wanted to avoid using any init system like hell.

Since systemd, I just do a service file.

So despite all its shortcommings, I now have a standard, well documented, init system that I can use accross all the distros I use, which happens to provide all the features I need.

I'm sure all the power user complaints about systemd are legitimate, but I really don't care about those. I just want an easy way to setup custom deamon on ubuntu and centos.

Provide a better system, that solve those problems, and I'll use it. I really don't care as long as my problem is solved.

onli
> So despite all its shortcommings, I now have a standard, well documented, init system that I can use accross all the distros I use, which happens to provide all the features I need.

Same couldn't be said about supervisor, the solution you already had running (well, maybe strike init)?

isostatic
systemd is far more than an init system though, it's a total rethink of how linux systems are controlled and managed.

Rewatch the first half of the video and imagine it was a bunch of random tools with proper names (like grep, ntpd, syslogd) etc, that had all been built separately but happen to nicely tie into each other, rather than "systemd-blah"

sametmax
It's not installed by default on any distro. I don't want to install something as basic as an init system.

And it's not installed by default because it doesn't solve the problem of a service dependancy graph.

onli
Sorry, but that first argument seems a bit random to me. Couldn't even use a proper editor like nano with that argumentation, having a usable editor available is after all also pretty basic.

I doubt that supervisor is not installed because of not solving something it evidently does not want to solve, but that argument at least looks valid to me. Just in case you are not aware, https://github.com/Supervisor/supervisor/issues/122 lists two solutions (wait_for_it.sh and a supervisor plugin that adds dependency resolution).

sametmax
I'm not tring to convince you, i'm sharing my pov. Systemd is here because of pov like this, and people not liking it end up with it anyway because they fail to aknowledge others pov.
isostatic
> proper editor like nano

Not sure what your point is (well ok, I am, and I agree), but you just lost

kbenson
I think "basic" in this case means "integral to the system". Many people don't want to replace the init system, nor the desktop display system, nor an authentication system (e.g. PAM).

The more systems something sits under the more places there are for it to cause a problem because it doesn't work quite how the upper components expect. At a bare minimum, you'll often run into problems with the included package management system (and how it may reload systems for security purposes after patching) if you don't take care.

nano sits on top of everything, and the few things that might rely on it use a very standard and not very extensible interface (the EDITOR environment var). It's not really equivalent at all.

Feb 13, 2019 · 17 points, 2 comments · submitted by tambourine_man
None
None
scartracs
This was a great talk from a FreeBSD dev praising systemd and putting it in a different perspective. He did a great job of explaining the relevant Unix history as well.
Guph
Good talk. I still hate systemd, to complex causing more issues than it claims to fix.(most of which I didn't see as problems) init could have gone modern without scattering it's files all over the file system, and turning into a borg monstrosity (cronyd,firewalld,journald,etc).
Feb 10, 2019 · 2 points, 0 comments · submitted by tosh
Feb 08, 2019 · 10 points, 0 comments · submitted by devnonymous
Feb 06, 2019 · 3 points, 0 comments · submitted by gcoleman
Jan 29, 2019 · voltagex_ on Systemd as tragedy
I'd recommend everyone watch the video: https://www.youtube.com/watch?v=o_AIw9bGogo
HN Theater is an independent project and is not operated by Y Combinator or any of the video hosting platforms linked to on this site.
~ yaj@
;laksdfhjdhksalkfj more things
yahnd.com ~ Privacy Policy ~
Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.