HN Books @HNBooksMonth

On that topic: I came to like John Ousterhout's notion (in [1]) of "deep" methods/classes/etc. He explains it like this: If you have a reusable piece of code, imagine drawing it as a rectangle. The width corresponds to the complexity of the interface, the depth corresponds to the complexity of the implementation.

Ousterhout is saying that we really want deep classes, i.e. the complexity of the implementation a construct hides should be considerably larger than the complexity it takes to use it. Which, at least to me, basically says "search for good abstractions". And what you're describing strikes me as a good way not to achieve that.

Of course, there's considerable wiggle room in that mental model, and one could still endlessly squabble about how (not) to break up the implementation.

[1]: https://www.amazon.com/Philosophy-Software-Design-John-Ouste...

⬐ BurningFrog

Sound about right!

I think the fallacy of the "Poltergeist Pattern" is to optimize solely on the - in itself valuable - metric of short simple methods, without consider anything else, like the complexity of using or reading the resulting code.

Sounds similar to Ousterhout's Philosophy of Software Design!

https://www.amazon.com.au/Philosophy-Software-Design-John-Ou...

The main point is that a software developer's job is to fix stuff so you end up introducing complexity, but you will have a much easier life if you realise when you're about to introduce more complexity, and try to minimise it by thinking about your implementation choices. KISS, I guess.

⬐ applepple

Nice, this is a broader way to explain the effectiveness of my CSS approach and many other approaches. IMO, Modern developers should read fewer books about tools and more books about software philosophy. It's far more important.

It's weird to think that philosophy can actually yield productivity gains... Many people think of philosophy as being the anti-thesis of productivity.

You’re right re: it’s early internals, and we probably both know people who still (mistakenly) complain about that. Conceptually the “everything is a string”(EIAS) model still holds though, even though it’s supported by dual-ported “objects” w native representation, and the running is often byte-compiled instead of string parsing. We’ve got early smart fans who saw the vision and possibilities like Karl Lehenbauer, Don Libes, Richard Hipp, and the current core team like Kevin Kenny, Don Porter, Donal Fellows, etc who deeply understand What Tcl Is, and strictly maintain it, even while pushing it ahead (TclObj, finally an in-core blessed object system, native database framework, Miguel Sopher’s(RIP) work on the stackless non-recursive engine (NRE) and coroutines that fell out of that work...

Back to Ousterhout though, you might dig his “Philosophy of Software Design”[0] book, or talks associated with it[1].

Lots to like in this space...

[0] https://www.amazon.com/Philosophy-Software-Design-John-Ouste...

[1] https://youtu.be/bmSAYlu0NcY

⬐ tyingq

Just bought the book. Thanks for replying with some valuable insights!

The book "A Philosophy of Software Design" should interest you then: https://www.amazon.com/t/dp/1732102201 It argues, among other things, that deep interfaces matter more than code complexity inside a module.

⬐ Chris_Newton

That one was the first software book I read in a while where I got to the end and felt like if I wrote a book myself then that is very close to what I would want it to say. I highly recommend it to anyone who has built up a bit of practical programming experience and wants to improve further.

Highly recommended this book:

A Philosophy of Software Engineering:

https://www.amazon.com/Philosophy-Software-Design-John-Ouste...

This is mentioned as one of the symptoms/appearance of bad code: Shallow/Passthrough methods, which results deep call chains, adding up cognitive overload. The author in this book recommends deep module over shallow module, which tends to end up with more cohesive code.

I think the best single observation about cognitive load is in Ousterhout's book A Philosophy of Software Design[1]. In the book he promotes the idea that classes should be "deep", such that their top-level surface API is small relative to the complexity they hide underneath.

This applies to the microservice/monolith debate as well. And it basically boils down to the observation that having lots of shallow services doesn't really reduce complexity. Each service may be simple unto itself, but the proliferation of many such services creates complexity at the next level of abstraction. Having well designed services with a simple API, but hide large amounts of complexity beneath, really reduces cognitive load for the system as a whole. And by "simple API" I think it's important to realize that this includes capturing as much of the complexity of error handling and exceptional cases as much as possible, so the user of the services has less to worry about when calling it.

[1]: https://www.amazon.com/Philosophy-Software-Design-John-Ouste...

⬐ save_ferris

I've noticed this as well. It's much easier to find the logic I'm looking for when I can easily remember where the domain of one class ends and another begins.

⬐ 29athrowaway

Encapsulation & information hiding in software design is an innovation from the 1970s.

⬐ averros

Every generation of new coder kids rediscovers what their parents already knew pretty well.

It's just arrogance and desire to hack away without actually being bothered to learn the craft.

⬐ jillesvangurp

Yes, Ousterhout's work is still a great read decades after he published.

What people forget when doing microservices, server-less, or other 'modern' ways of breaking up software into more or less independent things is that these are just variations of decades old ways of breaking stuff up. Whether you are doing OO, modules, DCOM components, or microservices, you always end up dealing with cohesiveness and coupling. Changing how you break stuff up does not change that. Breaking stuff up is a good thing but it also introduces cost depending on how you break things up.

In the case of microservices, the cost is automation overhead, deployment overhead, and runtime overhead. If your microservice is a simple thing, it might be vastly cheaper to build and run it as part of something else. I've actually gotten rid of most of the microservices and lambdas we used to have to get a grip on our deployment cost and complexity. We weren't getting a lot of value out of them being microservices. The work to get rid of this stuff was extremely straightforward.

⬐ touristtam

> In the case of microservices, the cost is automation overhead, deployment overhead, and runtime overhead.

Having just migrated a software to AWS, and seeing other project being re-engineered as lambdas, I can only related to this.

From the link, about comments:

  Use as explanation of intent.
  Use as clarification of code.
  Use as warning of consequences.

Here's a link to a well-known recent book, half of which is pretty much about why those points are a real and frequent thing, and why code should be much more thoroughly commented than it's recommended by the usual philosophies.

https://www.amazon.com/Philosophy-Software-Design-John-Ouste...

1. The Problem of Pain by CS Lewis 2. A Philosophy of Software Design https://www.amazon.com/dp/1732102201/ref=cm_sw_r_cp_api_i_EK... 3. Haskell from First Principles http://haskellbook.com 4. Les Miserables 5. Continuous Delivery https://continuousdelivery.com

Thanks everyone, the comments are much appreciated. Here's a list of books and other media resources recommended so far in the thread:

Robert C. Martin, Clean code: https://www.amazon.com/Clean-Code-Handbook-Software-Craftsma...

Vaughn Vernon, various: https://vaughnvernon.co/?page_id=168

Steve McConnell, Code Complete: https://www.amazon.com/Code-Complete-Practical-Handbook-Cons... 2

Clean coder: https://cleancoders.com/ videos

Hunt and Thomas, The Pragmatic Programmer: https://www.amazon.com/Pragmatic-Programmer-Journeyman-Maste...

Hitchhiker's Guide to Python: https://docs.python-guide.org/

Dustin Boswell The Art of Readable Code: https://www.amazon.com/Art-Readable-Code-Practical-Technique...

John Ousterhout, A Philosophy of Software Design: https://www.amazon.com/Philosophy-Software-Design-John-Ouste... This one looks particularly interesting, thanks AlexCoventry!

Kent Beck, Test Driven Development: https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/...

Dan Bader, Python Tricks: The Book: https://dbader.org/

Ian Sommerville, Software Engineering: https://www.amazon.com/Software-Engineering-10th-Ian-Sommerv...

Svilen Dobrev, various: http://www.svilendobrev.com/rabota/

⬐ blub

Skip anything by Robert Martin (clean coder series) and read at first Ousterhout and then McConnell instead.

Martin is well intentioned, but very dogmatic about some things like TDD, functions size, personal responsibility, etc. You need to already have some decent engineering experience to be able to detect and ignore the harmful stuff from his books.

⬐ n4r9

I'd like to re-emphasise sanderjd's point not to focus too much on reading books. I myself went from doing a PhD and a lectureship in mathematics (with some coding here and there) to a decent software engineering job in a smallish company. I've learnt everything on the fly by reading code, searching stack overflow, trying stuff out and coding alongside others. The great thing coming out of a PhD is not just that you have to be pretty smart to have done it: you now know you can grasp almost any aspect of human knowledge with sufficient brain racking. This is a vastly underrated piece of self-awareness which enables one to stay humble and tenacious.

⬐ AlexCoventry

Yeah, I think of a PhD as a kind of intellectual adolescence. It led me to far greater intellectual independence.

⬐ sanderjd

There are a lot of good recommendations here, and I certainly relate to the instinct to go to books when you're looking to level up a skill set, but I really think what you need is not a bunch of books to read, but a few people to watch do the work. The only real way to do that is to get a job alongside them. You can read the books at the same time; you can ask your new coworkers which recommendations they agree with and read those ones first.

⬐ fsloth

Yeah, software engineering is a craft, and generally the only way to learn those fast is to learn from others.

⬐ blub

It's not a craft, in its purest form it's an engineering discipline with specific rules, procedures and standards.

The crucial point is that most of us a doing programming, and not software engineering. Learning from others is hit or miss. One can certainly learn to program from others, but that's not enough to be able to do software engineering.

⬐ sanderjd

Convince me that its "purest form" is an engineering discipline rather than a craft. What distinguishes it from things that you would agree are crafts? Or are all crafts actually engineering disciplines in their purest form?

I think this is a pretty interesting question. Personally, when I was young, I would have said what you said: it's engineering, specifications go in and properly engineered finished product come out. I was proud that my CS program was contained in an engineering school and that all my friends were in other engineering disciplines. The longer I do this though, the more I think it's a craft: fuzzy idea is put to a skilled practitioner and through discussion and creativity, one or more artifacts are created to satisfy that idea. You could argue this is true of more traditional engineering disciplines as well, and I would probably agree. I'm not totally sure what the distinction between the two things is. So tell me what the distinction is and convince me what we do is more the one than the other.

For what it's worth though, apprenticeship is also a very important part of engineering education. At some point, you have to see people do the thing, regardless of whether it's engineering or a craft.

⬐ blub

When we say software development is a craft, we're saying that it's like shoemaking, pottery or woodworking.

Can the immense complexity of today and tomorrow's software be tamed by applying the same principles of building a cupboard? No, it requires an engineering mindset.

We're now limping along as an industry and it's not obvious because SW is bringing in massive amounts of money and we can basically get away with a lack of quality.

⬐ fsloth

"When we say software development is a craft, we're saying that it's like shoemaking, pottery or woodworking."

The point being, there are intricate details that are very hard to deliver in the traditional class-room oriented school environment with well defined requirements.

Those do not state anything about scalability. Crafts can scale - like for example how the old giant cathedrals and castles were built in middle-age europe.

They don't say anything about mindsets either. You need an engineers mindset to buid a cathedral or a castle.

The specific problem with crafts is that adapting to new requirements is a complete hit and miss process. The reason for this problem is the lack or proper theoretical framework in which to pose ones work and into which embed the requirements themselves.

The program verification people are working towards solving this problem, see for example Leslie Lamport's work in TLA+.

But until we have a general, mathematical proof backed compiler for requirements, as well as for the program implementation, we are pretty much stuck with craftsmen.

(Well, we have proof compilers but those are at the moment completely unusable for general programming since they are so complex to use.)

⬐ fsloth

"It's not a craft, in its purest form it's an engineering discipline with specific rules, procedures and standards."

Sorry, but I have to strongly disagree. In it's purest form the core of software engineering - i.e. programming is a craft. The other parts are mostly about creating processes so that craftsmen can create something together without stumbling into eachother.

The difference between a craft and engineering are numerous.

- engineers generally need a license

- engineering is about repeatability and creating dependable cost estimates

- engineers are required to study for years for a very good reason. You can be a rockstar programmer out of highschool.

Just having a bunch of cargo cult gibberish bound into a book does not make a craft into an engineering discipline.

It's harmfull to call programming engineering. Engineers have curriculums that can teach them pretty well what is expected of them once employed.

Not for programmers - or, well, software engineers. If there was even one curriculum that could churn out good programmers dependably, don't you think this model wouldn't be copied instantly elsewhere? If such a curruculum existed, do you think think software interviews would be filled with whiteboarding just to check out that the candidates understand even the basics?

I think this incapability to create a curriculum for actually creating good programmers is the best evidence that programming is a craft. It's such a complex topic that you can't create a mass curriculum that would serve all equally. Not with our current understanding, anyway. Maybe if we could teach everyone assembly, and Haskell , and have them implement compilers and languages as a standard things would be different.

The second best way to learn programming without being born a programmer savant is to learn from others while doing. Apprenticeship is the traditional way to train craftsmen.

Programming is so much more like a craft than engineering that it's best to call it a craft.

Craft is not a deragatory term. It just means we don't understand it theoretically well enough to teach it properly.

⬐ sanderjd

I tend to come down on your side of the craft vs. engineering debate, but then I disagree with basically your entire argument for it :)

You list three distinguishing features of an engineering discipline. The first and third swap cause and effect; a field doesn't become engineering because once it requires licensure and years of study. Surely you wouldn't agree that a successful campaign to require those things for software developers would bestow the status of engineering upon the work.

Your second point seems closer to the truth, but I'm not so sure it's true. If someone comes to a civil engineering firm and says, "design us a road between this city and that city", that is often a unique challenge because the terrain between those two cities is likely unique, maybe it requires a big bridge or a tunnel, which will be akin to but not identical to other bridges and tunnels you've built before.

⬐ fsloth

"The first and third swap cause and effect;"

Sorry, I wrote that in a hurry. I wasn't claiming either was a cause or effect. It was more of finding characteristics that we can use to identify one from another. I.e. following the argumentation "If it quacks like a duck and walks likes a duck it's likely a duck, and if it doesn't, we don't really have much evidence of the duckish quality of the observed thing".

So, I was not aiming to claim that licensure would turn software engineering into actual engineering. Rather, that the requirements of the field are so poorly understood in the general context that there would be very little to agree on the specific requirements. Poorly understood -> not engineering, really.

I totally agree with what you wrote above.

On the third point: I'm not claiming 100% truthiness to my argument, but it's pretty close. Software engineering projects are still among the riskiest ventures where you can think of investing capital in. If you want to build a road:

1. The language of the requirements are pretty well understood, from point A to B, this many lanes 2. Unless some unforeseen calamity arises, and you have the capital to pump to the project, eventually you will get the road

I think we can agree that these define any engineering project. Of course, engineeering is not cut and dried either - that's why you need to have actually trained professionals who can react to the events that come along as the project progresses.

I don't think 1. or 2. can hold for a software project in the general sense. Furthermore, you can end up accidentally, without anyones fault, wasting an arbitrary amount of capital on a feature that could, in the worst case, be replaced by a few lines of Python.

This poor quality of our general understanding of software development and lack of common language to describe anything means that most of the time software develoment is closer to R&D than engineering.

Generally, you can get better estimates when you are implementing a similar project the nth time. Like some general website, or a server backend. And in these instances you have the language to describe features and requirements. But in the general sense, software development isn't anything like this.

⬐ blub

Software development as practiced now by a huge number of individuals and companies is closer to a craft, but it can be and must be more than that if we want to be able to tackle the growing complexity of software and improve its overall barely adequate quality.

Crafts don't scale and are a poor fit for highly complex domains.

The curse of software development is its huge financial success, anemic legislative specification and the observed reality that customers will still buy poor quality software.

These are preventing the craft-like programming from turning into software engineering, but the craft is already failing to reach expectations: countless security disasters, unethical programmers enabling spying on millions, software literally killing users. This stuff will only get worse.

And finally, we do understand software engineering well enough to teach it properly. It's just not done, because it's not considered necessary when one can get by with a computer science degree, no degree or a bootcamp certificate.

⬐ fsloth

"And finally, we do understand software engineering well enough to teach it properly."

This is news to me. I would very much like a citation, please. Or do you mean applying formal proof verification to everything?

⬐ blub

Engineering doesn't mean using formal methods or specific fancy proofs, it's a systematic, disciplined quantifiable approach to software. It's described in an ISO standard and the more approachable IEEE SWEBOK.

The above is neither widely known (I only found out about it after many years of doing professional programming), nor is it necessary in order to be successful in the profession and/or make a lot of money.

Commercial software development is mostly a wild west and we're calling that craftsmanship.

⬐ fapjacks

I'll supplement this good advice by recommending pair programming. You will pick up a ton of stuff just sitting down at one keyboard with another programmer.

Another excellent resource is "A Philosophy of Software Design" [0], from John Ousterhout (known for Tcl/Tk, Raft, RAMCloud, Log-Structured File System). Like Niklaus Wirth, he is striving for designs that fight complexities and yield simple and powerful abstractions. This book is so much better than overrated books like "Code Complete" and "Clean Code" for a fraction of the page count.

[0] https://www.amazon.com/Philosophy-Software-Design-John-Ouste...

⬐ tmaly

What section of the book did you enjoy the most?

⬐ lboasso

I enjoyed most of it.

In particular:

  * Ch.  4 "Modules Should Be Deep" 
  * Ch. 10 "Define Errors Out Of Existence" 
  * Ch. 20 "Designing for Performance"

Link: https://www.amazon.com/Philosophy-Software-Design-John-Ouste...

Looks like it was just released!

⬐ davidw

Salvatore 'antirez' seems to think pretty highly of the book too; got a copy on his recommendation and am just starting to read it.

⬐ chubot

For those who haven't seen it:

http://antirez.com/articoli/tclmisunderstood.html

(Ousterhout is the creator of Tcl and both antirez and Richard Hipp of sqlite are Tcl programmers)

For the Tcl internals, was this John Ousterhout's code or some other codebase? Have you read his new (ish) book on Software Design? [1]

[1] https://www.amazon.com/Philosophy-Software-Design-John-Ouste...

⬐ antirez

Yep it was the original JO code base, but I think that when I read it (it was already Tcl 8) a lot of other people had rewritten major parts, but the standard and quality were set I guess, and also the core team was composed of people that knew how to write code. About the new book: ordered it via Amazon, but Amazon Italy for some time now for small things use the local postal service, that always fails, so it returned back... to Amazon. I'll have to purchase it again.