HN Books @HNBooksMonth

The best books of Hacker News.

Hacker News Comments on
Code Complete: A Practical Handbook of Software Construction, Second Edition

Steve McConnell · 32 HN comments
HN Books has aggregated all Hacker News stories and comments that mention "Code Complete: A Practical Handbook of Software Construction, Second Edition" by Steve McConnell.
View on Amazon [↗]
HN Books may receive an affiliate commission when you make purchases on sites after clicking through links on this page.
Amazon Summary
Widely considered one of the best practical guides to programming, Steve McConnell’s original code complete has been helping developers write better software for more than a decade. Now this classic book has been fully updated and revised with leading-edge practices—and hundreds of new code samples—illustrating the art and science of software construction. Capturing the body of knowledge available from research, academia, and everyday commercial practice, McConnell synthesizes the most effective techniques and must-know principles into clear, pragmatic guidance. No matter what your experience level, development environment, or project size, this book will inform and stimulate your thinking—and help you build the highest quality code. Discover the timeless techniques and strategies that help you: Design for minimum complexity and maximum creativity Reap the benefits of collaborative development Apply defensive programming techniques to reduce and flush out errors Exploit opportunities to refactor—or evolve—code, and do it safely Use construction practices that are right-weight for your project Debug problems quickly and effectively Resolve critical construction issues early and correctly Build quality into the beginning, middle, and end of your project .
HN Books Rankings
  • Ranked #14 all time · view

Hacker News Stories and Comments

All the comments and stories posted to Hacker News that reference this book.
Here are some pointers:

"Large-Scale C++ Software Design"

https://www.amazon.com/Large-Scale-Software-Design-John-Lako...

Although oriented towards C++, many architecture tips apply to other languages as well.

John Lakos is in the process of writing updated versions of the book.

"Large-Scale C++ Volume I: Process and Architecture"

https://www.amazon.com/Large-Scale-Architecture-Addison-Wesl...

"Large-Scale C++ Volume II: Design and Implementation"

https://www.amazon.com/Large-Scale-Implementation-Addison-We...

Then going back into the old days, you have

"Software Engineering in Modula 2: An Object Oriented Approach"

https://www.amazon.de/-/en/Jill-Hewitt/dp/0333515188

"Data Structures and Program Design in Modula-2"

https://www.amazon.de/Larry-R-Nyhoff/dp/0023886218/ref=sr_1_...

"Code Complete: A Practical Handbook of Software Construction"

https://www.amazon.com/dp/0735619670/ref=sr_1_1

"AntiPatterns: Refactoring Software, Architectures, and Projects in Crisis"

https://www.amazon.com/AntiPatterns-William-J-Brown/dp/04711...

"Component Software: Beyond Object-Oriented Programming"

https://www.amazon.com/Component-Software-Object-Oriented-Pr...

"Use Cases Combined With Booch/Omt/Uml: Process and Products"

https://www.amazon.com/Use-Cases-Combined-Booch-Omt/dp/01372...

Just some pointers to get you started.

While there is certainly something to be said for simplicity in a language, I think Go is too extreme in this regard. What it gains in simplicity it looses in expressiveness. This leads to more lines of code per feature, and bugs in software has been shown to be a function of LoCs [1,2,3].

One example of Go's lack of expressiveness is that loops are not abstracted away, e.g. you can't use higher level constructs like map and filter on collections. There are other languages that IMO strikes a better balance between simplicity and expressiveness, like Python and Elixir.

1. https://ieeexplore.ieee.org/document/1702967

2. https://link.springer.com/chapter/10.1007/978-1-4471-3003-1_...

3. https://www.amazon.com/gp/product/0735619670/ (Chapter "Developer Testing")

jerf
The results on code lines being directly related to bugs is getting old. Programming languages and practices have changed substantially in the past couple of decades. I'd want to see more recent evidence.

Besides, in practice I find lines of code in Go vs. Python are only maybe 20% different in many common cases, if you ignore the struct declarations which I have a hard time calling "code". (Being not a Turing complete language, that particular subset can't be considered vulnerable to the issues that Turing complete things can have.) I find myself very frequently wondering how many people complaining about all the lack of abstraction in Go have A: used it and B: used it to the point they're writing Go in Go. (Go just blows if you're trying to write Python in Go or something, or if you insist that you know better and must use map, filter, etc. rather than for loops. For all languages X, writing X in X is important, but Go is really graceless if you don't learn how.)

Now, if you do get to B, it is still true that Go does not quite abstract as "well" as some other languages, but it does abstract well enough that you can start having reasonably conversations about whether those other abstraction tools are actually paying their way in a lot of larger, shared code bases. If we throw out a ton of language complexity, but only pay a relatively small price, are they really carrying their weight? The answer will vary from code base to code base, but I think it's not necessarily some sort of horrid, unthinkably idea that not every program must be assaulted with the most powerful, complicated toolkit regardless of the price incurred by the complication.

coldtea
>The results on code lines being directly related to bugs is getting old. Programming languages and practices have changed substantially in the past couple of decades.

The results in those "past decades" used all kinds of languages, some far more advanced and modern than Go is today (e.g. Lisp, Smalltalk, Ada).

jerf
If we're going to quote science, we need to use it scientifically. It may be the case that all the changes in the decades since those studies are irrelevant, but we really can't just assume that. Even if we assume that the results of those studies are even relevant to real programming in the first place, since IIRC they, like almost every other study ever done on programming, did all their analysis on completely toy problems.
_ph_
As someone who programs both in Lisp and in Go (and I have used a little bit of Smalltalk), I have to comment your statement.

In a sense, Lisp is the most advanced language ever designed, and still, it is ignored by most of the programming community :). Equally, I think, Smalltalk does object orientation better than most languages which followed it.

Both languages are mostly dynamically typed, so comparisons to static typed languages need to take this into respect, they are different beasts. On the other side, I think Go has an interesting mix out of static and dynamic typed features, having interfaces for static contracts and interface{} as a fully dynamic type, which still offers full type safety at runtime.

Go structs have also some interesting properties, if you look at them beyond being containers. Any type can have methods, and structs can embed other structs and thus "inhert" their methods in a sense. This is not full blown object orientation, but gets you surprisingly far.

Un1corn
I'm not sure I would have called Python simple, it's a great language and it's not as complicated as something like C++ but it's not simple.

For example when should I use NamedTuple and when should I use a dataclass? Should I be using a list comprehension or a map/filter/reduce? When should I split that list comprehension? I have seen (and made myself) terribly long lines with only list comprehension and it was hideous.

fernandotakai
> Should I be using a list comprehension or a map/filter/reduce?

yes, use list comprehensions -- they are more readable.

>When should I split that list comprehension?

just use common sense. if you can read it and understand without thinking too much, that's good.

it's readability beats short code 100% of the time.

Scarbutt
You can write perfectly fine Python without any of the features you mentioned.
makapuf
Sure, but the more language that you may not use to write perfectly fine programs but have to know to know the whole language the more it seems superfluous. (See the whole drama over walrus operator). Python has accumulated a bit of rust also, old ways of doing things that were replaced with more modern ways, but that you should not use often. New languages or languages that don't evolve fast don't have them. python is becoming more and more complex (not tragically like c++) and v1.7 was indeed much simpler but it's still a nice language considering its age and scars.
erokar
When I said Python was simple I was talking about the concepts, functions and datatypes in the core of the language. I agree that if you include all the modules that come with Python it is a "big" language.
imode
To add, here's a nice article on implementing map from scratch: https://blog.burntsushi.net/type-parametric-functions-golang...

It illustrates what you have to pay to get this kind of functionality for arbitrary slices.

I wonder how this would change with generics.

alfalfasprout
And yet introducing higher level functionality also leads to more difficult to detect bugs. I'm not sure the "lines of code:errors" correlation truly applies to all languages.

That said, anecdotally I too have found it much easier to find errors visually in go programs due to its simplicity. I find the same is true of C vs. C++.

sagichmal
> One example of Go's lack of expressiveness is that loops are not abstracted away, e.g. you can't use higher level constructs like map and filter on collections.

This is (or was) deliberate, with the rationale being that it makes O(n) blocks of code trivial to identify in review.

Whether or not you buy that argument is a separate thing, of course.

coldtea
>This is (or was) deliberate, with the rationale being that it makes O(n) blocks of code trivial to identify in review.

Whereas, greping for .map(), .filter(), .reduce() wouldn't?

azth
> This is (or was) deliberate, with the rationale being that it makes O(n) blocks of code trivial to identify in review.

Like how

    for (int i = 0; i < strlen(str); i++) { ... }
is O(n)?
coldtea
Let's give them the benefit of the doubt, and consider that they meant that:

  for (int i = 0; i < strlen(str); i++) { ... }
could be grepped and easily discerned from:

  for (int i = 0; i < str_length; i++) { ... }
and generally, that every looping point could be easily identified.

After all, if you call functions inside the body, all bets are off as to what complexity the loop means anyway...

temac
So how does it work exactly? Are functions forbidden?
sagichmal
Functions (and methods) are the only way to "hide" computation and allocation in Go. That rubric makes reading Go code to build mechanical sympathy relatively straightforward.
h_r
Can you explain why you think it makes O(n) blocks trivial to identify?

How is such identification made easier by manually writing out a `for` loop applying a function foo to each element rather than writing `map foo myCollection`?

tfha
The way many people read code, it's much easier to overlook three characters than it is to overlook three lines with indentation.
h_r
That's quite an interesting, if frightening, take on it. Assuming a developer knows what `map` does, then I wouldn't have very much confidence in their reading comprehension if they somehow mentally skipped over the main higher order function being called on a three word line of code.

Would anyone expect such a developer to read and parse several more lines of code more reliably, in order to understand the algorithmic complexity? Seems unwarranted to me...

tfha
Humans make mistakes. People code review after exhausting days or late at night. Or sometimes they skim more than they intended to. You want to make problems in the code obvious everywhere that you can.

In programming, if you miss an important detail the repercussions can be high. In a million line codebase every idiom that's slightly more complex than it needs to be is going to result in dozens of additional bugs, sheerly because of the increased surface area for making mistakes with that idiom.

ernst_klim
> Humans make mistakes.

Yeah, and it's much easier to make mistake with a loop than with a map or filter.

h_r
I agree with everything you've said here but being mentally exhausted doesn't make it more likely that you can read even more lines of code in a more reliable fashion.

I think the clue to your thinking, for me, is in your description of the `map` HOF as "slightly more complex". Having years of experience with both paradigms, I've found that grokking a call to one of these fundamental building blocks (map, filter, reduce, fold, etc) is nearly instantaneous. We've all experienced reading prose where the author was excessively verbose when the same point could have been made succinctly. It feels the same way reading for loops once you get over the learning curve of these very basic functional constructs. You have to keep repeating that boilerplate endlessly and it's very tedious to keep writing and reading it.

kjksf
In every mainstream language I can think of `map foo myCollection` creates an intermediary map.

Memory allocation is so expensive that making that copy is often more expensive that calling `foo` on each element.

Sometimes making a copy is exactly what you need and there's no way around that cost (but hold that thought).

But I've also seen `sum map foo myCollection` so many times (especially in JavaScript).

Here you have a short, neat but also extremely wasteful way of doing things. I see it so frequently that I assume that many people are unaware of this cost (or completely disregard performance concerns).

If you were to write this imperatively, it would be obvious that you're making a copy and maybe you would stop and re-think your approach.

But there's more.

If you're paying attention to performance, an easy way to improve performance of `map foo myCollection` in e.g. Go is to pre-allocate the resulting array.

For large arrays, this avoid re-allocating of underlying memory over and over again, which is the best a `map` can do.

In imperative code those costs are more visible. When you have to type that code that does memory allocation, you suddenly realize that the equivalent of `map` is extremely wasteful.

LandR
>> In every mainstream language I can think of `map foo myCollection` creates an intermediary map.

Language support for Transducers can fix this, you can compose functions like map / filter / reduce over a collection and only hit each item once.

erik_seaberg
Even Java can do this. You only pay for another Collection<T> if you actually #collect the elements of a Stream<T>.
Gaelan
Is Rust mainstream? If so, this code:

    numbers.iter().map(|n| { ... }).sum();
Compiles to a plain loop, with no allocation.
steveklabnik
(You want for_each not map, by the way)
Gaelan
Wait, why? AFIAK for_each doesn't return another iterator, and I'm calling sum afterward.
steveklabnik
.... hacker news cut off the “sum” part, and so I didn’t see it. On my phone it just happened to clip it exactly long enough that it was still valid syntax, wow. Anyway, sorry and my bad!
h_r
I don't want to discuss this forever but I have a couple of comments:

Your points about efficiency are a separate topic entirely from the original claim that manually writing out an imperative solution makes it easier to see the algorithmic complexity. That was a surprising claim to me because, in my experience, if I understand what some HOF is doing, reading the code is even easier because there is less of it to wade through (and mental exhaustion doesn't make one easier to read vs the other).

> In every mainstream language I can think of `map foo myCollection` creates an intermediary map

You need to build up the final result, sure. Not an intermediary map but whatever structure (or functor) you're mapping over. That is the whole point of immutable data structures. Also when you're using persistent data structures, which all modern FP languages do, the cost of constructing the result can be far less than what you expect, especially if the result is lazy and you need only some of the results. There is a cost to immutability and if it's unbearable in some situation, fall back to in-place mutation but the semantics of these two approaches are definitely not the same.

> But I've also seen `sum map foo myCollection` so many times

Yeah... that should be a fold (reduce, whatever). :)

vhakulinen
> What it gains in simplicity it looses in expressiveness. This leads to more lines of code per feature, and bugs in software has been shown to be a function of LoCs

If the lack of expressiveness equals to greater amount of code and thus more bugs (in Go), does the same apply for languages that are more verbose (say, Java)? And if language is more expressive and less verbose (say, Python or Javascript maybe?), does it mean less bugs?

erokar
In general, yes. The more lines of code per feature, the more bugs per feature. Also, the guarantees you get from a typed language like Java does not seem to prevent bugs to a meaningful degree. Probably things like programming culture associated with a language is more important, like a strong preference for testing.

This is an interesting study on programming languages' effect on software quality: https://web.cs.ucdavis.edu/~filkov/papers/lang_github.pdf

fauigerzigerk
I wonder if everyone actually does initial testing and bug fixing after committing, because otherwise any effect of programming language choice might be gone before the code reaches the commit logs.

I'm not sure it makes sense to consider code correctness separately from productivity. You can probably get code written in any language to roughly similar levels of correctness. But how much work does it take to get there? That's what I'm really interested in.

fluffything
The OPs claim that it has been shown that bugs are a function of LoCs is unsubstantiated at best.

The number of bugs in a program correlates with the number of LoCs, but that's about it (correlation does not imply causation).

The claim is also quite absurd, since, for example, you can pre-process any C or C++ program ever written into a single line of code, yet this operation doesn't reduce the number of bugs in these programs, therefore the amount of bugs is not "just" a function of the amount of LoC.

Languages with better abstraction capabilities than Go (e.g. Rust or Python), might require programmers to type less, but the resulting lines of code are often dense. For example, I'm quite comfortable with Rust iterators and Python lists comprehensions, but it still takes me much longer to parse what code using these abstraction does, than code that just uses a simple for loop. It definitely feels great to cram a 10 LoC loop into a 1 LoC list comprehension in python, but reading that code 6 months later never felt that great, even for code that I've written myself. I just need to stop at those lines for much longer.

coldtea
>The number of bugs in a program correlates with the number of LoCs, but that's about it (correlation does not imply causation).

No, but it suggests it, and in this case, there's not many other options that could explain this than causation.

>The claim is also quite absurd, since, for example, you can pre-process any C or C++ program ever written into a single line of code, yet this operation doesn't reduce the number of bugs in these programs, therefore the amount of bugs is not "just" a function of the amount of LoC

That's an irrelevant strawman. When people speak of LoC they mean as to how many LoC are needed for various tasks based on the expressiveness of the language. Not whatever old LoC based on arbitrary adding needless lines (e.g. by preprocessing to inline function calls)...

littlestymaar
> The OPs claim that it has been shown that bugs are a function of LoCs is unsubstantiated at best.

But when looking at OP:

> bugs in software has been shown to be a function of LoCs [1,2,3].

He provided three different sources! Maybe it's wrong, or maybe you don't believe it, maybe it doesn't fit with your beliefs, but you can't say the OP claim is unsubstantiated when he provided academic sources on the subject!

pcwalton
Really? For loops are a nightmare. Here's one from SPEC 2006:

    for (dd=d[k=0]; k<16; dd=d[++k])
    {
        satd += (dd < 0 ? -dd : dd);
    }
The problem with for loops is that they are too flexible, and so there's more opportunity for misuse. Iterator functions are less flexible, and therefore easier to read.

And this excessive flexibility doesn't even make simple things simple. Try counting down from N to 0 inclusive using unsigned ints.

tfha
Loops in go are much less flexible than loops in C, you'd never see code like that in go.
pcwalton
In Go that loop could be:

    for k, dd = 0, d[0]; k < 16; k, dd = k + 1, d[k + 1] {
        ...
    }
I don't find that significantly more readable than the C version.
bfrog
Yeah a quick glance and I have no clue whats going on, tl;dr in review for sure
fluffything
Can you solve the exact same problem using Python list comprehensions or Rust iterators?

I'd like to see how solving this same problem in those languages is less of a nightmare.

I personally find that C loop quite readable compared with the same iteration being done in Rust.

pcwalton
This is a sum of absolute differences, so in Rust:

    let satd = d.iter().cloned().map(abs).sum();
As an added bonus, there's no undefined behavior!
eternalban
https://play.golang.org/p/dOYpmSK-Xor
dantillberg
I ported a fairly complex (thousands of lines of code, in use in production) program from Python to Go.

The Python version is around 1/3-1/2 the size of the Go version, but the clarity with which bugs are visible in the Go version is astonishing.

There's a lot of code in Python version that "looks right" and even "does kind of the right thing a lot of the time." But the Go version lays bare corner and edge cases that were not obvious in the concise Python version.

I agree that Go usage does suffer sometimes from lack of generics. I've sometimes used code generation in its place, and it works, but it's a poor solution. At the same time, the lack of generics has helped me avoid over-abstracting in other places, which is a common bane of mine whenever I work in more "expressive" languages.

Hi,

If you want to be a generalist, you may want to learn things which are useful independently of the programming language.

Some books that would qualify in my opinion (as examples):

- Code Complete: A Practical Handbook of Software Construction, Second Edition by Steve McConnell

https://www.amazon.com/Code-Complete-Practical-Handbook-Cons...

- Facts and Fallacies of Software Engineering by Robert L. Glass

https://www.amazon.com/Facts-Fallacies-Software-Engineering-...

Learning the different approches taken by multiple programming languages is certainly useful. It may not be that much relevant which language it is unless you want a job specifically in that language.

I can't speak for Google but I guess it is more relevant how familiar you are with software development practices and general knowledge about architecture, design, testing, algorithms to name a few than a specific language.

These are the timeless books and should be required reading

Pragmatic Programmer: http://amzn.com/020161622X

Code Complete 2: http://amzn.com/0735619670

rcavezza
Refactoring by Martin Fowler is a great one https://www.amazon.com/Refactoring-Improving-Existing-Addiso...

Design Patterns is also very well known https://www.amazon.com/Design-Patterns-Elements-Reusable-Obj...

Speaking from a software engineering perspective, here are a few that I found really valuable. I've seen them recommended by people I respect on many occasions.

Code Complete by Steve McConnell https://www.amazon.com/Code-Complete-Practical-Handbook-Cons...

The Mythical Man Month by Frederick Brooks https://www.amazon.com/Mythical-Man-Month-Software-Engineeri...

You could refresh your memories from the time before you started to complicate stuff by reading (or rereading) the marvelous book "code complete"

https://www.amazon.com/Code-Complete-Practical-Handbook-Cons...

amirsarancoder
Thanks, will read it.
ToFab123
Good luck :)
ToFab123
I have just ordered a copy for myself. I haven't read this book in more than 10 years, but it is the best computer book I have ever read. I have always said, that if I had my own company, then I would place a copy of this book on the table of all new employees. Reading this book should be their first assignment. It is a truly amazing book that anyone, despite experience level, platform of choice, project type and size, will benefit from reading.

You should read it.

Thanks everyone, the comments are much appreciated. Here's a list of books and other media resources recommended so far in the thread:

Robert C. Martin, Clean code: https://www.amazon.com/Clean-Code-Handbook-Software-Craftsma...

Vaughn Vernon, various: https://vaughnvernon.co/?page_id=168

Steve McConnell, Code Complete: https://www.amazon.com/Code-Complete-Practical-Handbook-Cons... 2

Clean coder: https://cleancoders.com/ videos

Hunt and Thomas, The Pragmatic Programmer: https://www.amazon.com/Pragmatic-Programmer-Journeyman-Maste...

Hitchhiker's Guide to Python: https://docs.python-guide.org/

Dustin Boswell The Art of Readable Code: https://www.amazon.com/Art-Readable-Code-Practical-Technique...

John Ousterhout, A Philosophy of Software Design: https://www.amazon.com/Philosophy-Software-Design-John-Ouste... This one looks particularly interesting, thanks AlexCoventry!

Kent Beck, Test Driven Development: https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/...

Dan Bader, Python Tricks: The Book: https://dbader.org/

Ian Sommerville, Software Engineering: https://www.amazon.com/Software-Engineering-10th-Ian-Sommerv...

Svilen Dobrev, various: http://www.svilendobrev.com/rabota/

blub
Skip anything by Robert Martin (clean coder series) and read at first Ousterhout and then McConnell instead.

Martin is well intentioned, but very dogmatic about some things like TDD, functions size, personal responsibility, etc. You need to already have some decent engineering experience to be able to detect and ignore the harmful stuff from his books.

n4r9
I'd like to re-emphasise sanderjd's point not to focus too much on reading books. I myself went from doing a PhD and a lectureship in mathematics (with some coding here and there) to a decent software engineering job in a smallish company. I've learnt everything on the fly by reading code, searching stack overflow, trying stuff out and coding alongside others. The great thing coming out of a PhD is not just that you have to be pretty smart to have done it: you now know you can grasp almost any aspect of human knowledge with sufficient brain racking. This is a vastly underrated piece of self-awareness which enables one to stay humble and tenacious.
AlexCoventry
Yeah, I think of a PhD as a kind of intellectual adolescence. It led me to far greater intellectual independence.
sanderjd
There are a lot of good recommendations here, and I certainly relate to the instinct to go to books when you're looking to level up a skill set, but I really think what you need is not a bunch of books to read, but a few people to watch do the work. The only real way to do that is to get a job alongside them. You can read the books at the same time; you can ask your new coworkers which recommendations they agree with and read those ones first.
fsloth
Yeah, software engineering is a craft, and generally the only way to learn those fast is to learn from others.
blub
It's not a craft, in its purest form it's an engineering discipline with specific rules, procedures and standards.

The crucial point is that most of us a doing programming, and not software engineering. Learning from others is hit or miss. One can certainly learn to program from others, but that's not enough to be able to do software engineering.

sanderjd
Convince me that its "purest form" is an engineering discipline rather than a craft. What distinguishes it from things that you would agree are crafts? Or are all crafts actually engineering disciplines in their purest form?

I think this is a pretty interesting question. Personally, when I was young, I would have said what you said: it's engineering, specifications go in and properly engineered finished product come out. I was proud that my CS program was contained in an engineering school and that all my friends were in other engineering disciplines. The longer I do this though, the more I think it's a craft: fuzzy idea is put to a skilled practitioner and through discussion and creativity, one or more artifacts are created to satisfy that idea. You could argue this is true of more traditional engineering disciplines as well, and I would probably agree. I'm not totally sure what the distinction between the two things is. So tell me what the distinction is and convince me what we do is more the one than the other.

For what it's worth though, apprenticeship is also a very important part of engineering education. At some point, you have to see people do the thing, regardless of whether it's engineering or a craft.

blub
When we say software development is a craft, we're saying that it's like shoemaking, pottery or woodworking.

Can the immense complexity of today and tomorrow's software be tamed by applying the same principles of building a cupboard? No, it requires an engineering mindset.

We're now limping along as an industry and it's not obvious because SW is bringing in massive amounts of money and we can basically get away with a lack of quality.

fsloth
"When we say software development is a craft, we're saying that it's like shoemaking, pottery or woodworking."

The point being, there are intricate details that are very hard to deliver in the traditional class-room oriented school environment with well defined requirements.

Those do not state anything about scalability. Crafts can scale - like for example how the old giant cathedrals and castles were built in middle-age europe.

They don't say anything about mindsets either. You need an engineers mindset to buid a cathedral or a castle.

The specific problem with crafts is that adapting to new requirements is a complete hit and miss process. The reason for this problem is the lack or proper theoretical framework in which to pose ones work and into which embed the requirements themselves.

The program verification people are working towards solving this problem, see for example Leslie Lamport's work in TLA+.

But until we have a general, mathematical proof backed compiler for requirements, as well as for the program implementation, we are pretty much stuck with craftsmen.

(Well, we have proof compilers but those are at the moment completely unusable for general programming since they are so complex to use.)

fsloth
"It's not a craft, in its purest form it's an engineering discipline with specific rules, procedures and standards."

Sorry, but I have to strongly disagree. In it's purest form the core of software engineering - i.e. programming is a craft. The other parts are mostly about creating processes so that craftsmen can create something together without stumbling into eachother.

The difference between a craft and engineering are numerous.

- engineers generally need a license

- engineering is about repeatability and creating dependable cost estimates

- engineers are required to study for years for a very good reason. You can be a rockstar programmer out of highschool.

Just having a bunch of cargo cult gibberish bound into a book does not make a craft into an engineering discipline.

It's harmfull to call programming engineering. Engineers have curriculums that can teach them pretty well what is expected of them once employed.

Not for programmers - or, well, software engineers. If there was even one curriculum that could churn out good programmers dependably, don't you think this model wouldn't be copied instantly elsewhere? If such a curruculum existed, do you think think software interviews would be filled with whiteboarding just to check out that the candidates understand even the basics?

I think this incapability to create a curriculum for actually creating good programmers is the best evidence that programming is a craft. It's such a complex topic that you can't create a mass curriculum that would serve all equally. Not with our current understanding, anyway. Maybe if we could teach everyone assembly, and Haskell , and have them implement compilers and languages as a standard things would be different.

The second best way to learn programming without being born a programmer savant is to learn from others while doing. Apprenticeship is the traditional way to train craftsmen.

Programming is so much more like a craft than engineering that it's best to call it a craft.

Craft is not a deragatory term. It just means we don't understand it theoretically well enough to teach it properly.

sanderjd
I tend to come down on your side of the craft vs. engineering debate, but then I disagree with basically your entire argument for it :)

You list three distinguishing features of an engineering discipline. The first and third swap cause and effect; a field doesn't become engineering because once it requires licensure and years of study. Surely you wouldn't agree that a successful campaign to require those things for software developers would bestow the status of engineering upon the work.

Your second point seems closer to the truth, but I'm not so sure it's true. If someone comes to a civil engineering firm and says, "design us a road between this city and that city", that is often a unique challenge because the terrain between those two cities is likely unique, maybe it requires a big bridge or a tunnel, which will be akin to but not identical to other bridges and tunnels you've built before.

fsloth
"The first and third swap cause and effect;"

Sorry, I wrote that in a hurry. I wasn't claiming either was a cause or effect. It was more of finding characteristics that we can use to identify one from another. I.e. following the argumentation "If it quacks like a duck and walks likes a duck it's likely a duck, and if it doesn't, we don't really have much evidence of the duckish quality of the observed thing".

So, I was not aiming to claim that licensure would turn software engineering into actual engineering. Rather, that the requirements of the field are so poorly understood in the general context that there would be very little to agree on the specific requirements. Poorly understood -> not engineering, really.

I totally agree with what you wrote above.

On the third point: I'm not claiming 100% truthiness to my argument, but it's pretty close. Software engineering projects are still among the riskiest ventures where you can think of investing capital in. If you want to build a road:

1. The language of the requirements are pretty well understood, from point A to B, this many lanes 2. Unless some unforeseen calamity arises, and you have the capital to pump to the project, eventually you will get the road

I think we can agree that these define any engineering project. Of course, engineeering is not cut and dried either - that's why you need to have actually trained professionals who can react to the events that come along as the project progresses.

I don't think 1. or 2. can hold for a software project in the general sense. Furthermore, you can end up accidentally, without anyones fault, wasting an arbitrary amount of capital on a feature that could, in the worst case, be replaced by a few lines of Python.

This poor quality of our general understanding of software development and lack of common language to describe anything means that most of the time software develoment is closer to R&D than engineering.

Generally, you can get better estimates when you are implementing a similar project the nth time. Like some general website, or a server backend. And in these instances you have the language to describe features and requirements. But in the general sense, software development isn't anything like this.

blub
Software development as practiced now by a huge number of individuals and companies is closer to a craft, but it can be and must be more than that if we want to be able to tackle the growing complexity of software and improve its overall barely adequate quality.

Crafts don't scale and are a poor fit for highly complex domains.

The curse of software development is its huge financial success, anemic legislative specification and the observed reality that customers will still buy poor quality software.

These are preventing the craft-like programming from turning into software engineering, but the craft is already failing to reach expectations: countless security disasters, unethical programmers enabling spying on millions, software literally killing users. This stuff will only get worse.

And finally, we do understand software engineering well enough to teach it properly. It's just not done, because it's not considered necessary when one can get by with a computer science degree, no degree or a bootcamp certificate.

fsloth
"And finally, we do understand software engineering well enough to teach it properly."

This is news to me. I would very much like a citation, please. Or do you mean applying formal proof verification to everything?

blub
Engineering doesn't mean using formal methods or specific fancy proofs, it's a systematic, disciplined quantifiable approach to software. It's described in an ISO standard and the more approachable IEEE SWEBOK.

The above is neither widely known (I only found out about it after many years of doing professional programming), nor is it necessary in order to be successful in the profession and/or make a lot of money.

Commercial software development is mostly a wild west and we're calling that craftsmanship.

fapjacks
I'll supplement this good advice by recommending pair programming. You will pick up a ton of stuff just sitting down at one keyboard with another programmer.
Recommended resources to consider:

* Code Complete https://www.amazon.com/Code-Complete-Practical-Handbook-Cons...

* Programming Pearls https://www.amazon.com/Programming-Pearls-2nd-Jon-Bentley/dp...

* Pragmatic Programmer https://www.amazon.com/Pragmatic-Programmer-Journeyman-Maste...

weej
One more I missed regarding review of architecture, code structure/design of OSS: The Architecture of Open Source Applications (Vol 1 & 2) https://www.amazon.com/Architecture-Open-Source-Applications...
wzel
Available for on-line reading at http://www.aosabook.org/en/
You could do worse than Steve McConnell's Code Complete - https://www.amazon.com/dp/0735619670.
Code Complete by Steve McConnell https://www.amazon.com/dp/0735619670
Safety1stClyde
I bought this book based on the endless internet recommendations. After reading it, I was mystified, completely mystified, why people recommended it so strongly.
travmatt
How experienced were you? It was really the first thing I read after beginner type tutorials, and helped me a great deal.
I used to begrudgingly practice martial arts. Computer programming is not like martial arts, at all. What you want to do sounds like the software craftsmanship approach:

https://en.wikipedia.org/wiki/Software_craftsmanship

Maybe look up more regarding that? I'm pretty skeptic about it...

IMO, the 'core programming concepts' (as in, skills that transcend language) most important to me, are those related to producing readable, maintainable code. Keeping your functions short, pure, and with one responsibility. Naming everything clearly. Making sure you are obeying DRY and KISS. Perhaps you would enjoy reading Code Complete:

https://www.amazon.com/Code-Complete-Practical-Handbook-Cons...

lagbaja
@ Mooseburger, I thank you for your comment.
My favorite general guidebook for writing good, readable, maintainable, defensive code is "Code Complete": http://www.amazon.com/Code-Complete-Practical-Handbook-Const...

It isn't domain specific but there is so much good advice in there around variable naming, code structure, general principles of bug-resistant coding that you will get a tremendous amount from it.

Amazon links for the lazy:

"The Mythical Man Month" - http://amzn.com/0201835959

"Peopleware: Productive Projects and Team" - http://amzn.com/0321934113

"The Pragmatic Programmer" - http://amzn.com/020161622X

"Managing the Unmanageable: Rules, Tools, and Insights for Managing Software People and Teams" - http://amzn.com/032182203X

"Joy Inc: How We Built a Workplace People Love" - http://amzn.com/1591845874

"A Lapsed Anarchist's Approach to Being a Better Leader" - http://amzn.com/0964895692

"Becoming a Technical Leader: An Organic Problem-Solving Approach" - http://amzn.com/0932633021

"Code Complete" - http://amzn.com/0735619670

owensmartin
Thanks for the shortened URLs. Question: I always obtain those by hitting "Share" and then copying the link. Is there a better way?
If I only had one suggestion it would be: http://en.wikipedia.org/wiki/The_Mythical_Man-Month

That book should be required reading for anyone on either side of the commonly asked question, "How do we build this?"

Maybe a second - it's a bit old (and almost 1000 pages) - but much of McConnell's CodeComplete2 still applies and has lots of great insights for software PM's: http://www.amazon.com/Code-Complete-Practical-Handbook-Const...

[edit grammer]

A selection of meditation resources for hackers:

OpenEEG (http://openeeg.sourceforge.net/doc/)

OpenSensors (http://sourceforge.net/p/opensensors/home/Home/)

Brain Refactoring (http://www.amazon.com/Code-Complete-Practical-Handbook-Const...)

derekja
OpenEEG is great, but getting pretty dated. Out much more recently is the OpenBCI project. http://openbci.com/
0xCMP
I see what you did there... "refactor your brain" to a bunch of programmers... (I. am. so. sorry. hn. I couldn't help my self)
rattray
HN has a very different tone than Reddit; this type of humor is less appreciated here. With that, though, welcome!
cmnzs
I have a feeling that third link is not supposed to be code complete?
logfromblammo
No, I selected that title as one of the most common and popular self-improvement resources available for software professionals. The implication is that reading it would do more to improve your thinking than a week-long meditation retreat.
As well as the extensively reviewed, recommended, and appraised "Code Complete" [0] by Steve McConnell

I'm working my way through it now (1+ year of professional experience) and it is a magnificent way to improve the quality of your code. I read it off and on, my goal is only 40 pages a week so that I'll make sure to find the time to do it (I'm doing a masters program and enjoy living in NYC too so setting huge goals doesn't work well for me).

Every time I crack it open, I find myself inspired to write better, clearer, and more concise code. Sometimes you just need a nudge to get back into doing things you already know you should be doing.

Finally, constantly learning, I think, is the best way to become a proficient, and then skillful professional software engineer. Many programmers become proficient and then level off. And that's good enough. But if you truly wanted to become one of the top 5% in your field you need to do something called deliberate practice. Reading 'Talent is Overrated' [1] really exposed me to the theory of constantly challenging yourself in order to grow. I really recommend it, I find myself trying to apply the theories to all areas of my life.

[0] - http://www.amazon.com/Code-Complete-Practical-Handbook-Const...

[1] - http://www.amazon.com/Talent-Overrated-Separates-World-Class...

Try having your team read these books:

http://www.amazon.com/Writing-Solid-Code-Microsoft-Programmi...

http://www.amazon.com/Practice-Programming-Addison-Wesley-Pr...

http://www.amazon.com/Code-Complete-Practical-Handbook-Const...

I know these are old books and are C and C++ oriented, but it helped me a lot during my formative years and helped me transition from being a decent programmer to being a decent engineer. They are short books which are well written and not very dense.

edgo
Code Complete has 960 pages, not in the short book category
Steve McConnell's Code Complete might be a good resource. Could you buy a copy for each team member and take an hour each week to read and discuss a chapter? That might get you on the road to better coding practices.

http://www.amazon.com/Code-Complete-Practical-Handbook-Const...

Code Complete: A Practical Handbook for Software Construction is considered one of the language agnostic classics on computer programming. It focuses on higher level concepts as they relate to the process and sequence of writing code. It covers just enough architecture and design to provide context and allow one to think intelligently about it.

[affiliate link] http://www.amazon.com/gp/product/0735619670/ref=as_li_tl?ie=...

[non-affiliate link] http://www.amazon.com/Code-Complete-Practical-Handbook-Const...

abhinavgujjar
I would highly recommend this as well. BUT - don't try to dive too much deeper into code patterns. The last decade has a massive spike in patterns and I can remember a lot of meetings wasted debating patterns.
Code Complete has pretty good bang per page, especially if you are beginner.

http://www.amazon.co.uk/Code-Complete-Practical-Handbook-Con...

abcd_f
This is the most disappointing programming book I have ever had a misfortune to buy. If you ever read any UML / Rational Software books - this is right up there in terms of sucking all joy out of programming, putting a blue collar and tie on you and then strangling you to death with soul-less trivialization of the software creation process. Horrible, horrible book.

If you like K&R books, stay the hell away from Code Complete. But make sure to read it though if you think programming is a career.

ocfx
it is a career
lukasm
I must strongly disagree. If you like K&R books, you will find a lot of useful tips like

if (!b) { a();} else { b();}

change to

if(b) {b();} else { a(); }

Code Complete 2 [1] was one of the first coding books I've read. As with anything else, it's good to look around (HN is a good place) for people who have problems with the book. I think I learn as much reading the commentary people make about books like that as I do from the book itself.

I think I've listened to every podcast on software engineering radio a few times [2]. The older ones are especially nice because they usually pick a specific topic and cover the high points. I liked that I could listen to it while I was driving, or otherwise not in front of a computer.

It's specific, but Javascript: The Good Parts is probably the most used book I have on my shelf. It has such a perfect amount of usable information in it. It's pretty great. Again, it's definitely worth looking up critiques and counterpoints.

I've also got Introduction to Algorithms, which I use as a reference, sometimes. I switched over to The Algorithm Design Manual [5] after I saw it referenced in an older Steve Yegge post [6]. I read through the intro and it seemed like a book that would be more appropriate from an autodidactic standpoint. I really have no idea if that's going to pan out, since I'm not that far into it, but we'll see, for sure. Doesn't kill me to have an extra algorithms book laying about, though, and I've always got intro to algorithms for cross reference. I've found that I really need to have as many sources available as possible when I'm learning alone. Usually I don't get something until the fifth person describes it from the tenth different angle.

That's most of what I can think of off hand. I really enjoyed The Joy of Clojure [7], though haven't checked out the newer version. Programming Collective Intelligence [8] is a fun book, and is what made me want to go back down the maths route to get more into machine learning.

And of course habitually reading hacker news for an hour or three every night :)

So that's my totally inexpert list of random stuff that I enjoy

[1] http://www.amazon.com/Code-Complete-Practical-Handbook-Const... [2] http://www.se-radio.net/ [3] http://www.amazon.com/JavaScript-Good-Parts-Douglas-Crockfor... [4] http://www.amazon.com/Introduction-Algorithms-Thomas-H-Corme... [5] http://www.amazon.com/Algorithm-Design-Manual-Steven-Skiena/... [6] http://steve-yegge.blogspot.com/2008/03/get-that-job-at-goog... [7] http://www.amazon.com/Joy-Clojure-Michael-Fogus/dp/161729141... [8] http://www.amazon.com/Programming-Collective-Intelligence-Bu...

When you start your post with "I have a high IQ so programming's easy for me", I indeed believe that working with other people might be a problem for you.

But I would still add that I'm doubtful that someone can work on large projects and not have communicating and working with other people be an actual part of programming (rather than that "other problem", "people").

If your projects need you to produce code that other people will modify, I would claim that your coding is going to be a matter of communication. Perhaps you write one-shot firmware for toasters or missiles so this doesn't apply. But I think any student of programming in general tends to see the task as a process of communication and not just cleverness. On this subject, I'd recommend Steve McConnell's Code complete.

http://www.amazon.com/Code-Complete-Practical-Handbook-Const...

yogo
Spot on. For the most part I think you can take anyone on HN to build 90% of the applications out there. After all, it mostly boils down CRUD (different levels), with the other 10% requiring specialized knowledge of algorithms and certain systems. All this to say that coding is the easy part. The more time-consuming part is figuring out exactly what the end users need, whether that involves communicating directly with them or with a project manager. After it's shipped? Even more communication because there will be some bug that you, or your test suite, would not have caught from an everyday user, and there's only one way to find out how to reproduce it...
joe_the_user
You mean 90% of the web applications out there.

The other situation where communication is important is once you have a large enough application that you have to start dealing with other people's code and design.

There is a lot of hellish bureaucracy in even the best of these situations but I think there is none-the-less some important learnings available there.

I did the same; taught myself css, php, javascript and quit my FTJ last Christmas. Best thing I ever did.

I also sent the following as advice to someone wanting to get into web dev:

"I was just thinking of 'easy ins' to the world of web development and a good source of information is http://news.ycombinator.com/ there's a lot of information from people who work in the world of tech startups and it's good information.

Also - if you are wanting to do php dev the key things to learn are: Software engineering techniques and practice - object oriented development and abstract patterns are key to how to think about good development. Database design and development (1st normal form, third normal form etc) Learn SQL. (SQL for dummies or similar is good for the basic commands and syntax etc.) Stackoverflow.com - it's the best source of help for software development on the internet. read books, the ones that come up again and again when people talk about learning to program:

http://www.amazon.co.uk/Code-Complete-Practical-Handbook-Con... http://www.amazon.co.uk/Mythical-Month-Essays-Software-Engin... http://www.amazon.co.uk/The-Pragmatic-Programmer-Andrew-Hunt...

also - look at github.com - that's where programmers keep their source code.

Learn about Object Oriented Programming, Design Patterns, MVC (which is a design pattern) is specifcally useful for web development.

Also - demand for javascript programmers will increase over the coming years because of things like jQuery and Ajax.

That's my starter for ten - if you are interested in a career as a web programmer.

If you want to focus more on the html/css design side you could do worse than focusing on one CMS - such as wordpress and learning it inside out - you could then offer that to clients and it's a good way to provide web sites cheaply with very little effort."

I got around this mindset along the years, by learning about the importance of lowering the maintenance costs, improving collaboration and not depending on any single person to understand any given module or system.

Books such as "Code Complete"[1] and "Clean Code"[2] should help you with that transition.

[1] http://www.amazon.com/Code-Complete-Practical-Handbook-Const...

[2] http://www.amazon.com/Clean-Code-Handbook-Software-Craftsman...

Nov 27, 2012 · robomartin on I’m writing my own OS
OK, if you don't have any real experience in low-level embedded coding (relevant to device drivers), RTOS or OS design in general, file systems, data structures, algorithms, interfaces, etc. And, if you have "hobby level" experience with Assembler, C and C++. And, if your intent is to write a desktop OS, from the ground up, without making use of existing technologies, drivers, file systems, memory management, POSIX, etc. Here's a list of books that could be considered required reading before you can really start to write specifications and code. Pick twenty of these and that might be a good start.

In no particular order:

1- http://www.amazon.com/C-Programming-Language-2nd-Edition/dp/...

2- http://www.amazon.com/The-Answer-Book-Solutions-Programming/...

3- http://www.amazon.com/The-Standard-Library-P-J-Plauger/dp/01...

4- http://www.amazon.com/C-Traps-Pitfalls-Andrew-Koenig/dp/0201...

5- http://www.amazon.com/Expert-Programming-Peter-van-Linden/dp...

6- http://www.amazon.com/Data-Structures-In-Noel-Kalicharan/dp/...

7- http://www.amazon.com/Data-Structures-Using-Aaron-Tenenbaum/...

8- http://www.amazon.com/Mastering-Algorithms-C-Kyle-Loudon/dp/...

9- http://www.amazon.com/Code-Complete-Practical-Handbook-Const...

10- http://www.amazon.com/Design-Patterns-Elements-Reusable-Obje...

11- http://www.amazon.com/The-Mythical-Man-Month-Engineering-Ann...

12- http://www.amazon.com/The-Programming-Language-4th-Edition/d...

13- http://www.amazon.com/The-Standard-Library-Tutorial-Referenc...

14- http://www.amazon.com/API-Design-C-Martin-Reddy/dp/012385003...

15- http://www.amazon.com/The-Linux-Programming-Interface-Handbo...

16- http://www.amazon.com/Computer-Systems-Programmers-Perspecti...

17- http://www.amazon.com/System-Programming-Unix-Adam-Hoover/dp...

18- http://www.amazon.com/Memory-Programming-Concept-Frantisek-F...

19- http://www.amazon.com/Memory-Management-Implementations-Prog...

20- http://www.amazon.com/UNIX-Filesystems-Evolution-Design-Impl...

21- http://www.amazon.com/PCI-System-Architecture-4th-Edition/dp...

22- http://www.amazon.com/Universal-Serial-System-Architecture-E...

23- http://www.amazon.com/Introduction-PCI-Express-Hardware-Deve...

24- http://www.amazon.com/Serial-Storage-Architecture-Applicatio...

25- http://www.amazon.com/SATA-Storage-Technology-Serial-ATA/dp/...

26- http://www.amazon.com/Beyond-BIOS-Developing-Extensible-Inte...

27- http://www.amazon.com/Professional-Assembly-Language-Program...

28- http://www.amazon.com/Linux-Kernel-Development-3rd-Edition/d...

29- http://www.amazon.com/Version-Control-Git-collaborative-deve...

30- http://www.amazon.com/Embedded-Software-Primer-David-Simon/d...

31- http://www.amazon.com/Programming-Embedded-Systems-C/dp/1565...

32- http://www.amazon.com/Making-Embedded-Systems-Patterns-Softw...

33- http://www.amazon.com/Operating-System-Concepts-Abraham-Silb...

34- http://www.amazon.com/Performance-Preemptive-Multitasking-Mi...

35- http://www.amazon.com/Design-Operating-System-Prentice-Hall-...

36- http://www.amazon.com/Unix-Network-Programming-Sockets-Netwo...

37- http://www.amazon.com/TCP-Illustrated-Volume-Addison-Wesley-...

38- http://www.amazon.com/TCP-IP-Illustrated-Vol-Implementation/...

39- http://www.amazon.com/TCP-Illustrated-Vol-Transactions-Proto...

40- http://www.amazon.com/User-Interface-Design-Programmers-Spol...

41- http://www.amazon.com/Designing-Interfaces-Jenifer-Tidwell/d...

42- http://www.amazon.com/Designing-Interfaces-Jenifer-Tidwell/d...

43- http://www.amazon.com/Programming-POSIX-Threads-David-Butenh...

44- http://www.intel.com/p/en_US/embedded/hwsw/software/hd-gma#d...

45- http://www.intel.com/content/www/us/en/processors/architectu...

46- http://www.intel.com/p/en_US/embedded/hwsw/hardware/core-b75...

47- http://www.hdmi.org/index.aspx

48- http://en.wikipedia.org/wiki/Digital_Visual_Interface

49- http://www.amazon.com/Essential-Device-Drivers-Sreekrishnan-...

50- http://www.amazon.com/Making-Embedded-Systems-Patterns-Softw...

51- http://www.amazon.com/Python-Programming-Introduction-Comput...

52- http://www.amazon.com/Practical-System-Design-Dominic-Giampa...

53- http://www.amazon.com/File-Systems-Structures-Thomas-Harbron...

54- ...well, I'll stop here.

Of course, the equivalent knowledge can be obtained by trial-and-error, which would take longer and might result in costly errors and imperfect design. The greater danger here is that a sole developer, without the feedback and interaction of even a small group of capable and experienced programmers could simply burn a lot of time repeating the mistakes made by those who have already trenched that territory.

If the goal is to write a small RTOS on a small but nicely-featured microcontroller, then the C books and the uC/OS book might be a good shove in the right direction. Things start getting complicated if you need to write such things as a full USB stack, PCIe subsystem, graphics drivers, etc.

tagada
The guy don't want to write a perfectly designed OS, neither he wants to design a RTOS by the way ... He didn't even specify whether his OS would be multitask. If not, no need for any scheduling at all, huh ... he just "writes his own OS". It's a fun experience, very rich and very teaching. But at least now, we all can admire the wideness of all your "unlimited knowledge" especially in terms of titles of books. Afterall, no needs for creating a new OS, if you get all your inspiration from things that already exists. You will end with Yet Another Nix ... Not sure he wants a YanOS, though.

Without talking about filesystems, drivers, memory managment or existing norms and standards (which it seems he wants to avoid ... which is not that stupid ... all having been designed 40 years ago ... who knows maybe he'll have a revolutionary idea, beginning from scratch), going from scratch with no exact idea of where you go could be a good thing. Definitely. You solve the problem only when you face them. Step by step, sequentially. Very virile, man.

I would advice the guy to start obviously with the boot loader. Having a look at the code of GRUB (v1), LILO, or better yet, on graphical ones (BURG, GAG or this good one XOSL which is completely under GUI - mouse pointer, up to 1600x1200 screens, and many supported filesystems, written in ASM ... it could actually be a hacking basis for a basic OS on itself) could be a great source of inspiration, and beginning to play directly with the graphical environment.

You could also have a look on these things http://viralpatel.net/taj/tutorial/hello_world_bootloader.ph... and of course on Kolibri OS http://kolibrios.org/en/?p=Screenshots

tagada
Blah, blah, blah ...

Better advice would be: Try, make mistakes, learn from them, continue.

Way, way, way less discouraging, pessimistic and above all pedantic.

derefr
> If the goal is to write a small RTOS on a small but nicely-featured microcontroller, then the C books and the uC/OS book might be a good shove in the right direction. Things start getting complicated if you need to write such things as a full USB stack, PCIe subsystem, graphics drivers, etc.

I've always wondered if there could be created some way to skip this step in [research] OS prototyping, by creating a shared library (exokernel?) of just drivers, while leaving the "design decisions" of the OS (system calls, memory management, scheduling, filesystems, &c.--you know, the things people get into OS development to play with) to the developer.

People already sort of do this by targeting an emulator like VirtualBox to begin with--by doing so, you only (initially) need one driver for each feature you want to add, and the emulator takes care of portability. But this approach can't be scaled up to a hypervisor (Xen) or KVM, because those expect their guest operating systems to also have relevant drivers for (at least some of) the hardware.

I'm wondering at this point if you could, say, fork Linux to strip it down to "just the drivers" to start such a project (possibly even continuing to merge in driver-related commits from upstream) or if this would be a meaningless proposition--how reliant are various drivers of an OS on OS kernel-level daemons that themselves rely on the particular implementation of OS process management, OS IPC, etc.? Could you code for the Linux driver-base without your project growing strongly isomorphic structures such as init, acpid, etc.?

Because, if you could--if the driver-base could just rely on a clean, standardized, exported C API from the rest of the kernel, then perhaps (and this is the starry-eyed dream of mine) we could move "hardware support development" to a separate project from "kernel development", and projects like HURD and Plan9 could "get off the ground" in terms of driver support.

robomartin
A lot depends on the platform. If the OS is for a WinTel motherboard it is one thing. If, however, the idea is to bypass driver development for a wide range of platforms it gets complicated.

In my experience one of the most painful aspects of bringing up an OS on a new platform is exactly this issue of drivers as well as file systems. A little google-ing quickly reveals that these are some of the areas where one might have to spend big bucks in the embedded world in order to license such modules as FFS (Flash File System) with wear leveling and other features as well as USB and networking stacks. Rolling your own as a solo developer or even a small team could very well fit into the definition of insanity. I have done a good chunk of a special purpose high-performance FFS. It was an all-absorbing project for months and, realistically, in the end, it did not match all of the capabilities of what could be had commercially.

This is where it is easy to justify moving into a more advanced platform in order to be able to leverage Embedded Linux. Here you get to benefit and leverage the work of tens of thousands of developers devoted to scratching very specific itches.

The down-side, of course, is that if what you need isn't implemented in the boad support package for the processor you happen to be working with, well, you are screwed. The idea that you can just write it yourself because it's Linux is only applicable if you or your team are well-versed in Linux dev at a low enough level. If that is not the case you are back to square one. If you have to go that route you have to hire an additional developer that knows this stuff inside out. That could mean $100K per year. So now your are, once again, back at square one: hiring a dev might actually be more exoensive than licensing a commercial OS with support, drivers, etc.

I was faced with exactly that conundrum a few years ago. We ended-up going with Windows CE (as ugly as that may sound). There are many reasons for that but the most compelling one may have been that we could identify an OEM board with the right I/O, features, form factor, price and full support for all of the drivers and subsystems we needed. In other words, we could focus on developing the actual product rather than having to dig deeper and deeper into low-level issues.

It'd be great if low level drivers could be universal and platform independent to the degree that they could be used as you suggest. Obviously VM-based platforms like Java can offer something like that so long as someone has done the low-level work for you. All that means is that you don't have to deal with the drivers.

To go a little further, part of the problem is that no standard interface exists to talk to chips. In other words, configuring and running a DVI transmitter, a serial port and a Bluetooth I/F are vastly different even when you might be doing some of the same things. Setting up data rates, clocks, etc. can be day and night from chip to chip.

I haven't really given it much thought. My knee-jerk reaction is that it would be very hard to crate a unified, discoverable, platform-independent mechanism to program chips. The closest one could possibly approach this idea would be if chip makers were expected to provide drivers written to a common interface. Well, not likely or practical.

Not an easy problem.

derefr
> It'd be great if low level drivers could be universal and platform independent to the degree that they could be used as you suggest. Obviously VM-based platforms like Java can offer something like that so long as someone has done the low-level work for you. All that means is that you don't have to deal with the drivers.

Another thought: if not just a package of drivers, then how stripped down (for the purpose of raw efficiency) could you make an operating system intended only to run an emulator (IA32, JVM, BEAM, whatever) for "your" operating system? Presumably you could strip away scheduling, memory security, etc. since the application VM could be handling those if it wanted to. Is there already a major project to do this for Linux?

Feb 20, 2012 · lars512 on On naming
Steve McConnell's book "Code Complete" was an eye-opener for me. It also has a whole chapter on variable naming, and is worth a read.

http://www.amazon.com/Code-Complete-Practical-Handbook-Const...

I am passionate about writing good code, and I'll try to offer some practical advice. As others said, the first step is understanding the importance of writing clean code and taking pride in your craft. By doing this, you are already ahead of most other programmers who don't really care. Now, how do you improve?

1) Read books.

With 4-5 years of experience, you already have a good intuition for "good" and "bad" code. Still, it doesn't hurt to learn more about it. There are a lot of good books on this subject.

The first is "Clean Code", by Robert C. Martin. It's the first book I read on this subject, and I learned a lot. Note that the book uses Java in the examples, but I think the ideas would apply to most other languages.

Its author, Uncle Bob ( http://en.wikipedia.org/wiki/Robert_Cecil_Martin ) has long been a proponent of writing clean, beautiful code. He recently did an interview for Xebia: http://blog.xebia.fr/2011/05/25/interview-avec-robert-martin... (French blog, but the interview is in English). In it, he advises:

"Well there are a number of books that talk about writing code well. Kent Beck wrote a book called “Implementation Patterns” very recently. It’s an excellent book all about software craftsmanship. The pragmatic programmers wrote a wonderful book in 2000 called “The pragmatic programmer”, again a wonderful book. Chad Fowler wrote a book call the “Passionate Programmer”. Again the book is about doing everything well. I wrote a book recently called “Clean Code” which is very specific about particular things you can do with your code to do it well. So there is a tremendous number of resources that are available for people who are interested in doing software craftsmanship well."

Out of those, disregard "The passionate Programmer". It's an okay book, but its focus is on building a good programmer career, not on code.

"Implementation Patterns" by Kent Beck is a great book with a lot of best practices when writing * Java * code. Less useful if you use another language.

"The pragmatic programmer" is a good book on software craftsmanship. I'm personally half-way through. There is a lot of good advice in it, but I often find myself thinking it's "common-sense". Maybe because I've already been exposed to most of the ideas by reading blogs? Still, it's a great book, with a lot of best practices. It's main focus is not code, though, so you might want to start with other books if your focus is on writing good code.

To these books, I'd add "Code Complete (2nd Edition)" (language-agnostic) and "Effective Java 2nd Edition" if you use Java.

Summary:

If you use Java, read:

  - Clean Code - http://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882
  - Effective Java 2nd Edition - http://www.amazon.com/Effective-Java-2nd-Joshua-Bloch/dp/0321356683
  - Implementation Patterns - http://www.amazon.com/Implementation-Patterns-Kent-Beck/dp/0321413091
  - Code Complete / Pragmatic Programmer
If you use another language, read:

  - Clean Code - http://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882
  - Code Complete - http://www.amazon.com/Code-Complete-Practical-Handbook-Construction/dp/0735619670
  - Pragmatic Programmer - http://www.amazon.com/Pragmatic-Programmer-Journeyman-Master/dp/020161622X
Other people might chime in with suggestions for other languages?

2) Practice.

Implement what you learned in the books. Keep improving.

3) Read other people's code.

Read good open source code. Not all code is good code. Ask around for OSS projects with good code in your language.

If you use Java, start with:

  - Google Guava - http://code.google.com/p/guava-libraries/
  - Google Guice - http://code.google.com/p/google-guice/
  - Spring Framework - http://www.springsource.org/
4) Practice.

5) Practice.

kevintaylor
I believe it is important to also have a view and strategy for your career. Read "Apprenticeship Patterns."

I highly recommend the other books, too.

Cheers, Kevin

mabid
Thank you so much for the comprehensive reply. You are right when you say that with 4-5 years of experience I know to some extent what good and bad code is. but sometimes I think a lot and write the best code that i can, and later after some time i realize; by looking at some one else's code or by myself knowing a new feature of the language i am using, that i could have done it in a much better way .. that is much more beautiful. My problem is that if my code doesnt look like poetry to me, clean readable understandable i feel the need to improve it.
HN Books is an independent project and is not operated by Y Combinator or Amazon.com.
~ [email protected]
;laksdfhjdhksalkfj more things
yahnd.com ~ Privacy Policy ~
Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.