HN Books @HNBooksMonth

"Design Patterns: Elements of Reusable Object-Oriented Software" (1994) is still going strong, in my opinion, and I don't think that will change in the coming 5-10 years.

[1] https://www.amazon.com/Design-Patterns-Elements-Reusable-Obj...

Object oriented programming and design patterns in particular get a bad rap these days, however, the original Design Patterns book [1] has a case study chapter about designing a WYSIWYG document editor. Also, one of the authors, Erich Gamma, joined Microsoft in 2011, and works on the Monaco suite of components that VS Code is built on top of. So, while I am sure there's a fair bit of difference in the years since they wrote that book, as well as the needs of implementation in JS, I'd say it's a fairly good deep dive into some of the topics from one of the actual architects behind it.

Fair warning though, it's a fairly hard book to read. For a lighter, more fun intro to design patterns in particular, I always recommend Game Programming Patterns [2]

[1] https://www.amazon.com/Design-Patterns-Elements-Reusable-Obj...

[2] https://gameprogrammingpatterns.com/contents.html

⬐ forinti

I was once tasked with writing a simple text editor. I knew this book inside out so I decided to try putting the text into a tree the way it describes.

This made selecting text quite hard. So I gave up and just put the whole thing into an array. It made everything a lot easier.

This codebase won an Oscar:

https://pbrt.org/

https://github.com/mmp/pbrt-v3

That said, IMHO, there is no "clean" C++ code. There are C++ codebases that use different styles, and their "quality" more or less is context sensitive.

Personally I felt the best tutorial to C++ were actually two other programming languages - Scheme and F#.

Scheme drove in the concept that it's totally fine to model things based on the shape of the data and that you don't need to create type based abstractions around every thing.

F# then demonstrated how a language with type system is supposed to work.

The problem with C++ is that the language is so verbose that unless you have an abbreviated model in your head how a type based language can be used to solve problems in best way, you will get lost in trivial C++ minutiae.

So, personally, I generally think "Am I solving a Scheme like problem or Standard ML like problem" and then try to apply C++ as simply as possible.

Several academics have created a career of how to patch C++/Java/C# with concepts that make them less fragile in a collaborative context:

https://en.wikipedia.org/wiki/Design_Patterns

https://www.amazon.com/Design-Patterns-Elements-Reusable-Obj...

In my opinion design patterns are not a fundamental concept, but rather provide common syntax for collaboration purposes for various patterns that are parts of language and more or less invisible in e.g. Scheme or F#. But if one is diving into C++ it's probably convenient to be familiar with these concepts.

⬐ CyberDildonics

PBRT won a technical achievement academy award because it was a reasonably complete renderer from start to finish while being not just open source but extensively documented and explained. The source though is not a great example of modern C++ at this point. It is based heavily on inheritance and uses lots of memory allocation and virtual function calls.

⬐ daniel-thompson

> It is based heavily on inheritance and uses lots of memory allocation and virtual function calls.

Interesting - can you talk a little bit about why those specific design choices are questionable from a modern C++ POV? How would you do it differently, given the problem & domain PBRT covers?

I ask because my mental "sweet spot" in terms of understanding and writing C++ is approximately C++11. It's pretty easy for me to go into symbol shock when I see some of the latest stuff in C++17 and C++20... and I suspect I'm not the only person that happens to.

⬐ CyberDildonics

The main points are not C++ features, but the way classes and inheritance are how everything is broken down.

A much better approach is to allocate large chunks of memory at one time and run through the arrays of floats directly. Instead of tracing and shading one path ray all the way through, finding out where it hits, running the brdf, looking up textures, casting another ray, etc. it is much better to do each stage in large chunks.

Going from data structures holding lots of virtual pointers to something like I described above can be a substantial improvement in speed and give a lot of clarity at the same time.

Many times programs with a lot of inheritance end up with their execution dictated by types calling other types' functions, which makes it more implicit and buried rather than linear in a top level function. It becomes similar in a way to callback hell where you are trying to track down how the program gets to a certain place. Many times it takes adding break points and walking back through the call stack to reverse engineer it, rather than looking at a single top level function that has a lot of steps.

⬐ fsloth

The source is aged a bit but

a) displays a way to put together a non-trivial C++ application

b) displays several extremely useful graphics related algorithms and patterns

c) Is documented on a level and quality few public equal code bases reach - so get the book first and then start reading the sources

⬐ daniel-thompson

Also note that PBRT v4 is out in early release: https://github.com/mmp/pbrt-v4

⬐ ncmncm

Each design pattern identifies a weakness in the language where it appears. If the language were stronger, the pattern would be either just a core language feature, or (better) a library component. Languages get stronger by being able to capture more patterns in libraries.

If your C++ code is "so verbose", you are Doing It Wrong, most likely by doing it the old way. C++98 was verbose.

⬐ fsloth

C++ code as written can be succint but the spec to describe what the code does can be really verbose.

So - verbose in terms of the spec which describes what the code actually does and mental chatter I have to do with myself when writing it "should I return a ref or a smart_ptr or a value or should I use move semantics..." whereas in F# it's just a ref and garbage collector will deal with it. So I can skip the mental chatter part.

> Ideally I'd like to find a more formalized treatment of "abstracting" in the vein of the GoF's Design Patterns.

Well [1] that book you're referring to, has changed my perspective on abstractions and how to communicate them to other people. There's probably no "Holy Grail" of abstraction guides better than that IMHO.

Designing and implementing useful and maintainable abstractions comes from experience in both consuming and designing abstractions for API's. Having empathy for both the consumers and future developer's of the API gives you a good perspective on how to structure the abstraction.

The best abstractions are from people who have used bad abstractions, and are usually incrementally developed over time...

[1] https://www.amazon.com.au/Design-Patterns-Elements-Reusable-...

I'm ready this book right now, and it's great. https://www.amazon.com/Design-Patterns-Elements-Reusable-Obj...

Refactoring by Martin Fowler is a great one https://www.amazon.com/Refactoring-Improving-Existing-Addiso...

Design Patterns is also very well known https://www.amazon.com/Design-Patterns-Elements-Reusable-Obj...

Nice list. Appreciate it. But I see there are a few amazing software books missing from the list such as:

- Clean Code (by "Uncle Bob")) [https://www.amazon.com/Clean-Code-Handbook-Software-Craftsma...]

- Design Patterns (by "Gang of 4") [https://www.amazon.com/Design-Patterns-Elements-Reusable-Obj...]

- Introduction to Algorithms (by "CLRS") [https://www.amazon.com/Introduction-Algorithms-3rd-MIT-Press...]

⬐ prani10

Introduction to Algorithms might not be the best thing to start with. The name is very much misleading

⬐ djaychela

Glad to see I'm not the only one who thinks this - it's one of the many areas I need to look at, and saw that recommended elsewhere... took a look on Amazon and got completely intimidated! If 1000+ pages is an introduction, what does the chapter and verse look like?

(I've started reading Grokking Algorithms this week, and it's been a much better introduction - I know it's not as in depth as some other books, but I'm making good progress with it and not drowning in complexity from the get go)

⬐ kevinskii

Don't let its size intimidate you. Introduction to Algorithms is quite approachable and its chapters are pretty well self-contained.

⬐ djaychela

Thanks - I'll keep that in mind (although I'm old and attempting to retrain, and have found lots of things more difficult than everyone else appears to!)

⬐ fuzzyset

Those are all there. With the nicknames as well.

⬐ None

None

⬐ None

None

⬐ renox

A book about the design patterns is a good idea, the original book by then the gang of 4? I disagree, IMHO this book isn't well written, now the question is: is there a better book on this topic?

⬐ None

None

⬐ zerogvt

Head first is a much better option IMO https://www.amazon.com/Head-First-Design-Patterns-Brain-Frie...

They meant the book Design Patterns https://www.amazon.com/Design-Patterns-Elements-Reusable-Obj...

⬐ jdmichal

The person I replied to, sideshowb, explicitly asked about both design patterns and Design Patterns. Regardless, that book was what elevated design patterns into common knowledge within the object-oriented world. Their fate is tied together; the book and the patterns described within are practically synonymous.

The classical Design Patterns book has a first chapter which takes you through the design of a text editor using the patterns provided in the book. If what you do is read the chapter and then the patterns referenced as you go and build the text editor based on their design you get exactly the sort of thing you are looking for. Its a different way of doing it than the entire book but arguably just in a different format for what is otherwise a reference book.

https://www.amazon.com/Design-Patterns-Elements-Reusable-Obj...

>, the good old Design Patterns book, responsible for more atrocious over-abstracted,

This is a misunderstanding of the DP book. It would be similar to saying that the existence of TVTropes.com is responsible for terrible scripts of tv shows and movies. Or, the existence of the Oxford English Dictionary is responsible for bad novels and useless documentation.

The DP book is a catalog (to gain awareness) and not a checklist (that you must do). It's a collection of observations about software structures out in the wild. It's not about prescription.

Even without the phrase "design patterns", it's easy to surmise that programmers out there are independently and unknowingly re-inventing the same higher-level structures (similar to tv tropes) but without giving them formal names. The DP book gives them formal names and hence, an attempt at shared vocabulary.

One can go to the amazon.com page for Design Patterns and click "Look inside" for the "First Pages" and see that the authors wanted to present a catalog: https://www.amazon.com/Design-Patterns-Elements-Reusable-Obj...

⬐ lkozma

Still, the original architecture "Design Patterns" book by Alexander (which this book is supposed to be the software analogy of) discusses "good practices", emphatically recommended for architects, city planners, etc. So it is maybe not that far-fetched to interpret these patterns as recommendations too.

⬐ ryandrake

Unfortunately, it tends to be _read_ as a how-to book rather than as a catalog of observations. I have actually heard from a professional software engineer, "We can't write the code this way because it's not using any known design pattern," said with a straight face. This is partially the mentality that gives us things like the classic AbstractSingletonProxyFactoryBean[1] class.

1: https://news.ycombinator.com/item?id=4549544

⬐ msluyter

I've seen this argument before, but I don't really buy it. Why did they subtitle their book "elements of reusable object oriented software" if they didn't intend to present "patterns" as "templates." (After all, "reusable" is a good thing, right? So if you follow these patterns your code will be reusable, hence, good.) Why did they use the word "pattern" at all? Why not name the book "Software Structures: observations about software out in the wild?"

In any event, it doesn't really matter what the original intent of the GoF book is because in practice, people treat design patterns as, well, what the word "pattern" would suggest: "something designed or used as a model for making things <a dressmaker's pattern>".

This is further indicated by the term "anti-pattern," which basically means, "something you should avoid." If you should avoid the "anti" then it stands to reason you should follow the "pattern."

For even more demonstrations of this fact, look at subsequent design patterns book, like one by the Head First. These explicitly treat patterns as improvements to be made over some previous attempt at a coding problem. Or hell, check out this description from https://sourcemaking.com/design_patterns -- "Design patterns can speed up the development process by providing tested, proven development paradigms." " Reusing design patterns helps to prevent subtle issues that can cause major problems and improves code readability for coders and architects familiar with the patterns."

Whether it was the original intent of the GoF or not, design patterns are mostly perceived to be _good things_ that should be adhered to.

Now, as many have suggested, the wisdom of reaching for design patterns has been increasingly viewed with skepticism, and rightly so. It'll be interesting to see over time how much weight they carry and where the equilibrium lies. My guess is that certain patterns will essentially fade from relevance (if your language has first class methods, do you really need Strategy, for example?) While some (adapter, facade) remain useful ways of handling various problems.

⬐ bphogan

The GoF book is responsible for a lot of bad decisions made by inexperienced devs because colleges insist on using it as a textbook. It's the textbook used in most Java-based CS programs for their Design Patterns courses. It was when I was in school and it still is now.

Mainstream developers who attend schools that produce Java developers believe that this is a recipe book. I have seen this book used at conference talks for nearly 20 years. Countless blog posts aimed at beginners reference this book as the way to solve problems.

The book itself is pretty explicit that they are not recipes, but that doesn't mean it isn't used that way.

Don't blame the book - blame the curriculum that's based on it.

⬐ agumonkey

A friend used to say that, he likes to share vocabulary, so they can avoid duplication. Which is entirely true, that said this book gave me nightmare, because I found the mindset so ceremonious; borderline off topic.

Giving name like flyweight to a simple "compression" scheme... The whole thing felt like "duh" to me. Sure it give common ground to people; but none of it felt like something to be written down, even less worshiped like it was.

I also strongly feel about the critics saying that it targets poor OOP languages (cpp, java <5), rendering the whole thing moot in other paradigms.

⬐ None

None

⬐ iak8god

> This is a misunderstanding of the DP book.

That may be, but it's a very common misunderstanding of the book by junior developers who read the book as a list of recipes to use anywhere they might be applicable.

Is it the book's fault that it's understood this way? I don't know. Regardless, the book + this persistent understanding and use of it are, as grabcocque says, "responsible for more atrocious over-abstracted, unreadable, hard to maintain Java code than anything else before or since."

⬐ dragonwriter

The facts that the book focuses on exactly the kind of patterns which are not abstractable into libraries in the popular OOP languages of the time (mostly true now, too), and that the implementation of each pattern is central to the presentation -- both of which arguably have quite good reasons based on the books intended role, to be sure -- I think promote this misunderstanding.

Certain other design patterns books in the software field (Patterns of Enterprise Application Architecture and Enterprise Integration Patterns, for examples that happen to be on the shelf next to me at the moment) don't share those features.

⬐ goostavos

I dunno if it's just _junior_ ones. I've encountered my share of "enterprise architects" that will use this book as though it were a bible when trying to justify their near-insane scribbling of boxes and lines on the white board.

⬐ iak8god

Yes, well, some of the juniors never repent. Plus it's a decent tactic for job security through obscurity...

⬐ dragonwriter

> >, the good old Design Patterns book, responsible for more atrocious over-abstracted,

> This is a misunderstanding of the DP book.

It is discussing an effect of the book, not the intent.

> It would be similar to saying that the existence of TVTropes.com is responsible for terrible scripts of tv shows and movies.

Which would be entirely accurate if script writers were widely using TVTropes pages as templates and creating bad scripts from it, regardless of what TVTropes is intended to be.

⬐ jasode

>It is discussing an effect of the book, not the intent.

Do you truly believe that specific DP book caused that effect?

I don't. The DP book is extremely dry reading and most programmers have not actually read it. ([In terms of readership/ownership ratio] It's a book like Stephen Hawking's "Brief History of Time" ... everybody knows about it and some might have it on the coffee table but very few actually read it.)

I do think the _phrase_ of "design patterns" (not the book) has become a sort of gravitational force such that any discussion about "overcomplicated design" seems to fall toward that particular phrase. Or put differently, "bad designs" is now a synonym of "design patterns" so much so such that you can't even say "design pattern" and have it be interpreted as a neutral term. It's become a trigger word.

>Which would be entirely accurate if script writers were widely using TVTropes pages as templates and creating bad scripts

Yes, I agree that would be equivalent. However, I don't think the existence of tvtropes compels directors like Michael Bay to repeat the "walk away calmly from explosion in the background" scene. The motivation to repeat that "cinematic pattern" comes from somewhere else. Likewise, the majority of bad/overcomplicated abstractions in source code come from somewhere else besides that particular DP book.

⬐ acdha

> Do you truly believe that specific DP book caused that effect? > > I don't. The DP book is extremely dry reading and most programmers have not actually read it.

This doesn't match my experience but I think that's because you're conflating two related ideas: while few people may have deeply read DP, that's not a requirement for it to have had a strong influence on them, any more than it is necessary for someone to have deeply read and understood their holy book to have strong religious beliefs.

I make that comparison because the worst projects I've seen were heavily based on DP but in a bad way because it was used more as a club than a means for self-improvement. Architects threw it around to bolster their status, a strong push towards ritual cargo culting rather than making sure the problem was well understood enough to know that the code being written matched the actual business case, and any criticism was met with a flak shield of “This is the best practice way according to DP”.

That's not necessarily DPs fault — although there is a discussion to be had about valuing comprehensibility if you want to make meaningful improvements — but it was definitely an effect shaping software development for the worse for over a decade.

⬐ bluGill

Most people have a copy of DP and have not read the whole thing. Most people have read it in that they have read a couple pages, the table of contents, and a couple random patterns. To some extent that is how the book should be read: you have a problem that a DP solves and so you read the section on that pattern to get it right.

The problem is most people have a superficial knowledge of the patterns in the book (a list is easy to put together along with a short summary so most people have read one). They then go looking for how to apply the patterns to their domain so they can check the pattern box, without thinking about the large picture. They never consider that patterns exist to solve a specific problem and if you don't have that problem you shouldn't use the pattern.

The classic example is the singleton: it solves a tricky problem that large systems sometimes have. However few people realize that a singleton is a fancy name for a global variable and have all the downsides of a global variable.

There are several patterns that are a specific way of creating a layer of abstraction. Abstraction layers are often very useful, but as the saying goes: "all problems in programing can be solved by adding a layer of abstraction - except too many layers of abstraction". I've seen cases of layers of abstraction that didn't solve any problems.

As such the original point: DP, by giving patterns names has created more harm than good. This is not the fault of the book though: the names were needed and the patterns are very useful when applied correctly. However before the book people would re-invent the patterns (badly) only when they needed the pattern.

⬐ ci5er

> However few people realize that a singleton is a fancy name for a global variable and have all the downsides of a global variable.

Really? It's been twenty years (maybe a bit more?) since I read the book, but I think this was pretty explicit, was it not?

⬐ None

None

⬐ dragonwriter

> Do you truly believe that specific DP book caused that effect

I believe that it, and specific choices made in the selection and presentation of patterns in it, contributed significantly.

> The DP book is extremely dry reading and most programmers have not actually read it.

Perhaps, but the influence of a book (especially one which directly inspires other books) often extends beyond its direct readership.

⬐ davik

A very weird comparison, the brief history of time is a popular science book aimed at laymen, it sacrifices precision for a larger audience and is not really a dry book.

⬐ jasode

Sorry for the confusion. (I've edited my comment for clarity so people don't think I was insulting SH.) I wasn't saying Hawking's prose was dry. That attribute was intended solely for the DP book. With ABHOT, I was comparing the ratio of readers aware of the book vs actually reading it. ABHOT has become sort of a "poster child"[1] among bibliophiles for books owned for vanity and I thought HN readers would get that reference.

[1] example from WSJ that mentions the common meme about ABHOT: https://www.wsj.com/articles/the-summers-most-unread-book-is...

⬐ dpdawson

If you own ABHOT and haven't read it, do yourself a favor and give it a shot. It's extremely interesting and a fairly easy read.

Context: My formal physics education ended in high school.

⬐ dwaltrip

I'll piggyback on you and highly recommend the "crash course astronomy" series on YouTube. It's 50 or so ~10 minute videos that does a broad overview of all of astronomy up to the current moment, starting back when the older civilizations first studied the stars just by looking up with their naked eyes.

It is fast paced and packs a lot of information into each episode, so I do end up rewinding sometimes. And being a passive video, I think retention won't be as high. However, the short videos are very approachable and bite sized. It's very well done and the host is quite entertaining. The past year, my interest in space and physics has increased a bunch. It feels like learning the deeper history and context in which all of humanity is embedded.

⬐ nickpsecurity

I'm actually with the others on this. I've been looking at CompSci papers, engineering methodologies, and real world code going back a long time. I saw a few patterns in use like Parnas' information hiding or state machines. I didn't see GoF patterns until I saw Java people using them while recommending the book. Also, many people in industry forgot about prior methodologies that got results without over-abstraction such as Cleanroom but were all over GoF patterns and latter Agile methodology. It seems their work was pretty influential.

⬐ galephico

do you know where I could find those other patterns?

⬐ nickpsecurity

You probably have already seen them. It was just things such as structured programming, decomposition into functions, layering with minimal loops between layers, interface checks for invariants, avoiding risky languages/constructs, and usage-based testing.

⬐ ssalazar

> Do you truly believe that specific DP book caused that effect?

GoF/Design Patterns was published in 1994. Java, the standard-bearer of overapplying Design Patterns, was first released in 1995, and it only grew into the AbstractAdapterFactoryImpl cliches after that. Based on that timeline Im not really sure how you can argue that GoF had nothing to do with DP madness, at least as exemplified by Java.

Just because GoF was originally meant to be descriptive does not mean that it can't be interpreted as prescriptive, especially by your stock work-a-day software engineer or peter-principle'd software architect. Despite whatever intentions the GoF may have had, by giving these patterns a name, they in a sense codified them, giving people something to point to when justifying otherwise poorly thought out design decisions.

> The DP book is extremely dry reading and most programmers have not actually read it.

The thing about GoF is that you don't really have to read it to apply it--and not actually reading it only increases the odds of mis-applying patterns if youve encountered them elsewhere.

⬐ mr_pink

> Do you truly believe that specific DP book caused that effect?

Yes, definitely. People used to cite the GOF book left and right while over-engineering crappy CRUD apps.

⬐ ajmurmann

I think you are hugely underestimating the influence this book had. This was the first catalog including these patterns and the authors gathered them by looking at source code from many, many projects and compiled common patterns they deemed useful into this book. They named the patterns and this book ultimately popularised them. The book was a huge success when it first came out and I know many people who have indeed read the book. I've even encountered it in university twice. Without this book many of these patterns might not have a name and hardly anybody would know them. They would be patterns in the way that they randomly occur in the wild, but not in the sense that they are templates that developers seek out to solve problems.

What i listed was just an example but you can verify quickly - https://www.amazon.com/dp/0201633612/?tag=devbookscom20-20 without opening the actual book.

And afaict - none of the patterns I listed are mentioned verbatim.

⬐ laxd

Seems like MVC is treated in GoF. http://blog.iandavis.com/2008/12/what-are-the-benefits-of-mv...

⬐ munificent

It talks a little about it, but it's not called out as a specific pattern with a chapter all its own.

There's always the classics:

https://www.amazon.com/Design-Patterns-Elements-Reusable-Obj...

https://www.amazon.com/Refactoring-Improving-Design-Existing...

EDIT: And it appears the website is a direct rip-off of these books.

⬐ wbillingsley

So much so, and for so long, that I'd always assumed it was authorised. It seems not.

⬐ icholy

FWIW I already have both of those.

⬐ petecox

"Refactoring to Patterns"

https://www.goodreads.com/book/show/85041.Refactoring_to_Pat...

Design Patterns: Elements of Reusable Object-Oriented Software http://www.amazon.com/Design-Patterns-Elements-Reusable-Obje...

Practical Object-Oriented Design in Ruby by Sandi Metz. http://www.poodr.com/

⬐ talles

Maybe it's a matter of taste, but I found the GoF book really boring (and a little self hyped TBH).

⬐ kat

If you want a good alternative to the GoF book try Head First Design Patterns. I findit much more approachable and much more enjoyable to read. http://www.amazon.com/Head-First-Design-Patterns-Freeman-ebo...

A remarkable classic http://www.amazon.com/Design-Patterns-Elements-Reusable-Obje...

And Kent Beck's http://www.amazon.com/Smalltalk-Best-Practice-Patterns-Kent/...

Dive into design patterns. Get yourself a good book listing them. The bible of all developers is the Gang Of Four (http://www.amazon.com/Design-Patterns-Elements-Reusable-Obje...), but there are plenty good ones out there that aren't as dry and academic.

Design patterns are the greatest tool there is for learning software design. If you know them, you can apply them, but much more importantly they show you how you can apply the basic principles of OO development to various problems. By learning them, you're learning the thought process of how they came to be, and you'll be able to apply those thought processes to your own problems. Consider it like learning algebra by doing exercises. You can read theory all you like, but unless you apply it it just won't click.

Just in case, here's the 'Gang of Four' book on design patterns: http://www.amazon.com/Design-Patterns-Elements-Reusable-Obje...

We've known this since at least 1994, when the GoF book [1] famously said:

"Favor object composition over class inheritance"

[1] http://www.amazon.com/Design-Patterns-Elements-Reusable-Obje...

⬐ drumdance

Ha, that's one of the few things I remember from that book. (Not knocking the book, just my poor memory.) I remember it being a big "a ha!" for a project I was working on.

Yes. I believe such a book exists (or should at least) for every language as well.

Effective Java — http://www.amazon.com/Effective-Java-Edition-Joshua-Bloch/dp...

POODR (Ruby) — http://www.poodr.com/

Javascript the good parts — http://www.amazon.com/JavaScript-Good-Parts-Douglas-Crockfor...

There is of course also the gang of 4 language agnostic classic on design patterns http://www.amazon.com/Design-Patterns-Elements-Reusable-Obje...

- Design Patterns: Elements of Reusable Object-Oriented Software http://www.amazon.com/Design-Patterns-Elements-Reusable-Obje...

- Effective C++: http://www.amazon.com/Effective-Specific-Improve-Programs-De...

OK, if you don't have any real experience in low-level embedded coding (relevant to device drivers), RTOS or OS design in general, file systems, data structures, algorithms, interfaces, etc. And, if you have "hobby level" experience with Assembler, C and C++. And, if your intent is to write a desktop OS, from the ground up, without making use of existing technologies, drivers, file systems, memory management, POSIX, etc. Here's a list of books that could be considered required reading before you can really start to write specifications and code. Pick twenty of these and that might be a good start.

In no particular order:

1- http://www.amazon.com/C-Programming-Language-2nd-Edition/dp/...

2- http://www.amazon.com/The-Answer-Book-Solutions-Programming/...

3- http://www.amazon.com/The-Standard-Library-P-J-Plauger/dp/01...

4- http://www.amazon.com/C-Traps-Pitfalls-Andrew-Koenig/dp/0201...

5- http://www.amazon.com/Expert-Programming-Peter-van-Linden/dp...

6- http://www.amazon.com/Data-Structures-In-Noel-Kalicharan/dp/...

7- http://www.amazon.com/Data-Structures-Using-Aaron-Tenenbaum/...

8- http://www.amazon.com/Mastering-Algorithms-C-Kyle-Loudon/dp/...

9- http://www.amazon.com/Code-Complete-Practical-Handbook-Const...

10- http://www.amazon.com/Design-Patterns-Elements-Reusable-Obje...

11- http://www.amazon.com/The-Mythical-Man-Month-Engineering-Ann...

12- http://www.amazon.com/The-Programming-Language-4th-Edition/d...

13- http://www.amazon.com/The-Standard-Library-Tutorial-Referenc...

14- http://www.amazon.com/API-Design-C-Martin-Reddy/dp/012385003...

15- http://www.amazon.com/The-Linux-Programming-Interface-Handbo...

16- http://www.amazon.com/Computer-Systems-Programmers-Perspecti...

17- http://www.amazon.com/System-Programming-Unix-Adam-Hoover/dp...

18- http://www.amazon.com/Memory-Programming-Concept-Frantisek-F...

19- http://www.amazon.com/Memory-Management-Implementations-Prog...

20- http://www.amazon.com/UNIX-Filesystems-Evolution-Design-Impl...

21- http://www.amazon.com/PCI-System-Architecture-4th-Edition/dp...

22- http://www.amazon.com/Universal-Serial-System-Architecture-E...

23- http://www.amazon.com/Introduction-PCI-Express-Hardware-Deve...

24- http://www.amazon.com/Serial-Storage-Architecture-Applicatio...

25- http://www.amazon.com/SATA-Storage-Technology-Serial-ATA/dp/...

26- http://www.amazon.com/Beyond-BIOS-Developing-Extensible-Inte...

27- http://www.amazon.com/Professional-Assembly-Language-Program...

28- http://www.amazon.com/Linux-Kernel-Development-3rd-Edition/d...

29- http://www.amazon.com/Version-Control-Git-collaborative-deve...

30- http://www.amazon.com/Embedded-Software-Primer-David-Simon/d...

31- http://www.amazon.com/Programming-Embedded-Systems-C/dp/1565...

32- http://www.amazon.com/Making-Embedded-Systems-Patterns-Softw...

33- http://www.amazon.com/Operating-System-Concepts-Abraham-Silb...

34- http://www.amazon.com/Performance-Preemptive-Multitasking-Mi...

35- http://www.amazon.com/Design-Operating-System-Prentice-Hall-...

36- http://www.amazon.com/Unix-Network-Programming-Sockets-Netwo...

37- http://www.amazon.com/TCP-Illustrated-Volume-Addison-Wesley-...

38- http://www.amazon.com/TCP-IP-Illustrated-Vol-Implementation/...

39- http://www.amazon.com/TCP-Illustrated-Vol-Transactions-Proto...

40- http://www.amazon.com/User-Interface-Design-Programmers-Spol...

41- http://www.amazon.com/Designing-Interfaces-Jenifer-Tidwell/d...

42- http://www.amazon.com/Designing-Interfaces-Jenifer-Tidwell/d...

43- http://www.amazon.com/Programming-POSIX-Threads-David-Butenh...

44- http://www.intel.com/p/en_US/embedded/hwsw/software/hd-gma#d...

45- http://www.intel.com/content/www/us/en/processors/architectu...

46- http://www.intel.com/p/en_US/embedded/hwsw/hardware/core-b75...

47- http://www.hdmi.org/index.aspx

48- http://en.wikipedia.org/wiki/Digital_Visual_Interface

49- http://www.amazon.com/Essential-Device-Drivers-Sreekrishnan-...

50- http://www.amazon.com/Making-Embedded-Systems-Patterns-Softw...

51- http://www.amazon.com/Python-Programming-Introduction-Comput...

52- http://www.amazon.com/Practical-System-Design-Dominic-Giampa...

53- http://www.amazon.com/File-Systems-Structures-Thomas-Harbron...

54- ...well, I'll stop here.

Of course, the equivalent knowledge can be obtained by trial-and-error, which would take longer and might result in costly errors and imperfect design. The greater danger here is that a sole developer, without the feedback and interaction of even a small group of capable and experienced programmers could simply burn a lot of time repeating the mistakes made by those who have already trenched that territory.

If the goal is to write a small RTOS on a small but nicely-featured microcontroller, then the C books and the uC/OS book might be a good shove in the right direction. Things start getting complicated if you need to write such things as a full USB stack, PCIe subsystem, graphics drivers, etc.

⬐ tagada

The guy don't want to write a perfectly designed OS, neither he wants to design a RTOS by the way ... He didn't even specify whether his OS would be multitask. If not, no need for any scheduling at all, huh ... he just "writes his own OS". It's a fun experience, very rich and very teaching. But at least now, we all can admire the wideness of all your "unlimited knowledge" especially in terms of titles of books. Afterall, no needs for creating a new OS, if you get all your inspiration from things that already exists. You will end with Yet Another Nix ... Not sure he wants a YanOS, though.

Without talking about filesystems, drivers, memory managment or existing norms and standards (which it seems he wants to avoid ... which is not that stupid ... all having been designed 40 years ago ... who knows maybe he'll have a revolutionary idea, beginning from scratch), going from scratch with no exact idea of where you go could be a good thing. Definitely. You solve the problem only when you face them. Step by step, sequentially. Very virile, man.

I would advice the guy to start obviously with the boot loader. Having a look at the code of GRUB (v1), LILO, or better yet, on graphical ones (BURG, GAG or this good one XOSL which is completely under GUI - mouse pointer, up to 1600x1200 screens, and many supported filesystems, written in ASM ... it could actually be a hacking basis for a basic OS on itself) could be a great source of inspiration, and beginning to play directly with the graphical environment.

You could also have a look on these things http://viralpatel.net/taj/tutorial/hello_world_bootloader.ph... and of course on Kolibri OS http://kolibrios.org/en/?p=Screenshots

⬐ tagada

Blah, blah, blah ...

Better advice would be: Try, make mistakes, learn from them, continue.

Way, way, way less discouraging, pessimistic and above all pedantic.

⬐ derefr

> If the goal is to write a small RTOS on a small but nicely-featured microcontroller, then the C books and the uC/OS book might be a good shove in the right direction. Things start getting complicated if you need to write such things as a full USB stack, PCIe subsystem, graphics drivers, etc.

I've always wondered if there could be created some way to skip this step in [research] OS prototyping, by creating a shared library (exokernel?) of just drivers, while leaving the "design decisions" of the OS (system calls, memory management, scheduling, filesystems, &c.--you know, the things people get into OS development to play with) to the developer.

People already sort of do this by targeting an emulator like VirtualBox to begin with--by doing so, you only (initially) need one driver for each feature you want to add, and the emulator takes care of portability. But this approach can't be scaled up to a hypervisor (Xen) or KVM, because those expect their guest operating systems to also have relevant drivers for (at least some of) the hardware.

I'm wondering at this point if you could, say, fork Linux to strip it down to "just the drivers" to start such a project (possibly even continuing to merge in driver-related commits from upstream) or if this would be a meaningless proposition--how reliant are various drivers of an OS on OS kernel-level daemons that themselves rely on the particular implementation of OS process management, OS IPC, etc.? Could you code for the Linux driver-base without your project growing strongly isomorphic structures such as init, acpid, etc.?

Because, if you could--if the driver-base could just rely on a clean, standardized, exported C API from the rest of the kernel, then perhaps (and this is the starry-eyed dream of mine) we could move "hardware support development" to a separate project from "kernel development", and projects like HURD and Plan9 could "get off the ground" in terms of driver support.

⬐ robomartin

A lot depends on the platform. If the OS is for a WinTel motherboard it is one thing. If, however, the idea is to bypass driver development for a wide range of platforms it gets complicated.

In my experience one of the most painful aspects of bringing up an OS on a new platform is exactly this issue of drivers as well as file systems. A little google-ing quickly reveals that these are some of the areas where one might have to spend big bucks in the embedded world in order to license such modules as FFS (Flash File System) with wear leveling and other features as well as USB and networking stacks. Rolling your own as a solo developer or even a small team could very well fit into the definition of insanity. I have done a good chunk of a special purpose high-performance FFS. It was an all-absorbing project for months and, realistically, in the end, it did not match all of the capabilities of what could be had commercially.

This is where it is easy to justify moving into a more advanced platform in order to be able to leverage Embedded Linux. Here you get to benefit and leverage the work of tens of thousands of developers devoted to scratching very specific itches.

The down-side, of course, is that if what you need isn't implemented in the boad support package for the processor you happen to be working with, well, you are screwed. The idea that you can just write it yourself because it's Linux is only applicable if you or your team are well-versed in Linux dev at a low enough level. If that is not the case you are back to square one. If you have to go that route you have to hire an additional developer that knows this stuff inside out. That could mean $100K per year. So now your are, once again, back at square one: hiring a dev might actually be more exoensive than licensing a commercial OS with support, drivers, etc.

I was faced with exactly that conundrum a few years ago. We ended-up going with Windows CE (as ugly as that may sound). There are many reasons for that but the most compelling one may have been that we could identify an OEM board with the right I/O, features, form factor, price and full support for all of the drivers and subsystems we needed. In other words, we could focus on developing the actual product rather than having to dig deeper and deeper into low-level issues.

It'd be great if low level drivers could be universal and platform independent to the degree that they could be used as you suggest. Obviously VM-based platforms like Java can offer something like that so long as someone has done the low-level work for you. All that means is that you don't have to deal with the drivers.

To go a little further, part of the problem is that no standard interface exists to talk to chips. In other words, configuring and running a DVI transmitter, a serial port and a Bluetooth I/F are vastly different even when you might be doing some of the same things. Setting up data rates, clocks, etc. can be day and night from chip to chip.

I haven't really given it much thought. My knee-jerk reaction is that it would be very hard to crate a unified, discoverable, platform-independent mechanism to program chips. The closest one could possibly approach this idea would be if chip makers were expected to provide drivers written to a common interface. Well, not likely or practical.

Not an easy problem.

⬐ derefr

> It'd be great if low level drivers could be universal and platform independent to the degree that they could be used as you suggest. Obviously VM-based platforms like Java can offer something like that so long as someone has done the low-level work for you. All that means is that you don't have to deal with the drivers.

Another thought: if not just a package of drivers, then how stripped down (for the purpose of raw efficiency) could you make an operating system intended only to run an emulator (IA32, JVM, BEAM, whatever) for "your" operating system? Presumably you could strip away scheduling, memory security, etc. since the application VM could be handling those if it wanted to. Is there already a major project to do this for Linux?

The only way to learn how to program is to just do it. Pick a language. Start with small programs. Learn the features and syntax of the language. When you feel you know the language well enough to write a program that uses most of its features, then pick an open source program and learn its code. Make some modifications to it. When you feel you're pretty comfortable in one language, pick a new language.

Rinse repeat. You'll find that once you have one language down, the process will go much faster with the second. The features and syntax of most languages are pretty similar.

You can pick any language you like to start with, but personally I'd recommend starting with a hard typed compiled language rather than a dynamic language. Dynamic languages are easy to learn, but its easy to be a lazy and bad programmer in them. They let you get away with too much. Hard typed compiled languages are a) much better at catching your errors and b) much harder to be a lazy programmer in. They don't let you get away with things nearly as much.

I'd recommend Java, seems to be the language a lot of people start with. It will also give you a good founding in open source programming. I started with C/C++ and I'm glad I did. But that was more of a challenge - had to learn pointers and memory management in my first language. And after that I took a little while to learn the open source mindset, since I'd started in a procedural language.

Once you have a good grasp of an object oriented language like Java (or Python) and what object oriented programming means, then I'd recommend reading the Gang of Four book on Design Patterns (http://www.amazon.com/Design-Patterns-Elements-Reusable-Obje...) and Martin Fowler's book on Refactoring (http://www.amazon.com/Refactoring-Improving-Design-Existing-...). Those will give you a pretty good basis in software design and maintenance.

In terms of recommended reading for learning a language, the O'Reilly books are nearly always good in my experience. Here's one for Java (Learning Java: http://www.amazon.com/Learning-Java-Patrick-Niemeyer/dp/0596...) and here's one for Python (Learning Python: http://www.amazon.com/Learning-Python-Powerful-Object-Orient...)

⬐ Female-hacker

Amazing. No Google search led me to an answer this helpful. Thank you for your input on languages also...it seems there is quite a debate between Java and Python.

⬐ SkyMarshal

I disagree with starting with Java. Rather I second coffeemug's advice in the following comment, focus on C, Lisp, and Haskell:

http://news.ycombinator.com/item?id=1747713

My first manager taught me something college had failed to (or that I failed to learn there), which is that programming languages come and go in trends, but they are all based on recurring fundamental concepts, which endure beyond any single language.

Most new programming languages are a remix or mashup of some or all of these concepts, so if you master those concepts you'll be able to easily pick up any other language.

It's like learning Latin as your first foreign language, and being able to quickly pick up any Latin-based language after that. C, Lisp, and Haskell are the Latin of the CS world.

Since you're at MIT, take the famous Structure & Interpretation of Computer Programs (used to be 6.001, not sure if it still is). http://groups.csail.mit.edu/mac/classes/6.001/abelson-sussma.... That'll get you started with Lisp (specifically the Scheme dialect).

For C, start with K&R (http://en.wikipedia.org/wiki/The_C_Programming_Language_(boo...).

For Haskell, Real World Haskell is good, and the e-version is free. (http://book.realworldhaskell.org/)

Joel Spolsky has an article worth reading on why not Java:

http://www.joelonsoftware.com/articles/ThePerilsofJavaSchool...

And of course PG's essays on Lisp are a must read:

http://www.paulgraham.com/lisp.html

Since it sounds like you're running a business you may feel pressured to learn Java or whatever your coders are currently using to get your business off the ground. But I heartily recommend resisting that pressure, doing it right, and mastering the enduring fundamentals first. It may cost you something in immediacy and expedience, but that's what school is for, especially one such as MIT. You should carefully weigh how you spend your time there, and unless you think you've got the next Facebook, focus on mastering the enduring concepts, not the trends. You'll be glad you did the rest of your life and career.

PS - there's little to nothing that Clojure (a Lisp dialect on the JVM) and Haskell can't do in the web space, so the cost in immediacy and expedience is slim to none anyway.

PPS - it didn't register till just now that you're an Econ Major not going into CS or Software Engineering as a career. If you just want a tried-and-true, versatile, easy-to-get-started-with language with tons of libraries and frameworks (reusable code) to hack in, huge and helpful community go with Python. Java is too bureaucratic a language for this purpose.

Start with Zed Shaw's excellent introduction, Learn Python the Hard Way (hard way = by doing lots of follow-along examples until it gets into your brain muscle memory), then go from there.

http://learnpythonthehardway.org/index

⬐ dbingham

Python's a good language, but it's a soft-typed interpreted language. Which means it will let you get away with certain things.

It is also has some differences in syntax from a lot of other languages that might give you some trouble when you switch. C/C++, Java, PHP, Perl, Javascript and many other languages share a lot of common syntax. Python's still really similar, but it deviates in some key ways.

It also tends to have really hard core adherents who will swear by it above and over anything else, which is why there is something of a debate. It really doesn't matter that much what language you start with.

One of the keys to being a good programmer is knowing your toolbox. And that includes knowing multiple languages. The best programmers don't have one favorite language. Rather they have a toolbox of a half a dozen or more, and they pick the best language for the task at hand.

⬐ Adam503

"...it will let you get away with certain things..."

...like reading what she just typed in? Yes. Python will let her "get away" with reading her code. ;-)

Here's a link to the Beginners Guide to Python...

http://wiki.python.org/moin/BeginnersGuide

I know no one like to read a text book, but good option would be picking up the Gang of Four design patterns book. http://www.amazon.com/Design-Patterns-Elements-Reusable-Obje... I recently read most of it (preparing for a final exam yay!) and the really nice thing about this book is the examples. Each pattern has a description, a list of pros and cons, a discussion of a piece of real software that used (or often pioneered) this pattern and then finally an example of C++ that uses it.

Honestly, the discussion of pros/cons and then the use in industry sections make this book a very good read. It won't be quite as entertaining as lippert's blog, but then again it is a Textbook

Some books to read:

http://www.amazon.com/Algorithm-Design-Jon-Kleinberg/dp/0321...

http://www.amazon.com/Design-Patterns-Object-Oriented-Addiso...

http://www.amazon.com/Refactoring-Improving-Existing-Addison...

http://www.artima.com/shop/forsale

Consider buying these books:

http://www.amazon.com/Refactoring-Improving-Design-Existing-...

http://www.amazon.com/Design-Patterns-Object-Oriented-Addiso...