HN Theater @HNTheaterMonth

I thought calling programmers "engineers" was very odd until I saw this great talk:

Real Software Engineering by Glenn Vanderburg

Lone Star Ruby Conference 2010 https://www.youtube.com/watch?v=NP9AIUT9nos

Software engineering as it's taught in universities simply doesn't work. It doesn't produce software systems of high quality, and it doesn't produce them for low cost. Sometimes, even when practiced rigorously, it doesn't produce systems at all.

That's odd, because in every other field, the term "engineering" is reserved for methods that work.

What then, does real software engineering look like? How can we consistently deliver high-quality systems to our customers and employers in a timely fashion and for a reasonable cost? In this session, we'll discuss where software engineering went wrong, and build the case that disciplined Agile methods, far from being "anti-engineering" (as they are often described), actually represent the best of engineering principles applied to the task of software development.

This talk describes how the waterfall development model was a caricature that got accidentally turned into reality: https://www.youtube.com/watch?v=NP9AIUT9nos "Lone Star Ruby Conference 2010 Real Software Engineering by Glenn Vanderburg"

This talks about the same topic: Lone Star Ruby Conference 2010: "Real Software Engineering" by Glenn Vanderburg: https://www.youtube.com/watch?v=NP9AIUT9nos

I always thought calling programmers engineers was very weird until I watched this great talk:

Real Software Engineering, Glenn Vanderburg, Lone Star Ruby 2010 https://www.youtube.com/watch?v=NP9AIUT9nos

which explains what various kinds of engineers actually do. Not at all dry and boring like it might sound! From the YouTube blurb:

Software engineering as it's taught in universities simply doesn't work. It doesn't produce software systems of high quality, and it doesn't produce them for low cost. Sometimes, even when practiced rigorously, it doesn't produce systems at all. That's odd, because in every other field, the term "engineering" is reserved for methods that work. What then, does real software engineering look like?

⬐ walshemj

Really I recall working for one on the big civil consultancies and we had one of our bridges fall of its supports once.

And lets not mention the new airport in Berlin (or the NYC tube lines) in the context of low cost.

⬐ yesenadam

> Really I recall working for one on the big civil consultancies and we had one of our bridges fall of its supports once.

> And lets not mention the new airport in Berlin (or the NYC tube lines) in the context of low cost.

That seems to me a low-quality comment, sorry. The lack of punctuation and the spelling mistakes made it hard to parse, and until you flesh it out more, I'm not sure precisely what point you were trying to make. And more mentioning, less "lets not mention"-ing, please.

⬐ walshemj

its two sentences mate ok first one it should be one of - I am dyslexic go sue me

⬐ kyuudou

Sounds like some of the cynicism behind Dijkstra's bit on software engineering[0]

"A number of these phenomena have been bundled under the name "Software Engineering". As economics is known as "The Miserable Science", software engineering should be known as "The Doomed Discipline", doomed because it cannot even approach its goal since its goal is self-contradictory. Software engineering, of course, presents itself as another worthy cause, but that is eyewash: if you carefully read its literature and analyse what its devotees actually do, you will discover that software engineering has accepted as its charter "How to program if you cannot."."

[0]https://www.cs.utexas.edu/users/EWD/transcriptions/EWD10xx/E... Edsger W. Dijkstra's "On the cruelty of really teaching computing science"

⬐ yesenadam

I was very sceptical too, but when I watched it, it all sounded very common-sensical.

I first came into contact with this article via https://www.youtube.com/watch?v=NP9AIUT9nos, which is a pretty decent analysis on how people misinterpreted the original concept.

⬐ gonzo41

I first came into contact with this article through this talk https://www.youtube.com/watch?v=csyL9EC0S0c

Its entertaining

⬐ musha68k

I first came into contact with this by way of Dave Farley's sobering "The Rationale for Continuous Delivery":

https://vimeo.com/147863304

He's essentially mocking our whole field for reading the paper lazily :) "This paper was a description of what not to do."

⬐ nickpsecurity

Probably Haskellers: always praising lazy evaluations. ;)

⬐ dang

It was Craig Larman who did the original research on this:

http://www.craiglarman.com/wiki/downloads/misc/history-of-it...

An old HN comment about it:

https://news.ycombinator.com/item?id=990463

⬐ nickpsecurity

That supports some of my guesses in my comment with a lot of context. Especially the government contractor part. Had anyone asked, I was going to reference Mills as an evolutionary step as well. I'll read the rest of it later.

Thanks for the paper! Getting many mental gaps filled in on our history this week. :)

⬐ dang

I can't remember if Larman includes this in the paper, but at a talk he said that he tracked down the guy who standardized waterfall for the DoD (in Boston IIRC) and when they met for lunch, the first thing he said to Larman was "I'm so sorry".

⬐ nickpsecurity

Well, at least he regretted the damage he did.

⬐ dang

In retrospect it was inevitable. The patterns of thinking that we take for granted around iteration and prototyping were still too alien. You have to get it before you can do it, and the types of programmers who were inclined to get it weren't in decision-making positions.

Another great story that Larman told was that he tracked down some of the programmers on famous waterfall projects that had succeeded, and found out that what they had really done was write code first, secretly, and then write up the requirements and design docs based on what they had learned. In other words they did things in the 'wrong' order but published them in the 'right' order.

⬐ nickpsecurity

"The patterns of thinking that we take for granted around iteration and prototyping were still too alien. You have to get it before you can do it, and the types of programmers who were inclined to get it weren't in decision-making positions."

Maybe I was too harsh on them. It does seem likely. Further, it probably came directly out of the nature of programming on expensive, shared, and often batched machines. Here's a great example of what programming looked like in the 60's (skip to 5:27):

https://vimeo.com/132192250

It wouldn't have been much better a bit after 1970 where many of the same concerns and processes would exist. I still think one has to ignore most of the Royce's paper to not pick up on iteration paradigm. But, I could easily see someone in that mentality sort of glossing over it, spotting a diagram of their current flow, giving it a name, and pushing it from there.

Finally read the paper you gave me. It was really neat to see iterative development kept getting independently invented from the 60's onward. It getting into the mainstream was an inevitability due to its inherent advantages. The majority just couldn't contain it within limited groups forever.

"and found out that what they had really done was write code first, secretly, and then write up the requirements and design docs based on what they had learned. In other words they did things in the 'wrong' order but published them in the 'right' order."

That's funny you say that: history is repeating in high-assurance field. Safety-critical design is like waterfall or spiral on steroids with specs, constraints on implementation, rigorous testing, analysis... you name it. To outsiders, it seems they're using a waterfall-like refinement strategy to build these things. Insiders trying to get Agile methods in there have countered that with an unusual supporting argument: successful projects already use an iterative process combining top-down and ground-up work that results in a linked, waterfall-like chain of deliverables. The actual work is often done differently, though, so why not allow that kind of development in the first place?

With your comment, I've heard that twice. Another quick one was Mills' people not being able to run their own code in Cleanroom. Sometimes it wasn't necessary but it has many benefits. So, of course they often ran their own code during prototyping phase to get an idea of how the final submission should be done. We'll all be better off when management lets their processes reflect the realities of development. At least it's already happening in some firms and spreading. :)

⬐ bklaasen

Brian Okken of the Python Testing podcast did a nice job explaining the importance of this paper: http://pythontesting.net/podcast/waterfall/

Summary of the links shared here:

http://blip.tv/clojure/michael-fogus-the-macronomicon-597023...

http://blog.fogus.me/2011/11/15/the-macronomicon-slides/

http://boingboing.net/2011/12/28/linguistics-turing-complete...

http://businessofsoftware.org/2010/06/don-norman-at-business...

http://channel9.msdn.com/Events/GoingNative/GoingNative-2012...

http://channel9.msdn.com/Shows/Going+Deep/Expert-to-Expert-R...

http://en.wikipedia.org/wiki/Leonard_Susskind

http://en.wikipedia.org/wiki/Sketchpad

http://en.wikipedia.org/wiki/The_Mother_of_All_Demos

http://io9.com/watch-a-series-of-seven-brilliant-lectures-by...

http://libarynth.org/selfgol

http://mollyrocket.com/9438

https://github.com/PharkMillups/killer-talks

http://skillsmatter.com/podcast/java-jee/radical-simplicity/...

http://stufftohelpyouout.blogspot.com/2009/07/great-talk-on-...

https://www.destroyallsoftware.com/talks/wat

https://www.youtube.com/watch?v=0JXhJyTo5V8

https://www.youtube.com/watch?v=0SARbwvhupQ

https://www.youtube.com/watch?v=3kEfedtQVOY

https://www.youtube.com/watch?v=bx3KuE7UjGA

https://www.youtube.com/watch?v=EGeN2IC7N0Q

https://www.youtube.com/watch?v=o9pEzgHorH0

https://www.youtube.com/watch?v=oKg1hTOQXoY

https://www.youtube.com/watch?v=RlkCdM_f3p4

https://www.youtube.com/watch?v=TgmA48fILq8

https://www.youtube.com/watch?v=yL_-1d9OSdk

https://www.youtube.com/watch?v=ZTC_RxWN_xo

http://vimeo.com/10260548

http://vimeo.com/36579366

http://vimeo.com/5047563

http://vimeo.com/7088524

http://vimeo.com/9270320

http://vpri.org/html/writings.php

http://www.confreaks.com/videos/1071-cascadiaruby2012-therap...

http://www.confreaks.com/videos/759-rubymidwest2011-keynote-...

http://www.dailymotion.com/video/xf88b5_jean-pierre-serre-wr...

http://www.infoq.com/presentations/Are-We-There-Yet-Rich-Hic...

http://www.infoq.com/presentations/click-crash-course-modern...

http://www.infoq.com/presentations/miniKanren

http://www.infoq.com/presentations/Simple-Made-Easy

http://www.infoq.com/presentations/Thinking-Parallel-Program...

http://www.infoq.com/presentations/Value-Identity-State-Rich...

http://www.infoq.com/presentations/We-Really-Dont-Know-How-T...

http://www.mvcconf.com/videos

http://www.slideshare.net/fogus/the-macronomicon-10171952

http://www.slideshare.net/sriprasanna/introduction-to-cluste...

http://www.tele-task.de/archive/lecture/overview/5819/

http://www.tele-task.de/archive/video/flash/14029/

http://www.w3.org/DesignIssues/Principles.html

http://www.youtube.com/watch?v=4LG-RtcSYUQ

http://www.youtube.com/watch?v=4XpnKHJAok8

http://www.youtube.com/watch?v=5WXYw4J4QOU

http://www.youtube.com/watch?v=a1zDuOPkMSw

http://www.youtube.com/watch?v=aAb7hSCtvGw

http://www.youtube.com/watch?v=agw-wlHGi0E

http://www.youtube.com/watch?v=_ahvzDzKdB0

http://www.youtube.com/watch?v=at7viw2KXak

http://www.youtube.com/watch?v=bx3KuE7UjGA

http://www.youtube.com/watch?v=cidchWg74Y4

http://www.youtube.com/watch?v=EjaGktVQdNg

http://www.youtube.com/watch?v=et8xNAc2ic8

http://www.youtube.com/watch?v=hQVTIJBZook

http://www.youtube.com/watch?v=HxaD_trXwRE

http://www.youtube.com/watch?v=j3mhkYbznBk

http://www.youtube.com/watch?v=KTJs-0EInW8

http://www.youtube.com/watch?v=kXEgk1Hdze0

http://www.youtube.com/watch?v=M7kEpw1tn50

http://www.youtube.com/watch?v=mOZqRJzE8xg

http://www.youtube.com/watch?v=neI_Pj558CY

http://www.youtube.com/watch?v=nG66hIhUdEU

http://www.youtube.com/watch?v=NGFhc8R_uO4

http://www.youtube.com/watch?v=Nii1n8PYLrc

http://www.youtube.com/watch?v=NP9AIUT9nos

http://www.youtube.com/watch?v=OB-bdWKwXsU&playnext=...

http://www.youtube.com/watch?v=oCZMoY3q2uM

http://www.youtube.com/watch?v=oKg1hTOQXoY

http://www.youtube.com/watch?v=Own-89vxYF8

http://www.youtube.com/watch?v=PUv66718DII

http://www.youtube.com/watch?v=qlzM3zcd-lk

http://www.youtube.com/watch?v=tx082gDwGcM

http://www.youtube.com/watch?v=v7nfN4bOOQI

http://www.youtube.com/watch?v=Vt8jyPqsmxE

http://www.youtube.com/watch?v=vUf75_MlOnw

http://www.youtube.com/watch?v=yJDv-zdhzMY

http://www.youtube.com/watch?v=yjPBkvYh-ss

http://www.youtube.com/watch?v=YX3iRjKj7C0

http://www.youtube.com/watch?v=ZAf9HK16F-A

http://www.youtube.com/watch?v=ZDR433b0HJY

http://youtu.be/lQAV3bPOYHo

http://yuiblog.com/crockford/

⬐ ricardobeat

And here are them with titles + thumbnails:

http://bl.ocks.org/ricardobeat/raw/5343140/

⬐ waqas-

how awesome are you? thanks

⬐ Expez

Thank you so much for this!

⬐ X4

This is cool :) Btw. the first link was somehow (re)moved. The blip.tv link is now: http://www.youtube.com/watch?v=0JXhJyTo5V8

Real Software Engineering, by Glenn Vandenburg. Not a perfect talk (especially the conclusions IMO), but a very good exploration of how some of the common beliefs in the field of software "engineering" came to be, and how something resembling actual engineering practice might be beneficial and practical.

Link: http://www.youtube.com/watch?v=NP9AIUT9nos

Abstract: "Software engineering as it's taught in universities simply doesn't work. It doesn't produce software systems of high quality, and it doesn't produce them for low cost. Sometimes, even when practiced rigorously, it doesn't produce systems at all.

That's odd, because in every other field, the term "engineering" is reserved for methods that work.

What then, does real software engineering look like? How can we consistently deliver high-quality systems to our customers and employers in a timely fashion and for a reasonable cost? In this session, we'll discuss where software engineering went wrong, and build the case that disciplined Agile methods, far from being "anti-engineering" (as they are often described), actually represent the best of engineering principles applied to the task of software development."