Eric Wise

Sponsors

The Lounge

Blogs I Read

Fun & Games

Advertisement

Images in this post missing? We recently lost them in a site migration. We're working to restore these as you read this. Should you need an image in an emergency, please contact us at imagehelp@codebetter.com
Rejecting Software Engineering

When I actually get some free time and surf around the blogosphere I often see people referring to software as "engineering".  I've always had a problem with this term because it implies things about software development that simply are not.  Let's take a look at Dictionary.com for a moment.  re: engineering

the art or science of making practical application of the knowledge of pure sciences, as physics or chemistry, as in the construction of engines, bridges, buildings, mines, ships, and chemical plants.

So here's where the whole software engineering thing falls down for me.  Building software is not like building a bridge.  It's just not.  In physical, "real world" engineering you have the laws of physics, near perfect information on durability, composition, balance, etc.  A programmer, as Fred Brooks puts it (The Mythical Man Month) is like a "poet who works only slightly removed from pure thought-stuff".  There is a plethora of languages and methods for achieving the same results in software development and none of them are exactly the same.  For something to be labeled as an engineering science in my opinion, it needs to have known values and be infinitely repeatable.  Software development meets neither of these.

Part of the issue is that programming systems are defined by the user's needs, which are infinitely varied across companies.  Once you step out of standard software packages and into the custom code world, every system is completely unique and is shaped by the users and by the individual humans working on the project.  This is one of the joys of working in software in my experience, since even problems that are similar to ones you've solved before can add new twists and features, keeping the work fresh and challenging.  At the same time though, this is what prevents me from referring to software development as engineering, because there is no silver bullet, no common standard or schema for doing things.  We are artists that are really good at math and logic.

Look at the estimation problems we have in software.  How many of you out there can truly, honestly, to the day pick out when you will be finished with a software project?  The bigger the project, the rarer this ability is.  Sure you can fudge your numbers and add padding so you can be reasonably sure you won't be late, but calling your complete date agressively is near impossible in large projects with many team members simply because you are working with thought-stuff and unlike building a bridge, you can't pull a template from the last 40 bridges you've built and nail down a relative time line.  Additionally, the human factor kicks in on large projects.  It has been well noted by researchers like Sackman, Grant, and Erikson that "very good professional programmers are 10 times as productive as poor ones".  This is also unlike building a building where the variance in productivity between say, people putting up drywall is far less than 10 times.  If software development was more like engineering you'd be able to walk in and say "oh, you want an invoice billing system, that'll take 500 hours" and you'd hit that target nearly every time.  This is far from reality.

Additionally, unlike in engineering where you want a free flow of information, I agree with David Parnas, who asserts that the various modules of code should be encapsulated with well defined interfaces, and that your programmers should be shielded from the reasoning, code, etc behind modules that they do not directly own.  I actually learned this the hard way at some past companies where developers across teams and modules got access to all project documents.  This lead to a lot of infighting where developers would cross team boundaries and try to impose their biases and design preferences upon other teams which was horrible for productivity.  The technique exposed by Parnas also lends itself better to separating your application into conceptual chunks, which makes it much easier to manage and parallel develop.

Regardless, my end point is that I don't think software development will ever be able to be defined as engineering in the traditional sense, and being that a company's use of software technology can lead to huge competitive advantages I really don't think we ever want it to be.


Posted 06-26-2007 10:39 AM by Eric Wise

[Advertisement]

Comments

Peter Ritchie wrote re: Rejecting Software Engineering
on 06-26-2007 11:24 AM

I disagree with your choice of definitions for "engineering".  Most definitions I've read include "applying scientific knowledge" or "the application of scientific principles".

Scientific principles or knowledge being that of a methodological process.

Its the methodological process that is considered the Engineering.

I'll agree, everyone who produces software (notice I didn't even qualify it with "write"...) is not performing Software Engineering.

Arguably, if software engineers could draw on a better/defined body-of-knowledge or discipline like Civil or Electrical Engineers, software would be in a better state.  But, I don't think that precludes the field of "Software Engineering".

Derik Whittaker wrote re: Rejecting Software Engineering
on 06-26-2007 11:42 AM

I could not agree more.  Software development is an art form, not a science.

The really sad thing is that business tend to not understand this, but developers do.  This differing mind set leads to many of project failures we know and read about.

Eric Wise wrote re: Rejecting Software Engineering
on 06-26-2007 11:47 AM

Peter,

While I see where you're coming from I'd have to disagree that programming is applying scientific knowledge.  In business programming, where I live, programming models _the perception_ of a process or _the perception_ of reality.  This is not science.  Nor is it often fact or truth. :)

DevKm wrote re: Rejecting Software Engineering
on 06-26-2007 12:10 PM

  Writing software is definitely  engineering - Only an immature one.  It took thousands of years for the construction industry to get to where it is today.  Some day, we will be able to estimate and schedule  Software Projects  just like Construction projects. Till then - Happy Churning.

Hadi Hariri wrote re: Rejecting Software Engineering
on 06-26-2007 12:34 PM

You mention that every software system is different. How does that disqualify it as being engineering? There is only a certain number of ways people want to cross a river. There's a certain number of techniques that have proven to work when building bridges, people follow these architectural "patterns". It doesn't mean that tomorrow someone can't come out with a new one. In software, the needs are just larger. The difference with software is that there are broader needs.

Regarding estimation, again, it's directly related to the needs of the customer. You can't ever give an exact figure or time line, but you can give pretty good estimates. This again applies to all engineering. Mishaps can happen and requirements can change when building a house too.

I agree that it can't be classified as manufacturing, but engineering, I think there are many aspects that resemble it. Is it art? Yes of course it is art, but so is building a bridge, so is building a house, they all involve a process of design.

Xor wrote re: Rejecting Software Engineering
on 06-26-2007 1:08 PM

I think both arguments hold and I know most of you are going to call this academia but I think the definition of engineering does hold for software. It's just the level that you are at is what lets you call yourself an engineer. If you’re just putting buttons and labels on a form or web page and have never even thougth of applying Demorgan's law on If-Statements that some developer hands to you and you have to know where that person is going wrong then you are not even close and may not even have the privilege of being called a programmer. Now if you are proving an algorithm using induction or any other type of proof technique “to” guarantee a curtain behavior for n case then you are now diving into a science. Remember that true programmers are known as Computer Scientists, we go through a lot of computing and logic that most “Physical” engineers go though to get there degree and should be shown some type of respect. You say that building a bridge is nothing like building software and that is true, but in both cases we have to make sure that it’s not going to fall under extreme circumstances as in too much load or abuse right? Well I think this is a counter example that should be thought about. On a brighter side, great article!!

Max wrote re: Rejecting Software Engineering
on 06-26-2007 2:37 PM

I find it amusing that our industry uses terminology such as architect, framework, and engineer; yet we deny ourselves the title of engineering.

Some people like to believe that their code creations are art, just like an architect would like to think the building he designed is art.  To some respects this is true, but in the end the architect still had to work in the boundaries of physics. We are not confined to boundaries of physics, but eventually time to market, testability, and stability of applications will be the factors that cause us to evolve into higher disciplined levels of engineering practices.

Peter Ritchie wrote re: Rejecting Software Engineering
on 06-26-2007 3:20 PM

Is this coming from IEEE Software magazine's Feb 2007 article by Raghavendra Rao Loka (basically saying the same thing; that software development is not software engineering)?

www.hacknot.info/.../showEntry is an interesting blog entry on that article and Steve McConnell's response...

Peter Ritchie wrote re: Rejecting Software Engineering
on 06-26-2007 3:22 PM

*some* software development may not be done by some people based on "scientific knowledge"...  But, by and large, software development is performed methodologically and using Scientific Method.

Peter Ritchie wrote re: Rejecting Software Engineering
on 06-26-2007 3:48 PM

Eric, Loka's article isn't publicly available (only to IEEE subscribers); but the abstract can be viewed here: ieeexplore.ieee.org/.../freeabs_all.jsp (subscribers would have access to the full text).

Steve McConnell's response is publicly available here: csdl.computer.org/.../r4006.pdf

It's similar to saying because the thousands of sub-contractors that build a bridge don't need to know anything about the laws of physics, durability, composition, balance, etc. to do their job (i.e. they're just pouring concrete, or welding iron, or painting), building bridges isn't engineering...

Max wrote re: Rejecting Software Engineering
on 06-26-2007 3:56 PM

Steve McConnell also explains in his blog:

blogs.construx.com/.../quot-engineering-quot-in-software.aspx

jdn wrote re: Rejecting Software Engineering
on 06-26-2007 9:33 PM

"*some* software development may not be done by some people based on "scientific knowledge"...  But, by and large, software development is performed methodologically and using Scientific Method."

Uh, 'Scientific Method' involves creating hypothesies and then coming up with tests to falsify them.

What the heck does this have to do with programming?

Peter Ritchie wrote re: Rejecting Software Engineering
on 06-27-2007 8:51 AM

jdn: you know exactly how everything needs to be written?  No, you decide how something needs to be written by methodologically researching what will and won't work before you start writing, performing tests to prove what will and won't work.

For a lot of people that comes as second nature; but that doesn't mean it's not scientific.

Andriy Solovey wrote re: Rejecting Software Engineering
on 06-27-2007 1:17 PM

I agree with author and think there are 2 main reasons why software is not completely engineering:

1. Traditional engineering deals with physical world, stable theories and proven scientific principles. Software development deals with unclear business and people needs, mental concepts  and empirical principles.

2. Software development relies a lot on the humans, how they can operate with these concepts and interpret these needs.

I have related post: softwarecreation.org/.../what-is-software-development

Jeremy D. Miller wrote re: Rejecting Software Engineering
on 06-27-2007 1:59 PM

Eric,

Fantastic post.  As an ex-engineer turned developer I'd like to quibble a little bit.  True, software development isn't always very scientific yet, but there are some parallels to engineering -- mostly because engineering itself is also partly craft rather than pure science.  As a practicing engineer you resort to empirical means, statistics, and models to help rationalize decisions when the underlying forces are simply too complex to accurately calculate.  That isn't all that different than the way software development is generally an exercise in adaptation and learning.

When I worked on 250 million dollar petrochemical projects the designs were very incremental and based on constant refinement from early models.  Yeah we used tons of number crunching and computer models, but intuition, experience, and constant learning fed into the final designs.

And by the way, software development is soooooo much more enjoyable than "real" engineering.  

Once again, great post Eric.

Gil Zilberfeld wrote re: Rejecting Software Engineering
on 06-28-2007 8:10 AM

Also from Steve McConnell, I think as a response to the IEEE article:

forums.construx.com/.../quot-engineering-quot-in-software.aspx

Andrew wrote re: Rejecting Software Engineering
on 06-28-2007 8:24 AM

I think your comparison to bridge building is somewhat lacking. Yes, you can look at the last 40 bridges you built and see how long they took, but to pretend that you can then estimate with 100% accuracy how long your current project will take is naive. For example, the new Wembley Stadium was finished both way over schedule and way over budget? There have been hundreds of stadiums built all over the world so surely by your reasoning Wembley should have been finished on time and on budget. It's easy to believe that software engineering is 'special' in some way which makes estimating more difficult but I'm not sure that that's just not an excuse for people not putting the appropriate effort and research into making their estimates.

Dr.Dichotomous wrote re: Rejecting Software Engineering
on 06-28-2007 8:47 AM

It strikes me as odd that people care whether software engineering is in fact engineering. If you wish to define engineering as being a physical/physics pursuit, then that your definition. Software design, development, maintenance, etc can go either way. On one hand you have z-prime style development that is more mathematical and strict, including efforts to consolidate concepts into "theory". On the other you have people who adore programming as an art form - similar to how some engineers (as you defined them) create physical structures that are more form than function.

The major difference is that people are willing to buy the "hobbyist bridge" because the state of the field is too immature to provide people with a "one size fits all" solution. If anything, I would encourage people to call proper software development methodologies (and their theoretical pursuit) less art and more engineering. Something needs to inspire people to go to school, learn how to properly do things, and to do them properly. Otherwise we will all be complaining about how shoddy the state of software is for a very, very long time.

Incidentally, it would be nice if you would have mentioned that Javascript is necessary to submit comments. It is a little annoying to type out an entire comment and learn I have to wait until I get to the office to submit the thing.

Guts wrote re: Rejecting Software Engineering
on 06-28-2007 9:04 AM

While you are picking at nits...

Whoever told you that engineering was "science" did you a big disservice.  Engineering is a craft by which you apply knowledge and technology and creativity to arrive at a practical end.  

Science is a methodology - a process that is applied like a tool by those who wish to prove or disprove a theory in a repeatable manner.

I've heard a lot of engineers blather on endlessly about how they disapprove of the term "software engineering".  I've yet to hear a reasonable argument as to why it cannot be applied - mostly it boils down to ignorance and a strangely protective clutch on the term "engineer".  I guess software is just the new kid on the block who isn't allowed into the club yet.

dlp wrote re: Rejecting Software Engineering
on 06-28-2007 9:05 AM

Curiously enough, David Parnas founded a Software Engineering program:

www.mcmaster.ca/.../software.htm

VeronicasLore.com » Rejecting Software Engineering wrote VeronicasLore.com » Rejecting Software Engineering
on 06-28-2007 9:14 AM

Pingback from  VeronicasLore.com »  Rejecting Software Engineering

William Woodall wrote re: Rejecting Software Engineering
on 06-28-2007 9:19 AM

I think most people can Identify that physicists, Architects, and laborers (all who on bridges) serve different roles, but fail to see the distinction between Computer Scientist and Software Engineers, when in fact they are synonymous.  Also Software is designed as much like a bridge is designed, through planning and application of knowledge.  The only difference is they play by the rules of physics where as we go by the rules of the development system.  And the stuff about software being art I agree with but you cannot tell an Architect that a house or bridge is any less art than your software, because it isn't.

KeithF PE wrote re: Rejecting Software Engineering
on 06-28-2007 9:44 AM

As a registered professional engineer currently working as an Oracle dba / developer, I see many similarities between engineering design and software development.

Embedded Dave wrote re: Rejecting Software Engineering
on 06-28-2007 10:57 AM

When you're writing software to build robots and control systems, it's multi-disciplinary engineering. You could say that COBOL-like programming or web design isn’t engineering and given the number of hacks out there, you’d be right. But 20 years as an embedded systems developer leads me to know that there is definitely “software engineering”.

Rob Low wrote re: Rejecting Software Engineering
on 06-28-2007 11:13 AM

I disagree with much of this.  For one, to say that we do not work under laws means you have never programmed.  Each language has a list of laws.  Just because a java software engineer uses a different list of rules than say a perl software engineer, does not mean that they do not exist.  Each language has characteristics just as every different type of metal does.  When you build a bridge with aluminum there are different things to consider than when you make it out of steel.  The same holds true for programming in different languages.  And to say that bridge building is not like building software is flat out wrong.  You start with a list of rules (programming language),  laws (logic, memory limits, latency, and user stupidity), tools (development software and code repositories), materials (computers, memory, networks), specifications and a some vision ...how is that any different than creating a bridge?

Also, to say:

"Part of the issue is that programming systems are defined by the user's needs, which are infinitely varied across companies."

and use that as a difference between creating an engine or bridge or a car and creating software makes no sense.  Aren't brides, engines, and cars defined by the user's needs?  Isn't it people's needs that causes engineers to create things?  Is every bridge the same?

I can't go into all of the other points I disagree with because I am "fake" engineering software right now but I assure you there are many.

Manuel Gonzalez wrote re: Rejecting Software Engineering
on 06-28-2007 11:17 AM

>> A programmer, as Fred Brooks puts it (The Mythical Man Month) IS LIKE A "POET (.....)".  There is a plethora of languages and methods for ACHIEVING THE SAME RESULTS in software development and none of them are exactly the same.

So, your poet is achieving the same results. Maybe he is also an engineer, then.

Steve McConnell wrote re: Rejecting Software Engineering
on 06-28-2007 12:20 PM

Eric seems like a bright guy and a good writer, but this post is unfortunately a good example of the kind of misinformation that people are spreading about software engineering.

While Eric's argument seems appealing and persuasive, his underlying assumptions (which he seems to present as facts) about real engineering are *way* off base. For example, his assertion early in his post that real engineering has "near perfect information on durability, composition, balance, etc." is idealized and not correct. When John Roebling designed the Brooklyn Bridge, for example, the properties of steel cables weren't well understood, and so he used a safety factor of FOUR in designing the cable supports for the bridge. Obviously that is not even close to "near perfect information."

Eric also says, "Look at the estimation problems we have in software" as if that somehow makes software different from other kinds of engineering. Can you say "Big Dig?" The largest "real engineering" project in recent memory, the Big Dig was originally estimated at $13 billion. The final cost was about $80 billion. In Seattle, the construction cost of the Mariner's baseball stadium ended up being nearly double the original estimates. There have been many, many cases like this, which I discuss in my book, Software Estimation: Demystifying the Black Art. Estimation error in software is not any better or worse than it is in other branches of engineering -- the key challenges arise when you have to estimate large, unique projects. Estimating the 40th similar house you build in a housing development is easy, but so is estimating the 40th similar customized version of a software product you deploy.

It's true that there are 10:1 differences in productivity among different programmers, but that's not unique to software either. There was an interesting study conducted in the 1970s that found that the 80/20 rule applies in virtually every discipline: 20% of the programmers write 80% of the code, 20% of the police detectives make 80% of the arrests, 20% of the NFL quarterbacks make 80% of the touchdown passes, 20% of writers write 80% of the best selling novels, etc. Eric's observation is valid, but it doesn't have anything to do with whether software is engineering.

Invoking David Parnas in an argument against treating software as engineering. Parnas has been one of the earliest and most prominent proponents of treating software as engineering. Indeed, he founded Canada's first undergraduate program in software engineering at McMaster University.

Eric's conclusion that "I don't think software development will ever be able to be defined as engineering in the traditional sense" is unfortunately a good example of the kind of uninformed opinion that I was talking about in my recent blog post. "Software Engineering Ignorance."

Software engineering already has been defined as engineering, we have an international standard that contains that definition, the field's two largest professional bodies have jointly adopted a professional code of conduct for software engineers, we have accreditation standards for university programs in software engineering, and we have numerous programs that have already been accredited.

The argument is basically similar to the old scientific proof that bumblebees can't fly. The logic seems convincing, except that they're already doing it!

Vladimir Levin wrote re: Rejecting Software Engineering
on 06-28-2007 12:32 PM

I think software development differs from civil engineering and architecture along 3 dimensions:

1) One can use engineering formulas to validate a design ahead of time. Software is not subject to the contraints of physics, so your ability to validate a design ahead of time is very limited - it's possible for well-defined algorithms, but that's about it.

2) Models are not nearly as useful in software development as they are in engineering. Models and simulations of bridges and buildings get you pretty close to the reality of your desing without actually having to build it.  Things like ERD diagrams and various UML diagrams, e.g. Class, Sequence, are far less effective.

3) Buildings and bridges are designed *against* change. Once you've built something like that, there are no significant changes possible. On the other hand, any worthwhile piece of software is undergoing constant change, often fairly substantial over time.Would anyone consider adding a few stories to an existing skyscraper, ro adding a lane to a bridge? I doubt it.

10x Software Development wrote Rumors of Software Engineering's Death are Greatly Exaggerated (aka Software Engineering Ignorance, Part II)
on 06-28-2007 1:16 PM

A reader of my previous blog post on Software Engineering Ignorance pointed me to Eric Wise's blog

Embedded Dave wrote re: Rejecting Software Engineering
on 06-28-2007 2:40 PM

Hey Steve, your numbers on the Big Dig are way off. Initial cost estimate was less than 2 billion and th final cost is approaching 15 billion (but your point is correct).

Gil Zilberfeld wrote re: Rejecting Software Engineering
on 06-28-2007 3:34 PM

I always saw engineering as application of known practices to solving problems. It holds true in electronics (I am an electronic engineer) and also in software.

Does the practices apply to every problem? No. That's where innovation comes in, and it builds on the foundation of the known practices. Again true in both every field I know and software.

So, I think software engineering is just that. I do think there's a difference than other engineering types: The penalty for building a bridge wrong (i.e. if it breaks) is  a lot more than building software wrong (i.e. if it breaks). For most enterprise software, there's an acceptable risk of not hitting the target. But according to the CHAOS report, this too is going the right way.

Robby Slaughter wrote re: Rejecting Software Engineering
on 06-28-2007 4:20 PM

Wise's comments echo the biggest problem facing software today---we treat it as a creative endeavor, not an engineering discipline.

I have outlined and rejected each of his points in a response here: www.robbyslaughter.com/blog

10x Software Development wrote Rumors of Software Engineering's Death are Greatly Exaggerated (aka Software Engineering Ignorance, Part II)
on 06-28-2007 6:46 PM

A reader of my previous blog post on Software Engineering Ignorance pointed me to Eric Wise's blog

Eric Wise wrote re: Rejecting Software Engineering
on 06-28-2007 9:52 PM

Eh, I don't see how you can say that software development is more engineering than a creative process.  It's more like designing than engineering just about any way you slice it.

The problem I have is the perception of Software Engineering as a literal metaphor.  Software as a disciplin has many key differences between nearly all other type of engineering and it continues to make me question whether software development is truly engineering.

Do software developers design softwares the way engineers design products?  By and large, no.  I can't get past the fundamental concept that we are working with virtual concepts and engineers create things not only to work in the physical world, but to be mass produced.

I agree that a lot of the problem may be the immaturity of the field.  I mean, when engineers go about designing that bridge we're talking about, you have a choice of a few well established types and toolsets, and most people agree on what to do when.  There is no such consensus in software design (waterfall, agile, tdd, language wars, zealot battles).  Why is it that the community can't agree on a few ways of going about software design that are best?  Is it immaturity or is it due to software design being more artistic in nature?  (despite being rooted in "science" since it's logic)

Engineering also has to deliver finished and usable products to be successful whereas in software we can constantly upgrade things, you can use a beta, etc.  In the other disciplins you simply can't ship a product that isn't finished, but we do it all the time.  If we are engineers, we're certainly not held to the same standards.

It's a bit unfair to call me on the Parnas thing since I diverged from my point of engineering because I started thinking about free flow of information.  I'm aware the guy has been a huge proponent for software as engineering, but I was trying to point out that even in his recommendation it skews from traditional engineering in that he recommends having good interfaces and coding against a black box.  A person designing a car engine certainly wouldn't accept hooking up his widget to a black box, he'd want to know what was in that box and what it was going to do to his widget.

Followup: Rejecting Software Engineering - Eric Wise wrote Followup: Rejecting Software Engineering - Eric Wise
on 06-28-2007 11:00 PM

Pingback from  Followup: Rejecting Software Engineering - Eric Wise

Dan Neuman wrote re: Rejecting Software Engineering
on 06-28-2007 11:59 PM

While Engineering is often described as Applied Science, I find in practice it is also Management of Complexity. A solid-state physicist could, in theory, design a radio, but I'd hate to see the differential equations. An engineer knows to use simplifying models instead, and knows how to vary the design to make it manufacturable without compromising the requirement significantly.

I see this happening gradually in software development: the use of objects to model the real world; higher-level languages to get the developer's thoughts down more quickly; frameworks to provide consistent glue to hold parts together; rapid prototyping systems to more readily capture client requirements. I believe that over time these tools will improve, and the productivity of good vs poor developers will equalize somewhat. When that happens (and when we get better at determining level-of-effort from requirements), cost and time estimations will improve.

In some sense, it is the developers who insist that what they do is an art that may be holding the field back. I suspect an "artist" would resist new tools that would mechanize what he does. It would make his work less fun. And more like real engineering.

I've sat in on presentations given by a CMM level 5 group designing software to fly on satellites and am pretty impressed by how well they  get things done. Mind you, their client knows enough to limit requirement changes. As the saying goes, developing to requirements is much like walking on water: they're both much easier when frozen.

Scott wrote re: Rejecting Software Engineering
on 06-29-2007 12:03 AM

" In the other disciplins you simply can't ship a product that isn't finished, but we do it all the time"

You should do a search for "Galloping Tacoma Narrows bridge". ;)

Andy wrote re: Rejecting Software Engineering
on 06-29-2007 5:29 AM

Science is not Engineering. Science is about the aquisition of knowledge, Engineering is about making things (often, but not always) using that knowledge.

If you reckon that we have near perfect knowledge of physical engineering, explain to me how concrete sets. Last I heard, mankind knew a lot