I spent a couple years as an engineer on large petrochemical construction projects.  A lot of the mental work in a large construction project is determining the order of construction for the various subsystems and physical structures of the plant.  In construction that order is more or less governed by physical realities and it’s a well understood problem domain.  In the world of software development determining the order of coding construction is more nebulous because we’re not bound by physical laws.  All the same though, the success of our projects is influenced by the order in which we code the pieces of the project. 

Framework Fool’s Gold

I’ll freely admit it; I’m a recovering “framework-aholic” that’s had some bouts with “pattern-itis” as well.  That’s what makes me qualified to say that one of the worst project anti-patterns in software development is building an elaborate framework first.  It’s a seductive path.  On one hand there’s the whispered belief that if we can just get the core framework right, building new services/screens/controllers/applications will be easy, right?  On the other hand, it’s just plain fun to exert your creativity to the maximum to build the ultimate metadata-driven, pluggable framework so new features can be added without any new code. 

There are a couple little problems with building frameworks upfront.  The main problem is that it very rarely works.  You’re essentially guessing what the framework needs to do, and even the best architects can easily guess wrong.  Even if you’re the best architect in the world, the folks feeding you the requirements are always changing their minds or coming to a different understanding of their needs.  Building an all-encompassing framework outside the scope of a specific business project almost always results in wasted effort.  The worst case scenario is that you’re framework won’t deliver any value and results in a lot of rework.  The best case scenario is that your framework does work, but you don’t reap much early feedback or have any opportunities to deploy the system early.

Too much abstraction with an overarching framework early on can make an application viscous.  Uncle Bob describes design viscosity as a malady that makes a design harder to preserve as changes are made.  If it’s easier to make a change by going around a framework or drilling exception cases into a framework with “if/then” statements, you’ve created a viscous framework that’s might be causing you more harm than good.  If you start to hear the phrases “can’t we just go around the framework?” or “this would be easier without the framework” or even worse “we need to get [Some Guy] to change the framework first” you’ll know that you’re going down a painful path with your framework.

The worst is when the framework turns into the mutant plant from the Little Shop of Horrors as it takes on a perverted life of its own.  “Feed me Architect, Feed me!”

To give you an anecdote that’ll get me in trouble when they read this, a part of my team this past spring was embarking on an overly elaborate, future-proofed architecture for our product rewrite.  They were having fits debugging and testing the application in large part due to some unnecessary technical infrastructure (and severe performance problems down the road).  About halfway through the project they decided to simplify their architecture.  The result was what they call the “Great Refactoring of Aught Five.”  Funny name aside, there’s no such thing as a big refactoring, only rewrites.  They would have clearly been better off starting simple without the abstractions and infrastructure (plus there’s still some dead code detritus lurking in the codebase we’ve got to kill off some day). 

I proscribe (now) to the view that you should harvest a framework from an application only after it has been proven to be useful or necessary.  When you do build a framework, don’t try to create something grander than what you need right now.  I’d also say that it’s much, much better to “dogfood” a new framework as part of a project rather than trying to write a “Hammer without a Nail” framework that doesn’t really fit the needs of any project. 

Slice the Work Vertically, Not Horizontally

The metaphor of software development as construction is common, but horribly wrong.  Approaching software development as constructing a structure on top of a foundation, then adding the finishing trim can be a slow, painful path to project failure.

Time and time again I’ve learned or observed that projects go much smoother when you build vertically instead of horizontally.  What I mean by this is that you build a new system by creating a feature at a time with the entire stack of UI, business logic, and database work to make the feature fully functional.  As you build new features you religiously eliminate duplication to avoid creating a Stovepipe anti-pattern and harvest reusable code.

One of the more disappointing project failures I’ve observed was the very first .Net pilot project in a VB6 shop.  I wasn’t involved with the project, but I had a stake in the project as it was going to prove the viability of .Net for application development and also be a pilot for doing iterative development.*  Unfortunately they blundered by crafting an iteration plan that called for them to write the entire data access layer first, the business layer second, then finally put together the user interface.  It failed miserably.  When the project was cancelled midway they didn’t have the slightest bit of deployable code because they had focused solely on the backend database.  One of the other things they found out was that the data access they coded wasn’t exactly what the business and UI layers needed.  In other words, they had wasted effort by speculatively creating code that turned out to be unnecessary. 

One of the best, positive lessons I’ve learned working on Agile projects is the importance of getting feedback early.  You can’t get much usable feedback from the end user when all you have to show is database diagrams or UML class diagrams (a word of advice, don’t ever show a customer a UML diagram unless it’s some kind of experiment in hypnosis).  Besides, you know damn well that the thing most likely to change is the user interface.  You’ll also understand any piece of the system better when you’ve built the entire chain of calls.  You simply cannot know any piece of code is done until its clients are complete.

I did a project about 3 years ago where I created an elaborate, overly generalized workflow framework first before creating any working user interface.  The framework worked fine and the system still humming along, but the lack of a UI became a problem.  The other members of the team didn’t truly understand the point of the framework because they didn’t have a set of screens to see the concrete usage of the framework.  The users and the project manager had no idea what our progress was because they couldn’t see anything working.  We didn’t get any usability feedback from our user base until dangerously late in the project.  Yeah, you can claim earned hours for the code you create, but what value does that really represent?  The Agile methods have gotten it exactly right by tracking progress by user stories completed, and defining that “done, done, done” for each user story requires customer approval.  And besides, the customer probably only counts a working user interface as progress anyway.

One of the desirable qualities of an iterative process is being able to deploy a partially complete system if the project has to come to an unexpected stop due to changing priorities or funding problems.  The only way to accomplish that goal is to build an application feature by feature rather than layer by layer.

Client First

Earlier this year there was a raging argument on the web on the merits of writing web services “Code First” (generate WSDL from code) versus “Contract First” (generate code from WSDL)** that I mostly ignored.  I think there’s a time and place for both approaches, but the one constant is to write (or at least simulate) the client first whenever possible.  The requirements and the signature of any service will be largely determined by the needs of its clients.  “Client First” doesn’t just pertain to web services, it useful for any class that’s a service provider to other classes.  If you’re doing Test Driven Development (or not I suppose), you might use a mock object or a stub to be a stand in for a service while you concentrate on creating the service client.   Once they’re completed, the unit tests for the service client will very accurately define the requirements for the service.  For the obvious example, don’t write data access or persistence code long before the business logic that needs to be persisted.  I know some SOA enthusiasts who have a “build it and they will come” mentality to creating web services left and right, but what’s the use of building something that nobody wants?

Configuration Last

When it is time to build some sort of configurable framework, my strongly worded advice is to concentrate on the classes that do work first, and worry about the configuration strategy second.  I read that years ago in Craig Larman’s excellent book on design patterns and I clearly see the wisdom in that advice.  StructureMap is a working dependency injection tool, but it started its life as the world’s greatest ORM tool.  I got bogged down obsessing with the configuration infrastructure and never wrote a single line of actual O/R mapping code.  Only later on did I take the abandoned configuration code and turn it into a dependency injection tool to use on a project.  I still wonder what would have happened if I’d been able to release a solid O/R mapping tool for .Net 3 years ago.

Besides, I think StructureMap and the other open source IoC/DI tools make it ridiculously easy to create configurable plugin strategies compared to what it took 3 or 4 years ago.  Just push all configuration information and dependencies in through constructors or setter properties and let the IoC/DI tool put things together for you later.

* This is a terrible example of working iteratively.  The idea of working iteratively and incrementally is to build fully working pieces of the application, one at a time.  What they did was just BDUF + Waterfall with some trappings of iterative development.

** I had a colleague in my old architecture team who thought all of us should be able to write WSDL by hand as a qualification for being on the team.  I blew him off while the others nodded fervently.  There’s a long list of holes in my resume I wish I had more familiarity with, but writing WSDL by hand isn’t on that list at all.