A Train of Thought – August 24th, 2008 Edition

Thank God I don’t have my old 3 hours a day of commuting into Manhattan, but I needed to spit out some little blog postlets in more than 140 characters at a time, so I present to you the latest Train of Thought.  Opinionated blathering ahead, comments are always open at the bottom.

 

Don’t Overreach in Your Designs

One of my older posts on The Last Responsible Moment and another post on evolutionary design were recently linked on some of the DZone/DotNetKicks sites and got some traffic.  I got some comments and emails to the effect of “we did evolutionary design and it bit us in the ass with all that refactoring and rewriting.”  Maybe, but let’s talk about how to do evolutionary design in a way that minimizes outright rework.  From my experience, the worst rework results from choosing elaborate abstractions upfront that turn out to be harmful.  The analogy that I like is trying to walk on slippery ground.  Anybody who’s walked across an icy patch or a muddy field this knows that the way to do it is to keep your feet as close to your center of gravity as possible by taking short steps.  If you take a big step you’re much more likely to slip and fall.  Design is the same way.  Bad things happen when you allow your design thinking and abstractions get ahead of your development and requirements.

We’re doing evolutionary design, and yes we have had to rewrite some functionality when we’ve found shortcomings in the design or simply found a better way to do it.  I would attribute the worst example of avoidable rework on our project to overreaching with some infrastructure outside of user stories.  We’re using the new ASP.Net MVC framework, and we didn’t like the way that it handles (or really doesn’t handle) the “M” part of the MVC triad.  We had one of those conversations that starts with “wouldn’t it be cool if…” and ended with one or both of us spending days of architectural spiking on an approach for screen synchronization – before we created our first web page.  We created the idea of a “ViewModel” that would represent screen state and help us to move data between the web page form elements and our Domain Model objects.  We wrote a very elaborate code generation scheme to automate a lot of the grunt coding.  As soon as we started to work on our first couple web pages we quickly realized that much of our ViewModel infrastructure was unnecessary or just plain wrong.  We effectively rewrote the ViewModel code generation in a simpler way and got on with the project.  Since then, we’ve extended the ViewModel code generation to add new behaviors on an as needed basis, but we haven’t had to rewrite any of it.

Just to head off the comments, I didn’t know about the BindingHelperExtensions in the MVC at the time (shame on me).  I don’t regret rolling our own infrastructure at all because I don’t think that BindingHelperExtensions is adequate, but I wish we’d played it a little smarter and put off the ViewModel code generation until we had a couple working pages to point out the real patterns.

What I’m trying to say here is to avoid speculative abstractions and fancy patterns outside of feedback from the real features and needs of the system.  It’s relatively painless to extend simple code for more elaborate usages than its original intentions, but it hurts to throw out or change elaborate code.  You can hedge your design bets by (almost) always starting simple.

 

Enabling Evolutionary Design

So, how do you do evolutionary design without incurring a lot of rework?  Here’s my recipe:

  • Worry a lot about cohesion and coupling as you work. 
  • Follow the Open/Closed Principle
  • Follow the Single Responsibility Principle
  • Use TDD or BDD for low level design.  First because it does more to ensure good cohesion and coupling on a class by class basis than any other technique, but also because the automated unit testing coverage left behind enables changes in the code to be cheaper in many cases.  Regression testing is a cost and risk associated with changing code is a considerable road block to making design improvements midstream.  If you reduce that cost and risk, evolutionary approaches are a lot more attractive.  That test coverage is one of the ways that TDD/BDD is more valuable a practice than merely applying some unit tests after the fact to strategic areas of the code.

From an old post:

One way to think about TDD is an analogy to Lego blocks. The Lego sets I had as a child were the very basic block shapes. Using a lot of little Lego pieces, you can build almost anything your imagination can create. If you buy a fancy Lego set that has a single large piece shaped like a pirate’s ship, all you can make is a variation of a pirate ship.

In that context I was talking about TDD, but I feel like the analogy holds very true for doing evolutionary design effectively.  Composing your system of small Lego pieces that can be rearranged is much better than using big monolithic pieces of code that are more likely to be modified later.

In the end, it really amounts to just design things well, all the time.  Unsurprisingly, I think that teams with strong software design skills are best equipped to do evolutionary design.

 

On Software Factories

Last week I was at an Open Spaces event in Colorado with a very diverse group of folks in a session that rambled around until “Software Factories” came up.  I stated, and not for the first time, that Software Factories are often Big Design Upfront dressed up in sexier new clothes.  I definitely think the software factory idea can work (with Ruby on Rails as exhibit A), but I think the activity of defining elaborate project and class templates upfront is risky or at least unoptimal.  A project team’s is going to have much less willingness to reconsider designs if design changes require changing the software factory templates.  To me, software factory techniques will succeed if and only if it’s easy to modify the factory automation guidance as the team works and learns more about their system.

My other point with software factories was that I think micro code generation (live templates, file templates, ReSharper tricks, scaffolding, etc.) where the developer is in complete control has a much better chance of succeeding than the elaborate factories that try to generate most of the application for you. 

 

Opinionated Software

Ruby on Rails introduced “Opinionated Software” into the common lexicon, but it’s been around for a while.  I think that my team is gaining some advantages from our design’s “opinions,” but what if you don’t like the opinions of your chosen framework?  Take CSLA.Net as an example.  I want absolutely nothing to do with CSLA.Net because I think its basic model is severely flawed, but I bet that it’s providing a lot of value for many teams.  That value is largely because CSLA.Net has firmly engrained “opinions” about how almost every common development task should be done.  I can’t use CSLA.Net, and a lot of the Microsoft tooling for that matter, because I don’t agree with the “opinions” baked into that tooling.  I’ll happily build my own infrastructure to support the way that *I* feel software should be created, or go around a piece of infrastructure I don’t agree with.  Heck, the MVC framework isn’t even released and we’ve already considerably diverged from its opinions.  Other developers will simply go with the flow of whatever tooling that their using and invest time into learning the idioms of that particular tool and not waste time questioning that tool. 

I think this comes down to a question of “go with the flow or change the course of the river.”  I’m a “change the course of the river” to the optimal path kind of guy, but I frequently wonder if it would be better to just give up and go with the flow.

 

TypeMock is only a Bullet of Ordinary Composition

I was out of pocket last week at a little open spaces event, so I missed most of the latest Twitter and blogging flareup of the TypeMock question.  I’ll repeat my opinion that there’s nothing inherently wrong with TypeMock itself, but I think that the rhetoric from TypeMock proponents is often harmful to the greater discussion of software design and practices.

TypeMock might be a better mocking framework than Rhino Mocks or Moq, but it does NOT change the fundamental rules of mock object usage.  Just because you can use TypeMock to mock a dependency doesn’t mean that it’s the right thing to do.  Let’s remember some of my rules of mock object usage:

  • Don’t ever try to mock chatty interfaces like ADO.Net or anything related to HttpContext because the effort to reward ratio is all wrong and you can never read those tests anyway. 
  • Be extremely cautious of mocking interfaces that you do not understand. 

The only thing that TypeMock changes is *how* the mock object is introduced into the code being tested.  If you really think that having separate interface definitions plus Dependency Injection is hard, then yeah, use TypeMock (an assertion that I would obviously dispute in this age of auto wiring, auto mocking containers, ReSharper, and convention driven configuration of IoC containers).  Just remember a couple things please:

  • Mocking in general isn’t going to be an effective technique with classes that aren’t cohesive or have a lot of semantic coupling with their dependencies.  In other words, interaction testing with any mock object is going to be painful with badly written code.  TypeMock simply doesn’t change that equation.  I’ve heard TypeMock put forward as a solution for unit testing legacy code.  In theory yes, but the reality that I’ve found is that interaction testing inside Legacy Code is an exercise in pain.  Most legacy code (and I’m using the Feathers definition of legacy code here) has very poor internal structure and poor separation of concerns.  Exactly the kind of code that you shouldn’t bother using interaction testing on.  I’d instead recommend surrounding Legacy Code with more coarse grained integration tests to preserve behavior first, then trying to modify the internal code to a better structure before writing fine grained unit tests.  Yes, it is possible to use TypeMock to “unit test” typical legacy code, but those tests would almost automatically be the type of overspecified unit tests that cause more harm than good.  The problem with legacy code is often the structure of the code more than the fact that it doesn’t have any unit tests.
  • Yes, you can unit test the class in question that news up its own dependencies and calls static methods, but you still have a very tight runtime coupling to those dependencies and the static method calls.  Regardless of your ability to unit test the class in question, that tight coupling can often be a problem.  Your ability to reuse those classes is compromised by the tight dependencies.  Your ability to practice evolutionary design is compromised because of the tight coupling.  Remember that Dependency Inversion and Inversion of Control have other benefits than just unit testing.

I think the TypeMock proponents are too focused on unit testing in a way.  I firmly believe that code that can’t be efficiently unit tested is almost automatically bad code (to me, testability == productivity).  However, code that can be unit tested isn’t necessarily good.

To recap, I don’t think there’s anything wrong with TypeMock per se, but I think that much of the TypeMock proponent’s rhetoric is irresponsible.  Just because TypeMock *can* do something, doesn’t mean that doing that something is a good idea. 

 

 

In Tribute to George Carlin

I couldn’t think of 7, and it’s a couple months late for a Carlin tribute, but here’s my list of the words or phrases that are henceforth banned from appearing in my blog or presence (starting right now).  Almost no conversation is going to be useful if it includes one of these words:

  • Mort – Apparently Microsoft is now referring to the developer formerly known as “Mort” as “Pragmatic Developers.”  Puh-leeze.  Everybody in the world thinks that they’re pragmatic, but yet we disagree on many significant directions in the best way to build software.  I was dead set against ALT.NET getting renamed “Pragmatic.Net” for the same reasons.  I gotta say though, “Pragmatic Developer” is much less a pejorative than “Mort” became and the typical “Joe Schmoe Developer who builds LOB apps at General Motors” line you hear from Microsoft employees.
  • Entity Framework – At least until there’s something new to say.  I’m liking that my attention lately has been on the advance of Fluent NHibernate instead of worrying about a tool that I’m very unlikely to use in the next 2-3 years.
  • Stored Procedures – I’ve seen nothing to change my opinion about sprocs for several years (good for edge cases and utility database scripts, bad everywhere else, i.e. 95%+ of the time I think sprocs are unnecessary)
  • TypeMock
     
  • “Vietnam of Software Development” – Most  overblown and misused analogy this side of Software as Construction.
  • “Software as Construction” – I worked on the engineering side of construction projects measured in the 100’s of million dollars and even billion dollar+ projects (and this was in the pre-W days when the USD was more than paper money), plus I worked for my father building houses as well.  I feel perfectly qualified to say that the “Software as Construction” analogy is an extremely poor fit.  Software as Manufacturing is better, but I bet that somebody will write a rant about that comparison in the next couple years.
  • Foo Considered Evil – It’s a cliche now
  • “Cargo Cult” – used as a magic talisman to win any argument, regardless of whether the use of the phrase is applicable or not.
  • “Your Emperor has no Clothes” – see above
  • “Jumped the Shark” – see above
  • “You should just use whatever is best for your project” – The intellectual equivalent of empty calories
  • “You’re just being dogmatic!” – Lamest way to try to win an argument.  Basically, this is code for “I’m pissed that you don’t agree with me so I’m just going to call you names and declare victory, so there!”
  • “You can just Refactor it later” – You can write simplistic code upfront and say you’ll refactor it later to eliminate duplication or handle more complicated cases as those cases arise, but you don’t write bad code on purpose.  You certainly don’t use Refactoring as an excuse to just not think about design.
  • “We’re refactoring” when the team really means “we’re rewriting that code altogether.”  There’s no such thing as a big refactoring.

 

 

Okay, I’m done.  Your turn:

About Jeremy Miller

Jeremy is the Chief Software Architect at Dovetail Software, the coolest ISV in Austin. Jeremy began his IT career writing "Shadow IT" applications to automate his engineering documentation, then wandered into software development because it looked like more fun. Jeremy is the author of the open source StructureMap tool for Dependency Injection with .Net, StoryTeller for supercharged acceptance testing in .Net, and one of the principal developers behind FubuMVC. Jeremy's thoughts on all things software can be found at The Shade Tree Developer at http://codebetter.com/jeremymiller.
This entry was posted in Uncategorized. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • paweł paul

    This is my first visit on this site and i found it very informative as you have provided here in this post. you need informations about I would like to be a part of this site for collecting some more info. I have bookmark this page and after work I read interesting things to update myself and to refresh my mind. I Would expect new posts from you .Many, many thanks! latest currency exchange rates

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Rob Williams

    In this case “Evolutionary” simply means “Incremental” It does not mean “throw spaghetti at the wall and see what sticks.” I’m sorry that the term caused you so much (unnecessary and silly) heartburn.

  • Rob Williams

    By the way, I totally agree with the statements regarding having “grandma” write software, with the specific emphasis on stored procedures.

    The issue is not whether any particular technology (such as stored procedures) is good or bad, the issue is that we need competent people paying adequate attention to the consequences as they use whatever technologies to build automated solutions.

    The trouble with stored procedures is that they are very difficult/expensive to create and maintain, especially compared to the alternatives. Stored procedures operate in a proprietary and relatively inaccessible environment (inside the database engine). The development tools are limited, testing is harder, and the code is tied to a specific vendor’s product.

    Such characteristics and their consequences should be understood by the programmer that uses the technology, and “grandma” simply never will bother (if she did, she has graduated to “programmer”). It is such knowledge and concern that distinguishes a “real” programmer from a “wannabee”, and therefore justifies the trust and the pay extended to a “professional”.

    On the other hand, there is still a place for the programmer who is unconcerned about such knowledge and consequences–toy programs, technology demos, self projects, explorations, etc. are all fine and necessary. Just be honest enough to stop someone from “betting the farm” on your work if you are not up for the responsibility yet.

  • Rob Williams

    I find it funny and quite hypocritical when people talk about “evolutionary design”, then describe a process filled with intelligent designers and intelligent design principles such as above.

    None of what is described here fits the term “evolution”. On the other hand, the closest thing to “evolutionary” programming seems to be the use of genetic algorithms, though many people debate whether the programmer’s survival criteria force a conclusion that this is another “intelligent” design approach.

    I also wonder why programmers don’t seem to feel insulted when their hard design work is attributed to the accident of evolution. I would be offended in the extreme to have the results of my work effectively attributed by someone to “blind dumb luck” = evolution. Have you too little self-respect not to do this to yourself and your team?

    I would suggest that you add “evolution” to your list of very bad/poor/useless/forbidden words/phrases, starting with this post.

  • joe

    “and this was in the pre-W days when the USD was more than paper money”

    Guess we know why you moved back to Austin. Commie :)

  • John "Z-Bo" Zabroski

    @Jeremy
    @Bad things happen when you allow your design thinking and abstractions get ahead of your development and requirements.

    No. Bad things *will* happen when you start envisioning requirements that don’t yet exist. Design is not about getting ahead — it’s about seeing macro operators that speedup turnaround time.

    However, there is a more basic truth: bad things tend to happen when you don’t see the consequences of your actions. Agile development is moderately successful because it can help overcome such blindness, but it does not give you eyesight. You need to start using your eyes to go beyond merely agile.

  • http://feeds.feedburner.com/chowyoureallyare Josh S.

    I just wanted to say that this was one of the best posts I have read in a long time. Opinionated yes, but you touched some many valid and substantial points, I just had to comment. Way to go. I love you analysis of CSLA, TypeMocks and Entity Framework and the whole ALT.NET movement. Loved it.

  • http://blog.troyd.net Troy DeMonbreun

    Don’t forget the highly religious:

    “If you’re not practicing / implementing X, you’re just creating a BigBallOfMud”

    BTW, love the “icy patch” analogy.

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Florian,

    I completely agree with you. At some point I’m posting on exactly that point, and how we’ve added quite a bit to the MVC on our own to add in all those opinions.

  • http://www.planet-xaml.net Florian Krüsch

    ASP.NET MVC’s extensibility is a good thing on the one hand. On the other hand, we will see as many flavours as there’s projects out there.

    That’s the big different to Rails, which has an opinion on everything it seems… how you implement the Model, how the Model interacts with validation etc.
    With ASP.NET there’s no convention (over configuration) on persistence, DI etc.
    The stuff you posted on the subject (common interface for DI, Repository interface etc) are very helpful in that direction, though.

  • Joseph Healy

    From the earliest days of agile and XP I’ve always been amused by reactionary responses to any suggestion that ‘architecture’, ‘frameworks’, and ‘big design’ might sometimes be needed and also by the fact that over the past decade the whole of the Alt.Net framework / platform has emerged (not to mention all the current framework / platform rush with MS Oslo, Google App Engine, Android, Amazon EC2, etc) in spite of a lot of rhetoric. All of them represent ‘frameworks’ / ‘platforms’, ‘big design’, and large-scale infrastructure development. Hell, Roger Jenning’s blog, chronicles the constant churning (i.e. endless, large-scale, joint community / corporate refactoring) just within persistence which is only one small corner of any overall enterprise framework / infrastructure / platform.

    I’m sure there are many new to our trade who would love to know exactly how does one thread the needle with no ‘big design’ on one side and no ‘big refactoring” on the other when building enterprise apps. How do you avoid ending up with a ‘found’ architecture and ‘found’ performance? And where and why have all these frameworks / platforms arisen from if not out of pressing, real world need for them?

    Also, if we’re all moving into a cloud-based / SaaS / PaaS future then it strikes me that someone, somewhere damn well better be doing a lot of ‘big design’, ‘architecture’, and heavy duty infrastructure lifting. Whenever I’ve takem a hard and honest look at these issues I always come back to considering it more a ‘shifting of concerns’ as part of the unavoidable conclusion that, overall, complexity simply can’t be eliminated per se. That in the end, we simply seek to hide and shufflle it around in an attempt to figure out who ultimately is going to deal with it – but, by design, accident, or luck, deal with it someone must. And while history has clearly shown we have a lousy track record attempting to do upfront enterprise-level design down to the last detail, I’m personally glad to see the talk swing back towards a more reasonable middle ground which better reflects the reality of what’s actually going on.

  • http://creedcultcode.blogspot.com Dale Smith

    @Charlie – I think I understand where you’re coming from. Our employers expect us to create software that meets their needs. Their needs are genuinely painful to them, and they expect us to alleviate their pain right now. And who can blame them? After all, they’re paying the bill. We all understand that pressure because we all live it daily.

    I certainly acknowledge the need to produce software quickly. But I think many developers, particularly in the .Net developer community, still need to acknowledge that maintainability has to be a primary feature of the software we create. We think that customers don’t care about maintainability, but I promise you they do. A lack of developer focus on quality impacts a customer 1) by dramatically increasing the amount of time it takes to produce new features, 2) by dramatically increasing the number of people and attention devoted to addressing issues that arise in production, 3) by dramatically increasing the number of developers a customer believes they have to maintain on staff, 4) and a whole bunch of other “dramatic increases” I’m leaving off right now. All of those things directly and significantly contribute to a customer’s overhead. Of course meeting the immediate requirements of a project is important, but a flexible design capable of easily incorporating changes is vital to a project’s survival, and perhaps even that of the company that pays for it. Not considering the maintenance cost of the design of a system is nothing short of professional malpractice.

    Stored procedures provide an easy way to put a system together quickly. They also provide an easy way to saddle yourself with a maintenance nightmare. I should know – one of the systems I work on has many, many business rules buried in stored procedures, and I’m working furiously right now to ferret out all those rules to put them into an API that can be reliably tested with an automated suite of tests. That suite of tests does lots of good things for me: 1) it drives me toward good design principals which make the system easier to maintain, almost as a side effect 2) it (hopefully) proves that my software does what I think it should do, 3) it provides a living communication tool between me and other developers on how the system works, 4) if you take it far enough, it can provide a living document you can use directly with your customers.

    Your grandmother is welcome to write as much code as she wants, and I hope she makes millions doing it. But if she doesn’t write it with an eye towards testability and maintainability, I hope I never have to touch it.

  • http://www.dualbotic.com/dasBlog Charlie Barker

    Why shouldn’t a granny be able to write software?
    I disagree that the ability to write software will always be the preserve of the technically minded. Obviously not device drivers or low level code there will have to be layers that hide the underlying complexity. The challenge is to make controlling software more accessible to the users. A lot of the time programmers are just translating user requirements into code it would be more efficient if the user could to this on their own. The problem most users face when trying to learn a new system is that there is nothing that let’s them visualize what’s going on inside the machine. These people aren’t stupid they are just do not speak Java or SQL or .Net. I think we are going to have to come up with something more intuitive than an RDBMS for storing data.

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Larry,

    Should have said granny wrote google maps with punch cards…

  • Larry Foulkrod

    My granny wrote google maps in notepad.

  • Brian Mavity

    Absolutely no effort should be expended to allow your granny to write software. Writing software is always going to be a complex process requiring constant learning and improvement on the part of the developer. Drag and drop designers that allow people to create software without thinking are harmful to all but the simplest projects. The sooner people accept that the sooner this ship will start pointing in the proper direction.

  • http://www.dualbotic.com/dasBlog Charlie Barker

    I worry when I read post’s like this that kid’s coming out of college or folk who are new to writing software are going to be bamboozled by the density of the “geek speak”. I have been coding for ten years and I struggled to follow the above. It would be helpful for myself and others if a context was provided, along the lines of… “When writing enterprise software approach X will offer Y in terms of Z”. They’re is often a fair amount of paranoia in the inexperienced developer’s mind about “Am I doing it right?”. For example you take the stance that Stored Procedures are unecessary, but that does not mean they cannot be used to write decent sofware that works. I would hate to think that someone reading your article goes into work tomorrow and scraps everything they have been working on for the last three months because they used stored procedures in their app. The point i’m making is, us lesser mortals rely upon the Scott Guthrie’s of this world to create the tools that allow us to create useful software in a reasonable amount of time. A modern car is a complex machine but my granny can drive one, software and the tools to write software should be like this.

  • http://samgentile.com/blogs/samgentile/archive/2008/08/25/thoughts-on-evolutionary-design.aspx Sam Gentile
  • Jeremy

    lol, touchy subject i see, will do some googlin.

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Jeremy,

    Way too many times. I will not get sucked into another discussion on the merits or drawbacks of that unmentionable subject.

    Just search my blog and you’ll find plenty of content, or go read the famous Frans Bouma post on the subject that will not be named.

  • Jeremy

    I’m intrigued by your comments on the necessity (or lack thereof) of sprocs. Have you blogged about this in the past?

    “Stored Procedures – I’ve seen nothing to change my opinion about sprocs for several years (good for edge cases and utility database scripts, bad everywhere else, i.e. 95%+ of the time I think sprocs are unnecessary) “

  • http://weblogs.asp.net/scottgu ScottGu

    Preview 5 of ASP.NET MVC is cooking along nicely and should be out shortly. One of the biggest areas of focus with this update is on form post scenarios – including action routing, validation, input form state management (when there is an error and you want to redisplay the form for the user to correct), and support for cleanly mapping form values to domain objects.

    I think you’ll find some useful out of the box features, as well as new extensibility points, that will help with your ViewModel approach above.

    Thanks,

    Scott

  • Greg

    “We’re refactoring” when the team really means “we’re rewriting that code altogether.”

    I have always called this refuctoring

  • http://creedcultcode.blogspot.com Dale Smith

    Lots of great quotes in this post, but this one’s my favorite: “There’s no such thing as a big refactoring.” We have abused the definition of the verb “to refactor” a little bit over the last year or so, and as a result my boss kind of gets the willies whenever we mention that some piece of code needs to be refactored. This is a very succinct way to say that we intend to change the undrelying implementation of an interface by taking multiple small testable steps.

    I agree with you that a tool like TypeMock is not necessarily a bad thing, but just another tool in the toolbox. However, as we’re trying to get our team up to speed on testing and refactoring, it sure is nice to use a mocking tool like Rhino.Mocks that forces us down the path of creating Feathers-style testability seams. We’re all degreed professionals in my shop, but it’s reassuring to have some guardrails up as we learn.