Patterns in Practice: A Retrospective

For most of the past two years I’ve written the “Patterns in Practice” column in MSDN magazine.  The January issue contained my last article and I thought it’d be good to write a little retrospective on the series.  It’s probably been a good opportunity for me and it seemed to be mostly well received (it’s not as much fun as blogging because you just don’t get as much immediate feedback, good or bad).  CodeBetter has always been frequented by a self-selected group (you can’t call it an echo chamber because of the way we argue) of developers who were already learning about Agile practices and the old software design concepts passed down to us from the “Gang of Four,” the Smalltalk world, and Uncle Bob’s SOLID writings from the 90’s.  The MSDN column has been nice in that it allowed me to write about some of these design concepts for a more mainstream audience who might not have as much exposure to these ideas.  Whether that worked or not, or made the slightest bit of impact, who knows?  It did pay for the down payment on my new car last year (and a root canal), so it’s all good to me;)

Some folks are going to argue that “software quality” doesn’t matter* because it’s really all about whether or not you’re building the right system with the right business plan.  Let’s assume right now that you’ve got the right business plan, good requirements, and even decent collaboration from your business partners (and manna streaming down from the ceilings whenever you get hungry at work because that’s just as likely as the preceding list).  You’ve still got to execute the project – and you probably aren’t going to build it all in one quick pass.  You’re going to build it incrementally.  You’re going to make mistakes, fix mistakes, and iterate (even if you’re working a theoretical waterfall).  Bad designs and bad code will slow your team down and delay your all important time to market.  Let’s be realistic here, you never have the perfect requirements.  Your business partners with the vision will need to iterate and refine their vision, and the entire product team is better off if the development team is technically able to efficiently deliver features that weren’t even imagined at project inception.  You can succeed with bad code, but all things being equal, I think you maximize your business’s chances of succeeding by taking software quality very seriously.

First off, let’s make the assumption that your software design really does matter.  It matters not because of ALT.NET or Software Craftmanship or Design Pattern purity, but because “good” code structures help development teams be productive now and over time while “bad” code structures bind a team up with “code viscosity.”  Let me be very clear here, I’m defining software quality as the structural qualities of code structure that enable a team to be productive within that codebase for an extended amount of time

I’ve been studying software design seriously for over a decade and building software for longer than that.  I’ve seen the exact same team founder in one codebase and generally succeed in another codebase.  I’ve worked in some codebases that felt like I was as light as a feather and functionality flowed into the system as fast as I could type.  I’ve worked in other systems where I wanted to tear my hair out because of how much work it took to do even simple seeming things.  Why were some codebases good and others pure frustration?  Offhand, I’ll give you five qualities or themes that separated the good codebases from the bad (not an exhaustive list of course):

  1. Orthogonality.  It’s a fancy word that just means being able to do or change or exercise one thing at a time.  It also means being able to understand or learn about one thing at a time in a system.
  2. Reversibility.  Technical or requirement decisions don’t have to be locked in stone.
  3. Feedback.  I think the best way to be successful building software is to assume that everything you do is wrong.  We need rapid feedback cycles to find and correct our inevitable mistakes.
  4. Iteration.  Your first idea is never your best idea.  Your business partners will need to refine their vision.  You need to be able to respond to both negative and positive feedback.
  5. Minimal Duplication.  Less duplication means easier change, less chance of making silly errors, and quicker iteration to get to the best product

These are the main themes in Patterns in Practice that I think are important.  Many of the design concepts like Design Patterns, the SOLID Principles, the GRASP Patterns, Code Smells, or the tools in Responsibility Driven Design are nothing but heuristics to help guide you to designs that exhibit the qualities I enumerated above or detect the absence of these qualities.

 

If you ask me what the essence of a good software

  1. The Open Closed Principle.  The “O” in SOLID.  Think about this question for a minute, what’s generally more productive for your team, writing all new code or breaking into the middle of existing code to make changes?  Which is more risky in terms of bugs and introducing cute new regression bugs?  Do you like having to trace through existing code to find the place to break in to change it, or starting from a fresh slate?  If you answered my loaded questions correctly, you sir or madam, are interested in learning about the Open Closed Principle as a design goal.  If you think the OCP just means having interfaces and plugin points for everything, then you need to read this article because that’s not the point.
  2. Object Role Stereotypes.  I’ve said many times that I think Responsibility Driven Design is one of the most valuable, yet overlooked design techniques you can possibly add to your design toolbox.  In the article on the Open Closed Principle I also mentioned the all important “Single Responsibility Principle” that more or less states that a class should have one and only one reason to change.  Great, but what’s a responsibility?  Object Role Stereotypes are a fantastic mental tool to help you understand and delineate the responsibilities within a design problem.  The research that I did in writing this article led to some significant architectural changes to StructureMap that opened up a lot of opportunities for extending and improving StructureMap.
  3. Cohesion And Coupling.  An explanation of what Orthogonality is and the practical impact of it on your code.  If you’re starting from ground zero in software design, I think the first two concepts you need to understand are cohesion and coupling.  Arguably, almost every design concept and tool in the whole Patterns in Practice series is about increasing cohesion and minimizing harmful coupling in your code.  It’s very important that you understand the impact of coupling in your code.  Know when it’s harmful coupling you should eliminate or relatively harmless coupling that would do more harm than good to abstract away.  A lot of harm is done in software development today due to zealous attempts to reduce coupling.
  4. Design For Testability.  It’s all about rapid feedback cycles and the ability to iterate.  How quickly can you go from doing something in your code to knowing that something was done correctly so you can turn your back on it?  It’s arguably just another way to think through the orthogonality of your architecture.
  5. Convention Over Configuration.  A design concept made popular in Ruby on Rails but increasingly important in .Net circles.  It’s a way to shrink your codebase down and to eliminate unnecessary duplication of “facts” in your system.  Makes iteration cheaper.
  6. Persistence Patterns.  I wasn’t particularly inspired by this topic, but hey, it’s valuable stuff to know if you’re going to use ORM’s.  NHibernate or Entity Framework users really need to understand these concepts to avoid some ugly pitfalls. At a minimum, make sure you understand what lazy loading and virtual proxies are all about.  Besides, it’s nice to understand what’s going on under the covers.  Once in a while you may want to use these patterns in your own code.
  7. The Unit Of Work Pattern And Persistence Ignorance.  I continued the discussion on Persistence Patterns and tried to explain what and why “Persistence Ignorance” is important and the real impact of persistence framework on your development efforts.  Judging from the recent blog posts from the Entity Framework on their new EF4 POCO implementation, they probably didn’t do more than skim this issue.  Okay, maybe that’s not fair, but if you read this article you might understand why I don’t think Entity Framework has really arrived as a viable option for me despite the dramatic improvements from EF1 to EF4.
  8. Incremental Delivery Through Continuous Design.  I can’t believe Howard let me get away with this one because I was specifically not to proselytize Agile development.  I tried to discuss the business value of being able to deliver incrementally and flexibly, then discussed the specific design issues supporting incremental delivery.
  9. Functional Programming for Everyday .NET Developers.  I learned a lot from writing this one.  I think that adding some functional programming techniques to my day to day coding has been very beneficial to my efforts.  In a lot of ways, FP usage can help reduce duplication by giving you a more fine grained mechanism for composition than OOP does and passing first class functions around often increases encapsulation of classes.  Continuation Passing Style is close to a daily technique for me now.
  10. Internal Domain Specific Languages.  Don’t assume that DSL means Oslo or whatever Whitehorse ended up becoming.  If you are interested in exploring the usage of Fluent Interfaces in your own architecture without a meltdown, here’s some design patterns to help you get started.  I think I could easily argue that articles 1-8 contained some bit of material that could help the average software team.  This one was just what *I* wanted to write about, so there.

 

The Future

I’m thoroughly burnt out on this for now and I’m distressingly behind on my book and mired in OSS project obligations.  I’m not touching it again for at least 6-9 months.  If I do start it up again, I think I want to focus on really basic concepts of OOP.  It’s my belief from 12+ years of development experience that the mass majority of developers, even “experienced” developers, cannot use object oriented programming or functional programming to their advantage.  I betcha that most .Net developers can give me a textbook glossary of OO terms like inheritance and polymorphism, but most of them probably don’t use these concepts in a way that helps them get work done.  It’s also my belief that most developers just never move beyond brute force procedural programming techniques.  If there ever is a next article, I’m tackling the “Anti-If” campaign with some basic Design Pattern usages to streamline your code by eliminating if/then branching code.

 

 

* Okay, I think you could argue with me that code quality doesn’t matter on small projects or projects that would be easier to rewrite later when and if they do need to change.  The only problem with that statement is that I’ve seen truly awful messes happen when those “throwaway” systems uncontrollably grew over time into big monsters.  My advice is to strive to reach a level of “unconscious competence” to where you naturally write high quality code and designs without going out of your way.

About Jeremy Miller

Jeremy is the Chief Software Architect at Dovetail Software, the coolest ISV in Austin. Jeremy began his IT career writing "Shadow IT" applications to automate his engineering documentation, then wandered into software development because it looked like more fun. Jeremy is the author of the open source StructureMap tool for Dependency Injection with .Net, StoryTeller for supercharged acceptance testing in .Net, and one of the principal developers behind FubuMVC. Jeremy's thoughts on all things software can be found at The Shade Tree Developer at http://codebetter.com/jeremymiller.
This entry was posted in Uncategorized. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • Boris Treukhov

    These articles are great – it’s sad that code formatting is gone on MSDN magazine site.

  • Joey Samonte

    Are you writing a book sir on software design/development?

  • http://codebetter.com/members/MichaelN/default.aspx Michael Nurre

    I really enjoyed reading your articles in MSDN magazine. Hopefully in the future you’ll find the time write more. Until then, I’ll continue to read your blog. Great Work!

  • Nitin

    summarized very nice

  • http://petesbloggerama.blogspot.com Peter Bromberg

    I thoroughly enjoyed your Patterns In Practice articles in MSDN Magazine. It is precisely because to topics covered were not “Mainstream” that made them more valuable. Thanks for the effort.

  • http://blog.invalidoperation.com Ray

    Good stuff as always.

    “I betcha that most .Net developers can give me a textbook glossary of OO terms like inheritance and polymorphism, but most of them probably don’t use these concepts in a way that helps them get work done. It’s also my belief that most developers just never move beyond brute force procedural programming techniques.”

    Yes. This.

  • http://weblogs.asp.net/jgalloway Jon Galloway

    Thanks so much for the Patterns In Practice series, as well as this good recap. I really like your postscript best – the goal is to get good at doing things right, so even when you’re rushed for time it’s just quicker to do it the right way.

  • Eric Malamisura

    Hey Jeremy I think you nailed it on the head, in my career of 15+ years of development I started off as one of those brute force procedural junkies. Eventually I matured into a true OOP and then a true OOP – TDD, and now I think its just the normal now, but its hard if not impossible to convince other developers why to use OOP. The benefit most of these “day programmers” see is that if they make it work, no matter the path they take they have accomplished their goal. Not only have I had extremely difficult time convincing developers to use OOP, but getting their heads to wrap around exactly what OOP is, thats difficult as well. I realized trying to explain TDD is a lost cause if they don’t even understand OOP concepts.

  • http://www.andrewcorkery.com Andrew Corkery

    Definitely book-marking this post. Great work on the series, I particularly enjoyed the post on Continuous Design. It’s a great introduction to the area.

  • dnndev

    awesome article – thank you very much!

  • http://codebetter.com/members/Patrick-Smacchia/default.aspx Patrick Smacchia

    Jeremy you wrote:

    >Pay attention to “constraints” on class stereotypes (Entity classes don’t touch infrastructure, infrastructure services don’t do business logic, UI classes don’t do persistence)

    I am curious to know how you advise to regularly enforce these large scale dependency constraints if you don’t use an automatic ruling tool like NDepend. Manually by verifying clean assemblies references in the Solution Explorer?

    We agree that if these constraint are not statically enforced once way on another, they will be violated sooner or later by mistake, aren’t we?

  • http://codebetter.com/members/Patrick-Smacchia/default.aspx Patrick Smacchia

    >You need to expand your idea of software design beyond one single, barely useful metric that just happens to make a pretty picture in your tool.

    As I wrote, I abide by tenets you quoted, and myself I wrote a lot of posts on functional-like/immutability/purity/side-effects free, design for testability, importance of code coverage, separation of concerns, API design, class responsibility, design for performance, code readability etc… Btw, NDepend address concretely most of these topics.

    But to me, details are much less important than big picture for one obvious reason. Details flaws are easy to fix, large scale organizational flaws are hard to fix.

    If I designed wrongly a class or a set of collaborative classes (it happens to me often) I can refactor it when needed in a few hours. And that’s how it should be because personally I cannot forecast future requirements and changes, even with solid Open/Close or Orthogonality principles in mind.

    If hundreds of classes end up using each other without any solid large scale organization (call it componentization, layering, pretty NDepend picture of namespaces, or any other way to refer to Acyclic Dependency Graph), no matter how many good patterns you put in it, it will become a daunting portion of code that cannot evolve anymore.

    At this stage the project is frozen for many weeks for a large scale refactoring. I would be surprised that you don’t know about this costly phenomenon because from what I saw in 2 decades of programming, it is the rule not the exception. This is why large scale structuring is my number one priority in software engineering.

  • http://codebetter.com/members/jmiller/default.aspx Jeremy D. Miller

    @Patrick,

    I don’t think you take a very holistic view of software design. From the criticism you lobbed at StructureMap last year, it seems like you are conflating good design with a pretty NDepend picture of namespace layering.

    How would you understand a design and keep it clean? Worry a lot about coupling and cohesion (even watch it with your tool since that’s all you seem to know). Pay very close attention to class responsibilities and stereotypes. Pay attention to “constraints” on class stereotypes (Entity classes don’t touch infrastructure, infrastructure services don’t do business logic, UI classes don’t do persistence). Namespace structure frankly has very little or nothing to do with “layering.” It’s just a way to organize or even “tag” classes as being a certain stereotype. Pay very close attention to the size of classes — again, since you only seem to see the world at statistics, use your CC numbers.

    You need to expand your idea of software design beyond one single, barely useful metric that just happens to make a pretty picture in your tool.

  • http://codebetter.com/members/Patrick-Smacchia/default.aspx Patrick Smacchia

    >I don’t agree with your definition of “good design” in the slightest.

    I don’t imagine any other way to prevent large chunk of entangled code, as we see in all sufficiently large application. I agree point-by-point with what you wrote, but something is missing: having plenty of good patterns applied at the micro-level doesn’t end up in a maintainable large scale organization.

  • http://codebetter.com/members/jmiller/default.aspx Jeremy D. Miller

    @Patrick,

    I might get around to blogging it later, but I think emphasizing “Layering” is relatively naive. I don’t agree with your definition of “good design” in the slightest.

  • http://codebetter.com/members/Patrick-Smacchia/default.aspx Patrick Smacchia

    IMHO the lack of large scale organization and the absence of dependency management is the biggest source of problems. This is a fact: Applications end up again and again with large portion of code completely entangled and unmaintainable.

    Talking about ‘cohesion and coupling’ is not enough, the 2 words ‘component’ and ‘layer’ would have deserved to be quoted at least once.

  • http://jopinblog.wordpress.com JohnOpincar

    Personally, I like to know “what’s going on under the covers” all the time. It’s what going on under the hood I don’t care about. :)

  • http://www.zubairahmed.net Zubair.NET!

    Very well written Jeremy.