Orthogonal Code

My employer, Finetix, added my blog to their main feed and my most recent post at the time just happened to be about Star Wars, so I thought I needed to put up my first "Grown Up" post of the year.  Before you read on, I am assuming some reader familiarity with the ideas of loose coupling, cohesive code, and encapsulation.

Continuing where I left off from On Writing Maintainable Code, it's time to talk about how to write code with a high degree of Orthogonality.  Applied to software design, Orthogonality is the ability to change a conceptual part of the software system while minimizing impact to other parts of the software system.  An orthogonal design separates different concerns of the system (data access, security, business logic) in a way that enables changes to be made in isolated areas of the code without rippling out.  

Over time, and even within the originating project, software systems will need to accommodate changes to both their behavior and infrastructure.  Later in the lifecycle, old dog software systems will need to learn new tricks for unforeseen requirements as the business environment changes.  Also, you stand a much higher chance of being able to reuse your existing code to perform this new functionality with a system that has been decomposed into orthogonal pieces.  My favorite analogy for orthogonal code is the Lego sets you had as a child.  You can use a small Lego brick to build almost anything.  A big piece shaped like a pirate ship can only be used to make a pirate ship.

The basic concepts for writing Orthogonal code are loose coupling and high cohesion.  These concepts are well known, but far too often ignored or not entirely appreciated.  Several years ago several of my "Architect" colleagues spent a week at an OOP bootcamp.  For the next couple weeks "Loosely Coupled" was worked into every single conversation that we had.  Two weeks after that they went back to designing web services that took SQL strings in their input messages and stuffing business logic into stored procedures.  I think they largely missed the point.

You've heard the mantra your entire software development career, as epitomized by this quote from Scott Bellware:

Loosely-coupled, highly-cohesive, ya-da, ya-da, ya-da, blah, blah, blah – the [developer's] eyes roll back into his head and the archi-speak fades into a nice, lulling drone.

Focus those eyes back to the front of your head and pay attention, because orthogonal code, maintainable code, testable code, understandable code — all of these goals are most likely accomplished by a strong focus on loose coupling and high cohesion resting on the bedrock of Encapsulation.  Achieve better cohesion by paying attention to "Tell, Don't Ask," the Law of Demeter, and the Single Responsibility Principle.  Supercharge the loose coupling of your code by applying the Dependency Inversion Principle and the Open/Closed Principle.  Get some testability into your code. 

Don't write this post off as just academic hot air, this is about consistently leaving the office on time and protecting your company's code assets because you'll be much more likely to smoothly sustain a forward progress with your existing code.   

 

Real World Example

To gain an appreciation for orthogonal code, or the lack thereof, let's look at some poorly designed code that does not exhibit orthogonality.  This isn't completely a contrived example either, this is largely taken and exaggerated from a preliminary version of a system that I later inherited.  I chose this example because this code was hard to change, and major changes and extensions were necessary in its first year of life.  Unfortunately, I've commonly run across code like this in my career.  Microsoft developers get a bad rap for having an inadequate command of good development principles and practices, but I've seen this type of tightly coupled, uncohesive code in several different programming languages, environments, and organizations.

Here's the scenario, the architecture is a Pipes and Filters structure that involves a series of message processors that take an Xml input, interpret that data and perform business validations and/or transform the input message to an outgoing message, and then put a serialized copy of the outgoing message onto an MSMQ queue.  The entry point method of one of the "Filter" classes looked like this:

        public void Transform(InputMessage input)

        {

            // BusinessLogicClass needs some reference data before it can do the transform

            // Use a derivation of the original Data Access Application Block from MS

           

            // First, gotta look up the connection string first

            string connectionString = ConfigurationManager

                .ConnectionStrings["ReferenceData"].ConnectionString;

 

            DataSet dataSet =

                SqlHelper.ExecuteDataSet("select * from sometable where some_field = some_property_of_input",

                                        connectionString);

 

            // do a bunch of business logic processing to transform Input to Output.  It's not shown here, but

            // the processMessage() function has calls to the data access classes interjected into the

            OutputMessage outputMessage = this.processMessage(input, dataSet);

 

            // then send the message on via MSMQ

 

            // Look up the Queue name first

            string queueName = ConfigurationManager.AppSettings["BusinessLogicClassQueue"]

            if (!MessageQueue.Exists(queueName))

            {

                MessageQueue.Create(queueName, true);

            }

 

            MessageQueue queue = new MessageQueue(queueName);

            string serializedOutput = serializeOutgoingMessage(outputMessage)

            queue.Send(serializedOutput);

        }

Fine, that code was easy to write in the first pass, but how about easy to change?  To understand?  To extend?  To test?  In reality this code is probably several times bigger because I omitted quite a bit of try/catch logic around the data access and queue management.  Not to mention that the actual business validation and transformation rules were complex.  We can easily spot some significant problems with this code as wewalk through some of the possible changes to this code base.  Let's talk about why a change could be difficult, and then in the sections below examine the application of orthogonal design principles to make these changes less painful.

  • Change the queueing mechanism.  Originally the system communicated between the logical filter classes running in multiple windows service processes by putting the outgoing Xml message into an MSMQ queue.  Reflecting this choice, our Transform() method directly uses the classes in the System.Messaging to talk to MSMQ. 
  • Change configuration mechanisms.  This method pulls its own configuration straight from the application config file.  If the application needs some more sophisticated configuration infrastructure, this code, and every other piece of code,
  • Change the database structure.  Ok, it's the year 2007, and in the post ASP classic world we don't embed sql directly in our business logic code anymore, but sometimes it's good to remember why we follow good design principles.  Changing the database structure could easily break the business logic.  Just as importantly, writing the business logic was harder than it had to be because you had to slog through the data access machinery along the way.  Even if the database structure doesn't change, you might want to take advantage of some sort of caching mechanism.  If you don't have an encapsulated data access library, you might end up needlessly repeating a large amount of boilerplate code around connection management (get connection string, open connection, attach transaction object, try/catch/finally exception handling).
  • Change the business logic.  Let's call this one a near certitude.  As I'll show later, reusing the business logic while leaving the database and messaging queue behind may be the important change that has to be made later.
  • Implement better (any) auditing or security.  Where would you put the security and auditing?  Everything seems to be jumbled together here.

 

Encapsulation

I've often heard encapsulation described as "shy" code.  Putting it another way, a lack of encapsulation is code walking around with its underwear on the outside.  Classes that conceal their inner workings are easier to use.  You don't have to do nearly as much hand holding with encapsulated classes because they know how to bootstrap themselves.

For a contrast in encapsulation, let's take a look at how the Filter.Transform() method performs its data access.  The original system used a modified copy of the original Data Access Application Block (DAAB) class.  The DAAB was an attempt to simplify data access by encapsulating some very common ADO.Net grunt work inside one monster sized class called SqlHelper.  The DAAB was popular, but I always thought it was lacking, mostly because it did not provide enough encapsulation of ADO.Net to form the core of a solid Data Access Layer.  Let's look at the signature of two of the most commonly used methods on the SqlHelper, then let's talk about where the DAAB just didn't go far enough.

public static int ExecuteNonQuery(string connectionString, CommandType commandType, string commandText)

public static int ExecuteNonQuery(SqlTransaction transaction, CommandType commandType, string commandText)

That's not so terrible, and it does, after all, encapsulate the creation and manipulation of SqlCommand and exception handling.  Take a look at what you *do* have to tell SqlHelper.  You have to go look up the connection string separately every single time.  If you need to do transactional sql statements (and you do), you're going to have to keep the SqlTransaction object around yourself and remember to pass it into the ExecuteNonQuery() overload method on every single call.  You also have to keep telling the SqlHelper what type of command you're using.  In the real life example system, that led to an absurd amount of repeated, noise code like this:

SqlHelper.ExecuteNonQuery(ConfigurationClass.GetConnectionString(), CommandType.Text, "some sql statement");

Why should I have to tell my Data Access Layer (DAL) what it's database connection string is?  What possible value is it to me to have to specify CommandType.Text repeatedly if that's the only CommandType I care about?  If I'm using transactions, I have to know about the Transaction class and shuttle it around between calls.  In my opinion, the DAAB simply did not provide enough value.  At most it was simply something that you could wrap further or use as sample code to write your own — and that's exactly what I did.

When I do have to use a classical DAL, I use a homegrown assembly that encapsulates all of my ADO.Net manipulation, mostly behind this core interface:

    public interface IDataSession

    {

       

 

        void BeginTransaction();

        void CommitTransaction();

        void RollbackTransaction();

 

        int ExecuteCommand(IDbCommand command);

        int ExecuteSql(string sql);

 

       

    }

The major advantage over something like DAAB is far more encapsulation of the underlying ADO.Net objects.  Using this interface as my core, I can concentrate solely on the sql being used inside my DAL instead of so much tedious ADO.Net code.  Consumers of the IDataSession interface don't have to know how to fetch the connection string or store a Transaction class somewhere because those details are encapsulated within the concrete implementations of IDataSession. 

In case you're curious, and you have far too much time on your hands, my homegrown DAL assembly is now part of the StructureMap code, but you'll have to pull it down out of the Subversion trunk (http://sourceforge.net/projects/structuremap).  Given a choice I'll use NHibernate for persistence instead, but I still use StructureMap.DataAccess for test automation and utility code.  I have not looked at the descendent version of the DAAB in the Enterprise Library, but I would have to assume that it's much improved from the DAAB.

 

Tell, Don't Ask

One way to detect or prevent leakages in encapsulation is to follow the Tell, Don't Ask principle.

…you should endeavor to tell objects what you want them to do; do not ask them questions about their state, make a decision, and then tell them what to do.

Here's an example violation of Tell, Don't Ask:

        public void SelectAPointByAskingDetails(DataPointSelector selector)

        {

            // ask DataPointSelector for all of its internal Point's

            Point[] points = selector.AllPoints;

 

            // ask DataPointSelector what it's SelectionType is,

            // then, depending on the SelectionType, pick the

            // Point out of the Point array and set the SelectedPoint

            // property on DataPointSelector

            switch(selector.SelectionType)

            {

                case SelectionType.First:

                    selector.SelectedPoint = points[0];

                    break;

 

                case SelectionType.Minumum:

                    // loop through all the points and

                    // find the "minimum" point, whatever that means

                    break;

            }           

        }

This "Asking" method knows a lot about the DataPointSelector, and it's making all of the decisions for DataPointSelector.  Following Tell, Don't Ask would lead us to encapsulate the details of Point selection into DataPointSelector.  After the encapsulation, the interaction with DataPointSelector is now:

        public void SelectAPointByTelling(DataPointSelector selector)

        {

            selector.SelectPoint();

        }

"Pragmatic" Andy Hunt explains "Tell, Don't Ask" very will in an episode of DotNetRocks.

You Will Follow the Law of Demeter!

I've always wanted to use that for a blog title, but I really don't think I can improve on the Wikipedia article on the Law of Demeter.  I mention it here because it's an important tool for increasing encapsulation in your code.  Roughly, following the Law of Demeter increases encapsulation by avoiding long messaging chains like TopClass.Child.Helper.Connection.Close().  Instead, open up a new Close() method on the TopClass and hide the entire chain of calls through its children.  It's nothing but making sure that TopClass is wearing its underwear (Child.Helper.etc) under its clothes where it belongs.

High Cohesion

Code that is highly cohesive groups strongly related code into a common class, namespace, or module.  On the flipside, a highly cohesive class is one that has narrowly focused responsibilities.  Code with low cohesion either scatters related code throughout the system, or leads to classes with unrelated responsibilities.  High Cohesion leads to code that is easier to understand, reduces duplication, enhances reusability, and generally makes code easier to maintain. 

The Filter class flunks a test of cohesion, and also implies some greater problems throughout the codebase.  To start fixing the Filter class, let's list the responsibilities of the Filter class.

  • Fetching configuration
  • Calling the database to get some reference data
  • Performs the business validation and transformation logic
  • Serializes the results into Xml
  • Creates the messaging queue if it does not already exist
  • Sends the output message to the outgoing messaging queue

Wait, you say.  The Filter class is probably the implementation of a single use case or story.  That's cohesive right?  Well, no, because even inside this single use case there's a mixture of business logic and system infrastructure.  Chances are very good that you will want to reuse pieces of the infrastructural code outside of this single use case.  Maybe more importantly, you want to avoid duplicating system infrastructure code so that you can accommodate changes in infrastructure more readily.

Let's take the immediate refactoring step of moving the messaging infrastructure code into a separate class and simply let Filter delegate to this new class.

    public class MessagingGateway

    {

        public void SendMessage(OutputMessage outputMessage)

        {

            // Look up the Queue name first

            string queueName = ConfigurationManager.AppSettings["BusinessLogicClassQueue"];

            if (!MessageQueue.Exists(queueName))

            {

                MessageQueue.Create(queueName, true);

            }

 

            MessageQueue queue = new MessageQueue(queueName);

            string serializedOutput = serializeOutgoingMessage(outputMessage);

            queue.Send(serializedOutput);

        }

 

        private string serializeOutgoingMessage(OutputMessage message)

        {

            return null;

        }

    }

Now we've gotten the details of the messaging infrastructure away from the rest of the flow of the Filter code in a class (MessagingGateway) that can be reused across multiple Filter classes.  If we wanted or needed to, we could change the serialization to a binary format, use a different security mechanism on the queues, easily add more auditing, or maybe switch to an entirely different technology.  Switching gears entirely, maybe you want to throw out the idea of queueing and run all of the Filter's in a single common AppDomain.  Assuming that any new version of MessagingGateway encapsulates the details of its implementation, every single one of these changes could take place without altering the code in the Filter classes.  That's orthogonality at it's best.

While doing research for this section I came across this great article from Meilir Page-Jones on Cohesion.  It's from an introductory textbook on Structured Programming of all things, but it's readily applicable to today's OOP and I would think SOA paradigms. 

Single Responsibility Principle

We improved the cohesion of the original Filter class by extracting out a class for MessagingGateway, but there are still unrelated responsibilities left in the Filter.Transform() method that could be pulled out.  The new MessagingGateway class itself could use some improvement.  So how many responsibilities can a class take on before cohesion is compromised?  Exactly one, and that's known as the Single Responsibility Principle.

A class should have only one reason to change.

Taking a closer look at the new MessagingGateway above, we see at least two reasons it may have to change.  The queueing itself obviously, but it MessagingGateway also pulls the address of the queue from the configuration file.  It's very conceivable that the configuration strategy of an application could change to something more sophisticated.  Specifically, I can remember a project where we could have collapsed 75 servers, all of which were a single point of failure, into 12-14 redundant servers, except that the configuration code was scattered throughout the application instead of being encapsulated into a cohesive module of the system.  Intermingling the code that retrieved configuration data with the rest of the system effectively eliminated a chance to remove a huge amount of support overhead while actually increasing uptime. 

Off the top of my head, I can think of two ways to remove the details of the configuration storage from the MessagingGateway class.  My strong preference is to simply poke in the queue name and whatever else configuration that the MessagingGateway needs into its constructor function (using StructureMap of course!).  The other way, just to avoid branching off into a discussion on IoC/DI tools or builder classes, is to pull the access to the configuration into a centralized class.  That class, and the new slightly simpler MessagingGateway, might look a bit like this:

    public class ConfigurationManager

    {

        public static string ConnectionString

        {

            get

            {

                return "something";

            }

        }

 

       

    }

 

    public class MessagingGateway

    {

        public void SendMessage(OutputMessage outputMessage)

        {

            // Look up the Queue name first

            string queueName = ConfigurationManager.QueueName;

           

        }

    }

Taking the Single Responsibility Principle knife to our original Filter.Transform() method leads to pushing some of the responsibilities out into these new classes: 

    public class DataAccessObjectForTheFilterClass

    {

        public ReferenceData GetReferenceData(InputMessage input)

        {

            // go grab the connection string, call some sort of sql,

            // and put the data into some sort of usable data structure

           

        }

    }

 

    public class BusinessValidationAndTransformationProcessor

    {

        public OutputMessage ProcessValidations(InputMessage message, ReferenceData referenceData)

        {

            // do a lot of business processing here, but only business processing

           

        }

    }

The new version of the Filter.Transform() method delegates to the new granular, cohesive classes: 

        public void Transform(InputMessage input)

        {

            // Get the reference data

            DataAccessObjectForTheFilterClass dao = new DataAccessObjectForTheFilterClass();

            ReferenceData referenceData = dao.GetReferenceData(input);

 

            // Do the validation and transformation

            BusinessValidationAndTransformationProcessor processor =

                new BusinessValidationAndTransformationProcessor();

            OutputMessage outputMessage = processor.ProcessValidations(input, referenceData);

 

            // Send out the outgoing message

            MessagingGateway messagingGateway = new MessagingGateway();

            messagingGateway.SendMessage(outputMessage);

        }

We pulled out the responsibility for configuration, data access, business validation, and the messaging infrastructure out of the main Filter class into cohesive classes.  At this point, the Filter class is merely a "Coordinator" stereotype from Responsibility Driven Design.  Centralizing the coordination of the various pieces into a dedicated coordinator is a great way to enhance the orthogonality of the code.

One of the client developers on my team asked me how do you know when you need to create a new class rather than add functionality to an existing class.  I responded with "One class, one responsibility."  That leads us into a question of just what *is* a responsibility.  My advice is to look for pieces of a class that don't seem to be related to the rest of the class and pull it out.  I think the class being discussed that day was the Application Controller class that manages the basic shell of our WinForms application.  Part of the UI is a panel of tabs for individual active views.  Our ApplicationController class was starting to collect quite a few methods around managing the contents of the TabControl that weren't related to the rest of the ApplicationController.  We pulled the functionality for managing the active view collection and the TabControl into a separate controller/view combination.  Instead of one really big schizophrenic controller class, we had two smaller controller classes, both with a focused intent.  There's a very important lesson in that.  Many times you'll find that more classes lead to a simpler design by minimizing the complexity of any one single class.

 

Loose Coupling

Back to the original Filter.Tranform() method, the business logic in the Filter class is tightly bound to the configuration of the system, the messaging infrastructure, and knows directly about the SQL that gets passed into SqlHelper.  In this design, the business logic inside of Filter is coupled very tightly to the infrastructure.  So what does that really mean?

Just to see if someone is reading my blog, here's another real world scenario.  Let's say you built your core software system 5-6 years ago before any of the Object/Relational Mapping persistance tools were considered mature.  To fill the gap you wrote your own proprietary ORM and everything is great — for awhile.  Fast forward to today and you realize that Hibernate or the forthcoming LINQ tools are significantly easier to use and have some functionality that you would like to have that isn't readily possible with your own homegrown ORM tool.  If your business logic, user interface, and service layer are tightly coupled to your homegrown ORM, changing to another tool may not be economical.  If you interspersed too much of the persistence code with the business logic or service layer code, you're effectively looking at a rewrite just to use the better tool.  That's not the end of the world, but you've just incurred an inefficiency that your competitors might not have by using the better tools.

You may absolutely not interpret this as a license to go build extra or unnecessary layers of abstraction on top of your persistence layer.  When I use NHibernate I do encapsulate it behind thin wrapper interfaces, but anything beyond that might be getting into the realm of silly.  Ted Neward recently skewered some folks for too many abstraction layers.  All I want to say here is to keep as much distance between your persistence and business logic code as you possibly can.

I've talked quite a bit about the coupling between the business logic and persistence code, but tight coupling might be lurking in a lot more corners of your system.

  • In the implementation of a web service or any other integration point, strongly coupling the functionality of that service to the transport mechanism or Xml format often turns out to have negative effects.  To the SOA enthusiasts of the world, I say to you that loose coupling doesn't stop once you're underneath the initial SOAP message.  Loose coupling is just as important, in the small, in the code within a service.
  • Singleton's or static classes are a very strong form of coupling
  • Calls within a class to the constructor of another class is a subtle form of strong coupling

Dependency Inversion Principle

Wait!  You just said that a call to a constructor is a form of tight coupling?  How?  When we last left the Filter.Transform() method we had decomposed the responsibilities of the original scenario into cohesive classes, but the Filter class is creating the concrete helper classes directly.

            MessagingGateway messagingGateway = new MessagingGateway();

            messagingGateway.SendMessage(outputMessage);

In the code above, Filter knows exactly the class that performs the messaging, and it's dependent upon that MessagingGateway.  We've moved all of the details around interacting with MSMQ into MessagingGateway, so we could change the transport layer without modifying Filter, but what if we really do need to use Filter both with MSMQ and a web services transport layer?  Simple, let's apply the Dependency Inversion Principle and extract out an interface for MessagingGateway that is independent on any of the concrete implementations of messaging transport.  The second step is to make Filter depend on the abstracted interface.  You do have to have a way to get the different implementations of IMessagingGateway into Filter, so we use just pass an instance of IMessagingGateway into Filter (Constructor Injection).

    public interface IMessagingGateway

    {

        void SendMessage(OutputMessage outputMessage);

    }

 

    public class MessagingGateway : IMessagingGateway

 

 

   

    public class Filter

    {

        private readonly IMessagingGateway _messagingGateway;

 

        // Push in the IMessagingGateway in the constructor function

        // so Filter doesn't even have to know which concrete

        // class is in here

        public Filter(IMessagingGateway messagingGateway)

        {

            _messagingGateway = messagingGateway;

        }

 

        public void Transform(InputMessage input)

        {

            // Do the things that Filter.Transform() does

            …

 

            _messagingGateway.SendMessage(outputMessage);

        }

    }

Now, we've got Filter receiving an abstracted IMessagingGateway from somewhere else, so it could work with completely different types of messaging infrastructure — or even work with no messaging.  Don't just stop with the messaging either, abstract away the data access and the business validation and transformation code as well.

Open/Closed Principle

We've busted the original Filter.Transform() method into smaller cohesive pieces, and decreased the coupling between the pieces by extracting out abstracted interfaces.  Now we can talk about the ultimate principle for creating maintainable code – the Open/Closed Principle (OCP)..  Verbatim, this principle is stated "Software entities (classes, modules, functions, etc.) should be open for extension, but closed for modification."  I used to think this only referred to the practice of creating plugin extensibility points in your code, but that's only part of the OCP story.  What I've learned since then is that the essence of the Open/Closed Principle is being able to add new functionality to an existing codebase by writing completely new code with minimal modification of existing code.  One of the major costs of extending an existing system is the potential for regression bugs in existing functionality.  The most reliable way to minimize the creation of regression bugs is to avoid modifying the existing code.  I"ve consistently observed that I'm always faster when I'm building all new code versus making modifications in existing code.

I've found that the OCP pattern is going to emerge in two basic ways:

  1. "Pluggability."  This is the obvious usage, but maybe not the most effective.  By "pluggability" I really just mean the ability to create a new class or group of classes to implement a new feature and make only minimal changes to the existing code to call into the new code.  For example, we can create new functionality in Filter.Transform() by injecting in all new implementations of IMessagingGateway.  If we've abstracted the business rules, we can even change strictly the business rules by creating a new class and simply passing it in at the proper place.  Following the OCP doesn't have to mean the full-blown ideal of runtime plugins (shameless plug:  Dependency Injection tools like StructureMap make this much easier and more reliable;)).
  2. The second, and possibly more powerful OCP usage, is recombining existing classes or components to create new functionality.  In the real world system behind my Filter.Transform() example, we had a need in a later project to use the transformation logic completely independent of the pipes and filters architecture in a brand new web service.  If the transformation logic is decoupled from the rest of the Filter.Transform() method, we can simply take that cohesive business validation class and use it in the entirely new context.  In real life that functionality was very strongly coupled to the pipes and filters architecture and I spent several days doing refactoring just to expose that functionality.

Think of this scenario as well from my new career in financial applications.  Say you have an analysis engine inside your web application that pulls its transactional data from a database and it works great for quite awhile.  Then one day your CEO comes in and says that they have a great opportunity if you can only mate the existing analysis engine to data entered into an Excel sheet so a trader can run simulations.  Can you do that?  If the analytic engine makes a lot of "pull" calls to the data access layer, or is tightly coupled in some other way to the database, you might miss out on that opportunity.  On the other hand, if in your existing architecture you simply "push" all of the data into the analytics engine that it needs, it's going to be relatively easy to take the cohesive analytics engine and use it in a completely new context.

 

 

Testability

A while back, Scott Bellware vociferously wrote that there is no sustainability without testability (and by inference, automated tests).  I could not agree more.  Part of modifying existing code is verifying both the correctness of the new code and regression testing the old code to prevent regression bugs.  Regression testing is a major cost and a significant risk to modifying existing software.  A high degree of test automation can alleviate these concerns, but an effective and efficient test automation strategy is only enabled by a good design — specifically an orthogonal structure that enables opportunities for focused whitebox testing.

There was a good discussion on the Test Driven Development group on Yahoo about the relationship between Testability and good design.  The discussion was kicked off by someone asking if there really was a linkage between testability and good design?  I emphatically believe that designs that are not testable are bad designs.  I'm not sure whether or not I'd say that testability is a first class goal in design, or simply a vital means towards making working code.  Regardless, testability alone is enough of a reason to strive for orthogonal code structures.  Code that is easy to test is easy to make work.

Testability largely coincides happily with the principles of the good design you were striving for anyway:  loose coupling and high cohesion.  Automated testing is much simpler when it's easy to isolate functionality (loose coupling) and when you can test one small thing at a time (cohesion).  One of the very significant values of using Test Driven Development is that the technique is very effective at sniffing out coupling or cohesion problems.  When it's hard to write the unit tests, it's usually indicative of low cohesion or tight coupling.

Putting it All Together

Alright, we know some of the principles of writing orthogonal code and we've taken a long look at the coupling and cohesion issues around a sample system.  As I said earlier, my sample scenario was from a real project I was part of two years ago.  Do your best Keith Jackson imitation here — in that real world system, the original team hit the "Granddaddy of Them All" – The Great Refactoring of Aught Five.  The pipes and filters architecture was killing them with complexity.  The code was slow, hard to deploy, difficult to troubleshoot, and nasty to test.  I don't remember how long it took, but they stopped and ripped out some of the pipes and filters support to collapse the application into a single AppDomain.  Performance went up, deployment got easier, and testing and troubleshooting improved somewhat. 

All the same though, I would call the Great Refactoring a mild failure because it could have been done more efficiently and left the code in a better state.  They didn't completely remove the pipes and filter motiff, largely because refactoring any farther would have been too difficult because the code was not orthogonal.  The time spent doing the Great Refactoring was pure overhead, because not a single bit of business functionality was added.  I say overhead because the refactoring could have been several times easier and quicker if they had religiously followed orthogonal design concepts from the very beginning.  With an orthogonal structure they could have thrown away the messaging infrastructure and the filter coordination code away, but reused the business logic and data access classes largely unchanged.  If the refactoring had been less risky, and orthogonal code would have mitigated the risk, they might have eased the refactoring by simply starting it early instead of waiting until they were backed into a corner.

Lastly, on the Pipes and Filters Scenario

I apologize to the original team for doing this at their expense, but they provided a near perfect object lesson on a couple fronts and I bet they'd agree with many of my conclusions two years later.  The original impetus for using the Pipes and Filters architecture being spread across multiple windows services was to be able to scale the system "vertically" across multiple systems. 

From my perspective, the project was jeopardized by committing far too early to a complex design upfront without any significant deviation or adaptation until things got really bad.  The entire skeleton of the system was layed out before doing any significant testing or even unit testing.  A large impetus for the XP YAGNI ideal is that it's considerably easier to start simple and add complexity as needed than it is to eliminate or change unnecessary complexity.  I think that building good systems is very largely dependent upon getting feedback and verification on your design early and often and then using those lessons.  Some of the team was used to heavy RUP style upfront design instead of an adaptive approach.  They dumped the UML, but didn't replace it with an adaptive design approach.  UML or not, pay attention to your design so you know when you're going off the rails.

Another lesson that we *all* need to take away from this is to keep learning and get some exposure to new ideas and different ways of thinking.  Consciously or unconsciously, the team lead on the project chose the pipes and filters model because he had a deep background in mainframe applications and it was a comfortable style for him.  Even after the big refactoring the pipes and filter model left a lot of "noise" code in the system that made the flow of the code very, very difficult to follow and generally bloated the codebase.  I still think that a Domain Model + Service Layer organization would have been much more suitable and far simpler.  This development motto from Bil Simser captures the myopia towards the comfortable for me:

"If all you do is what you know, you'll never grow."

The pipes and filters architecture included a lot of support for metadata driven composition of the filters along the way to make the addition of new workflows easier through configuration.  In reality the extra metadata-driven pipeline infrastructure just turned into a great deal of intellectual overhead in understanding and troubleshooting the system.  When it came time to build new functionality into the system, we struggled to reuse existing functionality because it was too tightly coupled to the pipeline extensibility mechanism.  Ironically, I felt that one of the biggest impediments to adding a new workflow to the system was the very extensibility mechanism put into the code for new workflows.

 

Wait, there's more… 

I've got much more to say on writing maintainable code in this ongoing series, but it's going to wait awhile.  I'm all blogged out for now.

About Jeremy Miller

Jeremy is the Chief Software Architect at Dovetail Software, the coolest ISV in Austin. Jeremy began his IT career writing "Shadow IT" applications to automate his engineering documentation, then wandered into software development because it looked like more fun. Jeremy is the author of the open source StructureMap tool for Dependency Injection with .Net, StoryTeller for supercharged acceptance testing in .Net, and one of the principal developers behind FubuMVC. Jeremy's thoughts on all things software can be found at The Shade Tree Developer at http://codebetter.com/jeremymiller.
This entry was posted in Maintainability. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • http://www.datarecoverytrial.com Data Recovery Software

    thank you for this article is very interesting, 

  • http://www.backlinkschecker.ws backlink checker

    I am really
    interested into this topic…..

  • Anonymous

    keylogger

    it’s very nice is actually a good post………

  • http://www.mahjonghub.com/ mah jong download

    Very interesting topic will bookmark your site to check if you write more about in the future

  • http://www.chatsitesi.biz/sehirler chat sitesi

    Cool post . .This means that there is no way for other people
    Thank you

  • http://www.portraitkingdom.com/portrait-paintings.html oil portrait paintings

    Orthogonal coding is really new to me. I guess because I wasn’t born to be a programmer. Hard coding is something that is really difficult for me to tackle. Thanks for your post. I’ve learned something new again.

  • http://www.portraitkingdom.com portrait artist

    Do you ever run out of things in your mind? You’re such a genius and it seems like you always exceed our expectations. What’s your secret? Where do you get all the inspiration?

  • http://www.factoryfast.com.au rugs

    Not just the technical aspects, but the entire thinking around your post is one that is sorely needed amongst many developers, architects, and business analysts. In my old company, the failure to think in this way resulted in just way too many headaches – deadlines not met, projects gone awry, and even spare parts going missing because of a system with too many holes and other things. The result of laziness upfront is money loss, especially when you’re looking at what you’re looking at here. Thank you for a very informative and excellent post!

  • http://colinjack.blogspot.com/ Colin Jack

    Really really good stuff, you thought of writing a book? :)

  • Jon Paul Davies

    Great article. More please.

  • http://haf.se Henrik Feldt

    Hello Jeremy,

    Before I go into the discussion of architecture — let me just ask a quick question — what is messaging?

    The article was a good read and indeed one of the few posts out there that deal with the problems and intricacies of software architecture on a deeper level than the (sometimes) academical mumbo-jumbo that’s indeed very easy to write. (i.e.: “You should use n-tier architecture and abstract the implementation details away from layers higher up in the architecture… ” — yes, but what does that mean?)

    I’ve been struggling with getting my own applications up to date in architecture wise, and reoccurring concepts that are talked about, but not really explained by people on the net, are inversion of control, service oriented architecture, domain driven design and implementation and design-pattern choices concerning O/R-mapping together with business rules and business constraints.

    #How would you implement business constraints?
    I’ve been creating my own exceptions, inherits from ApplicationException and checking for those in the presentation layer. I’ve gotten the feel that there is a large difference between data validation in the presentation/presenters-layers and business rule validation… Data validation being just checks, that the user inserts proper data, like e-mails, required fields, but when it comes down to checking the business constraints, like — the order in a e-commerce system must be associated with a customer — the presentation layer isn’t the best place to put it… Instead I cast these application-specific exceptions from the domain layer and check for them. Is this similar to what you do?

    #In what context and where would you place your Transform method?
    I recently did a project for a university class… To cut things short, there are application specific methods, like your transform method, or in the system we built, a method returning all the Course(s) possible to take for a Student, i.e.

    courses that the student were not actively taking (as could be found in Student’s ActiveCourses-property),

    courses the Student hadn’t already taken (As could be found by the class Result with an internal constructor which had 1 reference to the student, 1 reference to the course and 1 grade object, all publicly exposed,

    courses the Student had the requirements for, to take, as could be found in the RequiredCourses property of every Course-object and finally,

    courses that didn’t violate the fact that one Student should really just be allowed to read courses with credits aggregated to integer x.

    We had some choices: 1) Put the logic in the CourseDao that inherited a generic NHibernate data access object, by using stuff like “left join” and “outer join” etc with the high query language in NHibernate, or
    2) Put the logic in the Student domain object, where we’d have to reference the dao-layers in order to get all the course objects of the application (I didn’t pick this because we reasoned that the domain objects should indeed be free of data access logic like described in (1)), or
    3) create a mixture of a repository object and (what we named) a logic-objects, that indeed loads all the required domain objects from the dao-objects (man… objects!), and then looped through the courses of the system with a foreach and checked the business constraints in the loop.

    The reasoning we had behind this was that 1) was indeed a nice way to go, considering the indexing taking place in the database, that would speed up the search enormously compared to route 3. It would also be a lot easier to write because of the keywords (joins) in the data definition language (DDL -> HQL). However, it would put the business logic in the data-layer which is a violation of the orthogonal design, given that the business layer shouldn’t ever have a reference to either nhibernate or the data layer, but rather define all it’s required methods through the interfaces, which the data access objects in the data layer in turn, implemented, because then the data layer would suddenly be concerned with fetching application specific data.

    Summarily, we opted for path 1 because of the performance requirements; but what’s your take on it?

    #I’m also curious of the testing you’d do for the classes in this article, and how your team conducts its testing?
    – And do you know any great articles about (unit?) testing?

    Do you happen to know how the words domain object repository (for fetching domain objects), data access objects (for fetching domain objects) and service oriented architecture and its specific implementation details (which also seems to be used for fetching domain objects) mingle together? I’m having some trouble finding good information on this since it seems I have to buy a host of books in order to find out anything about it.

    #I’m also a bit curious on different takes on inversion of control, such as tutorials or articles; perhaps you could give me a hint?

    #How does service oriented architecture fit in with web services?

    #Would you put your security checks in the refactored Transform method you have (re-)written above, or would you do some nice tricks with Declarative programming with attributes on top of the class which contains the Transform method, or would you have the runtime and perhaps even the presentation layer do the security checks and assume that the calls to the transform method are secure? And what about the distinction of being able to rely on the presentation layer to supply data, i.e. the client is under your control, and that of the client being totally unreliable, like the web?

    #Would the transform method be a part of a larger middle-ware?

    Like you notice, I have quite a few questions :)… I would be very very interested in hearing your answers to them, or perhaps just your favorite questions :p!

    (I should really start blogging…)

    Regards
    Henrik

  • Girish Maskeri

    Excellent post! I have read quite a number of articles on maintainability but I think this is the first one to tease out the concepts using real life actual code and not some foo and bar examples.

    Great work!

  • http://kjellsj.blogspot.com KjellSJ

    I think the importance of testability as a measure of design quality and reusability/agility still is underrated by most developers, just think about the heated discussions about TDD last year. I think it is time to start talking about test smells and design smells in addition to the wellknown code smells, especially wrt service-orientation: http://kjellsj.blogspot.com/2007/01/test-smell-design-smell.html

  • http://lozanotek.com Javier Lozano

    This is an AWESOME read! I’m sending the link to everyone on my dev team!

  • Groover

    Nice way of tying a lot of hard core architecture concepts together. It usually takes a few years of building systems and reading patterns books to figure this stuff out on your own (at least for me).

  • Damon

    Great writing Jeremy. It’s fantastic to have someone put out some real world code to explain these concepts. You’ve raised the bar for blog posting in 2007.

  • http://blogs.giffordconsulting.com Tim Gifford

    Perfect!

  • http://www.base4.net/blog.aspx Alex James

    Excellent article…
    One thing though the link to StructureMap goes to a nasty place by accident.

  • dgon

    Terrific job in putting together several important cocepts.
    I will make this a must-read to all people in my team for them to realize what all those mean and why such things are important.

    Thanks a lot

  • http://haacked.com/ Haacked

    Well done!

  • http://exceptionz.wordpress.com Maruis Marais

    Jeremy is back with a great blog posts. Brilliant…

  • http://www.UdiDahan.com Udi Dahan – The Software Simplist

    You rock!