Over-Architecting Via Pure Fabrication and Indirection to Reduce Coupling

Unlike my wife who loves to go shopping with no real purpose in mind :), I avoid putting myself in situations where I could separate myself from my money without due cause. I just end up desiring or thinking about items that I have never had and am living quite comfortably and happily without. Where once my inner-pond was clear and calm, seeing new and improved toys causes the lines of need  vs. want to blur and I start trying to justify the need for a new toy.

The same idea is true in software development. The more I learn about new tools, techniques, patterns, and principles of software development, the more I try to use them even though I have been living "fine" thus far without them.

As an example, I have never been overly concerned with low coupling in the past, and in 99% of my applications it has never been a problem. In the other 1%, I was able to refactor with little effort to a design pattern that supported pluggable functionality. In the past year, however, I have started to see various classes and interfaces being added to my code to support looser coupling. I remember the effort of creating the classes, but have not realized any benefit from their existence because their creation wasn't based on actual requirements, but my desire to use the ever evolving best practices and patterns to make my software "more maintainable and flexible."

Low Coupling Principle

I have learned to see the low coupling principle as an "evil" in many cases. It is a con artist making you think your custom application or code needs to be pluggable or future-proof in many cases when 1) it doesn't, or 2) it is just as easy if not easier to refactor to a pluggable architecture later using known design patterns ( Refactoring to Patterns by Joshua Kerievsky ). Obviously if you are building frameworks and reusable components the idea of low coupling and coding to interfaces is important, but I have seen and done this myself in the past with custom / opinionated software as a "just-in-case." In more times than not, this just-in-case type of development leads to over-architected and hard to maintain software.

Once your skills have improved in OOP and refactoring and you are very much aware of design patterns to solve common problems, you can avoid doing things just-in-case because refactoring code to support new requirements is simpler. In fact, I conciously avoid adding code to support functionality unless it is "an absolute." This may sound obvious with people shouting YAGNI and KISS, but your code is a reflection of you as a developer. When later you move on to greener pastures, requirements change in the application, and another developer is maintaining your code, he might be saying you should have forseen this changing requirement and used the XXX Design Pattern to make this evolution easier.

Applying UML and Patterns – Pure Fabrication and Indirection

Getting back to this low coupling evil :), Applying UML and Patterns by Craig Larman has a couple of GRASP Patterns ( principles ) that are all about low coupling:

  • Pure Fabrication – Who is responsible when you are desperate, and do not want to violate high cohesion and low coupling? Assign a highly cohesive set of responsibilities to an artificial or convenience "behavior" class that does not represent a problem domain concept – something made up, in order to support high cohesion, low coupling, and reuse.
  • Indirection – How to assign responsibilities to avoid direct coupling? Assign the responsibility to an intermediate object to mediate between other components or services, so that they are not directly coupled.

I hadn't caught onto this when I originally read the book, but now that I have a better sensitivity to the overhead associated with low coupling, I start to see the words that imply only do this if absolute necessary as creating these artificial constructs leads to less pure and bloated code that blur the descriptive and self-documenting nature of a well crafted domain layer.

Only if absolutely necessary and based on in-your-face requirements do you add classes and behaviors to loosely couple your layers and partitions. Do you really need a factory, service locator, registry, etc. that returns an interface or can you couple directly to the class? Is there a requirement sitting in front of you now that justifies it? How long would it take you in ReSharper to extract the Interface, find usages, and replace the class with an interface if the need actually arises in the future? What is the chance of that need actually arising?

Domain-Driven Design and Repositories

I just finished reading Applying Domain-Driven Design and Patterns by Jimmy Nilsson. A really good book in my opinion and one I recommend. However, it still doesn't clear up for me one of the artificial constructs in Domain-Driven Design used to loosely couple the domain from the datastore – the Repository.

In the DDD community, there are many disagreements about what the repository can and should do and if it can be referenced by a domain object. Here we have a class that provides indirection for low coupling purposes, appears to be a first-class object in DDD, and yet people often still treat it as a "crazy-cousin" or only use it as an intermediate pass-thru object to NHibernate or a DAO.

Because I, personally, haven't got much value from the repository, I have tossed it out altogether in favor of a thick application layer above the domain layer handling use cases, application-specific rules, and interaction with the data access layer. I am sure others have used Repositories wisely, but until I see the light, such artificial constructs ( Repositories ) have been kicked to the curb as they only over-complicate my applications.

Principles Are Cagey Ideas – Expert C# 2005 Business Objects Example

Principles and best practices are cagey. For every principle suggesting you do "a", you will find at least one telling you not to do "a", instead do "b".

As an example, another book I recently read is Expert C# 2005 Business Objects, which is a followup to Expert C# Business Objects that discusses the CSLA .NET Framework. I will post a review of the new book in the near future.

Anyway, in the book, Rocky uses the principle of Encapsulation as well as other good reasons to suggest that the data access code for common CRUD methods be in the business object itself. The business object itself contains the code for fetching, deleting, etc. itself and any of its children. One could suggest this is good use of the Information Expert Grasp Pattern as well.

He briefly discusses why this is better than a competing principle, High Cohesion, that suggests using a separate persistent object in another layer. Some people would argue for high cohesion, suggesting it better to create a DAO in a data access layer that is responsible for all the data access needs of the object.

To each his own, but you can see how wishy-washy these principles can be taken.

Conclusion

Obviously when you have a need for loose coupling you need your artificial constructs. However, in general, objects and behavior based on pure fabrication and indirection only muddy-up your code. By sticking with various object-oriented principles, like single-responsibility principle and open-closed principle, you will be able to refactor youself to an architecture that supports looser-coupling where needed later while keeping your code as lean as possible today.

I am probably kicking in closed doors, but I needed to set myself straight once again and thought others might find it valuable :)

This entry was posted in Uncategorized. Bookmark the permalink. Follow any comments here with the RSS feed for this post.

23 Responses to Over-Architecting Via Pure Fabrication and Indirection to Reduce Coupling

  1. dhayden says:

    My thought is that client application #2 will call for the immediate refactoring of the code to a factory and hence the problem will not be as problematic as you imagine. However, if you already know about client applications that are forthcoming, a factory makes sense now as its addition would no longer be speculatory.

    I think we agree on this subject, Greg, and are only getting into semantics here.

  2. Greg says:

    The more important point is about the risk management side of the decision to go a given route. If adding a factory now costs me 5 minutes (even though it is not very necessary though it could be in the future) and the alternative refactor will dribble out into 72 client applications at a later point, risk management would say to provide the factory up front.

  3. dhayden says:

    Greg,

    I apologize for anything in this post that seems derogatory about DDD. To be honest, you have totally misunderstood my whole point of mentioning CSLA .NET, DDD, and Expert C# Business Objects.

    The minute point you are focusing on is to show how wishy-washy principles can be. I am explaining how you can use them to support any strategy and how principles conflict with each other. So where Rocky says he does something for encapsulation, I or anyone else can say he shouldn’t do it to be more cohesive. In all truth, I was actually playing devil’s advocate with Rocky on his point, not praising his framework.

    If you read further in my discussions on the Repository, I said I ( me ) have not made good use of them so I don’t use them. I also said that I am sure other people ( insert you in this group ) have made very good use of them. Personally, I think we are achieving the same goals in our solutions. What you do with a repository I achieve in other ways.

    I appreciate you mentioning that you thought I was being derogatory, because if you took it the wrong way I am sure others have as well. If there is one thing I am, it is extremely humble and very open-minded about architectures in seeing their strengths and weaknesses. I would never be derogatory.

    I will avoid mentioning DDD in the future so you don’t mistakenly lock horns with me again :)

  4. Greg says:

    “Just to make sure we are clear, I am not arguing CSLA, ActiveRecord, vs. DDD in this post. My example of CSLA was only to demonstrate how one can get principles to support any conclusion. My post wasn’t to discuss the merits of static finder methods in Active Record, for example, vs. a Repository in DDD.”

    This is true but your post made fairly derogatory comments on repository while many bolsterring comments for the AR/rocky’s comments.

    More to the point … I am putting forward a case that repository should be used as it is a very small investment and if you go down the other road you will have no clear method of refactorring if you need what repository would have given you to start with. I like to look at such things as cost vs risk. The example given includes a very small cost but a huge risk … I would rather eat the small expense up front than have a large (undetermined) risk at the tail end that would likely have me saying “this can’t be done” …

    A perfect example of this would be if later you decide you need to do something your OR mapper can’t do using a pattern such as AR. A refactor to the repository pattern at this point would be a huge change (quite likely large enough that it would be deemed not possible) as it effects not only the domain but every client of the domain.

  5. dhayden says:

    BigJimInDC,

    Thanks for the comments. We definitely agree.

    I do have a passion for TDD as well and use it on all but simple forms-over data applications. The topic is pretty well covered here on CodeBetter by people who are better at it than me, however, so I try to talk about other things :)

  6. BigJimInDC says:

    Dave,

    I couldn’t have written your reply to me any better than you did. Point being, we’re in complete agreement on everything you raised in the reply. I initially simply read your post as a statement about loose coupling in general, and did not necessarily pick up on “speculative design” being the core angle you were attacking the issue from.

    I believe that Greg’s points about the repository situation have more to do with a series of unstated logical leaps on his part (and many others), rather than any given promotion of premature or speculative design. IMHO, the use of patterns, repositories, etc is as speculative as deciding to use OO approaches over a strictly procedural approach. If you have already decided to use OO, then following what is laid out in Evans’ and Nilsson’s books is a very smart idea.

    In general, my point of bringing testability into the picture is that I firmly believe testability is a very good thing to aim for, much as Jeremy so eloquently stated. If you do not agree with that, or not as much as we do, then our opinions downstream will surely differ. But playing with the idea that you do agree with us, then potentially performing what you want to label as speculative design to support/enhance testability also becomes a very good thing.

  7. dhayden says:

    Greg,

    Thanks for the clarification.

    Just to make sure we are clear, I am not arguing CSLA, ActiveRecord, vs. DDD in this post. My example of CSLA was only to demonstrate how one can get principles to support any conclusion. My post wasn’t to discuss the merits of static finder methods in Active Record, for example, vs. a Repository in DDD.

    I think we are both saying the same thing, but in your case you have cause to really use the repository as they give you indirection that you actually use, versus someone who tosses in a repository just to encapsulate a DAO or O/R Mapper. And, although I understand that you do more than just testing with them, your description of their use sounds a lot for testing purposes, but that is obviously important.

    I think the fact that one adds infrastructure to support testability is obvious and necessary, and my point is to mainly focus on avoid un-necessary loose coupling based on future-proofing with no real requirements. As I said, others like yourself have used Repositories to their fullest, but I haven’t got much out of them. From what I can tell, the functionality you put in repositories are in my application controllers, etc., so it is probably just a difference of semantics.

  8. Greg says:

    I am far from saying that I use them only for testability … I am saying that it is a great by product and the alternative is a constant pain as the static method based systems do not support polymorphism.

    DDD p 152

    Repositories have many advantages including the following.

    -They present clients with a simple model for obtaining persistent objects and managing their life cycle
    -They decouple applications and domain design from persistence technology, multiple database strategies, or even multiple data sources
    -They communicate design decisions about object access [aggregate roots]
    -They allow easy substitution of a dummy implementation, for use in testing (typically using an in-memory collection)

    I use them for many things, as I stated I use them for testing (in memory collections). I also have in applications different versions of repositories (example a domain that runs with a prevalence layer and with an OR mapper depending on client configuration of the repositories).

    The static method methodology is doomed to fail in this case as you have lost polymorphism it is not feasable to have a domain run concurrently in multiple contexts unless whatever mechanism you are using in your static methods supports both contexts. The moment you start trying to reuse a domain you tend to end up with multiple contexts (like in the example of needing to work with both a prevalence layer and an OR mapper). Unit testing will often implicitly bring multiple contexts as you do not wish to hit the database (an OR mapper hitting an xml file is kind of a solution but I prefer the repeatability of having actual repositories who have their data loaded through code as depending on the xml file brings in another dependency for tests to work.

    Another nice side benefit dealing with testing is that I have a very clear line when it comes to integration vs unit testing. All of the domain is unit tested with in memory versions of the repositories (fast, easy setup/tear down, etc). My integration tests for my domain (data access) are simply making sure that my repositories work correctly (i.e. the tests I wrote for my abstract repository contract but run against a real data source). Providing my domain works with the in memory versions correctly and the repositories have been tested against their actual data source I can reasonably expect that the domain will now work with the data source after running my integration tests (providing the tests reasonably express the behavior of the repository)

    While my repositories are fairly thin objects to say an OR mapper they also give me the ability to change the data source etc that my domain is running against without having to change the domain itself.

    I think it should be noted here that the only real difference is the pulling out of the data access parts to another object to allow for polymorphism which in my mind is a good thing. I guess my reason for using repository is that it is very little extra code to begin with (putting the data access methods in their own class (following the single reponsibility principle) and creating an interface). By spending this very small amount of work up front I rid myself of the huge pain of having to try to support multiple concurrent contexts without it (which would in many cases require a source branch or lots of nasty if statements). While in many cases I may only use a single repository as my OR mapper handles things for me, it sure is nice to have the insurance policy of being able to change it if I have to, the benefit of this will only be recognized when you have to do it (remember that you have type coupling on all of those static methods in every client that accesses your domain).

  9. dhayden says:

    Greg,

    BigJimInDC does bring up a great point, but I don’t remember reading in Eric Evans’ and Jimmy Nilsson’s book anything about the repository be used just for testability. I don’t need the repository to use stubs for stubbing out subsystems like a database. I also don’t need the repository to support the concept of aggregates.

    Are you essentially using the repositories for testing purposes? One set of repositories for tests and another set for production? They essentially become a thin layer of indirection (pass-thru) between a database and say an XML file? Are they doing anything else?

  10. Greg says:

    David,

    BigJimInDC brings up a great point. I will pick on your respository example.

    For the repository example it is also important to note that there is a trade off to something like active record (especially in testability and reusability). One of the great bits about the repository pattern is that you can easily mock out a repository for when you are unit testing. A repository shows an outward contract of holding all items in memory (just make it actually hold all items in memory). Since it has been externalized you can quite easily make a repository that deals with other types of sources (as an example I have had repositories that deal witha prevalence layer) I can change them out that deal with repositories that use an OR mapper and most importantly I can use both versions without requiring a branch of the source.

    Aside from that the existence of a repository shows a design decision on that object. Namely that the object is considerred an aggregate root. With something more like what Rocky subscribes to every object is generally considerred an aggregate root (they also generally match 1-1 with database objects). I will stay away from the impedance discussion but I would guess that you are most likely sufferring from some problems as the other methodologies push you down a 1 model “solution” and frankly the database model is generally a poor model for a domain.

    “that suggests using a separate persistent object in another layer.”

    Or using the repository pattern and externalizing the accessor methods to another class so things such as polymorphism can be used on them.

    I also think its great that someone brings up the single responsibility principle since active record (or Rocky’s suggestion) as a whole is a violation of it. If you are following single resposibility you will end up with the repository pattern.

  11. jmiller says:

    David,

    I hear what you’re saying. I know it’s dogmatic, but I’d say that “Future-proofing” is best accomplished through squeaky clean code with a high degree of testability and automated test coverage. The odds of making any change easily are better that way than backing yourself into a corner by enabling only a certain kind of future change through pluggability. Besides, with clean, tested code you can always back in an extensibility point later.

    Good post, good discussion.

  12. dhayden says:

    Jeremy,

    Great point. It mainly happens with speculative design, but also with customers saying “at some point we will want feature xyz.” It is easy to get caught up in development for a future feature that may never really happen. Adding the extension point for that future feature can be a big piece of development work as well and slow up v1.0.

    I am reminding myself here that one has to build the code for now and add extensibility later as necessary. I know this. It is just easy to slip into old habits.

  13. dhayden says:

    BigJimInDC,

    I didn’t reference testing because my focus was on the evils of “speculative design” as Jeremy so elequently puts it. I see a lot of code that offers loose coupling and abstraction based on no real requirements. Various classes and interfaces being added based on pure fabrication and indirection that only muck the design and make it difficult to understand and overly complex to maintain. The intentions of making the application flexible and using design patterns where a problem doesn’t exist can cause headaches. This is so easy to slip into that I caught myself doing it during development and “scolded” myself :)

  14. jmiller says:

    David,

    How much of your problems with pluggability leading to unnecessary complexity are related to speculative designing upfront? If you’re trying to guess or plan for extensibility upfront it’s easy to go astray or overcomplicate things. What I’ve found is to just let the need for testability lead to the abstraction points. We put abstracted interfaces in every place where it’s convenient to substitute a mock object in tests. I think we actually get better extensibility now as a side effect of TDD than I did on previous projects where I obsessed over putting in the “right” extensibility points.

    I think that extensibility is going to follow from being religiously fastidious about assigning responsibilities with the Single Responsibility Principle.

  15. BigJimInDC says:

    Dave,

    So this is what I’m trying to figure out. In my exploration of TDD (I’m talking mostly test-driven, but not necessarily test-first), the CodeBetter blogs have been invaluable. So when I came across this post of yours, the confusion started to set in. Here’s why. I did not see the word “test” or “mock” anywhere in your post.

    From everything that I have uncovered about loose coupling and high cohesion, it has been all about testability. IMHO, one of the major short-comings that I had prior to the past 2 years of research into TDD is a complete lack of understanding about how to use interfaces, and even why they really needed to exist. “The day the light shown” was the day I finally came across mock objects. I still have not used them in practice, but I feel that I actually “get it” about them, and loose coupling, and high cohesion, and dependency injection, etc.

    I fully agree with your statements on over-architecting or over-engineering your solution for the sake of being “pluggable” without the requirements to do so. BUT IMHO, if you are doing so for the sake of testability, then loose coupling principles becomes invaluable.

    Thoughts???

  16. dhayden says:

    Fregas,

    I think your point about viewing persistence as a service is a valid and recommended strategy for many applications. I also understand Rocky’s viewpoints about the value of encapsulating the persistence logic inside the business object.

    I think the interesting thing is that you can find various principles, best practices, and reasons for doing one or the other. If the architecture could go either way, I typically go for the easiest to maintain if there happens to be one.

  17. dhayden says:

    Arnon Rotem-Gal-Oz,

    I do re-use code from existing applications when possible and try to write code so it is reusable. I also use code generation, 3rd party components and open-source solutions when applicable and appropriate.

    I have found, however, that when doing this one can overly complicate a customer’s solution using this cookie cutter approach. If I am the only person who will maintain the application this might be fine, but another developer might find the “overly-architected solution” difficult to maintain and understand.

    Since I mainly come in and develop the application and don’t have the burden of always maintaining it, I am very sensitive about keeping the software very maintainable, lean, and easy to understand.

    However, I totally understand your points and agree with the strategy.

  18. Fregas says:

    Having worked with CSLA, I have to say I abandoned Rocky’s approach, even though I favored it at one time. Having all the data access code in or even called by business objects really bloats them. It is possible to get the “High Cohesion” and the Encapsulation at the same time, depending on what framework or O/R Mapper you are using. For example, in the Wilson O/R Mapper, your objects’ data can be neatly encapsulated behind public properties and methods, and only by reflection or thru implementation of a specific interface does the object reveal its internal data to the O/R Mapper engine (which in this case is effectively the DAL) for persistence purposes. So your data is generally not exposed by data access methods or public fields/properties but resting safely behind the business layer. Finally, the business object can respond to events that tell it when it is about to be inserted, updated, deleted, etc. so that it can perform business logic related to persistence, but it doesn’t necessarily have to know about how its being persisted, such as the drivers, sql statements, or database used.

    So in my opinion, this meets both principles: The business object is well encapsulated, by limiting exposure of its internals ONLY to the DAL/ORM Layer. The business object also has high cohesion, in that it only handles its own business logic, and not all the mechanics of data access and persistence.

    Of course, there is no reason you have to use an O/R Mapper to meet these needs, you could write your own BLL and DAL using this kind of structure.

    Other than that, I agree with you. There is no point in every one of my domain classes to implement a matching interface detailing all their properties, and have each domain class that references another do so only thru that interface rather than by concrete types. That level of flexibility/loose coupling just isn’t necessary in most cases. But its good to have the tools and the know how for when it IS necessary or useful.

    Most OO pattern books tend to tell you the trade-offs in using a pattern as well. You have to make a judgement call when you apply a pattern, and not just use it repeatedly for its own sake. Thats kind of the “pre-mature” optimzation/abstraction evil.

  19. dhayden says:

    Great point, Scott.

    The application layer tends to be rather transaction script by definition, but it mainly delegates the work to domain objects. I try to keep it as thin as possible and delegate most, if not all, of the work to domain objects and lower layers. If it gets too thick, it does become the slippery slope to the “Anemic Domain Model.”

    In my case, the repositories were doing almost nothing, so their loss doesn’t make the application layer much thicker, and I find the code easier to maintain, understand, and re-use – until I learn new techniques that is :)

  20. To add to what Scott said
    1. Do you always start new projects as if it is the first time you code – don’t you have anything you take from your previous ones? You don’t want to future-proof just for kicks but if you know (looking at past projects) that you can reuse something, or know that this is something that is more volatile etc
    .
    2. Software spends much more time in maintenance that in development (if it is any good) – again you don’t have to make your solution ever so flexible (to the point it would take ages to complete and be overly complex) but if you look at your past projects there are probably some conclusions you can draw as to what get changed more often

    Arnon

  21. ScottBellware says:

    If you eschew the Repository concept, or any domain-specific data access interface, do you end up with a collection of transaction scripts that contain logic that might otherwise rest cohesively in domain objects? Is this the slippery slope to what Fowler called out as the “Anemic Domain Model” design smell?

  22. Nice article – welcome back.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>