CodeBetter.Com
CodeBetter.Com
RSS 2.0 via Feedburner
           Do you Twitter? Follow us @CodeBetter

Karl Seguin

developer @ Fuel Industries ottawa, ontario

November 2007 - Posts

  • Foundations of Programming - pt 2 – Domain Domain Domain

    Intruduction
    Starting this kind of series by talking about domain driven design and object oriented programming is rather predictable. At first I thought I could avoid the topic for at least a couple posts, but that would do both you and me a great disservice. There are a limited number of practical ways to design the core of your system. A very common approach for .NET developers is to use a data-centric model. There's a good chance that you're already an expert with this approach – having mastered nested repeaters, the ever-useful ItemDataBound event and skillfully navigating DataRelations. Another solution which is the norm for Java developers and quickly gaining speed in the .NET community favors a domain-centric approach.


    DDD2
    What do I mean by data and domain-centric approaches? Data-centric generally means that you build your system around your understanding of the data you'll be interacting with. The typical approach is to first model your database by creating all the tables, columns and foreign key relationships, and then mimicking this in C#/VB.NET. The reason this is so popular amongst .NET developers is that Microsoft spent a lot of time automating the mimicking process with DataAdapters, DataSets and DataTables. We all know that given a table with data in it, we can have a website or windows application up and running in less than 5 minutes with just a few lines of code. The focus is all about the data – which in a lot of cases is actually a good idea. This approach is sometimes called data driven development.

    Domain-centric design or, as it's more commonly called, domain driven design (DDD), focuses on the problem domain as a whole – which not only includes the data, but also the behavior. So we not only focus on the fact that an employee has a FirstName, but also on the fact that he or she can get a Raise. The Problem Domain is just a fancy way of saying the business you're building a system for. The tool we use is object oriented programming (OOP) – and just because you're using an object-oriented language like C# or VB.NET doesn't mean you're necessarily doing OOP.

    The above descriptions are somewhat misleading – it somehow implies that if you were using DataSets you wouldn't care about, or be able to provide, the behavior of giving employees a raise. Of course that isn't at all the case – in fact it'd be pretty trivial to do. A data-centric system isn't void of behavior nor does it treat them as an after though. DDD is simply better suited at handling complex systems in a more maintainable way for a number of reasons – all of which we'll cover in following posts. This doesn't make domain driven better than data driven – it simply makes domain driven better than data driven in some cases and the reverse is also true. You've probably read all of this before, and in the end, you simply have to make a leap of faith and tentatively accept what we preach – at least enough so that you can judge for yourself.

    (It may be crude and a little contradictory to what I said in my introduction, but the debate between The MSDN Way and ALT.NET could be summed up as a battle between data driven and domain driven design. True ALT.NETers though, ought to appreciate that data-driven is indeed the right choice in some situations. I think much of the hostility between the “camps” is that Microsoft disproportionately favors data-driven design despite the fact that it doesn't fit well with what most .NET developers are doing (enterprise development), and, when improperly used, results in less maintainable code. Many programmers, both inside and outside the .NET community, are probably scratching their heads trying to understand why Microsoft insists on going against conventional wisdom and clumsily playing follow the leader with a 5+ year lag (witness the recent announcement of a MVC pattern slated for 2008)).


    Users, Clients and Stakeholders
    Something which I take very seriously from Agile development is the close interaction the development team has with clients and users. In fact, whenever possible, I don't see it as the development team and the client, but a single entity: the team. Whether you're fortunate enough or not to be in such a situation (sometimes lawyers get in the way, sometimes clients aren't available for that much commitment, etc.) it's important to understand what everyone brings to the table. The client is the person who pays the bills and as such, should make the final decisions about features and priorities. Users actually use the system. Clients are oftentimes users, but rarely are they the only user. A website for example might have anonymous users, registered users, moderators and administrators. Finally, stakeholders consist of anyone with a stake in the system. The same website might have a sister or parent site, advertisers, PR or domain experts.

    Clients have a very hard job. They have to objectively prioritize the features everyone wants, including their own and deal with their finite budget. Obviously they'll make wrong choices, maybe because they don't fully understand a user's need, maybe because you made a mistake in the information you provided, or maybe because they improperly give higher priority to their own needs over everyone elses (a lot like the big screen TV I bought my girlfriend for her birthday). As a developer, it's your job to help them out as much as possible and deliver on their needs.

    Whether you're building a commercial system or not, the ultimate measure of its success will likely be how users feel about it. So while you're working closely with your client, hopefully both of you are working towards your users needs. If you and your client are serious about building systems for users, I strongly encourage you to read up on User Stories – a good place to start is Mike Cohn's excellent User Stories Applied.

    Finally, and the main reason this little section exists, are domain experts. Domain experts are the people who know all the ins and outs about the world your system will live in. I was recently part of a very large development project for a financial institute and there were literally hundreds of domain experts most of which being economists or accountants. These are people who are as enthusiastic about what they do as you are about programming. Anyone can be a domain expert – a clients, a user, a stakeholder and, eventually, even you. Your reliance on domain experts grows with the complexity of a system.


    The Domain Object
    As I said earlier, object oriented programming is the tool we'll use to make our domain-centric design come to life. Specifically, we'll rely on the power of classes and encapsulation. In this part we'll focus on the basics of classes and some tricks to get started – many developers will already know everything covered here. We won't cover persistence (talking to the database) just yet. If you're new to this kind of design, you might find yourself constantly wondering about the database and data access code. Try not to worry about it too much. In the next part we'll cover the basics of persistence, and in following parts, we'll look at persistence in even greater depth.

    The idea behind domain driven design is to build your system in a manner that's reflective of the actual problem domain you are trying to solve. This is where domain experts come into play – they'll help you understand how the system currently works (even if it's a manual paper process) and how it ought to work. At first you'll be overwhelmed by their knowledge – they'll talk about things you've never heard about and be surprised by your dumbfounded look. They'll use so many acronyms and special words that'll you'll begin to question whether or not you're up to the task. Ultimately, this is the true purpose of an enterprise developer – to understand the problem domain. You already know how to program, but do you know how to program the specific inventory system you're being asked to do? Someone has to learn someone else's world, and if domain experts learn to program, we're all out of jobs.

    Anyone who's gone through the above knows that learning a new business is the most complicated part of any programming job. For that reason, there are real benefits to making our code resemble, as much as possible, the domain. Essentially what I'm talking about is communication. If your users are talking about Strategic Outcomes, which a month ago meant nothing to you, and your code talks about StrategicOutcome then some of the ambiguity and much of the potential misinterpretation is cleaned up. Many people, myself included, believe that a good place to start is with key noun-words that your business experts and users use. If you were building a system for a car dealership and you talked to a salesman (who is likely both a user and a domain expert), he'll undoubtedly talk about Clients, Cars, Models, Packages and Upgrades, Payments and so on. As these are the core of his business, it's logical that they be the core of your system. Beyond noun-words is the convergence on the language of the business – which has come to be known as the ubiquitous language (ubiquitous means present everywhere). The idea being that a single shared language between users and system is easier to maintain and less likely to be misinterpreted.

    Exactly how you start is really up to you. Doing Domain Driven Design doesn't necessarily mean you have to start with modeling the domain (although it's a good idea!), but rather it means that you should focus on the domain and let it drive your decisions. At first you may very well start with your data model, when we explore test driven development we'll take a different approach to building a system that fits very well with DDD. For now though, let's assume we've spoken to our client and a few salespeople, we've realized that a major pain-point is keeping track of the inter-dependency between upgrade options. The first thing we'll do is create four classes:

    public class Car { }
    public class Mode { }
    public class Package { }
    public class Upgrade { }
    
    Next we'll fill-in these classes with some safe assumptions:
    using System.Collections.Generic;
    using System.Collections.ObjectModel;
    
    public class Car
    {
       private Model _model;
       private List<Upgrade> _upgrades;
    
       public void Add(Upgrade upgrade)
       {
          //todo
       }
    }
    
    public class Model
    {
       private int _id;
       private int _year;
       private string _name;
    
       public ReadOnlyCollection<Upgrade> GetAvailableUpgrades()
       {
          //todo
          return null;
       }
    }
    
    public class Upgrade
    {
       private int _id;
       private string _name;
       
       public ReadOnlyCollection<Upgrade> RequiredUpgrades
       {
          get
          {
             //todo
             return null;
          } 
       }
    }
    
    Things are quite simple. We've added some pretty traditional fields (id, name), some references (both Cars and Models have Upgrades), and an Add function to the Car class. Now we can make slight modifications and start writing a bit of actual behavior.
    using System.Collections.Generic;
    using System.Collections.ObjectModel;
    
    public class Car
    {
      private Model _model;
      //todo where to initialize this?
      private List<Upgrade> _upgrades;
    
      public void Add(Upgrade upgrade)
      {
         _upgrades.Add(upgrade);
      }
      public ReadOnlyCollection<Upgrade> MissingUpgradeDependencies()
      {
         List<Upgrade> missingUpgrades = new List<Upgrade>();
         foreach (Upgrade upgrade in _upgrades)
         {
            foreach (Upgrade dependentUpgrade in upgrade.RequiredUpgrades)
            {
               if (!_upgrades.Contains(dependentUpgrade) && !missingUpgrades.Contains(dependentUpgrade))
               {
                  missingUpgrades.Add(dependentUpgrade);
               }
            }
         }
         return missingUpgrades.AsReadOnly();
      }
    }
    
    First, we've implemented the Add method. Next we've implemented a method that lets us retrieve all missing upgrades. Again, this is just a first step; the next step could be to track which upgrades are responsible for causing missing upgrades, i.e. You must select 4 Wheel Drive to go with your Traction Control; however, we'll stop for now. The purpose was just to highlight how we might get started and what that start might look like.


    UI
    You might have noticed that we haven't talked about UIs yet. That's because our domain is independent of the presentation layer – it can be used to power a website, a windows application or a windows service. The last thing you want to do is intermixed your presentation and domain logic. Doing so won't only result in hard-to-change and hard-to-test code, but it'll also make it impossible to re-use our logic across multiple UIs (which might not be a concern, but readability and maintainability always is). Sadly though, that's exactly what many ASP.NET developers do – intermix their UI and domain layer. I'd even say it's common to see behavior throughout ASP.NET button click handlers and page load events. The ASP.NET page framework is meant to control the ASP.NET UI – not to implement behavior. The click event of the Save button shouldn't validate complex business rules (or worse, hit the database directly), rather its purpose is to modify the ASP.NET page based on the results on the domain layer – maybe it ought to redirect to another page, display some error messages or request additional information.

    Remember, you want to write cohesive code. Your ASP.NET logic should focus on doing one thing and doing it well – I doubt anyone will disagree that it has to manage the page, which means it can't do domain functionality. Also, logic placed in codebehind will typically violate the Don't Repeat Yourself principal, simply because of how difficult it is to reuse the code inside an aspx.cs file.
    With that said, you can't wait too long to start working on your UI. First of all, we want to get client and user feedback as early and often as possible. I doubt they'll be very impressed if we send them a bunch of .cs/.vb files with our classes. Secondly, making actual use of your domain layer is going to reveal some flaws and awkwardness. For example, the disconnected nature of the web might mean we have to make little changes to our pure OO world in order to achieve a better user experience. In my experience, unit tests are too narrow to catch these quirks while they are plainly visible as you create your real UI.

    You'll also be happy to know that ASP.NET and WinForms deal with domain-centric code just as well as with data-centric classes. You can databind to any .NET collection, use sessions and caches like you normally do, and anything else you're used to doing. In fact, out of everything, the impact on the UI is probably the least significant. Of course, it shouldn't surprise you to know that ALT.NET'ers also think you should keep your mind open when it comes to your presentation engine. The ASP.NET Page Framework isn't necessary the best tool for the job – a lot of us consider it unnecessarily complicated and brittle. We'll talk about this more in a later part, but if you're interested to find out more, I suggest you look at MonoRails (which is a Rails framework for .NET) and find out about the new page framework being released by Microsoft in 2008. The last thing I want is for anyone to get discouraged with the vastness of changes, so for now, let's get back on topic.


    Tricks and Tips
    We'll finish off by looking at some useful things we can do with classes. We'll only cover the tip of the iceberg, but hopefully the information will help you get off on the right foot.


    Factory Pattern
    What do we do when a Client buys a new Car? Obviously we need to create a new instance of Car and specify the model. The traditional way to do this is to use a constructor and simply instantiate a new object with the new keyword. A different approach is to use a factory to create the instance:

    using System.Collections.Generic;
    public class Car
    {
      private Model _model;      
      private List<Upgrade> _upgrades;
    
      private Car()
      {
         _upgrades = new List<Upgrade>();
      }
      public static Car CreateCar(Model model)
      {
         Car car = new Car();
         car._model = model;
         return car;
      }
    }
    

    There are two advantages to this approach. First, we can return a null object, which is impossible to do with a constructor – this may or may not be useful in your particular case. Secondly, if there are a lot of different ways to create an object, it gives you the change to provide more meaningful function names. The first example that comes to mind is when you want to create an instance of a User class, you'll likely have User.CreateByCredentials(string username, string password), User.CreateById(int id) and User.GetUsersByRole(string role). You can accomplish the same functionality with constructor overloading, but rarely with the same clarity. Truth be told, I always have a hard time deciding which to use, so it's really a matter of taste and gut feeling.


    Interfaces
    Interfaces will play a big part in helping us create maintainable code. We'll use them to decouple our code as well as create mock classes for unit testing. An interface is a contract which any implementing classes must adhere to. Let's say that we want to encapsulate all our database communication inside a class called SqlServerDataAccess such as:

    using System.Collections.Generic;
    internal class SqlServerDataAccess
    {
      internal List<Upgrade> RetrieveAllUpgrades()
      {
         //todo implement
         return null;
      }
    }
    
    public class Sample
    {
      public void SampleMethod()
      {
         SqlServerDataAccess da = new SqlServerDataAccess();
         List<Upgrade> upgrades = da.RetrieveAllUpgrades();
      }
    }
    
    You can see that the sample code at the bottom has a direct reference to SqlServerDataAccess – as would the many other methods that need to communicate with the database. This highly coupled code is problematic to change and difficult to test (we can't test SampleMethod without having a fully functional RetrieveAllUpgrades method). We can relieve this tight coupling by programming against an interface instead:
    using System.Collections.Generic;
    internal interface IDataAccess
    {
      List<Upgrade> RetrieveAllUpgrades();
    }
    
    internal class DataAccess
    {
      internal static IDataAccess CreateInstance()
      {
         return new SqlServerDataAccess();
      }
    }
    
    internal class SqlServerDataAccess : IDataAccess
    {
      public List<Upgrade> RetrieveAllUpgrades()
      {
         //todo implement
         return null;
      }
    }
    
    public class Sample
    {
      public void SampleMethod()
      {
         IDataAccess da = DataAccess.CreateInstance();
         List<Upgrade> upgrades = da.RetrieveAllUpgrades();
      }
    }
    
    We've introduced the interface along with a helper class to return an instance of that interface. If we want to change our implementation, say to an OracleDataAccess, we simply create the new Oracle class, make sure it implements the interface, and change the helper class to return it instead. Rather than having to change multiple (possibly hundreds), we simply have to change one.

    This is only a simple example of how we can use interfaces to help our cause. We can beef up the code by dynamically instantiating our class via configuration data or introducing framework specially tailored for the job (which is exactly what we're going to do). We'll often favor programming against interfaces over actual classes, so if you aren't familiar with them, I'd suggest you do some extra reading.


    Information Hiding and Encapsulation
    Information hiding is the principle that design decisions should be hidden from other components of your system. It's generally a good idea to be as secretive as possible when building classes and components so that changes to implementation don't impact other classes and components. Encapsulation is an OOP implementation of information hiding. Essentially it means that your objects data (the fields) and as much as the implementation should not be accessible to other classes. The most common example is making fields private with public properties. Even better is to ask yourself if the _id field even needs a public property to begin with.


    Access Modifiers

    As you focus on writing classes that encapsulate the behavior of the business a rich API is going to emerge for your UI to consume. It's a good idea to keep this API clean and understandable. The simplest method is to keep your API small by hiding all but the most necessary methods. Some methods clearly need to be public and others private, but if ever you aren't sure, pick a more restrictive access modifier and only change it when necessary. I make good use of the internal modifier on many of my methods and properties. Internal members are only visible to other members within the same assembly – so if you're physically separating your layers across multiple assemblies (which is generally a good idea), you'll greatly minimize your API.


    Conclusion

    The reason enterprise development exists is that no single off-the-shelf product can successfully solve the needs of a complex system. There are simply too many odd or intertwined requirements and business rules. To date, no paradigm has been better suited to the task than object oriented programming. In fact, OOP was designed with the specific purpose of letting developers model actual systems in code. It may still be difficult to see the long-term value of domain driven design. Sharing a common language with your client and users in addition to having greater testability may not seem necessary. Hopefully as you go through the remaining parts and experiment on your own, you'll start adopting some of the concepts and tweaking them to fit your and your clients needs.

  • Foundations of Programming - Part 1 - Introduction

    Preamble
    A few years ago I was fortunate enough to turn a corner in my programming career. The opportunity for solid mentoring presented itself, and I took full advantage of it. Within the space of a few months, my programming skills grew exponentially and over the last couple years, I've continued to refine my art. Doubtless I still have much to learn, and five years from now I'll look back on the code I write today and feel embarrassed. I used to be confident in my programming skill, but only once I accepted that I knew very little, and likely always would, did I start to actually understand.

    My Foundations of Programming series is a collection of posts which focus on helping enthusiastic programmers help themselves. Throughout the series we'll look at a number of topics typically discussed in far too much depth to be of much use to anyone except those who already know about them. I've always seen two dominate forces in the .NET world, one heavily driven by Microsoft as a natural progression of VB6 and classic ASP (commonly referred to as The MSDN Way) and the other heavily driven by core object oriented practices and influenced by some of the best Java projects/concepts (known as ALT.NET).

    In reality, the two aren't really comparable. The MSDN Way loosely defines a specific way to build a system down to each individual method call (after all, isn't the API reference documentation the only reason any of us visit MSDN?) Whereas ALT.NET focuses on more abstract topics while providing specific implementation. As Jeremy Miller puts it: the .Net community has put too much focus on learning API and framework details and not enough emphasis on design and coding fundamentals.  For a relevant and concrete example, The MSDN Way heavily favors the use of DataSets and DataTables for all database communication. ALT.NET however, focuses on discussions about persistence design patterns, object-relational impendence mismatch as well as specific implementations such as NHibernate (O/R Mapping), MonoRail (ActiveRecord) as well as DataSets and DataTables. In other words and despite what many people think, ALT.NET isn't about ALTernatives to The MSDN Way, but rather a belief that developers should know and understand alternative solutions and approaches of which The MSDN Way is part of.

    Of course, it's plain from the above description that going the ALT.NET route requires a far greater commitment as well as a wider base of knowledge. The learning curve is steep and helpful resources are just now starting to emerge (which is the reason I decided to start this series). However, the rewards are worthwhile; for me, my professional success has resulted in greater personal happiness.

    Design Goals
    Although simplistic, every programming decision I make is largely based on maintainability. Maintainability is the cornerstone of enterprise development. Frequent CodeBetter readers are likely sick of hearing about it, but there's a good reason we talk about maintainability so often – it's the key to being a great software developer. I can think of a couple reasons why it's such an important design factor. First, both studies and first hand experience tell us that systems spend a considerable amount of time (over 50%) in a maintenance state - be it changes, bug fixes or support. Second, the growing adoption of iterative development means that changes and features are continuously made to existing code (and even if you haven't adopted iterative development such as Agile, your clients are likely still asking you to make all types of changes.) In short, a maintainable solution not only reduces your cost, but also increases the number and quality of features you'll be able to deliver.

    Even if you're relatively new to programming, there's a good chance you've already started forming opinions about what is and isn't maintainable from your experience working with others, taking over someone's application, or even trying to fix something you wrote a couple months ago. One of the most important things you can do is consciously take note when something doesn't seem quite right and google around for better solutions. For example, those of us who spent years programming in classic-ASP knew that the tight integration between code and HTML wasn't ideal.

    Creating maintainable code isn't the most trivial thing. As you get started, you'll need to be extra diligent until things start to become more natural. As you might have suspected, we aren't the firsts to put some thought into creating maintainable code. To this end, there are some sound ideologies you ought to familiarize yourself with. As we go through them, take time to consider each one in depth, google them for extra background and insight, and, most importantly, try to see how they might apply to a recent project you worked on.

    Simplicity
    The ultimate tool in making your code maintainable is to keep it as simple as possible. A common belief is that in order to be maintainable, a system needs to be engineered upfront to accommodate any possible change request. I've seen systems built on meta-repositories (tables with a Key column and a Value column), or complex XML configurations, that are meant to handle any changes a client might throw at the team. Not only do these systems tend to have serious technical limitation (performance can be orders of magnitude slower), but they almost always fail in what they set out to do (we'll look at this more when we talk about YAGNI). In my experience, the true path to flexibility is to keep a system as simple as possible, so that you, or another developer, can easily read your code, understand it, and make the necessary change. Why build a configurable rules engine when all you want to do is check that a username is the correct length? In a later part, we'll see how Test Driven Development can help us achieve a high level of simplicity by making sure we focus on what our client is paying us to do.

    YAGNI
    You Aren't Going to Need It is an Extreme Programming belief that you shouldn't build something now because you think you're going to need it in the future. Experience tells us that you probably won't actually need it, or you'll need something slightly different. You can spend a month building an amazingly flexible system just to have a simple 2 line email from a client make it totally useless. Just the other day I started working on an open-ended reporting engine to learn that I had misunderstood an email and what the client really wanted was a single daily report that ended up taking 15 minutes to build.

    Last Responsible Moment
    The idea behind Last Responsible Moment is that you defer building something until you absolutely have to. Admittedly, in some cases, the latest responsible moment is very early on in the development phase. This concept is tightly coupled with YAGNI, in that even if you really DO need it, you should still wait to write it until you can't wait any longer. This gives you, and your client, time to make sure you really DO need it after all, and hopefully reduces the number of changes you'll have to make while and after development.

    DRY
    Code duplication can cause developers major headaches. They not only make it harder to change code (because you have to find all the places that do the same thing), but also have the potential to introduce serious bugs and make it unnecessarily hard for new developers to jump onboard. By following the Don't Repeat Yourself (DRY) principal throughout the lifetime of a system (user stories, design, code, unit tests and documentation) you'll end up with cleaner and more maintainable code. Keep in mind that the concept goes beyond copy-and-paste and aims at eliminating duplicate functionality/behavior in all forms. Object encapsulation and highly cohesive code can help us reduce duplication.

    Explicitness and Cohesion
    It sounds straightforward, but it's important to make sure that your code does exactly what it says it's going to do. This means that functions and variables should be named appropriately and using standardized casing and, when necessary, adequate documentation be provided. A Producer class ought to do exactly what you, other developers in the team and your client think it should. Additionally, your classes and methods should be highly cohesive – that is, they should have a singularity of purpose. If you find yourself writing a Customer class which is starting to manage order data, there's a good chance you need to create an Order class. Classes responsible for a multitude of distinct components quickly become unmanageable. In the next part, we'll look at object oriented programming's capabilities when it comes to creating explicit and cohesive code.

    Coupling
    Coupling occurs when two classes depend on each other. When possible, you want to reduce coupling in order to minimize the impact caused by changes, and increase your code's testability. Reducing or even removing coupling is actually easier than most people think; there are strategies and tools to help you. The trick is to be able to identify undesirable coupling. We'll cover coupling in detail in a later part.

    Unit Tests and Continuous Integration
    Unit Testing and Continuous Integration (commonly referred to as CI) are yet another topic we have to defer for a later time. There are two things that are important for you to know beforehand. First, both are paramount in order to achieve our goal of highly maintainable code. Unit tests empower developers with an unbelievable amount of confidence. The amount of refactoring and feature changes you're able/willing to make when you have safety net of hundreds or thousands of automated tests that validate you haven't broken anything is unbelievable. Secondly, if you aren't willing to adopt, or at least try, unit testing, you're wasting your time reading this. Much of what we'll cover is squarely aimed at improving the testability of our code.

    Conclusion
    Although this first part was void of any actual code, we did managed to cover quite a few items. Since I want this to be more hands-on than theoretical, we'll dive head first into actual code from here on end. Hopefully we've already managed to clear up some of the buzz words you've been hearing so much about lately. The next couple parts will lay the foundation for the rest of our work by covering OOP and persistence at a high level. Until then, I hope you spend some time researching some of the key words I've thrown around. Since your own experience is your best tool, think about your recent and current projects and try to list things that didn't work out well as well as those that did.

     

    Posted Nov 26 2007, 08:59 AM by karl with 31 comment(s)
    Filed under:
More Posts