Sponsored By Aspose - File Format APIs for .NET

Aspose are the market leader of .NET APIs for file business formats – natively work with DOCX, XLSX, PPT, PDF, MSG, MPP, images formats and many more!

Code Contracts – TDD in a DbC World

Lately, I’ve been talking about the new feature coming to .NET 4.0, Code Contracts, which is to bring Design by Contract (DbC) idioms to all .NET languages as part of the base class library.  Last week, I attended QCon, where Greg Young, one of my CodeBetter cohorts, gave a talk titled "TDD in a DbC World" in which he talked about that they are not in conflict, but instead are complementary to a test first mentality.  Both improve upon the usage of the other.

One of the questions that came up during some of my blog posts and even at QCon for that matter.  Many times, DbC gets pegged as being in conflict with TDD, and that many people who espouse the use of DbC aren’t doing TDD, and I don’t think that’s necessarily true.  Let’s look at where each fits into the general picture.

 

Where Does Design by Contract Fit?

The first thing we need to remember about DbC is that it is specified to create constraints through the use of contracts.  This gives us the ability to specify acceptable input, state constraints of invariants, and output that can be guaranteed.  Not only to specify this, but also statically verify this behavior through the compiler, as well as at run time.  That’s the most important part is the proving.  Think of it as as a nun from Catholic school who will come up behind you and rap your knuckles should you violate the contract, which is a constant reminder that you are not living up to your end of the deal.

DbC, when applied properly, serves as documentation.  The edge cases of your tests can then melt away as you realize what is and is not acceptable input, state and output from the API.  At this point, many of those test cases on constraints for input, output and state could be eliminated due to it being superfluous with our contracts in place.  Looking at the following code using Spec# to write these preconditions, what’s the value of this test?

// Functionality under test
public void Add(Employee employee)
  requires employee != null otherwise ArgumentNullException;
  ensures Count == old(Count) + 1;
{
  // Adds employee here
}

[Fact]
public void AddWithNullEmployee_ShouldThrowException()

  // Assert
  Assert.Throws<ArgumentNullException>(
    () => employees.Add(null));
}

 

The question then comes up, what are we actually creating these tests for anyway?  Edge cases and constraints, or actual business value?  What I’m not advocating is deleting this test, but de-emphasizing its role using DbC constructs and to utilize the compilers warnings as guides.  This test can also serve as documentation going forward.

 

Thinking About BDD

What we need to think about when it comes to TDD, when done properly is testing behavior of our system.  This is where the real business value is, and not in the constraints, and the edge cases, so writing tests for those constraints doesn’t make sense.  Maybe then it’s better to talk about this as Behavior Driven Development (BDD) because ultimately, that’s what our tests are flushing out with our design and leave the word Test Driven Development at the door as the name really doesn’t fit.

What we need to test is behavior and the rich interactions that take place, which is something that DbC can’t necessarily ensure.  As well, the contracts can’t verify the business value of said test either.  The rule should be, DbC for your constraints, and TDD for the business value.

Now, how does it fit together?  Well, let’s walk through a hyperbole of a a situation to see what is DbC and what is TDD/BDD.  For example, let’s take the example of an application that launches a nuclear missile and destroys some land mass.  What in there is DbC and what is TDD?

// DbC
public TargetResult LaunchMissile(Target target)
  requires Target != null otherwise ArgumentNullException;
  ensures Button.IsPressed;
  ensures TargetResult != null;
{
  // Some logic to destroy civilization
}
 

What we’ve defined for DbC is our constraints for our given functionality under test.  Then what might TDD be in this situation?  Well, let’s look at the two major ways of thought around the issue of being either a classicist or a mockist in terms of our testing style.

// Classicist
[Fact] 
public void LaunchMissile_ShouldDestroyTarget() 

  var target = // Sets up target
  var missile = new FakeMissile(); 

  var result = launcher.LaunchMissile(target); 

  Assert.True(missle.WasLaunched);
  Assert.True(result.IsDestroyed);
}

// Mockist
[Fact] 
public void LaunchMissile_ShouldDestroyTarget() 

  var target = // Sets up target
  var missile = new Mock<IMissile>(); 
  missile.Expect(x => x.SetTarget(target));
  missile.Expect(x => x.Launch()); 

  var result = launcher.LaunchMissile(target); 

  missile.VerifyAll(); 
}

 

Let’s move forward with the discussion towards some general ways of tackling the issues described here.

 

Moving Forward

The question may arise how to tell the difference between a constraint and a behavior?  Here’s one example that might make it more clear.  If an integer that is passed into my functionality under test is out of range and exceptional, then it is a constraint.  If, however, that an integer out of range is ok because another rule might apply instead, then it is behavior.

Moving forward with development, how do we do it?  As I mentioned before, we have two options here:

  1. Define contract first for API level designs for well known behaviors, then define tests, and then finally implement the logic to get the test to pass.
  2. Define tests first, implement the logic, get to pass, then refactor constraints to the contract.

For most situations, we’ll go with the second option and then over time, refine and then narrow the contracts.  Then if we can, we can generalize these contracts through interfaces to define constraints that apply to all implementations. 

 

Conclusion

As developers start to discover the upcoming Code Contracts for .NET 4.0, it’s important to understand not only Design by Contract, and TDD, but how they can complement each other.  For business value, we as developers need to flush out our design through the use of TDD, and we can refactor into using contracts for our constraints over time.  With these methodologies in place, the edge cases will melt away and help us ensure DRY and write more concise code directly having business value.

Once again, pick up the latest version of the Code Contracts for .NET and give the team feedback.

This entry was posted in C#, DBC, Spec#, TDD/BDD. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • http://colinjack.lostechies.com Colin Jack

    @Greg
    No I understand you, no need to be patronizing! :)

    What I’m saying is that yes the compiler will warn me and that I love, however I’d also want the option of being able to write a spec that ensures a particular contract is in place (if I do X I get Y). If I can’t do that then refactoring of the code (not the specs) becomes less safe. In many cases that would be OK but I’ve used DBC in situations where failure to ensure a precondition was met could cause a real mess for the user/developer/system.

    In those cases I want the tests as a safety net aspect which was why above I suggested that we might want another syntax for asserting that contracts are in place.

    Take the current situation, I write code to throw a particular exception if I pass in a particular value (or range of values). I don’t write that tests just to get my code coverage up, and although it does fulfill the tests as documents aim what actually helps is that if I remove that precondition the test will fail.

    “Per the rest, changing contracts is EXACTLY the same as changing/deleting tests.”

    Not really, your tests currently act as a safety net for your refactoring. I understand you can screw this up but generally I don’t find people do that.

  • Greg

    btw: I am packing my camera as I am doing my talk again at devteach … maybe the video can add to the discussion …

    Cheers,

    Greg

  • Greg

    @colin jack you seem to be missing something big …

    If you had a contract for something let’s say that a parameter could not be null … you couldn’t write a test to show it failing. The compiler would complain that it broke the constraint!

    Per the rest, changing contracts is EXACTLY the same as changing/deleting tests.

  • http://colinjack.lostechies.com Colin Jack

    @Matthew
    I didn’t say you did suggest we delete the tests, but Greg pointed out tests might no longer compile (if I understand correctly).

    So I’m getting what you’re saying, and I like the idea of contracts but if the contracts themselves can’t be “tested” then refactoring becomes risky (which might be fine in 90% of cases, but….).

  • http://podwysocki.codebetter.com Matthew.Podwysocki

    @Colin

    At no point did I mention that deleting a test would be the best option. Instead, you could leave them in there and still receive compiler warnings, or pragma them out as you go along.

    The point I’m trying to make here is that as we go along, there are certain constraints that we can extract and put into contracts to have statically checked as compiler errors/warnings. Note that I didn’t say business behavior, but API level behavior such as when adding, the count goes up, etc.

    This also serves nicely as documentation for the constraints in Intellisense (or at least should be). To gain the insight of 3rd party libraries like this as for the behaviors is quite appealing in a way.

    I’m enjoying the debate here and that’s what this post was intended to seek.

    Matt

  • http://colinjack.lostechies.com Colin Jack

    > I can’t recall a single instance where I accidentally deleted a contract. < Delete/updating contracts that do not themselves have tests would, in fairness, leave me a little uncomfortable but thats probably influenced by my test-first mentality. I'll bow to your Eiffel experience here I guess! :)

    > A much bigger concern in my mind would be the effort of keeping unit and contract tests in sync as refactorings take place. <

    Refactoring is supported by tests, so I don’t find this “redundancy” at all un-natural.

  • http://colinjack.lostechies.com Colin Jack

    @ colin
    > Can’t the same be said for the implications of
    > deleting a test?

    No, not really. If I have a really key contract and a test for it then if I delete the test then it is unfortunate but the behavior is still correct. If I then also mess up the contract then I’m out of luck.

  • http://podwysocki.codebetter.com Matthew.Podwysocki

    @Steve,

    I have to disagree as well in regards to caring less about TDD if you have a good implementation of DbC. I feel that TDD approaches the better design from the outside and helps flush out the design of your APIs as a whole. You cannot get that with DbC alone. There is immense benefit to coming at it with the test first mentality because you’re thinking of the basic principles of design. I prefer to refactor and then further refine my contracts as I go along.

    Matt

  • Greg

    @Steve

    >Truth be told, give me a good implementation of DbC and I could really care less about TDD.

    I disagree here. There is still benefit in writing code from the outside in in terms of API design etc.

    Cheers,

    Greg

  • Greg

    @ colin

    I think the point is if you accidentally delete the contract, or delete it without considering implications, then you’re only safe if you have tests in place to catch the mistake.

    Can’t the same be said for the implications of deleting a test? Frankly you shouldn’t be doing either without a large understanding of what you are doing. At least the prover will catch places in some cases that are screwed up when you delete a contract, you have no such safety net with tests.

  • Steve Thompson

    >>
    I think the point is if you accidentally delete the contract, or delete it without considering implications, then you’re only safe if you have tests in place to catch the mistake.
    <<

    It has been some time since I last worked with Eiffel, but in those days I was making a large number of refactorings to my projects, and I can’t recall a single instance where I accidentally deleted a contract. I won’t say that this could never happen, as it obviously could, but I think this is remote enough that it doesn’t justify the redundancy of a unit test.

    A much bigger concern in my mind would be the effort of keeping unit and contract tests in sync as refactorings take place.

  • http://colinjack.lostechies.com Colin Jack

    > Defining the contracts *is* defining the tests.

    I think the point is if you accidentally delete the contract, or delete it without considering implications, then you’re only safe if you have tests in place to catch the mistake.

  • Steve Thompson

    >>
    Define contract first for API level designs for well known behaviors, then define tests, and then finally implement the logic to get the test to pass.
    < <

    Defining the contracts *is* defining the tests. You do it as you create methods, prior to defining their implementation, as a way of stating what must be true before your algorithm can do its job, and what must be true once this algorithm is finished. In so doing you are clarifying the behavior of the code much better than if you were to create some other module and start down the road of writing a traditional unit test.

    Truth be told, give me a good implementation of DbC and I could really care less about TDD. DbC gets me thinking about the kinds of things that make a well rounded class, such as the queries I would want to get answers to in pre/post conditions. This is a very important point that leads to very well factored code that is easy to read and think about, and I find that with TDD this is usually not the case in teams with which I've worked.

    I feel that with a great implementation of DbC, a la what I could achieve in Eiffel years ago, unit testing will fall out of vogue. Not only is it overly brittle (especially when coupled with mocks) but it has always meant cognitive dissonance because tests and code are not integrated. In the final analysis DbC is easier to write, makes the purpose/assumptions of an algorithm clear, and is a more convincing proposition for most programmers.

    >>
    Assert.True(missle.WasLaunched);
    Assert.True(result.IsDestroyed);
    <<

    And of course, this could have been stated equally well as part of your postcondition, rendering the ‘classicist’ unit test useless, unless there is something I am missing.

  • http://scottic.us Scott Bellware

    Greg,

    > If the constraint were a valid business concern that required a test to
    > express then don’t make it a requires and write a test for it.

    What kind of declarative DbC constraints are left in the code when the above is applied as a rule?

  • http://colinjack.lostechies.com Colin Jack

    First off I haven’t played with the new contracts code so I could be well off, but I’ll comment none the less :)

    I think Scott raises a very valid point, sure TDD/BDD is about driving the design but the tests/specs also act as a safety net for refactoring.

    During refactoring it would be all too easy to accidentally remove a constraint. Sometimes this wouldn’t be a big disaster but sometimes it could cause real issues so I think the tests will still have value in some cases.

    Could we not just have a new syntax/verifier:

    Assert.Contract.Disallows(employees.Add(null)).ExceptionIs();

    Not sure this syntax would work well because if there were multiple arguments we’d want to say that we expect an ArgumentNullexception if the *first* argument (or the argument called “employee”) is null but you get the idea.

  • Greg

    Scott you seem to be missing the point that the test would not even compile with the prover.

    The prover would say that the test breaks the constraint and should be fixed.

    If the constraint were a valid business concern that required a test to express then don’t make it a requires and write a test for it.

    Cheers,

    Greg

  • http://scottic.us Scott Bellware

    > what’s the value of this test?

    Same value as if you replaced the new “requires” keyword with an if block.