Last week I was presenting a session covering some practices and tooling we were using to build NDepend at BuildStuff Lithuania 2013. One of the essential practice we are using is Code Contracts and really I was surprised when I asked who was using Code Contracts in the audience. Not even 5 hands up in like 150 pro developers, that are among developer++ since they attend professional conference.
So I do really hope with this blog post to convince you, dear reader, that you should use Code Contracts in order to drastically reduce the number of bugs that hit production and to reduce the overall maintenance cost of the code base you are working on, by like an order of magnitude.
I’d like to underline that Code Contracts is a coding practice not an implementation. In .NET there are two main ways to define a contracts : through the Microsoft Code Contracts Framework and through System.Diagnostics Debug.Assert(). These are the two main alternatives and I’ll discuss pro and cons of both later on.
What is a contract
- A contract is an assertion in the code.
- If a contract is violated (hence if the condition asserted is false) we consider that there must be a bug.
The important thing here is that if a contract is violated, you (the developer) should be notified of it somehow, to be able to be aware of the bug and get information to fix it.
Another important thing to note, often not well understood, is that if a contract is violated, you shouldn’t try to recover your application. In such situation the application state is certainly corrupted and there is nothing to do about it, except being agile and humble, and try to fix the bug as soon as possible for future executions.
A very common contract, is when one asserts that a method parameters is not null. With MS Code Contract syntax it looks like:
With Debug.Assert() it looks like:
Do you notice how a contract is also a form of documentation? Everytime I look at a method without contracts I wonder what values are acceptable. For example with the method below I can certainly imagine that the one that has written this method didn’t expect str to be null, but this reasoning is just a guess and maybe the null value is acceptable and the method has just a bug.
I admit that I am a bit extremist, if a null value is acceptable, I put a comment explicitly stating that null is valid. But since I think that null values should be banished, it is pretty rare.
But I digressed. While contracts in C# are I’d estimate 66% about avoiding NullReferenceException, there are plenty of situations where you want to assert something else about your states:
- str.Length > 2
- str == ‘.’
- i > 10
- validate an invariant state inside a loop
What is NOT a contract
Often developers are doing something like that and say it is contracts:
But personally I don’t consider that throwing a common exception is a contract. I have several reasons for that:
- It is perfectly legal to throw an exception without asserting that there is a bug.
- An exception can be caught, and hence you might not be notified of it.
- Moreover, when an exception is caught, the application can try to recover, and it can be pretty dangerous to work with data potentially corrupted by a bug.
In a word, an exception is a behavior of your program, it is not a contract! A contract is not a behavior, it is an artefact around your code like a unit-test for example, but wait…
Unit-Tests vs. Contracts
…a unit-tests is made of assertions right? Basically a unit-test define a state, throw this state to a portion of your code, and then assert something on the result. If a unit-tests assertions is violated you can consider that there is a bug (in the application, or in the unit-test). And certainly if a unit-test assertion is violated, you’d like to be aware about it. You might even want to fail a build for that, right?
For all these reasons, I consider that contracts and unit-tests are pretty much the same thing. Both are assertions about your code states obtained while running the code. Who care about when and how the code is executed (debugging time, unit-tests time, smoke tests time, production…), remember the important thing is to be notified of assertions violations, because it means that there is a bug.
Notice a corollary: When a unit-test exercise a code that contains contracts, if a contract is violated you should be notified. Such situation means that there is a bug, either in the exercised code, either in the test code, but there is a bug somewhere and tests must not violate contract! For example it means that if your method asserts that the first argument is not null, then you shouldn’t write a test passing a null reference for this argument.
In 2013 every educated developers do write unit-tests, but almost none of them write contracts, hence this post! We are very very few claiming that contracts and unit-tests are the same thing. I really do hope the situation will evolve in the future because really the benefits of using both massively are outstanding!
Benefits of Contracts
Remember, when a unit-test fails or when a contract is violated it means that there is a bug! Frankly, the opposite is almost true. After using massively contracts and and unit-tests for years, I can say that the absence of assertions violations, pretty means that you have no bugs. Of course there are still a few minor pesky bugs. But by using massively contracts and unit-tests (with very high coverage), maintenance cost is cut by an order of magnitude, really.
Another point that few realizes, is that unit-tests is all about code covered by tests. You can write 1.000 unit-tests and still cover only 10% of the code to be exercised. The number of unit-tests is meaningless. What matters is the percentage of code covered by tests, and also the number of assertions verified. And covering code full of contracts with your unit-tests, is a unique way to multiply the number of assertions checked. So a benefit of contracts lies also in the fact that it makes much higher the number of assertions checked at unit-tests time. By using contracts you make unit-tests much more effective. And this is cool, because while unit tests takes time to be written, contracts takes no time to be written once you took the habit.
At NDepend we use contracts also to test the UI. We have a single huge integration test, that triggers everything that can be triggered in the NDepens UI (and I hope we can agree it is a sophisticated UI). This integration test takes 30 seconds to run. It checks literally millions of assertions, and still this integration test itself just contain just a few assertions. So in practice, contracts are also an excellent and original way to test automatically UI.
And finally, we saw that contracts are powerful documentation.
I do really hope that these benefits sounds good to you! Yes, code contracts work, and they work really well!
Contract and tooling
Contracts makes the job of static analyzer tool easier. For example below, a simple asserts makes Resharper clever enough to notify you of dead code, or of a luring NullRefException.
This benefits isn’t an anecdote. I believe that part of the next decade big things for developers, will be static analyzer detecting non-trivial bugs in the IDE. As you can see Resharper already does it pretty well. But it still doesn’t try to check for states from the chain of callers methods and the chain of called methods (yes it does it for .NET Fx methods and few other cases which is pretty cool, but it is still limited). Contracts is what will make such tool clever enough why? Because with contracts you drastically reduce the number of states the static analyzer have to check, and reducing numbers in an exponential algorithm makes all the difference between something that works practically and something that works theoretically. And hence the tool can focus and report only real buggy cases.
In the same vein, there is also the great Pex tool. I have huge hopes in it. For now there is just a lite version for VS2012 and VS2013, but I guess much more is coming.
Also, soon the C# and VB.NET compiler will be opened thanks to the Roslyn project. It will be easier than ever to write static analyzer because we’ll get all the syntax analysis for free. In other words Roslyn will let tools vendor focus on semantic analysis and this sounds very promising to me.
As a side note, one especially cool situation where I use contract daily, is when you switch over an enumeration with say 3 values. Instead of having 3 cases, one for each value, I like to have 2 cases and one default case, asserting the third value in the default case: The benefit is that the switch flow is free from any ambiguity. Both the C# compiler and Resharper understands it and if all cases returns, it won’t ask for a return statement after the switch ! Notice that the compiler magic isn’t coming from the contract itself. But the cool benefit of the contract is that in the default block we can be certain of the status argument value.
MS Code Contract vs. Debug.Assert()
So what should you use? What do we use at NDepend after years of practices? Actually we do use both technologies, dealing with their pros and cons. MS Code Contracts have huge advantages over Debug.Assert():
- The framework is nicely done, with many well named methods to handle all scenarios (Require(), Ensure(), Invariant(), ValueAtReturn()…).
- The framework let’s put contract on interfaces members.
- The framework works nicely with documentation tooling, and there are facilities to include contracts assertion inside the doc itself.
- Some checks can be made at static analysis time, to chase automatically for pesky bugs like finding case where a null value can be passed to a method argument that has a contract.
But in practice MS Code Contracts has a single one significant drawback that doesn’t make it competitive with Debug.Assert(). In most scenarios: it double and even triple compilation time, because by using MS Code Contracts assemblies get rewritten once compiled by the C# or VB.NET compiler. This rewrite is needed because MS Code Contracts is doing not natural act on code (like executable code on interface declaration, that needs to be weaved inside the impl methods).
Because long compilation time is not acceptable, in practice, what I’d recommend, is to use contracts on the public surface API of your application (like NDepend.API for us, just one small assembly), and Debug.Assert() everywhere else! Apparently the C# team is considering compiling MS Code Contracts directly in C#6 (source Wesner Moise). This would be really cool, and would certainly make me change my mind and forget definitely about the usage of Debug.Assert().
You’ll notice that in the list of advantages I didn’t put the fact that with MS Code Contracts, assertions can be checked during production. Indeed Debug.Assert() assertions are removed from assemblies compiled in Release mode. Personally I consider this like an advantage because:
- Keeping thousands of assertions checks in production code, can and actually do, significantly impact performance.
- I don’t consider clients like testers. But still clients do experience bugs, crash and exception reportd (typically automatically reported). Our job is to avoid the client to experience any bug and from my experience, a well detailed exception report is as good as a contract report in 80% of situations.
Btw, one detail point I didn’t mention, is that a MS Code Contract assertion violated throw a ContractException (which is not a public type) that can hardly be caught. And if you think about it, it makes sense. Jon Skeet explains it well in his StackOverflow answer.
As you certainly know, a Debug.Assert() assertion violated triggers a scary blocking dialog that lets attach a debugger. It is especially convenient for notifying a bug and for starting investigating when running code compiled in DEBUG mode.
A bit of History
DbC or Design by Contract has been invented by Dr Bertrand Meyer in 1986 when he specified the Eiffel language. So contract is not something new and I regret so much that when Microsoft built .NET at the end of the 90’s, they didn’t considered including contract in the IL and Common Type System. Especially taking account the fact that M.Meyer was working closely with the .NET team at that time.
As we saw, Contracts is closely related to null reference check, which still represent like half of the bugs on earth! We know that the null reference concept was deemed as a billions dollars mistake by Tony Hoare its inventor. Here also I regret that the .NET Common Type System doesn’t come with a way to statically enforce non-null reference. I’ve read somewhere that it was the biggest Anders Helsberg regret (if anybody has an url on that… cannot find it anymore, but I am sure I read it … ok it is here thanks Antonio for the link ). And I remember in 2004 when I was MVP C# trying to convince Anders to enhance the CTS with non-null references. The one and only argument against wasn’t so much about feasibility, but about the fact that it would create a tsunami of breaking changes in the .NET Framework 1.1 and one very cool thing of .NET is that breaking changes are pretty limited!
Pff…. it was a much longer post than what I expected, but if you read this conclusion I really do hope you’ve learn something, and that you are curious about using Code Contracts from now to see how much benefits it can bring.