I wrote in a past post about my passion for code 100% covered by tests. High
Test Coverage Ratio is a good thing, Anyway!
To summarize the main motivations exposed in the
- Using code contract coupled with high coverage ratio leads to
- A bug discovered in tested code is easier to fix.
- High test coverage guides mechanically the developer to potential
flaws in the code.
- Code fully covered by test is less subject to code erosion.
An essential note is that whatever the API used to
implement contract (.NET 4 Code Contract, Debug.Assert(…)…)
automatic tests shouldn’t provoke a contract violation. A contract
violation means buggy code. If a test execution violates a contract
either the code or the test or both should be fixed. The idea is that
when practicing full coverage by tests driven development
contracts are utterly important for checking correctness that is not
necessarily checked by tests.
Today, one year after having written this post, I
realize that I am even more extremist concerning full code coverage
by tests coupled with code contract. I attest that the extra-cost of covering all
branches of my code + writing as many contracts as I can is well
balanced by the time and effort saved. A lot of development resource is
saved thanks to the avalanche of flaws discovered early at development
time. I attest that the number of bugs discovered afterthought in
such code is, dare I say it, null! After having written set of
collaborative classes 100% covered by tests I feel the great impression
of a job done, what’s next?
Unfortunately, not all code is covereable by tests
and I wrote on this topic in the past: Dealing
with Code Un-Coverable by Tests
In this post, I explained a solution we used with NDepend to both
segregate and tag code uncoverable by tests.
Making sure that 100%
test coverage ratio won’t decrease in the future
Actually we have a cool trick to keep
coverage ratio to its maximum. In the assembly NDepend.CQL.dll
there is a public attribute class named FullCoveredAttribute.
This attribute can tags assembly, class, struct, method and
constructor. We use this attribute mostly to tag classes 100% covered by
automatic tests. Under NCoverExplorer this looks like:
We did half the job here, to develop a class
100% covered by tests with contracts. But the other half of the job is
to make sure that future changes in the class will be also covered. This
is where the FullCovered attribute comes into play. Code Query
Language (CQL) can deal both with attribute tagging and
code coverage ratio metrics. Hence, it is really easy to write a CQL
rule to check that classes tagged with the attribute FullCovered
is also 100% covered by tests:
WARN IF Count > 0 IN
I can attest that this
rule warns regularly, reminding the team that some classes has been
refactored and that some new code hasn’t been properly covered. Clearly,
having the FullCovered attribute tagging the class declaration
is not a high enough indication for developers to care for full
coverage. An automatic verification process must takes place through the
execution of the CQL rule.
Notice that nothing prevents you from having your own
attribute class instead of NDepend.CQL.FullCoveredAttribute.
As a final note, concerning coverage technology we are
happy users of NCover coupled with TestDriven.NET
for VS integration and NUnit as
a test Fx. What I appreciate is the awesomely tiny overhead of NCover
on code execution and the numerous convenient tricks of TestDriven.NET!