I am really
glad to announce that now NDepend supports test coverage
metrics:
PercentageCoverage, NbLinesOfCodeCovered and NbLinesOfCodeNotCovered, defined on methods, types,
namespaces and assemblies.
There is
also the metric PercentageBranchCoverage defined on methods.
Coverage
data are gathered from coverage files emitted by NCover 2.x and Visual Studio Team System 2005
or 2008.
So why is
it so great that NDepend now supports these metrics? After all both
VisualStudio and NCoverExplorer present facilities to churn
coverage data. The cool thing is that integrating coverage metrics inside
NDepend and the CQL language opens up a wide range of brand new possibilities.
Continuously check for
coverage
Typically,
you spend time making sure that your class YourNamespace.YourClass
is 100% covered. But how can you make sure that in the future your class will
remain 100% covered across versions and evolutions? With NDepend and CQL 1.7
you just have to define the CQL rule:
WARN IF Count > 0 IN SELECT TYPES WHERE FullNameIs "YourNamespace.YourClass" AND PercentageCoverage < 100
As we are
heavily using this facility on the code of NDepend itself, we found convenient
to define an attribute NDepend.CQL.FullCoveredAttribute
in the assembly NDepend.CQL.dll, Then we defined the rule: make sure that all types tagged with this attribute are 100%
covered. With CQL this results in:
WARN IF Count > 0 IN SELECT TYPES WHERE HasAttribute "NDepend.CQL.FullCoveredAttribute" AND PercentageCoverage < 100
This is a
great way to scale on hundreds of classes fully covered. But also it helps
documenting the source code: when a developer is about to tweak such a class,
she can’t even forget covering all her modifications.
Alternatively,
if you think in terms of namespaces or assemblies thoroughly covered, you can
write such a rule:
WARN IF Count > 0 IN SELECT NAMESPACES WHERE NameIs "YourNamespace" AND PercentageCoverage < 100
Or:
WARN IF Count > 0 IN SELECT ASSEMBLIES WHERE HasAttribute "NDepend.CQL.FullCoveredAttribute" AND PercentageCoverage < 100
Because
assembly has support for attribute while namespace don’t.
Focus coverage on sensitive
code
What is sensitive
code? In my last blog entry on practices to tend toward bug-free code I explained that sensitive code
that will likely contains most of bugs you’ll get within the next release is:
- Methods
that have been added since the last release.
- Methods
that have been refactored since the last release.
With CQL,
you can write rules to be warned when such sensitive code is not 100% covered.
// Methods that have been added must be fully covered
WARN IF Count >
0 IN SELECT METHODS WHERE WasAdded AND PercentageCoverage < 100
// Methods that have been refactored must be fully covered
WARN IF Count >
0 IN SELECT METHODS WHERE CodeWasChanged AND PercentageCoverage < 100
// Complex Methods must be fully covered
WARN IF Count >
0 IN SELECT METHODS WHERE
(CyclomaticComplexity >
15 OR NestingDepth > 5) AND
PercentageCoverage < 100
You can
also use the metric MethodRank to know which method is sensitive,
in the sense that if it contains a bug, this will likely be a catastrophic bug.
The trick is that methods with high MethodRank are those that are the most
used.
// The 100 most important methods must be fully covered
WARN IF Count >
0 IN SELECT
TOP 100 METHODS WHERE
PercentageCoverage < 100 ORDER BY MethodRank
DESC
Get an intuitive feeling of
how your code base is covered
The visual
facility supported by NDepend to display a set of code element that matches
some condition is especially useful when it comes to evaluate what is well or poorly
covered. For example, the following picture shows the methods fully covered in
NDepend. They represent 22.726 lines of code on 58.955.
It is
interesting to compare this with the set of methods at least a bit covered (i.e
%coverage > 0). They represent 32.790 lines of code on 58.955.
Deal with ‘uncoverable code’
We all know
that 100% coverage of an entire code base is not feasable. Even if you are
really enthusiast about testing, some case just cannot be tested automatically.
Typically, a method that call MessageBox.Show(…)
cannot be covered by unit tests because it requires a user action. This is why
NDepend has support for what we call uncoverable
code. Project properties accept an attribute
that will tag uncoverable methods, uncoverable classes or uncoverable assemblies. Such tagged code
element won’t be taken account when computing coverage statistics. This will
let you write full coverage rule without being bothered by uncoverable code.
Smart merge of coverage files
Code coverage data is often spread across numerous coverage files. Thus NDepend
accepts a list of coverage files as input. A coverage file has a merge option
OR XOR AND or NONE.
Most of the
time you’ll use the option OR to merge your coverage files. But if you want to
know what is only covered by a particular file, you’ll need the option XOR. And
if you want to know what is the intersection of what is covered by all coverage
files, you’ll use the option AND. And finally the option NONE will be used to
temporarily disable some coverage files.
You won’t
have to re-analyze your project to play with these options because they can
be computed on the fly on an analysis currently loaded.
If you wish
to see all these facilities live, have a look at this 3 minutes screencast.
Also,
details on how to import coverage data from coverage files are available here.