Code UnCoverable by Tests
Even for test-coverage addict (as me), there is some code that simply cannot be covered by tests. An example?
The call to MessageBox.Show() cannot be tested automatically since it is blocking. Of course we could mock calls to MessageBox.Show(), but at the end of the day, there will be at least one call to this method in the code base, one call that cannot be covered by tests.
This example is not isolated. There are a multitude of cases where code reacts to things that cannot be automatized, such as exception that cannot be reproduced…
…but also some environment settings, folder or file browsing by user, showing a modal custom dialog and more.
As we explained above, while using mocking can help reduce the surface of code not-coverable by tests, it can’t help to get a code base 100% covered. When exploring test coverage results, with NCoverExplorer, Visual Studio Team System or NDepend, this problem will lead to classes 98% or so covered. It is taking time and it is error prone to figured out each time you are exploring coverage results, that these are false alert. This is clearly anti-agile! What we want is a clean 100% coverage result!
NCover and NDepend are tackling this problem the same way: you can exclude some methods from coverage results through a dedicated (and eventually custom) attribute.
NCover knows about this attribute through the /ea command line option as describe here by Jamie Cansdale. Code tagged with this attribute will be excluded from coverage statistics and won’t disturb while exploring test coverage results. For example, if all methods of the class Foo are 100% covered except the method MessageBoxShow(), if the method MessageBoxShow() is tagged with the coverage exclude attribute then the class Foo will be shown as 100% covered. A great side-effect is that developers reviewing the code will know about this coverage exclusion.
With NDepend, the coverage excluding attribute can be provided through the import coverage file dialog as shown below. This will have the same effect as with NCover. For convenience, the assembly NDepend.CQL.dll can be linked from your project. This assembly contains the dedicated attribute: NDepend.CQL.UncoverableByTestAttribute.
If you prefer you can provide your own attribute.
Actually the burden of exploring coverage results to make sure that some classes or namespaces should be 100% covered is not really agile. It is manual, time consuming, error-prone and cannot be capitalized for future iterations.
NDepend can help automate this task with just 2 CQL rules:
// <Name>Types 100% covered by tests</Name> WARN IF Count > 0 IN SELECT TYPES WHERE HasAttribute "NDepend.CQL.FullCoveredAttribute" AND PercentageCoverage < 100
// <Name>Methods 100% covered by tests</Name> WARN IF Count > 0 IN SELECT METHODS WHERE HasAttribute "NDepend.CQL.FullCoveredAttribute" AND PercentageCoverage < 100
Basically, these rules will check automatically that all types and methods tagged with the attribute FullCoveredAttribute are indeed 100% covered. Insert these rules in your build process and you’ll be informed as soon as a lack of test coverage is luring. Here also, as a bonus side-effect developer reviewing or refactoring the code will know instantly which part of the code is supposed to be 100% covered. For applying this trick to the code base of NDepend since the code coverage feature is available, I can testify that it is a really, really time saving trick.
For convenience, the assembly NDepend.CQL.dll can be linked from your project and it contains the dedicated attribute: NDepend.CQL.FullCoveredAttribute (and also the attribute NDepend.CQL.MoreThan95PercentCoveredAttribute).
Of course, CQL rules can be readily tweaked to work with a custom attribute.