Patrick Smacchia [MVP C#]

Sponsors

The Lounge

Advertisement

Images in this post missing? We recently lost them in a site migration. We're working to restore these as you read this. Should you need an image in an emergency, please contact us at imagehelp@codebetter.com
Make the most of your test coverage data

 

I am really glad to announce that now NDepend supports test coverage metrics:

PercentageCoverage, NbLinesOfCodeCovered and NbLinesOfCodeNotCovered, defined on methods, types, namespaces and assemblies.

There is also the metric PercentageBranchCoverage defined on methods.

 

Coverage data are gathered from coverage files emitted by NCover 2.x and Visual Studio Team System 2005 or 2008.

 

So why is it so great that NDepend now supports these metrics? After all both VisualStudio and NCoverExplorer present facilities to churn coverage data. The cool thing is that integrating coverage metrics inside NDepend and the CQL language opens up a wide range of brand new possibilities.

 

 

Continuously check for coverage

 

Typically, you spend time making sure that your class YourNamespace.YourClass is 100% covered. But how can you make sure that in the future your class will remain 100% covered across versions and evolutions? With NDepend and CQL 1.7 you just have to define the CQL rule:

 

WARN IF Count > 0 IN SELECT TYPES WHERE FullNameIs "YourNamespace.YourClass" AND PercentageCoverage < 100

 

As we are heavily using this facility on the code of NDepend itself, we found convenient to define an attribute NDepend.CQL.FullCoveredAttribute in the assembly NDepend.CQL.dll, Then we defined the rule: make sure that all types tagged with this attribute are 100% covered. With CQL this results in:

 

WARN IF Count > 0 IN SELECT TYPES WHERE HasAttribute "NDepend.CQL.FullCoveredAttribute" AND PercentageCoverage < 100

 

This is a great way to scale on hundreds of classes fully covered. But also it helps documenting the source code: when a developer is about to tweak such a class, she can’t even forget covering all her modifications.

 

Alternatively, if you think in terms of namespaces or assemblies thoroughly covered, you can write such a rule:

 

WARN IF Count > 0 IN SELECT NAMESPACES WHERE NameIs "YourNamespace" AND PercentageCoverage < 100

 

Or:

 

WARN IF Count > 0 IN SELECT ASSEMBLIES WHERE HasAttribute "NDepend.CQL.FullCoveredAttribute" AND PercentageCoverage < 100

 

Because assembly has support for attribute while namespace don’t.

 

 

 

Focus coverage on sensitive code

 

What is sensitive code? In my last blog entry on practices to tend toward bug-free code I explained that sensitive code that will likely contains most of bugs you’ll get within the next release is:

  • Methods that have been added since the last release.
  • Methods that have been refactored since the last release.
  • Complex methods.

 

With CQL, you can write rules to be warned when such sensitive code is not 100% covered.

 

// Methods that have been added must be fully covered

WARN IF Count > 0 IN SELECT METHODS WHERE WasAdded AND PercentageCoverage < 100

 

// Methods that have been refactored must be fully covered

WARN IF Count > 0 IN SELECT METHODS WHERE CodeWasChanged AND PercentageCoverage < 100

 

// Complex Methods must be fully covered

 WARN IF Count > 0 IN SELECT METHODS WHERE

(CyclomaticComplexity > 15 OR NestingDepth > 5) AND

PercentageCoverage < 100

 

You can also use the metric MethodRank to know which method is sensitive, in the sense that if it contains a bug, this will likely be a catastrophic bug. The trick is that methods with high MethodRank are those that are the most used.

 

// The 100 most important methods must be fully covered

 WARN IF Count > 0 IN SELECT TOP 100 METHODS WHERE

PercentageCoverage < 100 ORDER BY MethodRank DESC

 

 

Get an intuitive feeling of how your code base is covered

 

The visual facility supported by NDepend to display a set of code element that matches some condition is especially useful when it comes to evaluate what is well or poorly covered. For example, the following picture shows the methods fully covered in NDepend. They represent 22.726 lines of code on 58.955.

 


 

It is interesting to compare this with the set of methods at least a bit covered (i.e %coverage > 0). They represent 32.790 lines of code on 58.955.

 

 

 


 

 

Deal with ‘uncoverable code’

 

We all know that 100% coverage of an entire code base is not feasable. Even if you are really enthusiast about testing, some case just cannot be tested automatically. Typically, a method that call MessageBox.Show(…) cannot be covered by unit tests because it requires a user action. This is why NDepend has support for what we call uncoverable code. Project properties accept an attribute that will tag uncoverable methods, uncoverable classes or uncoverable assemblies. Such tagged code element won’t be taken account when computing coverage statistics. This will let you write full coverage rule without being bothered by uncoverable code.

 

 

 

 

Smart merge of coverage files

 

Code coverage data is often spread across numerous coverage files. Thus NDepend accepts a list of coverage files as input. A coverage file has a merge option OR XOR AND or NONE.

 

Most of the time you’ll use the option OR to merge your coverage files. But if you want to know what is only covered by a particular file, you’ll need the option XOR. And if you want to know what is the intersection of what is covered by all coverage files, you’ll use the option AND. And finally the option NONE will be used to temporarily disable some coverage files.

 

 

 

You won’t have to re-analyze your project to play with these options because they can be computed on the fly on an analysis currently loaded.

 

If you wish to see all these facilities live, have a look at this 3 minutes screencast.

 

Also, details on how to import coverage data from coverage files are available here.

 


Posted Wed, Apr 9 2008 10:28 AM by Patrick Smacchia

[Advertisement]

Comments

Patrick Smacchia [MVP C#] wrote NDepend presentation at MVP Summit
on Wed, Apr 9 2008 6:32 AM

I will present NDepend during the MVP summit. The schedule is one hour Wed 17th at 5pm in MSCC/ Hood

DotNetKicks.com wrote Make the most of your test coverage data
on Wed, Apr 9 2008 7:36 AM

You've been kicked (a good thing) - Trackback from DotNetKicks.com

Dew Drop - April 9, 2008 | Alvin Ashcraft's Morning Dew wrote Dew Drop - April 9, 2008 | Alvin Ashcraft's Morning Dew
on Wed, Apr 9 2008 9:04 AM

Pingback from  Dew Drop - April 9, 2008 | Alvin Ashcraft's Morning Dew

Paul Kinlan’s Development Blog » Development Link Goodness 10th April 2008 wrote Paul Kinlan&#8217;s Development Blog &raquo; Development Link Goodness 10th April 2008
on Thu, Apr 10 2008 7:53 AM

Pingback from  Paul Kinlan&#8217;s Development Blog &raquo; Development Link Goodness 10th April 2008

Mind Gravy » Blog Archive » links for 2008-04-10 wrote Mind Gravy &raquo; Blog Archive &raquo; links for 2008-04-10
on Thu, Apr 10 2008 8:35 AM

Pingback from  Mind Gravy  &raquo; Blog Archive   &raquo; links for 2008-04-10

Reflective Perspective - Chris Alcock » The Morning Brew #71 wrote Reflective Perspective - Chris Alcock &raquo; The Morning Brew #71
on Fri, Apr 11 2008 3:16 AM

Pingback from  Reflective Perspective - Chris Alcock  &raquo; The Morning Brew #71

Coverage » Make the most of your test coverage data wrote Coverage &raquo; Make the most of your test coverage data
on Sat, Apr 19 2008 4:18 AM

Pingback from  Coverage &raquo; Make the most of your test coverage data

Resveratrol wrote Resveratrol
on Tue, Apr 29 2008 4:32 PM

The content of this website belongs to a private person, www. blog. co. uk is not responsible for the content of this website.

Patrick Smacchia [MVP C#] wrote Write Active Conventions on your Code Base
on Sun, May 11 2008 4:53 PM

Recently, both Glenn Block and Ayende wrote about how to define some sort of active conventions about

Community Blogs wrote Write Active Conventions on your Code Base
on Mon, May 12 2008 3:45 AM

Recently, both Glenn Block and Ayende wrote about how to define some sort of active conventions about

free data entry tests wrote free data entry tests
on Tue, May 13 2008 2:42 PM

Pingback from  free data entry tests

Some new cool features in NDepend « C# Disciples wrote Some new cool features in NDepend &laquo; C# Disciples
on Wed, May 28 2008 3:21 PM

Pingback from  Some new cool features in NDepend &laquo; C# Disciples

Elegant Code » NDepend - Static Analysis Made in Heaven wrote Elegant Code &raquo; NDepend - Static Analysis Made in Heaven
on Mon, Jun 2 2008 5:38 PM

Pingback from  Elegant Code &raquo; NDepend - Static Analysis Made in Heaven

Patrick Smacchia [MVP C#] wrote NCover v3 rocks!
on Mon, Apr 13 2009 3:49 PM

We've just move forward the NDepend code coverage under NCover v3 and it rocks! At first glance I

Rob Conery wrote You’re Not Your Data Access
on Thu, Jun 11 2009 11:52 PM

Seems I touched off a bit of a “swirl” with a comment I made on my last blog post: I think, in general

uberVU - social comments wrote Social comments and analytics for this post
on Sat, Dec 5 2009 6:06 PM

This post was mentioned on Twitter by officialdjs: "THE RED CARPET AFFAIR"SAT DEC.19TH @ CIRCOLO $10 B4 11PM ON

GLIST @ OFFCIALEVENTS1@GMAIL.COM http://is.gd/5aL RT

Add a Comment

(required)  
(optional)
(required)  
Remember Me?