Agreed. Code metrics merely open your eyes to what you are doing and not doing. It's then the developers' job to fix what is important.

That is the metrics only value: Insight.

Right, it's a metric. Complaining about the information it gives you is like complaining about temperature not giving you the position and velocity of all molecules. That's the point of the metric: to reduce the view on a system to a single value. It does not characterize the system, it measures something. Only by adding other metrics can you get a more real idea of the underlying system. The difficulty is to know what and how much you need to know.

So the question is, how does one convince the FDA?

Hearing about this the last couple days I realized that it's not really the coverage number itself that I usually pay attention to but rather what it's doing. When it's falling I wonder what's happening. When it's rising I feel a bit more comfortable.

It also helps that I fully trust the testing capabilities of my coworkers.

Every so often, somebody points out how bad of a metric code coverage is . And of course, on its own

I don't put much importance on the coverage "metric" but looking at code coverage as a whole is certainly important. Identifying areas that aren't covered is critically important, but also the number of times each element is "touched" can be equally important especially when CC is unavoidable. The metric may say that nearly every line is touched, but that certainly lends nothing to assessing whether each combination has some form of coverage.

Bad tests can spoof any effort to mine information out of coverage so it's certainly *not* enough to just say "The code is safe, the code is clean, so long as those bars are mostly green."  Personally I review as much of the code, and unit tests as time affords me.

Thank you for submitting this cool story - Trackback from PimpThisBlog.com

Like anything, I think metrics are best used and studied and moderation. When coming on to a new project, you can glean a lot of insight just by getting code coverage. It helps to diagnose problems, not resolve them.

Some people I know are perfectionists when it comes to this metric. 100% code coverage in and of itself is kind of a myth - you still have integration issues as well as potential false positives with mocks and stubs.  You described it really well when you said it can act like a "security blanket"

This post was mentioned on Twitter by laribee: Posted - Code Coverage: What Is It Good For? http://rurl.org/23m7

To be honest, you really have to look at functional coverage rather than actual coverage made by writing tests.

People can (wrongly) engineer tests that will check down different branching points in code, but don't actually identify the fact that something should or shouldn't happen.

This is where imho, BDD is a good route to take, as it describes the functionality of a piece of code in the given a set of circumstances.

Composite metrics are definitely the way to go. The more different ways you are able to measure your code quality the better. I've been playing with integrating NDepend and NCover into Teamcity and have come up with some nice visual dashboards:

blog.robbowley.net/.../visualising-the-internal-quality-of-software-part-1

Metrics in software and physics

Hi Dave,

We do a weekly webcast on testing called "This week in testing" (learn.typemock.com/.../episode-4-animals-that-talk-back.html)

and we've mentioned this post (and the one about the cost of losing a developer).

Please come and watch, and if you like what you see, please tell other people as well.

Thanks,

Gil Zilberfeld

Typemock

I second the comment by a few previous readers that code coverage tools can reveal some important insights, including:

* The 'riskiest' areas in your codebase:  A combination of low coverage and high compexity definitely warrants investigation.

* Loss of code coverage over time is an indicator that code is changing but tests are not, which can be problematic.

IMHO, much of the negativity towards code coverage metrics is due to their misuse.  The statistics can easily be used to instill a false sense of security in stakeholders who don't understand that touching a line of code w/ a test doesn't make it bug-proof.  I'd prefer to see code coverage metrics as a tool used within the development team to prioritise where to invest in new and updated unit and functional tests.