100% code coverage to the rescue

Every single programming day, abiding by the simple idea… if a class is covered by tests, cover it 100%… sheds light early on many potential bugs and problems, before they end up screwing at production time.

For example, today I worked with a XML parser class that was 100% covered. After a refactoring session, it was not 100% covered anymore. But still all tests were green! The uncovered sequence points are shown in red:

Something was wrong since all these seven return false; sequence points used to be covered before the refactoring occurred! After investigating, it appeared that the line TryGetIntegerFromNexTAttribute(TAG_FIRST_STATEMENT_LINE… was added during the refactoring session. However unit-tests were not modified to take account of this new behavior. The 7 unit-tests concerning the 7 return false; sequence points, were all green, but it was because they were missing the TAG_FIRST_STATEMENT_LINE xml attribute.

These 7 unit-tests were out-of-sync with the code and if the 100% coverage ratio was not checked, we had no way to figured out this issue. Having green unit-tests out-of-sync is a bug in the unit-tests set, not in the code base itself. But this situation can certainly lead to more serious problems in the future.

Hence having green unit-tests is not enough. Classes covered must be - and must remain - 100% covered.

Btw, to check continuously that a class is 100% covered, we tag 100% covered classes with a FullCovered attribute…

…and we have a corresponding CQL rule : Types tagged with FullCovered attribute should be 100% covered by tests. An extra benefit is that, thanks to the FullCovered attribute, a developer working with a 100% covered class is aware that a complete test suite exists for this class.

This entry was posted in Uncategorized. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • http://www.facebook.com/wasimelatrash Wasim Elatrash

    i think that , 100% code coverage not applicable in the practice , every time you have more and more test cases . so .. what you think ?

  • Anonymous

    Sure, 100% coverage is not enough, 100% coverage goes hands in hands with code contracts.

    If your code is 100% covered + has as many contracts as possible + you made sure a contract fails at test-time if violated, then the chance such code contains a bug is very low. This is the theory part, I had chance to write in the past, here I wanted to expose a small practical real-world case.

  • Torkel

    I am not sure I agree with your conclusion (the title that is). I can think of a lot of exampels where the tests would give 100% code coverage but not be safe for refactoring, it all depends on the what asserts are made. The coverage figure just tells you lines was reached and executed, not that the test actually verified the expected behavior.

  • Anonymous

    No, the problem was that despite there were still green, some unit-tests were out-of-sync.

    >Having green unit-tests out-of-sync is a bug in the unit-tests set, not in the code base itself. But this situation can certainly lead to more serious problems in the future.

  • Fschwiet

    Was there a bug in the production code discovered by the tests in this case?

  • http://duraid.blogspot.com/ Darage

    OMG Patrick .. you’re very boring