Introduction
Some other bloggers have asked how much unit test code (as a percentage of the total project’s code) should be written. I have certainly asked that question myself! So I decided to find out.
The point of this research was to determine roughly how much unit test code there was in a given project, expressed as a percentage. I know that lines of code aren’t the greatest thing in the world to count, but if you program your unit test code the same way you do your application code (and you certainly should), then making generalizations based on project percentages should be ok. Please note that all projects were developed in.NET.
I also wanted to find out what the percentage of unit test code was in real life projects. I did not want statistics from some book using contrived examples which “should apply to your projects.” Worse yet, I didn’t want uninformed speculation!
Procedure
Requiring access to source code, I turned to open source for data. I reviewed quite a few popular open source software projects, and ended up with eight projects that had discernable unit test code. I did not use data from my own projects yet, as that would obviously bias the findings.
All of these projects had GUIs, and the total lines of code counted included any GUI code. Lines of code were only counted for class files (cs and vb files). Lines of code ranged from 3000 to 50000.
Results
Here are the results:
|
Average |
39.0% |
|
Median |
39.9% |
|
High |
55.9% |
|
Low |
18.4% |
|
Standard Deviation |
13.7% |
The average was 39 percent of the total code base was unit test code. Probably more accurate is the median, which was 39.9 percent. The highest individual project percentage was 55.9 percent, and the lowest project (that still had unit tests) was 18.4 percent. The standard deviation was 13.7 percent.
The average and standard deviation together tell you the range in which most projects fall. Assuming a normal distribution curve, 95 percent of all projects will have unit test code ranging from 12 to 66 percent. The normal curve is a safe assumption, however the problem lies in the fact that the sample I have now might not be large enough to accurately represent the population.
Analysis/Recommendation
If you are doing test driven development, use a simple line counter program to see what percentage of your code base is unit test code. If it is much less than 20 percent, you probably aren’t testing enough. If it is much more than 66 percent, your test code is probably becoming an out-of-control mess.
Note that I did not follow the 95th percentile figures exactly. Also, unit test code percentages often vary depending on the type of the application. Fancier GUIs tend to require a lot of code which is often not as well tested with tools like NUnit or csUnit.
Anything else?
If you know of any open source or freeware projects I should include, please leave them in the comments!