Qualities of a Good Unit Test

Posted by Jeremy Miller on July 20, 2005
Many developers, including me sorry to say, treat unit test code as a second class citizen. After all, NUnit test fixture classes aren’t going into production, so why should you put a lot of time, effort, and thought into them? An unfortunate reality is that unit tests are rarely “write-only” code. Badly written unit tests or interdependent unit tests hamper refactoring or adding new code to the system. I’ve heard of teams that were afraid to add new code to a system because the changes would introduce test failures. One of the advantages of automated unit testing should be the ability to safely modify existing code. Bad unit tests turn this advantage on its head.

I’m giving an internal presentation today on mock objects and I prepared a slide on the “Qualities of a Good Unit Test.” I actually couldn’t find much in the way of resources on the web for this, so I made up the list below. I’d love to hear from the rest of you what your best practices are for unit tests.

Unit Tests Should Be Atomic

A unit test should only test a small piece of functionality. This falls inline with the idea of getting into a rhythm of “write a little test, write a little code.” A litmus test is to ask yourself if any part of the unit test could stand alone in a separate unit test. Another feedback loop for your unit testing quality is the amount of time you spend with the debugger. If your unit tests are coarse, a test failure is more difficult to find. If the unit test exercises a small amount of code, the test failure cause can usually be spotted very quickly.

Order Independent and Isolated

There should never be an order dependency, intentional or not, between unit tests. Problems arise when one unit test leaves some kind of dirty data laying around. Testing against a database or some sort of static cache is a common culprit. The theme of the day was mocking (actually Episode III), so put evil stateful things behind nice, deterministic mocks or stubs.

Intention Revealing

A unit test can, and should, be a valuable form of documentation. At best, a unit test should explain the intended usage and function of a class. At worst, the unit test should still be easy to debug in the case of a regression failure. Excessive data setup can obfuscate a unit test beyond any hope of comprehension.

Easy to Setup

How easily can I set up an environment to run the tests? If the answer is “not very,” I might never get the tests running. As much as possible, decouple unit tests from external dependencies like databases and web services by using mocks or stubs. When you do test against an external dependency, the dependency better be setup correctly by the automated build.

Runs Fast

You’re never going to run the test just once. For the sake of constant feedback, successful continuous integration, and your sanity, make sure your tests run quickly. Thirty minute build and unit test cycles are nothing but friction. Just to keep in the mock rut, mock things that make network calls to speed up your testing. I like to segregate connected tests into a seperate assembly. This way you can quickly run the majority of your unit tests on demand without the lag from network calls.

There’s surely some obvious things I’m missing, but I’m short on sleep (and a midnight movie awaits) at the moment so that’s all I’ve got.

About Jeremy Miller

Jeremy is the Chief Software Architect at Dovetail Software, the coolest ISV in Austin. Jeremy began his IT career writing "Shadow IT" applications to automate his engineering documentation, then wandered into software development because it looked like more fun. Jeremy is the author of the open source StructureMap tool for Dependency Injection with .Net, StoryTeller for supercharged acceptance testing in .Net, and one of the principal developers behind FubuMVC. Jeremy's thoughts on all things software can be found at The Shade Tree Developer at http://codebetter.com/jeremymiller.
This entry was posted in Test Driven Development. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • http://www.jeffreypalermo.com Jeffrey Palermo

    Awesome points. I see a lot of “unit” test code that is really just big, ugly automated tests. This test code touches all kinds of dependencies, and that’s very hard to keep false negatives from happening. Keeping tests to one component at a time is much better. Atomic is the word. If a component needs outside data, mock it.

  • http://blog.griffincaprio.com Griffin Caprio

    Interesting read. I originally proposed something similar in a unit testing article I wrote for CODE Magazine.

    http://blog.griffincaprio.com/blog/PermaLink.aspx?guid=19de0d91-41a9-4b38-85af-5a3aae9c2108

  • http://codebetter.com/blogs/john.papa johnpapa

    Hey Jeremy,

    Interesting read …. some good points in here.

    I am curious to read what you think about the situation of setting up a unit test where you are testing a method of a class the gets a single entity (or null if no match is found). Let’s say the class is Customer and it has a method like this:

    public Customer GetEntity(int custID)

    In your unit test you can pass in a valid integer, say 123, to get a valid Customer. Or you can query the database to find the first customer using a SELECT TOP 1 CustomerID FROM Customers statement. Or you can pass invalid IDs in to test failure. But my point is how do you feel about supplying the ID for the method within the unit test (possibly in a Setup method for the test fixture)? The issue I see is that if someone deletes that customer later, your tests will fail until you change the ID in your test fixture class. Just curious as to your thoughts on this.

  • http://www.jeffreypalermo.com Jeffrey Palermo

    John,
    If the unit test supplies the integer, then the database would not be on the test stack. The data tier would be mocked. If the database were on the testing stack, then this would be an integration test and would require database setup to ensure that the database is in a known, reliable state before running this test. Each integration test has to be responsible for its own setup to avoid false positives/negatives.

  • Jeremy D. Miller

    John,

    You have to be in a known database state first. The most reliable way to do this is to setup the database state within the TestFixture class. There’s another quality I missed here – “Self-Contained.”

    A.) It makes your tests more robust because they aren’t reliant on something happening outside the test

    B.) Troubleshooting unit tests and understanding the unit test should be easier if all the pieces of the test data is visible in the same file.

    It’s also a really good thing to have separate databases for each developer and the build machine to avoid database clashes and test data pollution.

    For mapper testing like this I do something like:

    DELETE FROM CUSTOMER

    Customer customer1 = new Customer();
    // fill customer1 properties

    CustomerMapper mapper = new CustomerMapper();
    mapper.Save(customer1);

    Customer customer2 = mapper.GetEntity(customer2.ID);
    // compare customer1 & customer2

    You can compromise occasionally by doing the setup in a NAnt build or database script that runs before the tests.

  • http://www.agileadvice.com/ Mishkin Berteig

    I’ve written a slightly more generic Qualities of an Ideal Test: http://www.agileadvice.com/archives/2005/05/the_qualities_o.html