Fast Tests, Slow Tests, Integration Tests, Unit Tests

I don’t use the terms unit test and integration test much anymore – especially not in code artifacts.  I don’t group test types into separate Visual Studio projects.

In late 2005, Michael Feathers formalized some guidelines for defining the responsibilities of a unit test.  The guidelines help us to understand when we’re writing unit tests and when we’re writing integration tests.

The subtitle of his post is, “Don’t let slow tests bog you down.”  We should be able to run fast tests independently of slow tests to validate that the system still works according to its design and without resisting running the tests because the test suite is slow.  We do this by isolating classes along their seams, often leaning on tools like mock object frameworks.

He says about a unit test, that it isn’t a unit test if:

  • It talks to the database
  • It communicates across the network
  • It touches the file system
  • It can’t run at the same time as any of your other unit tests
  • You have to do special things to your environment (such as editing config files) to run it.

I’ve often found a gray area in test suites where a test that would traditionally fit the definition of a unit test, but would actually execute quite a fast test.  Based on the common knowledge of the definition of a unit test, I couldn’t move these fast tests into my suite of fast unit tests.  Doing so would be confusing, and I like to do what I can to avoid arbitrarily introducing confusion into my codebases.  I’ve also had test classes that start as fast unit tests, and for one reason or another they become slow integration tests (sometimes by mistake, and sometimes only until they or their target code can be fixed).  I tend to move these into my integration test suite until the accidental design degradations can be set straight.

Previously, I would use two Visual Studio projects to group tests, one unit test project and one integration test project.  Areas of functionality in an application would have test classes in both projects, slow tests in the integration test project and fast tests in the unit test project.  Moving tests from the fast suite to the slow suite meant physically moving them from one Visual Studio project to another, and hopefully remembering to change their namespaces.

On my current project, we have all of our tests in a single Visual Studio project.  We tag test classes as fast or slow using NUnit’s Category attribute.  We have a target in our NAnt build that runs fast tests, and a target that runs slow tests.

There’s a great side effect of using tagging rather than visual studio projects.  Our test project is now organized by area of functionality.  We have a namespace (folder) for tests on model, tests, on controllers, etc.  All of the tests for any given area of the system are organized for editing and navigation in one place.  And, we can organize our tests for execution with tags.

Those tests that sometimes fall in the gray area are no longer in a gray area and when we need to turn a slow test into a fast test (or vice-versa), we change the tag.

Michael Feather’s guidelines are still the best definition of a unit test, but we tend more and more to use the terms unit test and integration test as pattern language rather than implementation cook book recipes.

This entry was posted in Design, Testing, Tools. Bookmark the permalink. Follow any comments here with the RSS feed for this post.

Leave a Reply