Compile => CI => Tests, or "How to compile first, test later"

A common piece of feedback we got for early reviews of Brownfield Application Development is that the automated testing chapter should come before the automated build/continuous integration chapter. This was an awesome bit of feedback mostly because we could ignore it and instead get some air time on my blog on why we’re doing it this way.

When I come into a project, my most immediate goal is usually to get it into a continuous integration environment as soon as possible. And the bare minimum for that is to get the source code and compile it. If there are tests and they pass, that’s an added bonus. I’ll add them to the CI process afterward.

For me, it comes down to treating your environment just as incrementally as your code. I want my CI environment up and running now. Maybe the tests pass, maybe they don’t. If I dump them in there now, that adds a variable into the mix I don’t want to deal with at the moment.

There’s some element of personal preference, I suppose. But it’s also borne from experience. At one time, I had some minor frustration trying to do everything in the automated build before dumping it on the CI server and having it fail. Just like we check in smaller bits of code more often, I like to enhance my CI process in small chunks more often.

Incidentally, CI is why we cover version control first. Not because we think you don’t know anything about version control, but so that you can get your source repository prepared for continuous integration. Any hack can dump code into SVN. The important thing is whether you can get it back in an immediately useful state. Continuous integration can help determine if it’s immediately useful.

Alas, the rest of the feedback for the book was not so easily dismissed. Damn you people and your “values”.

Kyle the Edited

This entry was posted in Brownfield, Continuous Integration. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • Jim Cooper

    @Neil Mosafi

    Anytime anyone says “do something” with the qualifiers “always” or “never” they are almost certainly wrong :-)

    You have to consider why you are doing CI and unit tests in the first place.

    Unit tests are more important than CI, because without them, there is far less benefit to doing CI at all.

    The developers should certainly be writing unit tests, regardless of whether the CI machine runs them yet or not (which is how I read what Kyle wrote).

    Also, I’ve worked in a place where just getting an extra machine set up and on the network was so difficult (because of information security issues, work scheduling etc), that it meant waiting weeks to months for a CI server.

    Automated deployment is most likely something you can defer until later, even though that may not be ideal. In the aforementioned company, it took many months before we were able to automatically deploy to the various testing and production environments. In my current job, there is nowhere to deploy my current project to, at the moment.

    So write unit tests first (you are doing TDD/BDD aren’t you?), setup CI as soon as you can (which may be early on if you have the resources).

    But debugging CI build scripts can be difficult, so it does make sense to get the CI checkout working first, then the build, then the unit test runner, because that’s the order in which they are needed – can’t run the tests until the build works, can’t do a build until check out works.

    So I think that (almost) 100% of the time, tests come first. Just not on the CI machine.

  • Richard Wright

    I totally agree with the order you do this.
    I always create a VS solution that does nothing and get that compiling on a separate build machine using CI, in my case TeamCity.
    When the compile works I then add a trivial test and check that a failing test breaks the build and a passing test allows the build to proceed. Then I can create unit and integration tests to my heart’s content.

  • Neil Mosafi

    Continuous Integration definitely comes before tests if you are starting without either. 100%. I’d probably say automated deployment does too, as you need to be able to release easily and often.

    Once you are doing so, the value of test will become more apparently as those frequent releases will become more stable with more test coverage.