Using Continuous Integration? Better do the "Check In Dance"

So you’re making the move to install a Continuous Integration process on your project.  Most of your early attention is going to be spent on getting the CI server up and the build scripts put together and functioning.  That part of the process in the .Net world is getting easier as the tools have matured and the accumulated knowledge available online has multiplied.  My company is moving to full blown CI and TDD on our main systems soon.  Other than getting migrated off of VSS, I’m not too worried about the infrastructure setup.  What I am worried about are the developer practices. 


One of the key practices in agile processes is Collective Code Ownership.  Collective ownership can make a project team much stronger, but there are some inconvenient side effects in coding construction.  Because agile iteration work is mostly scheduled by business need and not by what is convenient for us the developers, it’s not unusual for two or more active coding threads to be simultaneously changing some subset of the code.  The idea of continuous design* implies that even big architectural changes can be happening on an ongoing process.  The key point is that things are changing and that can be risky.  CI is one of the enabling processes that supports both continuous design and collective ownership, but you’ve got to take advantage of it.  The CI build process is a mechanism to find and eliminate problems from code changes as early as possible.  The key for developers to get the most out of CI without being burned is to be disciplined about their “Check In Dance.”


Basic Guidelines



  • Check in as often as you can.  Try to reach stopping points as often as you can.  This goes back to the basic agile philosophy of making small changes and immediately verifying the small change.  When you’re doing Test Driven Development you strive to keep “Red Bar” periods as short as possible.  The same kind of thinking applies to code check-ins.  Make small changes and see the impact on the rest of the code immediately.  Merging code will be less painful the more frequently a team integrates their code.
  • Avoid stale code.  If you have to keep code out for any length of time, make sure you are getting everyone else’s changes.  Try really hard not to keep code out overnight.  If you’re using shared developer workstations, put some sort of sign on the workstation that there is outstanding code on the box.  I’ve seen several XP zealots swear that they’ll throw away any code left overnight.  Personally, I think that’s just a silly case of “I’m more agile than thou,” but it’s still a bad idea to leave code out overnight if you can help it.
  • Don’t ever check into or out of a busted build.  Checking in might make it harder to fix the build because it will cloud the underlying reason for the build, and you can’t really know if your changes are valid.
  • Communicate and negotiate check in’s to the rest of the team.  Frequently the complexity of a merge can be dependent upon who goes first.  Some teams will use some kind of toy as a “check in token” to ensure that there is never more than one set of updates in any CI build.  Pay attention to what the rest of the team is doing too.
  • If you’re working on fixing the build, let the rest of the team know.
  • DON’T LEAVE THE BUILD BROKEN OVERNIGHT.  That’s also an occasional excuse to your wife on why you’re home late from work.  Use with caution though.
  • Not every member of the team needs to be a full-fledged “BuildMaster,” but every developer needs to know how to execute a build locally and troubleshoot a broken build.  If you’re suckered into being the technical lead, make sure every team member is up to speed on the build.

 


The “Check In Dance”


The best practice for effective CI is to perform the integration on a developer workstation before that code escapes into the build server wild.  It’s okay to break the build once in awhile.  One of my former colleagues used to say that the CI build *should* break occasionally just to know it’s actually working.  What’s not okay is to leave the build in a broken state.  That slows down the rest of the team by preventing them from checking in or out.  Even worse, somebody might accidentally update their workstation with the broken build and get into an unknown state.  If you follow these dance steps, you can minimize build breaks and run more smoothly.  Besides, it’s embarassing to have the “Shame Card” on your desk.



  1. Let the rest of the team know a change is coming if it’s a significant update.
  2. Get the latest code from source control. 
  3. Do a merge on any conflicts.
  4. Run the build locally and fix any problems found.
  5. Commit the changes to source control.
  6. Stop coding until the build passes.
  7. If the build breaks, drop everything else and fix the build.

Here’s a tip on annoying the crap out of your teammates.  I found a MIDI file of the “Chicken Dance” song one time and commenced playing that out loud everytime I checked in.  It was funny for about a day or two.


 


* I detest the term “Emergent Design.”  That term makes it sound like you just spit out a bunch of code like a new age painter randomly splattering paint on a canvass and voila — a coherent design will spontaneously create itself.  When I worked with the High Priests of Agile, I used to frequently get chewed out and told to “just let the design emerge.”  To this day I have no earthly idea what that means.  I think it means just write lots of “it/then/else” statements until the code practically screams for restructuring, then have a big, heated hullabaloo meeting to agonize about the proposed refactoring.  Agile processes can be just as stupid as any CMM-driven waterfall if you’re not thinking straight.

About Jeremy Miller

Jeremy is the Chief Software Architect at Dovetail Software, the coolest ISV in Austin. Jeremy began his IT career writing "Shadow IT" applications to automate his engineering documentation, then wandered into software development because it looked like more fun. Jeremy is the author of the open source StructureMap tool for Dependency Injection with .Net, StoryTeller for supercharged acceptance testing in .Net, and one of the principal developers behind FubuMVC. Jeremy's thoughts on all things software can be found at The Shade Tree Developer at http://codebetter.com/jeremymiller.
This entry was posted in Continuous Integration, Test Driven Development. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • ScottMuc

    I still reference this blog post every few months. Can’t believe how relevant it still is after 8 years!

  • Billy

    Of course “Emergent Design” is desirable. It keeps the backlog full of work to do (refactoring of “technical debt”) which keeps people busy (employed), rather than focusing on creating real value for the business. It’s fun to right code on someone else’s dollar.

  • John

    “I detest the term “Emergent Design.” 
    Oh, I couldn’t agree more!

  • Duncan

    Years later, this is still a great article! I have just printed for my team (bring them into the CI world). I made one small change:

    • Get the latest code from source control. (EDIT: this seems to be the hardest part of the process to remember. DO NOT check in code unless you have updated and merged RECENTLY)

    I have seen a few instances where people checked in without a local merge and screwed things up. Best to stress that during the learning stages so it becomes a “habit inviolate”.

  • http://www.technicaldeveloper.com A Wright

    Nearly 5 years later, and this advice is still relevant today. Thanks for a great article!

  • http://creedcultcode.blogspot.com Dale Smith

    Three years on, and it’s still an amazingly relevant post. I just forwarded the Check In Dance and a link to this post to one of my coworkers who has not quite learned the joy of keeping the build clean.

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    But yet Alex, my team and I are happily able to do emergent design on a daily basis. Some design before coding, sure. All of it? No way.

  • Alex

    I’m into a Continuous Integration process and it’s a total nightmare. Unluckily I inherited the project so there’s little I can other than being patient and find out why the h*ll isn’t the workspace compiling today.
    I agree with you when you say that most of the time, the CI process becomes the real goal and people forget that we are here… to design and write software. Not just be cool and say “wow… it’s integrating continuously!!!”

    As for the Emergent Design… well… people believing in ED may also believe to Santa Claus. If what you need to acheive is a frontend-somebackend-database project it may sound likely the design grows up itself. For serious enterprise projects, analysis and design are mandatory practices before ANY line of code is actually written.

  • http://www.viewtier.com Slava Imeshev

    Nice post. Here is another collection of best practices that we have accumulated over years: http://www.viewtier.com/support/presentations/continuous_integration_benefits_challenges_best_practices.pdf

  • http://www.jefspalace.be/cms Jef Patat
  • http://codebetter.com/blogs/jeremy.miller jmiller

    Thanks for the link Colin. Good to hear from you.

  • Colin Kershaw

    I think Bill had a great post that addresses the all too often overlooked focus on People:

    http://www.williamcaputo.com/archives/000069.html

    “In short,” as he says, “… non-agile … views process as having a larger impact on a project’s success than its people…”.

  • http://codebetter.com/blogs/jeremy.miller jmiller

    Ranjan, I really only meant to avoid the extreme of “emergent” design. I would never support a BDUF approach. I prefer Jim Shore’s description of “Continuous Design” (http://www.martinfowler.com/ieeeSoftware/continuousDesign.pdf). The key in my mind is to be thinking about the design and trying to improve every single day. Our development practice is still much closer to XP than something like RUP, and it works for us. I just think the zealot, purist approach to XP can be stupid. Design philosophy is a full range of approaches, not a binary switch, and there are plenty of folks out there doing XP and Agile well.

    Regardless of approach, you’re gonna want skilled developers that are working responsibly. NO process or design technique will ever change that fact. To Jeffrey’s point, you really want the full team to be responsible for design or at least participate. If your developers need to be “coralled” you’re already screwed out of the gate.

  • http://www.jeffreypalermo.com Jeffrey Palermo

    Emergent design. All the developers have to be very skilled for that to happen. So many programmers get things working and are content with crap that works. Agile assumes that the team consists of motivated, capable people. To do Agile development, you have to have the people first.

  • http://www.codebetter.com/blogs/ranjan.sakalley/ rsakalley

    While I am not a full fledged follower of Agile, (Emergent Design being one of the major reasons for that) especially for long term projects, I feel Emergent design is like letting a horse run within the circular wooden boundary. If you fail to define these, I think more things would break than make. So defining the boundaries is a must, and then let the developers loose, for the micro level emergent design, which actually comes out of TDD and Refactoring, nothing more nothing less. This may keep improving your macro level design, or may not.

  • http://www.jeffreypalermo.com Jeffrey Palermo

    These are some great tips. I use them at work all the time.