Microsoft’s recommendations for Test Driven Development are wrong!

Okay, wrong is too harsh, but I made you look.  Microsoft has published a list of recommendations for doing Test Driven Development in VS 2005 at http://msdn2.microsoft.com/en-us/library/ms182521.aspx.  Before
I start bashing MS here, let me say that it *is* a good thing that MS
is even talking about best practices like unit testing and build
automation.  That being said though,
Microsoft seems to be trying to redefine TDD to match up with their
tools instead of creating tools that support TDD best practices.

 

Let’s begin…

 

            “3.) Make
a list of tests that will verify the requirements. A complete list of
tests for a particular feature area describes the requirements of that
feature area unambiguously and completely.

            “4.) File work items for feature requirements and for the tests that need to be written.

 

Quickly
making a list of unit tests to write in a coding session is often a
useful way to break the TDD ice and get started, but don’t ever assume
that that list is final or correct.  It is nice to get a full set of acceptance tests for the feature before you start the coding, but writing all of the unit tests before starting coding is not efficient.  My experience has been consistent.  Anytime
you write several unit tests in a row before you start implementing the
code you end up wasting effort because you always learn something in
the first test that invalidates the next unit test.  Michael Feathers in his retort talks about how you use the feedback from one unit test to write the next test.  I couldn’t agree more. 

 

Oh, and creating a “work item” for each unit test I’m going to write?  I don’t think so.  Can you say “unnecessary overhead?”

 

            “6.) Define
the interfaces and classes for your feature or requirement. You can add
a minimum of code, just enough to compile. Consider using the Class
Designer to follow this step

            “7.) Generate tests from your interfaces and classes. For more information

 

No, no, no.  Autogeneration
of the unit tests from interfaces or classes might sound nice, but it
isn’t really correct and might not be all that useful.  It’s also not Test Driven Development.  TDD is using unit tests to drive the design of the code.  One
of the “crossing the Rubicon” moments in learning TDD is when you learn
to define the interface of a class inside the unit test.  That’s right, use the unit test to define the intended function of a new piece of code, and then write the code to make the unit test compile and pass (ReSharper will very happily create the method stub for you).  I
still do some paper and pencil work before coding, but by and large
I’ve found that I have better results when you are determining a
class’s signature while writing the unit tests.  Writing
unit tests first will inevitably simplify the usage of a class (because
you’ll want to minimize the mechanics of creating the unit tests). 
I certainly don’t think that you throw away traditional design
techniques when you do TDD, but you can’t ever let the code and design
get very far ahead of the unit tests.

 

Another thing to keep
in mind with the autogeneration of unit tests is that unit tests do not
map one to one with methods and properties.  You don’t test methods one by one.  You test logical pathways through the code.

 

One of the key things to success in TDD is to very deliberately design for testability.  The best way to accomplish testability is to write code “Test First”.  If
you design and create a lot of code before you switch to retroactively
applying unit tests you’ll often find that the unit tests are hard to
write.  Also, you will probably not achieve the same level of code coverage that you would if you code test first.  Actually, let me put this more strongly – retrofitting existing code with unit tests is hard.  It’s perfectly acceptable to spend some time contemplating the design before coding, but do write the unit tests first.  TDD will go so much better when you write the tests first.

 

My single biggest
irritation with Microsoft’s guidance to .Net developers and their
development tools is Microsoft often seems to ignore anything
outside of the Redmond campus.  The advice they’ve given is contrary to existing best practices for TDD that have resulted from experience.  Patterns and Practices team, are you out there?  I know you guys are using Agile methods because I read your blogs.  Do y’all really buy off on the MS TDD “recommendations?”  Can you do something about this? 

 

My single biggest irritation with the .Net development community is our over reliance on Microsoft for development guidance. 

A.)  We need to diversify our sources of knowledge

B.)   We need to question any and all advice from Microsoft (or anyone else) and think for ourselves

C.)   We need to have an active conversation with ourselves on best practices

 

By the way, you can do TDD and Continuous Integration with free open source tools that might actually work better than the VSTS alternatives.  For doing rapid TDD, the consensus view seems to be that the combination of NUnit or MbUnit and TestDriven.Net is superior in productivity to the new VS 2005 unit testing capabilities.

 

There’s a thread active on the Test Driven Development message board about the MS TDD guidance that’s entertaining.  Also read Peter Hancock’s response

About Jeremy Miller

Jeremy is the Chief Software Architect at Dovetail Software, the coolest ISV in Austin. Jeremy began his IT career writing "Shadow IT" applications to automate his engineering documentation, then wandered into software development because it looked like more fun. Jeremy is the author of the open source StructureMap tool for Dependency Injection with .Net, StoryTeller for supercharged acceptance testing in .Net, and one of the principal developers behind FubuMVC. Jeremy's thoughts on all things software can be found at The Shade Tree Developer at http://codebetter.com/jeremymiller.
This entry was posted in Test Driven Development. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • Jeremy D. Miller

    At least I didn’t use “the path not taken” cliche. Point well taken Will.

  • will

    c’mon people. this “crossing the rubicon” allusion, analogy, whatever is becoming badly overused. you’re not having any “crossing the rubicon” moments in TDD. (that is, a point of no return). in fact, doing something like TDD requires discipline — there is no point of no return, you can always backslide into writing untested code. people are starting to use that phrase as if it meant a moment of epiphany or sudden comprehension, but it’s not that at all.

  • Jeremy D. Miller

    James Shore puts it a bit better (and stronger) here –> http://www.jamesshore.com/Blog/Microsoft-Gets-TDD-Completely-Wrong.html

  • http://www.jeffreypalermo.com Jeffrey Palermo
  • Jeremy D. Miller

    I just noticed Uncle Bob’s rant about this at http://www.butunclebob.com/ArticleS.UncleBob.MsTdd