If…

Posted by Jeremy Miller on October 14, 2008

you write a fairly complicated integration test to exercise some the interplay between different classes and some brand new DSL code…

and the test passes on the first try…

is the explanation:

 

A) You’re damn good.  Clean design.  Good unit tests.  Master of the programming universe is you.

or…..

B) Your test is wrong!

 

While you, dear reader, ponder this question I shall now proceed to double check my test code.

About Jeremy Miller

Jeremy is the Chief Software Architect at Dovetail Software, the coolest ISV in Austin. Jeremy began his IT career writing "Shadow IT" applications to automate his engineering documentation, then wandered into software development because it looked like more fun. Jeremy is the author of the open source StructureMap tool for Dependency Injection with .Net, StoryTeller for supercharged acceptance testing in .Net, and one of the principal developers behind FubuMVC. Jeremy's thoughts on all things software can be found at The Shade Tree Developer at http://codebetter.com/jeremymiller.
This entry was posted in Uncategorized. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • Chauncey Smith

    My test is usually Wrong.

  • Jarrod

    Red, Green, Refactor

    If your test never goes red, then you may have written a test that can never fail.

  • http://www.felipe-lopes.com Felipe Lopes

    Yes, master!

    Kidding.

  • http://www.bluespire.com Rob

    c) You’re still confused about how to use Rhino.Mocks AAA syntax ;)

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Rob,

    We’ve had a couple “Come to Jesus” meetings already about how we’re going to use Rhino Mocks going forward (classic mode or the new AAA, and how does that AAA thing work exactly?). I’m lukewarm on the whole AAA thing.

  • http://geekswithblogs.net/LessonsLearned/Default.aspx Randolpho

    That is the age old question:

    Who tests the test?

  • http://graysmatter.codivation.com Justice~!

    Maybe for poor schlubs the answer is b) but when you are a master of your domain like myself (get it? master of your domain? DSL? See what I did there?) a) is the only true way.

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Randolpho,

    I’ll give you my automatic comeback to that one: “Pay attention to what you’re doing”

  • http://geekswithblogs.net/LessonsLearned/Default.aspx Randolpho

    But if you pay attention to what you’re doing, do you really need tests? :-D

    Anyway, I was more paraphrasing “Who watches the Watchmen?” than I was attempting to take a stab at unit testing or TDD.

  • http://www.adronbhall.name Adron

    The first test is supposed to fail, then you go and make it green.

    Red-Green-Refactor -> it’s right over there on the right side of your site. :)

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    Adron–>

    It was an INTEGRATION test — like the blog post states rather clearly. As in, all the individual classes had been dev’d in a TDD manner first.

  • http://shadowcoding.blogspot.com/ Erik
  • http://www.ayende.com/Blog/ Ayende Rahien

    I remember having this exact issue with Rhino Mocks about 2.5 years back, I couldn’t believe that the test would pass and spent several hours debugging that.
    Spooky

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Ayende,

    The one that’s really scaring me at work is our new Rules engine / Rules DSL. We unit tested the hell out of all the little bitty parts, but we’ve literally gone 500 integrated tests w/o finding a bug. Lots of false negatives, but no bugs.

    Nobody on the team actually believes that we’re 100% bug free. Just wondering about what we’re missing.

  • http://www.ayende.com/Blog/ Ayende Rahien

    For a long time, I actually believed that Rhino Mocks _was_ bug free.
    And it was, for the set of scenarios that I used it for.
    The bugs that were found usually had to do with:
    – Using the API with the wrong order or method calls
    – Using the API with nasty CLR features such as COM interop, generics, out parameters and such.

    A language tends to be very simple to build. You’ll see the pain for real when you give it to someone who isn’t aware of the underlying assumptions that you don’t realize that you made.

  • Ganesh

    IMHO if anything works fine at the very first step, it is bound to fail very fast as well

  • Ian Cooper

    @Jeremy

    Do you have any QA guys. I just ask because there mind set is different, They test for failure whereas devs test for success. it’s an advantage of getting them to author acceptance tests or do explotatory testing.

    Although I sometimes wonder why anyone in their right mind would do what some testers do to break the product.

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Ian,

    Testers, what are those? Basically, I wasn’t able to recruit the one and only tester I trust in Austin, so we decided to go dev-only. The obvious drawback is making ourselves do adequate exploratory testing. The upside is a better test automation story.

  • http://www.lucasgoodwin.com Lucas Goodwin

    I run into this from time to time (tests succeeding out of the gate).

    I’ve found more often then not that the integration test is testing a scenario that’s already being addiquately covered by unit tests on the underlying parts.

    Or I wrote the test in a way that used my assumptions to my advantage that basically insured it was testing a known good scenario.

    btw, this form is broken in IE7 today (Javascript errors)