Test what you’re testing

I use the phrase in the title usually to refer to using mock objects in a useless way (mocking ADO.Net or Active Directory or calls to any API that you don’t understand).  When you’re choosing whether to write a small integration test or a mocked test, just think what would give you the actual value.  If the mocked test doesn’t actually prove that whatever you’re doing works, then it isn’t useful. 

Today we got a little reminder about this rule.  By accretion, we are building a little Fluent Interface that uses lambda tricks to configure NHibernate mappings without having to expose my tender eyes to angle bracket hell.  Behind the scenes, it still generates good old fashioned hbm Xml, but at least I don’t have to see it.  Today we hit our first case of a somewhat common mapping scenario.  We had a unit test that specified that the Fluent Interface built the Xml exactly the way we intended it to for this scenario, but as soon as we tried to use it, NHibernate threw a 50 line stacktrace exception.  Lesson (re)learned here, we probably need to skip the intermediate unit test to worry about the intended effect of saving the object.  The Xml was only an intermediate deliverable, and not the real goal.

About Jeremy Miller

Jeremy is the Chief Software Architect at Dovetail Software, the coolest ISV in Austin. Jeremy began his IT career writing "Shadow IT" applications to automate his engineering documentation, then wandered into software development because it looked like more fun. Jeremy is the author of the open source StructureMap tool for Dependency Injection with .Net, StoryTeller for supercharged acceptance testing in .Net, and one of the principal developers behind FubuMVC. Jeremy's thoughts on all things software can be found at The Shade Tree Developer at http://codebetter.com/jeremymiller.
This entry was posted in Test Driven Development. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • http://colinjack.blogspot.com Colin Jack

    > Given the choice between writing Xml with strings or ripping through something with Intellisense, which would you rather have?

    I get you, but with the schema setup I don’t really find it that difficult to write the XML. Deciding how to map from my classes to the tables is sometimes tricky, but once we’ve decided actually writing out the mapping is usually relatively easy and afterwards I’ve generally found the mapping files readable. It would be useful if we got refactoring support though, but its not a killer.

    > Plus we’re doing quite a bit of “convention over configuration” along the way as well (setting up id’s, foreign key naming, taking care of enumeration mapping, pulling some metadata out of embedded
    validation stuff, etc.)

    Interesting, just being nosy what do you mean on the enumeration mapping? I’m also guessing on the validation this is where you’ve used attributes to define your simpler validation rules?

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Colin,

    We hit inheritance yesterday. I stuffed in support for the one table per hierarchy (all I needed in my case), and it worked out well.

    “I must admit I don’t find that the complexity/readability of the HBM mapping files themselves is too bad,”

    Given the choice between writing Xml with strings or ripping through something with Intellisense, which would you rather have? Plus we’re doing quite a bit of “convention over configuration” along the way as well (setting up id’s, foreign key naming, taking care of enumeration mapping, pulling some metadata out of embedded validation stuff, etc.)

    The inheritance is here:

    public class CategoryMap : DomainClassMap
    {
    public CategoryMap()
    {
    Map(x => x.Description);
    DiscriminateSubClassesOnColumn(“Type”)
    .SubClass().IsIdentifiedBy(“Widget”).MapSubClassColumns(map => { })

    .SubClass
    ().IsIdentifiedBy(“Area”)
    .MapSubClassColumns(map =>
    {
    map.Map(x => x.TrackThickness);
    })

    .SubClass().IsIdentifiedBy(“Length”)
    .MapSubClassColumns(map =>
    {
    map.Map(x => x.TrackOD);
    });

    }
    }

  • http://colinjack.blogspot.com Colin Jack

    Out of interest, is the plan to make it support mapping things like inheritance and so on longer term?

    Be interested to see the result but I must admit I don’t find that the complexity/readability of the HBM mapping files themselves is too bad, other than in situations where the mappings involved are inherent complex (inheritance and some of the more complex dictionary mappings spring to mind).

  • Sheraz

    Could we also see the xml it ends up generating.

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Sheraz,

    We will be Open Sourcing it at some point, but here’s a sample (we generate the database from the Domain Model, so the column names are derived from the object properties):

    public class SalesTransactionMap : DomainClassMap
    {
    public SalesTransactionMap()
    {
    Map(x => x.BuyerName);
    Map(x => x.DeliveryDate);
    Map(x => x.DeliveryNotes);
    Map(x => x.ExpirationDate);
    Map(x => x.OrderDate);
    Map(x => x.QuoteDate);
    Map(x => x.QuoteOrder);
    Map(x => x.TransactionId).ValueIsAutoNumber();
    Map(x => x.CustomerPurchaseOrder);
    Map(x => x.JobName);

    References(x => x.Customer);
    References(x => x.CustomerContact);
    References(x => x.SalesRepresentative);
    References(x => x.Warehouse);
    References(x => x.CustomerJob);

    Component(x => x.DeliveryDetails, m => configureDetails(m));

    HasMany(x => x.Items);
    }

    private void configureDetails(ComponentPart deliveryMap)
    {
    deliveryMap.Map(x => x.Mechanism);
    deliveryMap.Map(x => x.DeliveryName);
    deliveryMap.Map(x => x.Distance);
    deliveryMap.Map(x => x.CarrierName);

    deliveryMap.References(x => x.CustomerDeliveryAddress);
    deliveryMap.Component

    (x => x.Address, m =>
    {
    m.Map(x => x.AddressLine1);
    m.Map(x => x.AddressLine2);
    m.Map(x => x.AddressLine3);
    m.Map(x => x.CityName);
    m.Map(x => x.CountryName);
    m.References(x => x.State);
    m.References(x => x.PostalCode);
    });
    }
    }

  • Sheraz

    Are we gonna get to see how your fluent interface syntex look like to configure Mapping files sometime soon? I used Object Mapper in the past which is a free tool to create mapping files and it works fine but again each time you need to change anything, you have to deal with xml files. I’d be nice to see how much readability we get with your fluent interface syntex.

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @cristian,

    Sure, it did help to flush out the syntax. What I’ve learned with Fluent Interfaces though is that it’s best to go with more coarse grained tests to avoid over specification.

    @Chad,

    Jeremy made the exact same mistake earlier, only you weren’t around to see it blow up in my face;)

    @Ruud,

    I’m not testing NHibernate per se, I’m testing whether or not my particular interaction with NHibernate was done correctly (and it wasn’t).

  • http://chadmyers.lostechies.com Chad Myers

    We have an FI that builds up the XML mapping for NHibernate to help take away some of the pain of building, maintaining, and testing the XML mappings for NHibernate.

    Unfortunately, one of the things I wrote generated an XML element for an entity’s NHibernate HBM mapping XML file, but it build the XML incorrectly. The test I wrote verified that the generator produced the correct XML which was, in fact, NOT correct.

    So it was a cycle of stupidity where both the code and the test were wrong and the test only verified that the code did what it was supposed to do; That is, generate incorrect XML.

    Not until actually loading the HBM XML into NHibernate did we (Jeremy) detect my mistake.

    So Jeremy is calling into question whether that test I wrote was worth anything or not. Perhaps it provides some valid in that it was a basic sanity check, but it didn’t prove that the mapping worked.

    Is that another test? Perhaps. Or maybe it’s just more efficient to try to load the HBM into NHibernate’s config object and see what’s up and cut out the (incorrect) middle-man.

  • Brian Chiasson

    Not sure what you’re saying here, maybe it’s a lack of experience with NHibernate. Are you saying that your Mock hid the fact that you had a problem because you mocked something out behind the scenes? If so, in my experience I tend to write two sets of tests (potentially separate test libraries). One that does the unit testing with mocking and another that is a straight integration that ensures the whole thing is wired up properly.

  • cristian

    But hey, didn’t the intermediate unit test help you create your fluent api more quickly?

  • http://www.sitechno.com/blog Ruurd Boeke

    Aren’t you afraid that you will be testing nHibernate then? It has it’s own set of tests.