Jeremy D. Miller -- The Shade Tree Developer

Sponsors

The Lounge

News

Advertisement

Images in this post missing? We recently lost them in a site migration. We're working to restore these as you read this. Should you need an image in an emergency, please contact us at imagehelp@codebetter.com
Spiking a new syntax for integration tests. Whaddaya think?

Hello, my name is Jeremy, and I sometimes write unit tests specifications that are uglier than sin.  Specifically, when I started to look back at the interaction style tests I've been writing over the past couple years I'm finding a bunch of unit tests that are indecipherable to me, the author of those very tests.  I came to the conclusion that I needed to start writing my interaction tests in a different fashion to promote readability -- and hence better maintainability. 

I was really concerned about readability more than anything else, but a quicker way to write integration tests would be nice to.  To that end, I've been following the Eleutian guys' posts on their AutoMockingContainer strategy.  Honestly, I've been dubious about the value of the AutoMockingContainer idea.  I get the mechanical savings, but worried that it was too much magic that would obfuscate the unit tests somewhat.  I've got the StructureMap code open for its upcoming 2.5 release, so I decided to stick in some facility for "AutoMocking." 

The auto mocking stuff was easy enough to do, but I wanted to go farther in terms of readability.  I've found that one of the factors related to integration tests being confusing to newbies and hard to read for anyone is simply the order in which things happen.  In the traditional RhinoMocks record/replay model I call a bunch of expectations on the mock objects, then perform the actual action on the class under test, then call VerifyAll().  It's Yoda syntax.  Instead, what if we could express the tests in a more natural order of Given blah, when I execute blah, this stuff should happen.

Here's what I came up with in my spike.  This is working code, but I don't want to talk about the mechanics of the underlying code.  I'm mostly interested in what y'all think about the syntax and flow of the test.  I'm purposely leaving out comments or trying to explain the grammar upfront because I want to see how "soluble" this syntax is if you're looking at it cold.

This is a rewrite of an existing unit test in StoryTeller.  Upon looking at the code, I think there was a simpler way to write the code, but that's a refactoring for another day.  The new and old code effectively express the exact same unit test.  The auto mocking does do a lot to cut down on monotonous setup code.  I definitely think this code is going to be smoother in C# 3.0, but I want to release a 2.0 version before I do that.

The New Code

        [Test]
        public void Execute_the_command()
        {
            string theFragmentName = "Fragment1";
            Fragment theNewFragment = new Fragment(theFragmentName);
            
            GivenThatThe<Suite>().Is(new Suite());

            Given = delegate {
                That.Calling(Service<IInputCommand>().PropertyValue).Return(theFragmentName);              
            };

            WhenExecuting = delegate  {
                AddNamedFragmentCommand.Execute();
            };

            TheInteractionShouldBe = delegate {
                Call<ISystemUnderTest>().Save(theNewFragment);
                Call<ILeafNode>().RefreshNodes();
                Call<IFragmentPresenter>().StartEditing(theNewFragment, Member<ISystemUnderTest>());
                Call<IApplicationController>().OpenScreen(Member<IFragmentPresenter>());
            };

            Then = delegate {
                Assert.IsNotNull(Member<Suite>().FindFragment(theFragmentName));           
            };
        }

The Old Code for the Same Test

        [SetUp]
        public void SetUp()
        {
            _mocks = new MockRepository();
            _node = _mocks.CreateMock<ILeafNode>();
            _suite = new Suite();
            _systemUnderTest = _mocks.CreateMock<ISystemUnderTest>();
            _inputDialog = _mocks.CreateMock<IInputCommand>();

            _applicationController = _mocks.CreateMock<IApplicationController>();

            _command =
                new AddNamedFragmentCommand(_node, _suite, _systemUnderTest, _inputDialog, _applicationController);
        }

        #endregion

        private MockRepository _mocks;
        private ILeafNode _node;
        private Suite _suite;
        private ISystemUnderTest _systemUnderTest;
        private IInputCommand _inputDialog;
        private AddNamedFragmentCommand _command;
        private IApplicationController _applicationController;

        [Test]
        public void Execute()
        {
            string theFragmentName = "Fragment1";
            And fragmentConstraint = new And(
                new TypeOf(typeof (Fragment)),
                new PropertyIs("Name", theFragmentName)
                );

            IFragmentPresenter fragmentPresenter = _mocks.CreateMock<IFragmentPresenter>();
            ObjectFactory.InjectStub(typeof (IFragmentPresenter), fragmentPresenter);


            Expect.Call(_inputDialog.PropertyValue).Return(theFragmentName);


            // Save the fragment
            _systemUnderTest.Save(null);
            LastCall.Constraints(fragmentConstraint);

            _node.RefreshNodes();

            // Open the screen
            fragmentPresenter.StartEditing(null, _systemUnderTest);
            LastCall.Constraints(fragmentConstraint, new Equal(_systemUnderTest));
            _applicationController.OpenScreen(fragmentPresenter);

            _mocks.ReplayAll();
            _command.Execute();
            _mocks.VerifyAll();

            Assert.IsNotNull(_suite.FindFragment(theFragmentName));
        }

 


Posted 01-17-2008 2:27 AM by Jeremy D. Miller

[Advertisement]

Comments

jason wrote re: Spiking a new syntax for integration tests. Whaddaya think?
on 01-17-2008 4:19 AM

The new code is much better than the old one, but... but... is it really readable and easy to follow. I don't think so. I must admit, I took the approach of "skim through the test and see if I follow without the need of going back and re-read", and this failed. A newbee will also. Complicated tests will not survive and sooner or later they becomes zombies. Making tests easy to read is very, very hard. Especially when the business to be tested is complicated, and having said that, I have no suggestions on how to make your test easier to read :-(

ISerializable - Roy Osherove's Blog wrote The evolution of unit testing and syntax
on 01-17-2008 6:23 AM

The semantics of how you write a unit test, the basic syntax, or Domain Specific language of how we write

Igor Brejc wrote re: Spiking a new syntax for integration tests. Whaddaya think?
on 01-17-2008 7:01 AM

I agree with jason. Tests using mocks will always be as complicated as the system that is being tested. As someone who's seen your code snippet for the first time, I still had to make an effort to understand it.

My approach to increasing readability is by using code comments, both as headers of test methods describing what is being tested and how (if it is really complicated) and also inline comments for certain things which may not be obvious.

I have the same problem with forgetting what some old test code was all about ("old" in my case means few months ) and code comments do help. But I support your effort if you think it helps.

Kyle Baley wrote re: Spiking a new syntax for integration tests. Whaddaya think?
on 01-17-2008 9:14 AM

It's hard to say after a first glance. The Given...WhenExecuting...TheInteractionShouldBe...Then is easy enough to follow but the stuff inside each of these blocks isn't right off the bat. But I do get the impression it would be easy to get used to.

The thing is that it looks even more magical than the AutoMockingContainer. Having said that, I like the magic of AutoMockingContainer so I could get used to what you're doing pretty easily. It's nice that I don't even need to retrieve a reference to the mocked objects directly. Others may argue it's hard to figure out what it's calling all these methods on in the TheInterfactionShouldBe section.

Side note: I find using the Record and Play methods in Rhino.Mocks goes a long way to adding readability:

using ( _mocks.Record )

{

   // Set expectations

}

using ( _mocks.Play( ) )

{

   // Replay code

}

Still requires knowledge of mocking but it's dead simple to see how the Replay/Verify code is separated.

Christopher Bennage wrote re: Spiking a new syntax for integration tests. Whaddaya think?
on 01-17-2008 9:53 AM

I think C# is getting in the way: all of the <> () delegate ...

Jeremy D. Miller wrote re: Spiking a new syntax for integration tests. Whaddaya think?
on 01-17-2008 9:59 AM

@Christopher,

Yeah.  Chad Myers played with it a bit using C# 3.0 lambdas and it still had a lot of code noise.  This is looking like it's going to end up being yet another "C# just cannot be Ruby" idea.

Troy DeMonbreun wrote re: Spiking a new syntax for integration tests. Whaddaya think?
on 01-17-2008 4:55 PM

"Yoda syntax" - Genious wit!

Mike Suarez wrote re: Spiking a new syntax for integration tests. Whaddaya think?
on 01-17-2008 7:47 PM

I think the new approach is very stylish but not that readable. I do something like your old code but with more refactors for readability. Here’s how I would refactor it.

1. Rename _command to subject. It’s odd that there’s a _systemUnderTest there already n’ it’s mocked. The first line in my BehaviourTest is the subject (or sut declaration).

AddNamedFragmentCommand subject;

2. Extract method givenTheFragment(string name). I use assume instead of given but at the end it the same, setup a stub for the test.

3. Extract method expectToSave(Fragment fragment).

4. Extract method expectToRefreshTreeNodes().

5. Extract method expectToStartEditing(Fragment fragment).

6. Extract method expectToOpenScreen().

7. Extract method assertThatExistFragment(string name).

Then all those methods go into regions and all mocks into another region called main cast and the test would look like:

public class AddNamedFragmentCommandTest : BehaviourTest {

private AddNamedFragmentCommand subject;

#region main cast

#region fixture setup

#region given

#region expectations

#region assertions

[TestMethod]

public void AddNamedFragmentCommand_adds_new_Fragment(){

            string name = "Fragment1";

Fragment fragment = new Fragment(name);

givenTheFragment(name);

expectToSave(fragment);

expectToRefreshTreeNodes();

expectToOpenScreen();

subject.execute;

assertThatExistFragment(name).

}

}

*: I hide ReplayAll in the last expectCall. Shame on me, but takes away the noise from the test. The VerifyAll() call is hidden in the [TestCleanup].

Javier Lozano wrote re: Spiking a new syntax for integration tests. Whaddaya think?
on 01-18-2008 10:36 AM

I totally agree with jason that the new syntax is a bit harder to read but easy to understand since it "reads" as if you were expressing it verbally.  However, like Chris said, C# does "get in the way" since, like you said, "it's not Ruby".

For my tests, I'm currently using the AutoMockingContainer that's part of Rhino.Testing and it's working pretty well for me in conjunction with the Record and Playback syntax.  For example, here's  snippet:

using (Record)

{

   PatientEntity patient = Mock<PatientEntity>();

   PatientEntityDTO dto = Mock<PatientEntityDTO>();

   info.PatientEntity = dto;

   IEntityTranslator<PatientEntityDTO, PatientEntity> translator =

       Get<IEntityTranslator<PatientEntityDTO, PatientEntity>>();

   Expect.Call(translator.ConvertTo(dto)).Return(patient);

   IFormXmlProvider<PatientEntity> xmlProvider = Get<IFormXmlProvider<PatientEntity>>();

   Expect.Call(xmlProvider.GetFormXml(patient, null)).Return(eventXml);

   LastCall.IgnoreArguments();

}

My Get<T>() method is part of my AutoMockingFixture class that handles all the "dirty" syntax needed to create mocks or get items from the container.

Granted my tests are probably not as complex as yours.

Jeremy D. Miller wrote re: Spiking a new syntax for integration tests. Whaddaya think?
on 01-18-2008 10:50 AM

@Javier,

I have an automocking container that basically does the same thing as building the mocks and class under test as the Get<T>.  I just wanted to flirt with going farther with a new mini language that's built around the automocking container.  I think I'm going to try it just a little bit more, but only with C#3.

"Granted my tests are probably not as complex as yours."

That might be a design problem in *my* code.

Mike Suarez wrote re: Spiking a new syntax for integration tests. Whaddaya think?
on 01-18-2008 11:45 AM

I was showing this post to a coworker and I did a bit more of refactor. Here’s the result:

[TestMethod]

public void AddNamedFragmentCommand_adds_new_Fragment(){

 givenTheFragment(name);

 expectToSave(fragment);

 expectToRefreshTreeNodes();

 expectToShowInScreen(fragment);

 subject.execute();

 assertThatExistFragment(name).

}

Colin Jack wrote re: Spiking a new syntax for integration tests. Whaddaya think?
on 01-18-2008 12:20 PM

You seem to be using integration and interaction interchangeably in this article, is that intentional?

I don't find the new one particularly readable. Its probably better but still not that understandable and Mike Suarez's solution seems to me to be more readable.

Jeremy D. Miller wrote re: Spiking a new syntax for integration tests. Whaddaya think?
on 01-18-2008 12:41 PM

@Colin,

That was me being sloppy.  It should always read "interaction."  I thought I caught that.

Mike's is clearly more readable, but a bit more work.  I was trying to make the mechanics easier too.

Mike Suarez wrote re: Spiking a new syntax for integration tests. Whaddaya think?
on 01-18-2008 1:23 PM

@Jeremy

I don’t actually refactor into those given/expect/assert methods. I write then in Pseudo Code style, not as comments (like in Code Complete) but as real functions and Resharper generates then. The extra effort is moving the pack into a region but it pays off in readability and Test as living documentation. What you see in a class after collapsing all is something like:

[TestClass]

public class AddNamedFragmentCommandTest : BehaviourTest {

private AddNamedFragmentCommand subject;

#region main cast

#region supporting roles

#region given

#region expectations

#region assertions

[TestMethod]

public void AddNamedFragmentCommand_adds_new_Fragment(){}

}

*: fragment and name are declared in the supporting roles section.

You can start expanding accordingly your depth interest. If that’s going too far I’m not so sure. A friend recommended throwing the Symetry patterm from Becks new book. Plus I’d like to add the “treat your tests as your production code” + Custom Assertions from Meszaros’s . Anyways I’m a n00b and I tend to go too far. So it’s a good sign that you like my approach as more readable but it looks like is not worst the effort. Thanks

Jeremy D. Miller wrote re: Spiking a new syntax for integration tests. Whaddaya think?
on 01-18-2008 6:54 PM

@Mike,

Gotcha.  I think I like that.  I've done that a bit with Extract Method refactorings, but I think you're on to something.

Colin Jack wrote re: Spiking a new syntax for integration tests. Whaddaya think?
on 01-19-2008 2:06 PM

Yeah I like that too, testing by intention at the high level seems to me like a good way to go.

The evolution of Unit Testing Syntax and Semantics | Developer Home wrote The evolution of Unit Testing Syntax and Semantics | Developer Home
on 05-22-2008 6:33 PM

Pingback from  The evolution of Unit Testing Syntax and Semantics | Developer Home

Bob on Medical Device Software » Blog Archive » Drop, Spike… Huh? wrote Bob on Medical Device Software &raquo; Blog Archive &raquo; Drop, Spike&#8230; Huh?
on 05-24-2008 7:46 PM

Pingback from  Bob on Medical Device Software  &raquo; Blog Archive   &raquo; Drop, Spike&#8230; Huh?

Recent Links Tagged With "rhinomocks" - JabberTags wrote Recent Links Tagged With "rhinomocks" - JabberTags
on 03-05-2009 4:34 PM

Pingback from  Recent Links Tagged With "rhinomocks" - JabberTags

Add a Comment

(required)  
(optional)
(required)  
Remember Me?