Introducing Machine.Specifications (or MSpec for short)

As some of you who follow me on twitter know, I’ve been working on Yet Another Context/Specification Framework as an experiment. Yeah, I know we already have NSpec and NBehave, and they’re great and all, but MSpec takes things on from a slightly different angle, and it’s just an experiment (for now). Here’s a sample Description:

[Description]
public class Transferring_between_from_account_and_to_account
{
  static Account fromAccount;
  static Account toAccount;

  Context before_each =()=>
  {
    fromAccount = new Account {Balance = 1m};
    toAccount = new Account {Balance = 1m};
  };
  
  When the_transfer_is_made =()=>
  {
    fromAccount.Transfer(1m, toAccount);
  };
   
  It should_debit_the_from_account_by_the_amount_transferred =()=>
  {
    fromAccount.Balance.ShouldEqual(0m);
  };

  It should_credit_the_to_account_by_the_amount_transferred =()=>
  {
    toAccount.Balance.ShouldEqual(2m);
  };
}

And a TestDriven.NET run:

------ Test started: Assembly: Machine.Specifications.Example.dll ------

Transferring between from account and to account
  When the transfer is made
    * It should debit the from account by the amount transferred
    * It should credit the to account by the amount transferred


2 passed, 0 failed, 0 skipped, took 0.79 seconds.

Err, What?

Different eh? The idea was heavily inspired by Scott Bellware‘s SpecUnit.Net framework he showed at the ALT.NET conference. It also took heavy cues from RSpec and my insanity. I realize that the the code doesn’t look much like C# code and I’m OK with that. Many have and will ask why I don’t just use Boo or RSpec w/ IronRuby eventually or even one of the existing Context/Spec/BDD frameworks. Those are good questions, but my main motivations are tooling and syntax. I enjoy the tooling I get in C# and I personally like the syntax in this library considering the limitations imposed by C#.

How’s it work?

The simplest way to describe it is to compare it to a normal *Unit style testing framework:

  • Description = TestContext
  • Context before_each = SetUp
  • Context before_all = SetUpFixture
  • Context after_each = TearDown
  • Context after_all = TearDownFixture
  • When = Also SetUp, but happens after Context before_each
  • It = Test

Rather than methods and attributes, MSpec uses named delegates and anonymous functions. The only reason for this is readability. You’ll also notice that the fields used in the context are static. This is necessary so that the anonymous functions in the field initializers can access them. Probably the first thing you noticed is the =()=> construct. I won’t mention the names that this was given on twitter, but I think it’s an acceptable thing to have to deal with in exchange for the cleanliness of the rest of the syntax.

Ok, you’re crazy, but how do I try it?

First, this is a very rough cut. Everything is subject to change as we experiment with the language. That said, here’s how you play with it:

  1. Grab the drop here.
  2. Extract it somewhere. Put it somewhere semi-permanent because the TestDriven.NET runner will need a static location for the MSpec TDNet Runner.
  3. If you want TestDriven.NET support, run InstallTDNetRunner.bat
  4. Check out the example in Machine.Specifications.Example. Note that you can run with TD.NET.
  5. Create a project of your own. Just add Machine.Specifications.dll and get started.
  6. Send me feedback! Leave comments, email me, tweet me, whatever.

Also, this is part of Machine, so feel free to take a look at the code and/or submit patches. There’s also a Gallio adapter in there, but I didn’t include it in the release as it’s not quite polished enough yet. If you’re interested in it, talk to me. Special thanks to Scott Bellware, Jeff Brown and Jamie Cansdale for their help and support.

This entry was posted in bdd, frameworks, machine. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • Anonymous

    Everyone, if you’ve recently read this post and then wondered why you can’t even get this example to work, it’s because some of the words used were changed (“When”, is no more for example). Read the updated post here http://codebetter.com/aaronjensen/2008/09/02/mspec-v0-2/, which is more accurate.

  • Aaron Jensen

    There’s a command line runner, you could certainly use that from the command line or add it as an external tool and use that from within VS. Once you get up to a certain # of specs, time them. Then time what it takes to run one spec. Count the # of times you run all specs per day and use that data to show your boss how much money you’ll save if he buys you TD.Net or R# :)

  • GW

    Hi Aaron,

    Do you have to use a test runner such as TestDriven.NET or Resharper? I have next to zero chance of my boss paying for a licence and am wondering if mSpec works with just VS2010?

    Thanks

    Garth

  • Pingback: The Cutting Ledge » Blog Archive » Teach Yourself TDD

  • GW

    “to contribe” – the act of making a contrived contribution.
    :)

  • http://codebetter.com/members/aaronjensen/default.aspx aaronjensen

    You could try compiling the command line runner under Mono. I’ve never done it, but I’m not using anything I can think of that would cause it to not work. Let me know how it goes, thanks!

  • Mike

    I’m using Linux and Monodevelop – is there a test runner that fits that environment?

  • ErikF

    typo: contribed should be contrived :)

  • ErikF

    After reviewing several BDD frameworks for .Net, MSpec has become my favorite. The syntax is so clean and readable. The only puzzle for me was using my code to document the entire user story, not just scenario/assertions – this was something that comes out-of-the-box with NBehave and StoryQ. To that end, I contribed a simple solution that I have used on a few projects. It may be of interest to people who appreciate the readability of MSpec: http://www.codeproject.com/KB/testing/user-story-structure.aspx.

  • http://code.google.com/p/dotnetxp/w/list Kirk

    I have yet another BDD framework for .NET called CSSharp. It attempts to force better coding standards and conventions when writing unit tests by keeping each test to three lines of code. The actual code to implement the test is kept in a separate test state class allowing the developer and reader to focus on the test meaning and human requirements. If anyone is interested, the google code project is here: http://code.google.com/p/dotnetxp/w/list. I would appreciate any feedback.

  • http://www.abhobley.co.uk Assembly Machines Fixtures

    Its a piece of nice work but initially it looks strange . I would love to read your further posts.

  • http://marcinobel.com Marcin Obel

    You can find some useful Resharper templates for MSpec here: http://marcinobel.com/index.php/resharpers-live-templates-for-mspec-bdd-framework/

  • http://codebetter.com/members/aaronjensen/default.aspx aaronjensen

    Thanks Jonny. No plans for parameterized tests, those are different types of tests and as such should use a more suited framework. MSpec isn’t a golden hammer :)

  • JonnyRoc

    I’m lovin’ it!!

    Are there any plans to accomodate parameterized tests?

  • http://www.kodart.com Arthur

    Hi, that’s great! I was also in the same direction making ArtSpec, but your approach is better!

    I would suggest some changes. Instead of:

    Context before_each = SetUp
    Context before_all = SetUpFixture
    Context after_each = TearDown
    Context after_all = TearDownFixture

    introduce:

    Before each = () => {…}
    Before all = () => {…}

    same for:
    After each/all

    Best regards,
    Arthur

  • http://codebetter.com/members/aaronjensen/default.aspx aaronjensen

    Hey Ori,

    I actually released 0.2 several months ago: http://codebetter.com/blogs/aaron.jensen/archive/2008/09/02/mspec-v0-2.aspx

    And since then all changes have been in the actual repository without much announcement:

    http://github.com/machine/machine.specifications/tree/master

    Though I’m probably due for another post on it with the latest info… Thanks!

  • Ori

    Heya guys, promising start on a exciting test framework, just one thing, long time no updates, what gives is 0.1 it ??

  • http://codebetter.com/members/aaronjensen/default.aspx aaronjensen

    Paul, TestDriven.NET

  • http://www.LinkedTraining.com.au Paul Linton

    What’s TD.NET? http://www.td.net is not very interesting nor is http://www.tdnet.com and nothing on the first page of a google search for “td.net” sheds any light

  • http://codebetter.com/members/aaronjensen/default.aspx aaronjensen

    Paul, Thanks! Are you running TD.NET? If so you can just run with debugger. If not, I’d recommend it :) R#’s runner can run with debugger as well. If you’re just running the console runner you’ll have a harder time debugging, but you could always use Debugger.Break() and attach.

  • http://www.LinkedTraining.com.au Paul Linton

    Rob Conery opened my eyes to MSpec and I’m loving it. I have followed all his suggestions and created a toolbar button and html reports.
    How do I debug a failing specification? Setting a breakpoint on the failing line within the Because delegate doesn’t work.
    What am I missing?

  • http://codebetter.com/members/aaronjensen/default.aspx aaronjensen

    Hi Eduardo,

    This looks like a versioning issue. I’m guessing you have some old mspec assemblies laying around somewhere (perhaps the td.net runner if this is from td.net). I’d rebuild mspec and reregister the td.net runner from the built location.

  • http://www.menoncello.net Eduardo Menoncello

    Hi Aaron, I am testing the machine specification, but I have a problem… when I run my tests, the tests break like that:

    TestCase ‘N:Machine.Specifications.Example’
    failed: Method not found: ‘System.Collections.Generic.IEnumerable`1 Machine.Specifications.Explorers.AssemblyExplorer.FindDescriptionsIn(System.Reflection.Assembly, System.String)’.

    thanks for the help

  • Aaron Jensen

    Michael,

    TD.NET allows you to print whatever you like when implementing a custom runner. I just used that.

    It’s not super flexible (errors always print the same, along w/ the stack trace) so I save the errors until the end in TD.NET.

  • Michael

    Hi Aaron,

    You might have already mentioned that somewhere but I can’t seem to find it.. How do you manage to have a pretty print output after you run your test in TestDriven.NET. ?

    Thanks,
    Take care

  • Aaron Jensen

    James,

    I hear you. It’s definitely a stretch of C#. The benefits of using C# are clear though. The tool and the framework support make it worth it in my opinion. Like I said in the post though, this is an experiment. I’ve got no ego behind this and I’m not pushing for widespread adoption. I’m just kind of dropping it on the community and seeing if it sticks at all :)

  • http://www.jamesthigpen.com James Thigpen

    This is very awesome. However…

    While I understand your choice for doing it in C#, I feel that by not using a language that would allow for a more fluent DSL syntax, you have created something that will hamper it’s adoption.

    The way you are using C# goes against the grain of how users are taught to use C#. static fields and what look like fancy function but end in semicolons. If I brought this to my team it’d be very confusing for them because it *is* C# but it’s not the C# they’re use to.

    That could be just dumped into the “learning curve” bucket, but I see it being a recurring problem with the context switching required between MSpec and C# when bouncing between tests and code.

    It could be argued that the context switching between MPsech C# and “regular” C# when bouncing back and forth between tests and code is no more grating than if it were a completely separate DSL, but I feel that all the syntactic sugar required to make this work prevents me from forgetting that it is C#. There is a certain amount of muscle memory in “private Account fromAccount;”

  • http://blogs.codehaus.org/people/bamboo Rodrigo B. de Oliveira

    That’s pretty cool.

  • http://www.sergiopereira.com/blog Sergio Pereira

    This is brilliant, Aaron.

  • http://blog.bits-in-motion.com/ Jeff Brown

    Neat!
    The =()=> notation with fields is a clever way to work around the fact that we can’t otherwise define new “shapes” for methods.

    hmmm… I can see interesting things coming out of this…

  • http://rhouston.lostechies.com Ray Houston

    Nice work man!

  • http://colinjack.blogspot.com Colin Jack

    I’m not normally one that gets excited by new syntactic sugar but I saw the example you put on twitter and immediately liked it. Very cool.

    Best thing was I looked at it and thought “It”, what’s that about? I was expecting it to be some mad work around but when I downloaded the Machine codebase I found it was really elegant and understandable. Made me feel dumb (in a good way).

    Anyway to me the “=()=>” is not a major issue, it’s at the end of the line and easily ignored. The static field thing threw me a little but I guess with the BDD style of small focussed fixtures it won’t be an issue.

    Only thing that I’m still unsure of is whether just having [Description] is that useful, would I not also want to use something like this group my specifications (especially given that with the context/specification approach I might end up with lots of fixtures for one higher level behavior such as funds transfer). Guess it depends on how you use these specifications but finding ways to group them seems to be important.

    My first impression is also that you are not necessarily covering exactly the same area as NBehave, my feeling it is xBehave is _supposed_ to be most suitable for higher level story tests where as to me the xSpec approach is more generally applicable.