How about a StoryTeller Preview Release!

 

Ok, so first I betcha wanna know what StoryTeller is and why you might care.  StoryTeller is my attempt at creating a tool that’s useful for creating “Executable Requirements” in .Net development.  The Hanselman caught me talking about StoryTeller and dreaming the impossible dream on Channel 9 when it was still pretty crude, but tonight I’m ready to start showing it off to the world and proving that the dream of Executable Requirements is fully possible.  In short, executable requirements are detailed requirements expressed as human readable, automated acceptance tests.  Think of StoryTeller as a tool specifically engineered to rapidly create External DSL’s to specify and test the behavior of a .Net application.

Previously, the only game in town for this in the .Net space has been to use FitNesse or try out a Ruby tool like Cucumber to drive .Net code.  While Cucumber/Ruby tooling will definitely give you readable tests/specifications, there is inevitably some friction from using Ruby to drive an underlying system written in C# (or VB.Net or F#).  With FitNesse you can certainly write the Fixture testing code in C#, but FitNesse has issues.  I’ve used FitNesse very extensively on past projects and came away with a bad taste in my mouth.  I like the potential and the conceptual usage of FitNesse, but I always found the mechanics to be time consuming and error prone (and don’t you even dare commenting that it’s just a “communication” problem with my teams).  I started the StoryTeller project three years ago to build a better test editing and management tool for the FitNesse engine.  Along the way I realized that I was never going to be able to fulfill my vision with the FitNesse engine the way it was and broke down last fall and started over from scratch.  My team at work has dogfooded StoryTeller for the past 6 months and I think it’s finally ready for at least a preview release.

I’ll be blogging much more about StoryTeller over the next 10 days or so, but this should be enough for early adopters to get started.  Ask questions in the comments here if you would, and that’ll let me know what I need to write up.  Tomorrow I’ll do the smackdown post and talk about why and how StoryTeller is going to make FitNesse completely obsolete in the .Net ecosystem.

StoryTeller is released under the Apache 2.0 license, and is only available for .Net 3.5.  I’ve very much optimized it for .Net 3.5 usage.

 

Getting Started

The first thing to do is to download the binaries from Trigris here.  Next, you might want to pull down the source code to look at the StoryTeller.Samples and StoryTeller.Gallery projects for plenty of sample tests and fixtures.  You can download the source code via Subversion here (uid is guest, password is blank for anonymous access).  Once you unzip the binaries, let’s start up a sample project for StoryTeller testing.  The first thing to do is to add a reference to the StoryTeller.dll library:

ST-AddReference

Next, I would suggest that you add two folders under the root for “Tests” and for “Fixtures” like this:

ST-ProjectStructure

You certainly don’t have to put the actual test folders and files under a Visual Studio project, but we’ve found it to be helpful just to use VisualSVN to get the files checked in.  Once that’s done, let’s talk about the pieces:

  1. A TestRunner.  Your “system under test” probably needs some sort of bootstrapping before it can be executed.  In StoryTeller, you build a custom TestRunner class by inheriting from the basic TestRunner class and overriding the setUp() and tearDown() methods.  You also use the TestRunner object to locate and find the…
  2. Fixtures and Grammars – Before you write tests, you need a vocabulary with which to express the tests, and something behind that vocabulary that actually exercises the system under test and records the actual results.  Each action or assertion in a StoryTeller is performed by a “Grammar.”  A related collection of grammars are implemented or collected in a Fixture class (very similar to FitNesse).
  3. Projects.  StoryTeller needs a very small “project” file just to tell the StoryTeller engine where the application directory of the system under test is, the root directory for the test files, and the custom Test Runner to use to run tests.
  4. StoryTellerRunner.  A command line runner for automated build integration
  5. StoryTellerUI.exe.  An xUnit flavored test editing and execution tool written in Jeremy style WPF.
  6. History file.  StoryTeller is pretty crude at the moment, so you’ll have to help the StoryTellerUI.exe project along a little bit by editing a project history file called “history.xml” in the same directory as the StoryTellerUI.exe file.

 

 

Building a TestRunner

There isn’t much to building a TestRunner.  Simply inherit from the StoryTeller.TestRunner class, specify which Fixtures to use (if it looks like StructureMap, it’s because it’s there under the covers), and override the setUp/tearDown methods as appropriate.

    // Before I do anything else,

    public class GalleryTestRunner : TestRunner

    {

        private SystemUnderTest _system;

 

        // Automatically scans the containing assembly for any public classes

        // that implement IFixture and makes them available for use

        public GalleryTestRunner()

            : base(x => x.AddFixturesFromThisAssembly())

        {

        }

 

        protected override void setUp(ITestContext context)

        {

            // Do any necessary bootstrapping just before a test run

            // ITestContext is effectively an IoC container, so you

            // might be registering your application services here

            _system = new SystemUnderTest();

            context.Store(_system);

        }

 

        protected override void tearDown(ITestContext context)

        {

            // Do any post-Test run cleanup

            _system.CleanUp();

        }

 

        protected override void setUpEnvironment()

        {

            // This method runs once before the very first test.

            // We use this method to spin up the Selenium RC

            // proxy server and Selenium session

        }

 

        protected override void tearDownEnvironment()

        {

            // This method runs once after all tests are executed

        }

    }

 

 

 

Creating a Project File

Your project file needs to follow this format:

<?xml version=1.0?>

<Project xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance xmlns:xsd=http://www.w3.org/2001/XMLSchema>

  <BinaryFolder>bin\debug</BinaryFolder>

  <TestFolder>Tests</TestFolder>

  <TimeoutInSeconds>30</TimeoutInSeconds>

  <TestRunnerTypeName>StoryTeller.Gallery.GalleryTestRunner, StoryTeller.Gallery</TestRunnerTypeName>

  <Name>Gallery</Name>

</Project>

The TestRunnerTypeName is the assembly qualified name of the custom test runner.  “TestFolder” tells StoryTeller where to find the test files and “BinaryFolder” tells StoryTeller where the base application directory should be during testing.  Both folders can be specified as either an absolute rooted path or as a relative path from the project file.

 

 

The History File

Edit and put this file in the StoryTeller binary directory with the name “history.xml.”  This is mandatory for the preview, but there will eventually be some wizard-y stuff to do it for you.

<?xml version=1.0?>

<ProjectHistory xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance xmlns:xsd=http://www.w3.org/2001/XMLSchema>

<Projects>

  <ProjectToken>

    <Name>Gallery</Name>

    <Filename>..\..\..\StoryTeller.Gallery\gallery.xml</Filename>

  </ProjectToken>

    <ProjectToken>

      <Name>Math</Name>

      <Filename>..\..\..\..\samples\math.xml</Filename>

    </ProjectToken>

    <ProjectToken>

      <Name>Grammars</Name>

      <Filename>..\..\..\..\samples\grammars.xml</Filename>

    </ProjectToken>

    <ProjectToken>

      <Name>Dovetail V5</Name>

      <Filename>C:\code\Blue\StoryTeller.xml</Filename>

    </ProjectToken>

  </Projects>

 

</ProjectHistory>

 

 

Fixtures and Grammars

Ok, enough with the set up, let’s write ourselves some test Fixtures.  Let’s say we’re coding up a new calculator software package (because the world really needs that).  Our “system under test” might be a class with this signature:

    public interface ICalculator

    {

        double Value { get; set; }

        void MultiplyBy(double value);

        void DivideBy(double value);

        void Add(double value);

        void Subtract(double value);

    }

 

The tester, customer, and I go up to a whiteboard and start to sketch out the vocabulary we want to use to specify behavior.  We initially decide that we want these “grammars:”

  1. Start with #
  2. Add #
  3. Subtract #
  4. Divide by #
  5. Multiply by #
  6. The value should be #

Now that we’ve agreed on how we want the tests to read, I create a fixture class like this below:

 

    public class CalculatorFixture : Fixture

    {

        private Calculator _calculator = new Calculator();

 

        [FormatAs("Start with {value}")]

        public void StartWith(double value)

        {

            _calculator.Value = value;

        }

 

        [FormatAs("Add {value}")]

        public void Add(double value)

        {

            _calculator.Add(value);

        }

 

        [FormatAs("Subtract {value}")]

        public void Subtract(double value)

        {

            _calculator.Subtract(value);

        }

 

        [FormatAs("Multiply by {value}")]

        public void MultiplyBy(double value)

        {

            _calculator.MultiplyBy(value);

        }

 

        [FormatAs("Divide by {value}")]

        public void DivideBy(double value)

        {

            _calculator.DivideBy(value);

        }

 

        [FormatAs("The value should be {value}")]

        [return: AliasAs("value")]

        public double TheValueShouldBe()

        {

            return _calculator.Value;

        }

    }

I now fire up the UI (that’s pointed at our project file through the history.xml file) and add a new test:

ST-AddTest

Which brings up this screen once I enter the new test name in a dialog not shown here:

ST-TestScreen

From the test screen I first select a new section using the CalculatorFixture by clicking on the left “action pane” selection for “Add Calculator”

ST-TestWithSection

Now, I’ll add some grammar steps to the test and fill in the blanks to edit the test:

ST-EditSentences

 

Right here and now is the biggest single advantage of StoryTeller over FitNesse.  I’m editing a test with an intelligent form that conforms to the grammars that I specified.  All I have to do is fill in the blanks by quickly tabbing through the textboxes in the test.  After I run the test, I get these results:

ST-SuccessfulTest

 

What Can It Do?

I’ve shown the very easiest usage of StoryTeller.  In the coming days I’ll show the full gamut of grammar authoring as well as discuss how to manage StoryTeller tests inside your configuration management and continuous integration schemes.

About Jeremy Miller

Jeremy is the Chief Software Architect at Dovetail Software, the coolest ISV in Austin. Jeremy began his IT career writing "Shadow IT" applications to automate his engineering documentation, then wandered into software development because it looked like more fun. Jeremy is the author of the open source StructureMap tool for Dependency Injection with .Net, StoryTeller for supercharged acceptance testing in .Net, and one of the principal developers behind FubuMVC. Jeremy's thoughts on all things software can be found at The Shade Tree Developer at http://codebetter.com/jeremymiller.
This entry was posted in StoryTeller. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • Larry T

    Hi Jeremy,

    First off – nice work! I see this being a very useful addition to our test framework!

    Also wondering if you plan to add some Nant integration ability – I’m trying to use TestRunner to exec my tests but its just not the cleanest when tying to track different acceptance and regression tests.

    Thanks

  • Indrani

    Hi Jeremy,

    Thank you for the StoryTeller. How can I use the StoryTellerRunner to run my prdefined tests automatically ? Can you please provide some step by step “what to do”s ?

    Thanks.

  • http://www.kscastings.com/pro/Cnc-Screw.htm Cnc Screw

    keep the good work ,so great.

  • dru

    awesome stuff

  • http://www.drivensoftware.net Kevin Trethewey

    Hi Jeremy,

    Very exciting release – this is something I have been waiting/praying for!

    I have been through the samples and written a few tests of my own, and I really like the ideas here. Would it be possible for you to post some real world implementation examples? I don’t need the production code that they actually test, I just want to get a feel for the structure and patterns that you are using in your implementation.

    Thanks again!

  • http://thetestingblog.com Marisa Seal

    Hi Jeremy,

    I am really eager to get started with StoryTeller – but the one thing that’s preventing me from being super-excited is that I’m wondering how data (from the DB) verifications might be structured.

    I’m imagining “The first name in the Person table should be Maria” and “The last name in the Person table should be Melchiore” etc. and that seems really cumbersome to me.

    Do you have a suggestion for general approach for this? Or would you share what your team does?

    There is probably a simple solution I don’t see yet – I hope that’s the case because StoryTeller looks frickin’ cool!

    Marisa

  • John Dhom

    Hey Jeremy,

    Gratz on releasing the preview.

    Just an fyi… StoryTeller.Gallery.csproj is missing in svn.

    /jhd

  • http://thedwalker.blogspot.com Thedric Walker

    Wow, it looks like .Net has finally gotten a cucumberesque testing framework.

  • http://shane.jscconsulting.ca Shane Courtrille

    Took so little time to get working I was in shock..

  • http://www.mickdelaney.com mick delaney

    well done jeremy… good to see the .net world catching up with the ruby guys.. :-)

  • Dan F

    Dude! Looks awesome

  • http://apobekiaris.blogspot.com Apostolis Bekiaris

    Revolutionary!!!

  • http://www.coderjournal.com Nick Berardi

    Looks promising.

  • http://gratdevel.blogspot.com Neil Kerkin

    Great to see Storyteller progressing.

    Would love to see a “who does what when” style writeup of how you are using storyteller in your process/workflow.

    Also a more in-depth example would be greatly appreciated.

  • http://scottlittlewood.blogspot.com Scott Littlewood

    Great news ! Looking forward to trying this out for a small project. I’ll definitely be interested in seeing where this goes.

  • Ken

    I can’t wait to start playing with it! Great work!

  • http://twitter.com/pnschofield Paul Schofield

    And there was much rejoicing!

  • http://www.cprieto.com cristian

    Awesome! keep the good work man!

  • http://www.cprieto.com cristian

    AWESOME!!! Keep the good work!

  • http://phatboyg.lostechies.com/ Chris Patterson

    Awesome and congratulations on getting it up!

    I’m going to see how we can build fixtures to test MassTransit sagas!