Ok, so first I betcha wanna know what StoryTeller is and why you might care. StoryTeller is my attempt at creating a tool that’s useful for creating “Executable Requirements” in .Net development. The Hanselman caught me talking about StoryTeller and dreaming the impossible dream on Channel 9 when it was still pretty crude, but tonight I’m ready to start showing it off to the world and proving that the dream of Executable Requirements is fully possible. In short, executable requirements are detailed requirements expressed as human readable, automated acceptance tests. Think of StoryTeller as a tool specifically engineered to rapidly create External DSL’s to specify and test the behavior of a .Net application.
Previously, the only game in town for this in the .Net space has been to use FitNesse or try out a Ruby tool like Cucumber to drive .Net code. While Cucumber/Ruby tooling will definitely give you readable tests/specifications, there is inevitably some friction from using Ruby to drive an underlying system written in C# (or VB.Net or F#). With FitNesse you can certainly write the Fixture testing code in C#, but FitNesse has issues. I’ve used FitNesse very extensively on past projects and came away with a bad taste in my mouth. I like the potential and the conceptual usage of FitNesse, but I always found the mechanics to be time consuming and error prone (and don’t you even dare commenting that it’s just a “communication” problem with my teams). I started the StoryTeller project three years ago to build a better test editing and management tool for the FitNesse engine. Along the way I realized that I was never going to be able to fulfill my vision with the FitNesse engine the way it was and broke down last fall and started over from scratch. My team at work has dogfooded StoryTeller for the past 6 months and I think it’s finally ready for at least a preview release.
I’ll be blogging much more about StoryTeller over the next 10 days or so, but this should be enough for early adopters to get started. Ask questions in the comments here if you would, and that’ll let me know what I need to write up. Tomorrow I’ll do the smackdown post and talk about why and how StoryTeller is going to make FitNesse completely obsolete in the .Net ecosystem.
StoryTeller is released under the Apache 2.0 license, and is only available for .Net 3.5. I've very much optimized it for .Net 3.5 usage.
Getting Started
The first thing to do is to download the binaries from Trigris here. Next, you might want to pull down the source code to look at the StoryTeller.Samples and StoryTeller.Gallery projects for plenty of sample tests and fixtures. You can download the source code via Subversion here (uid is guest, password is blank for anonymous access). Once you unzip the binaries, let’s start up a sample project for StoryTeller testing. The first thing to do is to add a reference to the StoryTeller.dll library:
Next, I would suggest that you add two folders under the root for “Tests” and for “Fixtures” like this:
You certainly don’t have to put the actual test folders and files under a Visual Studio project, but we’ve found it to be helpful just to use VisualSVN to get the files checked in. Once that’s done, let’s talk about the pieces:
- A TestRunner. Your “system under test” probably needs some sort of bootstrapping before it can be executed. In StoryTeller, you build a custom TestRunner class by inheriting from the basic TestRunner class and overriding the setUp() and tearDown() methods. You also use the TestRunner object to locate and find the…
- Fixtures and Grammars – Before you write tests, you need a vocabulary with which to express the tests, and something behind that vocabulary that actually exercises the system under test and records the actual results. Each action or assertion in a StoryTeller is performed by a “Grammar.” A related collection of grammars are implemented or collected in a Fixture class (very similar to FitNesse).
- Projects. StoryTeller needs a very small “project” file just to tell the StoryTeller engine where the application directory of the system under test is, the root directory for the test files, and the custom Test Runner to use to run tests.
- StoryTellerRunner. A command line runner for automated build integration
- StoryTellerUI.exe. An xUnit flavored test editing and execution tool written in Jeremy style WPF.
- History file. StoryTeller is pretty crude at the moment, so you’ll have to help the StoryTellerUI.exe project along a little bit by editing a project history file called “history.xml” in the same directory as the StoryTellerUI.exe file.
Building a TestRunner
There isn’t much to building a TestRunner. Simply inherit from the StoryTeller.TestRunner class, specify which Fixtures to use (if it looks like StructureMap, it’s because it’s there under the covers), and override the setUp/tearDown methods as appropriate.
// Before I do anything else,
public class GalleryTestRunner : TestRunner
{
private SystemUnderTest _system;
// Automatically scans the containing assembly for any public classes
// that implement IFixture and makes them available for use
public GalleryTestRunner()
: base(x => x.AddFixturesFromThisAssembly())
{
}
protected override void setUp(ITestContext context)
{
// Do any necessary bootstrapping just before a test run
// ITestContext is effectively an IoC container, so you
// might be registering your application services here
_system = new SystemUnderTest();
context.Store(_system);
}
protected override void tearDown(ITestContext context)
{
// Do any post-Test run cleanup
_system.CleanUp();
}
protected override void setUpEnvironment()
{
// This method runs once before the very first test.
// We use this method to spin up the Selenium RC
// proxy server and Selenium session
}
protected override void tearDownEnvironment()
{
// This method runs once after all tests are executed
}
}
Creating a Project File
Your project file needs to follow this format:
<?xml version="1.0"?>
<Project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<BinaryFolder>bin\debug</BinaryFolder>
<TestFolder>Tests</TestFolder>
<TimeoutInSeconds>30</TimeoutInSeconds>
<TestRunnerTypeName>StoryTeller.Gallery.GalleryTestRunner, StoryTeller.Gallery</TestRunnerTypeName>
<Name>Gallery</Name>
</Project>
The TestRunnerTypeName is the assembly qualified name of the custom test runner. “TestFolder” tells StoryTeller where to find the test files and “BinaryFolder” tells StoryTeller where the base application directory should be during testing. Both folders can be specified as either an absolute rooted path or as a relative path from the project file.
The History File
Edit and put this file in the StoryTeller binary directory with the name “history.xml.” This is mandatory for the preview, but there will eventually be some wizard-y stuff to do it for you.
<?xml version="1.0"?>
<ProjectHistory xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Projects>
<ProjectToken>
<Name>Gallery</Name>
<Filename>..\..\..\StoryTeller.Gallery\gallery.xml</Filename>
</ProjectToken>
<ProjectToken>
<Name>Math</Name>
<Filename>..\..\..\..\samples\math.xml</Filename>
</ProjectToken>
<ProjectToken>
<Name>Grammars</Name>
<Filename>..\..\..\..\samples\grammars.xml</Filename>
</ProjectToken>
<ProjectToken>
<Name>Dovetail V5</Name>
<Filename>C:\code\Blue\StoryTeller.xml</Filename>
</ProjectToken>
</Projects>
</ProjectHistory>
Fixtures and Grammars
Ok, enough with the set up, let’s write ourselves some test Fixtures. Let’s say we’re coding up a new calculator software package (because the world really needs that). Our “system under test” might be a class with this signature:
public interface ICalculator
{
double Value { get; set; }
void MultiplyBy(double value);
void DivideBy(double value);
void Add(double value);
void Subtract(double value);
}
The tester, customer, and I go up to a whiteboard and start to sketch out the vocabulary we want to use to specify behavior. We initially decide that we want these “grammars:”
- Start with #
- Add #
- Subtract #
- Divide by #
- Multiply by #
- The value should be #
Now that we’ve agreed on how we want the tests to read, I create a fixture class like this below:
public class CalculatorFixture : Fixture
{
private Calculator _calculator = new Calculator();
[FormatAs("Start with {value}")]
public void StartWith(double value)
{
_calculator.Value = value;
}
[FormatAs("Add {value}")]
public void Add(double value)
{
_calculator.Add(value);
}
[FormatAs("Subtract {value}")]
public void Subtract(double value)
{
_calculator.Subtract(value);
}
[FormatAs("Multiply by {value}")]
public void MultiplyBy(double value)
{
_calculator.MultiplyBy(value);
}
[FormatAs("Divide by {value}")]
public void DivideBy(double value)
{
_calculator.DivideBy(value);
}
[FormatAs("The value should be {value}")]
[return: AliasAs("value")]
public double TheValueShouldBe()
{
return _calculator.Value;
}
}
I now fire up the UI (that’s pointed at our project file through the history.xml file) and add a new test:
Which brings up this screen once I enter the new test name in a dialog not shown here:
From the test screen I first select a new section using the CalculatorFixture by clicking on the left “action pane” selection for “Add Calculator”
Now, I’ll add some grammar steps to the test and fill in the blanks to edit the test:
Right here and now is the biggest single advantage of StoryTeller over FitNesse. I’m editing a test with an intelligent form that conforms to the grammars that I specified. All I have to do is fill in the blanks by quickly tabbing through the textboxes in the test. After I run the test, I get these results:
What Can It Do?
I’ve shown the very easiest usage of StoryTeller. In the coming days I’ll show the full gamut of grammar authoring as well as discuss how to manage StoryTeller tests inside your configuration management and continuous integration schemes.
Posted
Mon, Aug 24 2009 11:03 PM
by
Jeremy D. Miller