Introduction to using the StoryTeller Alpha #1

The first ship is away!  The first ship is away!   Download the StoryTeller v0.50 at http://storyteller.tigris.org/servlets/ProjectDocumentList?folderID=0.

At Scott Bellware's request I'm making an alpha release of StoryTeller today along with a little introduction to the capabilities that are built into StoryTeller so far.  If you're willing to forgo the Wiki portion of FitNesse and edit html by an editor, StoryTeller is ready to go.  For those of us with an investment in FitNesse, there are also some tools in StoryTeller to supplement the usage of FitNesse to smooth out some of the kinks of FitNesse.  Just to be clear, StoryTeller still uses the exact same .Net version of the FIT engine used in FitNesse (http://fitnessedotnet.sourceforge.net).   The only caveat is that StoryTeller will be .Net 2.0+ from the onset and FitnesseDotNet is not yet ready to make that leap.  StoryTeller might temporarily come with a forked version of the FIT engine that we use on my current project that has added support for .Net 2.0 specific features (generic types, nullable types, and enums).  Here's a summary of the functionality in this release:

  • Organize FIT tests for an application by a nested Suite hierarchy
  • SetUp & TearDown support
  • Run tests inside of Visual Studio.Net
    • Pull the test definition from a FitNesse (or any other web) url
    • Embed Wiki text inside your code
    • Define the test and table structure programmatically
  • A drop in replacement for FitNesse's TestRunner.exe
  • Run tests from NAnt
    • by tag
    • from FitNesse files
  • Run tests in a clean AppDomain
  • The UserInterface is included, but all it does at this point is run a single test at one time.  If you want to play with the client, I suggest that you do it from the VS.Net solution to avoid changing configuration.

 

I am assuming a basic familiarity with FIT testing for the rest of the post.  See FitNesse.org for more background on FIT concepts.

 

Domain Terms

StoryTeller's core domain model is a composite pattern consisting of classes implementing the ILeaf interface.  Many aggregate operations take advantage of the Composite & Visitor Pattern combination (more on that later).  The individual classes/terms are:

  • Table – Stores and structures the information for a single html table within a FIT test
  • Row – A row of cells inside a Table
  • Cell – A single cell within a Table row
  • Comment – A piece of freeform text within a Test.  At this point, StoryTeller does not support any kind of markup inside of Comments.
  • Test – Represents a logical Test (duh).  Contains a mixture of Comment's, Table's, and Include's
  • Fragment – A subclass of Test.  Represents a reusable "fragment" of FIT html that can be included inside Test's.  StoryTeller follows the same convention as FitNesse for "SetUp" and "TearDown" fragments
  • Include – a reference inside a Test to a Fragment object
  • Suite – A related group of Test's.  Can also include other child Suite's in an n-deep composite structure
  • SystemUnderTest – The logical "System" being tested.  Instances of the ISystemUnderTest interface know how to persist and retrieve their test data, and execute tests. 

 

File Structure

In it's default mode, StoryTeller structures tests into a hierarchical file structure.  The top folder is the "SystemUnderTest" folder.  Inside this folder are folders for the top level Suite's and a FixtureLibrary file.  Roughly, a system folder is going to look like this:

  • Root
    • FixtureLibrary.xml
    • Suite #1
    • Suite #2

Inside the source tree is a folder called \TestData\SimpleTestApplication that can be used as an example.

FixtureLibrary

One of the changes from FitNesse to StoryTeller is the manner in which Fixture classes are aliased inside the FIT tables.  My current concept is to have an object that represents the entire aliasing of Fixture classes as well as information related to editing Fixture tables by Fixture type.  At this point, all the FixtureLibrary class supports is registering all of the concrete Fixture types within a single assembly into the FIT engine.  The point being that if you have a Fixture class called "MyAssembly.Namespace1.Namespace2.Namespace3.MyFixture," you can reference it in FIT tables as simple "MyFixture" instead of the full name that might scare off your testers.  Create a file called "FixtureLibrary.xml" in the root directory of the system to list the assemblies that contain FIT Fixture classes like this:

<?xml version="1.0"?>

<FixtureLibrary xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">

  <Assemblies>

    <Assembly>StoryTeller.TestFixtures1</Assembly>

    <Assembly>StoryTeller.TestFixtures2</Assembly>

  </Assemblies>

</FixtureLibrary>

 It's simply an Xml-serialized memento of the FixtureLibrary class.

 

Suite

At this point StoryTeller requires a file named "Suite.xml" in each Suite folder that contains the Suite "Key" (same as the folder name).  That file looks like this:

<?xml version="1.0"?>

<Suite>

  <Key>Grandchild</Key>

  <Tags />

</Suite>

 I'll explain the tagging later

The Suite folder might look like this:

  • Parent Suite
    • Child Suite #1
    • Child Suite #2
    • Test1.htm
    • Test2.htm
    • Setup.fragment
    • TearDown.fragment
    • DataSetup.fragment 

 

 

Test 

I do think I'll create a mode later that lets you save tests in the FitNesse Wiki text format, but for right now the tests are stored in minimal XHTML (StoryTeller parses the file as XML!) files with an "htm" extension like this one:

<html>

    <head>

        <!– Test "Key" should be in the title node –>

        <title>The First Test</title>

    </head>

    <body>

 

        <h2>The First Test</h2>

 

        <!– The tags are read from any <li> node with text of form {key} = {value} –>

        <h4>Tags</h4>

        <ol>

            <li>category = regression</li>

            <li>category1 = regression1</li>

        </ol>

 

        <!– Comments need to be nested inside <p> tags –>

        <p>Some comment</p>

 

        <!– Tables are in <table> elements (surprise), all formatting will be ignored –>

        <Table border="1">

           <tbody>

               <TR>

                   <td>A</td>

                   <td>B</td>

               </TR>

               <tr>

                   <TD>C</TD>

                   <td>D</td>

               </tr>

           </tbody>

        </Table>

 

        <p>Another comment</p>

    </body>

</html>

 

Fragment

Fragments are formatted exactly like Test's, but saved to a ".fragment" extension.  In the course of writing this post I realized that I don't yet support reading and writing Include references to the test html.  I think that this will be added soon.

 

Tagging

You can add tags to both Suites and Tests to organize and find Tests by category.  Tests and child Suites inherit tags from their parent Suites if not explicitly defined in the Test.  Some usages for tagging might be:

  • The FIT workflow that I try to use is to define FIT tests that are "in flight" as Acceptance tests, and FIT tests from previously completed stories as Regression tests.  The primary reason being that failures in Regression tests will cause a FIT run to fail, while Acceptance tests will not.
  • Maybe you group Tests into Suites by component or feature.  You might use tags to further categorize the tests by user story or task.
  • Assigning Tests to individuals.  You might tag Tests to indicate to the Testers that the tests are incorrect, or the Testers might tag a Test to let developers know that the Test is not complete and safe to ignore.

 

NAnt Tasks

I've built a couple of custom NAnt tasks as entry points to the StoryTeller engine.  If there's any demand I'll add MSBuild equivalents as well.  To use any of these tasks, StoryTeller.dll and StoryTeller.Tasks.dll must be reachable by NAnt.  Make it easier on yourself and just copy the dll's into the same folder as the NAnt executable.  Right off the bat, if you're using FitNesse, here's an easy way to run FitNesse tests from a NAnt build inside of CruiseControl.Net.

    <storyteller.runfitnesse

      failonerror="false"

      systemName="Fitnesse Test Run"

      description="Regression Tests"

      fixtureLibraryFile="${fixture.library.file}"

      binaryFolder="${test.app.dir}"

      testFolder="${fitnesse.dir}"

      outputFile="${results.dir}\RegressionTestsOutput.htm"

      />

"testFolder" is the directory containing the FitNesse tests you want to run.  "binaryFolder" is the directory that contains the binary assemblies and configuration files.  The task works by first importing the FitNesse tests into the StoryTeller internal structure, then executes the tests by creating a new AppDomain with the ApplicationBase pointing to the "binaryFolder."  StoryTeller.dll must be deployed to the "binaryFolder." 

Import an existing FitNesse test folder to StoryTeller with:

    <storyteller.importfitnesse

      fitnesseFolder="${fitnesse.dir}"

      destinationFolder="${imported.test.dir}"

      />

  

Finally, you can run actual StoryTeller test structures with:

    <storyteller.testbatch

      description="All Tests"

      outputFile="${results.dir}\RunAllTests.results.htm"

      binaryFolder="${test.app.dir}"

      testFolder="${sample.test.dir}"

      />

 

    <storyteller.testbatch

      description="All Tests"

      outputFile="${results.dir}\RunWithTags.results.htm"

      binaryFolder="${test.app.dir}"

      testFolder="${sample.test.dir}">

        <tag category="Story" value="Story1"/>

    >

 

 

In writing this up I noticed that I haven't added support for running a batch of tests by Suite. That will be added in the next alpha.

Running FitNesse Tests from NUnit

Finally, the one feature, as simple as it is, that will make even the most diehard FitNesse fan use StoryTeller — running FitNesse tests from within NUnit tests to debug or just to run the FitNesse tests locally without deploying the code or importing the FitNesse tests.

Run a FitNesse Test Locally 

        [Test]

        public void RunFromWebPage()

        {

            TestRunner runner = new TestRunner();

            runner.ReferenceAssembly(Assembly.GetExecutingAssembly().FullName);

 

            TestResult result = runner.RunFromWebPage("http://MyServer:8080/SuiteAcceptanceTests/Test001");

            result.AssertSuccess();

 

            // if you need to,

            result.OpenResultsInBrowser();

        }

Build the FitNesse Test in Code

            TestRunner runner = new TestRunner();

            runner.ReferenceAssembly(Assembly.GetExecutingAssembly().FullName);

 

            Test test = new Test();

            test.AddTable(typeof (ArithmeticFixture));

            test.AddRow("X,Y,Add()");

            test.AddRow("2,2,4");

            test.AddRow("A,4,6");

 

            TestResult result = runner.ExecuteTest(test);

            result.AssertSuccess();

Lastly, with Wiki Text

        [Test]

        public void ExecuteWikiText()

        {

            TestRunner runner = new TestRunner();

            runner.ReferenceFixtureType(typeof (ArithmeticFixture));

 

            TestResult result =

                runner.ExecuteWikiText(

                    @"

                !|ArithmeticFixture|

                |X|Y|Add()|

                |2|2|4|

                |2|3|5|

                |2|3|6|

                        ");

 

            Assert.AreEqual(2, result.Counts.Right);

            Assert.AreEqual(1, result.Counts.Wrong);

        }

About Jeremy Miller

Jeremy is the Chief Software Architect at Dovetail Software, the coolest ISV in Austin. Jeremy began his IT career writing "Shadow IT" applications to automate his engineering documentation, then wandered into software development because it looked like more fun. Jeremy is the author of the open source StructureMap tool for Dependency Injection with .Net, StoryTeller for supercharged acceptance testing in .Net, and one of the principal developers behind FubuMVC. Jeremy's thoughts on all things software can be found at The Shade Tree Developer at http://codebetter.com/jeremymiller.
This entry was posted in Uncategorized. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • EPiddy

    Jeremy, Any progress on this one? We’ve found it useful in order to get our “Actual Code Coverage” from NCover by wrapping our FitNesse suites/pages in NUnit classes (like in the Run Fitnesse Tests Locally section above).

    For projects that have tons of FitNesse tests but are lacking somewhat on NUnit tests, it has been great for finding gaps that we haven’t covered (or thought we covered) in our acceptance testing.

    Keep up the good work!

  • http://www.bellware.net ScottBellware

    Looking forward to playing with it this week. Thanks!