I’ve gotten pinged 4-5 times in the past week about just where I’m at with StoryTeller development. I think I’m finally closing in on something that I could release at least as a preview. Just to prove that I’m still working on it, here’s a sneek preview of the tool. I’m tentatively doing another Elegant Code Cast with David Starr on StoryTeller soon, so just pile on with any questions.
Quick History
I’ve had a love/hate relationship with NFit and/or FitNesse for quite some time. In many ways, I still believe that “Executable Specifications” are the “Holy Grail” of Agile/Lean software development and that belief naturally led me to using first NFit and then FitNesse. However, I increasingly found FitNesse rather painful to use and impossible to transition to client teams. My frustration with FitNesse eventially led me to start a new OSS project called StoryTeller as a replacement for the test editing and management functionality of FitNesse, but keeping the underlying Fit engine. I coded on that for about 2 years with no more than middling enthusiasm and minimal success before I finally threw my hands up in the air over the FitNesse testing engine and junked the whole StoryTeller project.
The New StoryTeller
Late last year I started a complete ground up rewrite of StoryTeller with a totally different testing engine that we’re using to write acceptance tests and integration tests at Dovetail. I haven’t made an official release yet, but as of right now StoryTeller has:
- A testing engine that is a superset of the basic FitNesse Fixture types (ColumnFixture, RowFixture, DoFixture, ArrayFixture, etc.). The engine supports a mix of flow-based and table-based testing.
- Customizable type coercion
- A command line runner for use in an automated build
- A way to tag tests as either “Acceptance” or “Regression” for testing lifecycle (Regression test failures cause a build failure, Acceptance test failures are simply reported)
- A User Interface tool that acts as a runner and an interactive tool for test editing
How does this relate to other tools?
- NUnit/MbUnit/xUnit.Net — There is some overlap, but an xUnit test will never be terribly readable to non coders, and that’s what StoryTeller is striving for. I see xUnit tools being used for unit tests, StoryTeller for acceptance testing, and integration testing being a gray area where you might go either way on a case by case basis.
- FitNesse — I obviously think StoryTeller is a far superior tool to FitNesse, or I’d just be using that as is. FitNesse is the closest analogue to StoryTeller.
- Machine.Spec or Cucumber or RSpec — Kind of solves the same problems in very different ways, but StoryTeller is more geared towards integrated acceptance testing. I wouldn’t care to use StoryTeller for straight up TDD at the low level. I guess the short answer on this comparison is “I don’t know.”
- Oslo — One of the frequent examples of Oslo I’ve seen is a DSL over the top of WatiN for web testing — and that’s great, but how about being able to quickly churn out little one off testing DSL’s?. In many ways StoryTeller is a tool for creating External DSL’s for testing your application code. That’s a direct overlap with Oslo (depending on which Oslo team member is speaking and to what audience), but at this time I feel like Oslo would require far more effort than StoryTeller in order to build a behavioral, runtime DSL to do testing. StoryTeller grammars allow you to quickly bridge the gap between how you want to express your test specifications and the system under test. Oslo would require you to parse and interpret those nasty, nasty AST’s to perform the actual functionality.
The Main Screen:
Woohoo! You can see all of the tests, you’ve got a status bar to show you what’s running, what’s queued up, and when the StoryTeller UI picked up the binaries (it does use a FileSystemWatcher like NUnit to watch for the system under test binary getting updated).
At the top, you have options to filter the tests shown in the test tree on the left:
Uh, and if that looks a lot like the ReSharper unit test runner, let’s say that imitation is the sincerest form of flattery.
You can also control the testing runtime like so:
I’m particularly proud of these features. Sometimes (most always), an integrated test can be slow to execute or Selenium can hang or who knows what can happen. StoryTeller will allow you to specify that tests should be aborted the first time they encounter an exception or a test failure instead of running the entire test as it normally does. You can also change the test timeout on the fly.
StoryTeller can show you the tests that are queued up to execute:
Yes, that screen is butt ugly so far, but it gives you the ability to track progress, clear tests out of the queue if you change your mind about running them, and abort the executing test.
Lastly, how about an actual test? Here’s the preview mode:
and the Results mode:
Like I said, this is just a sneak peek. I’ll have much more information out soon as I feel it’s becoming usable.
And no, I don’t think my problems with FitNesse are a team structure or a communication problem. Please don’t insult my intelligence (again) by changing the subject. The problem with FitNesse is mechanical.
