Sponsored By Aspose - File Format APIs for .NET

Aspose are the market leader of .NET APIs for file business formats – natively work with DOCX, XLSX, PPT, PDF, MSG, MPP, images formats and many more!

StoryTeller is Dead. Long live StoryTeller!

I worked on and off again for about 2 years on a new OSS project called StoryTeller.  My original goal was to create a better editing and test management tool for Fit tests than the FitNesse Wiki approach.  I had *some* success, and I was able to use the rough StoryTeller UI on a couple projects.  Even in its rough form, I think StoryTeller was already better than FitNesse (as long as you knew what things on the UI that were not ready to be clicked on;)  All the same though, I haven’t found the tool to be all that successful, and my enthusiasm for the project has waned.  All of this is just an explanation for me announcing that I’m abandoning the original StoryTeller approach altogether.

For those of you who have asked me where the old code went, I’ve shelved the old code in a new SVN branch at:  http://storyteller.tigris.org/svn/storyteller/branches/old (for anonymous access, the user id is “guest” and the password is blank).

 

I still believe in Fit from a conceptual standpoint, but I’ve simply had it with the Fit mechanics.  Last week I started StoryTeller over from scratch, but this time I’m ditching Fit altogether and building a new test automation engine from scratch.  I don’t have a lot of information on it at the moment, but I’m hopeful that I’ll be able to demonstrate the new StoryTeller in action as soon as KaizenConf in three weeks. 

 

Some Facts:

  • StoryTeller is being conceived as a dedicated tool for crafting external testing DSL’s, and a UI for editing that DSL.  The test editing will be done mostly in forms rather than in text.  The point is to make test editing as guided as possible to make editing more efficient and less error prone.
  • I’m building a brand new testing engine from the ground up.  I think the new engine will make it easier for .Net developers to write the Fixture code and make it much easier to expose the metadata that the UI will need to create the test visualizations and test editors
  • I’m pitching StoryTeller as an automated testing tool that creates human readable specifications rather than a tool to automate specifications.  It’s a subtle shift in focus, but my experience has been that analysts and business partners either can’t or won’t participate in Fit test writing.  Hell, getting business people to even look at Fit tests has been a trial.  StoryTeller New is being built to be as easy to use as possible for reasonably able developers.  Specifically, it’s getting built for my team and my project with the hope that it will be useful for somebody else too.
  • The UI will be built from the ground up in WPF.  I’m hoping to use the UI for sample code in my book on UI patterns.  I think WPF is miles ahead of WinForms in its ability to do dynamic layout, and I’m going to push that ability as far as it will go.

 

 

Why I’m abandoning the FitnesseDotNet engine

I feel like I owe a bit of an explanation for this decision.  You can find most of the reasoning at Why I’m Suddenly Down on Fit/FitNesse.  The final nail in the coffin for me was way too many conversations like the following:

“Well first, make this a ColumnFixture and override the Execute() and Reset() methods.  Then override the DoRows() method, but be sure to call base.DoRows() before you add your code.  Then write a DoFixture class with a method that returns your new ColumnFixture”

Blech.  I love the concept behind Fit, but the implementation leaves something to be desired.  It’s just plain weird, and far too laborious to craft Fixture classes.  We partially beat the issue by doing quite a bit of code generation, but I’m in the camp that says code generation is a border line code smell.  In bullet point form, here’s why I think it’s time to move beyond Fit:

  • Test writing with Fit is very failure prone.  Fit is very sensitive to formatting errors and there isn’t a good way to discover syntax errors without running the tests.
  • The test management tools for Fit suck
  • The extensibility mechanisms (ICellHandler etc.) are odd and hard to understand, but understand them you must
  • It simply takes far too much knowledge about how the Fit engine works to write Fixture classes.  Very few people are willing to make that investment. 
  • The FitnesseDotNet engine has become far too coupled to FitNesse.  The last time I pulled a new build, I had to fork the code to get it to run outside of the FitNesse Wiki engine. 

 

Other Things

There’s a lot of things shaking out there in the world of test automation.  I’m not sure that I’d say that StoryTeller will be a direct competitor to the xBehave and xSpec tools.  ThoughtWorks just released Twist, and it looks roughly like what I’m envisioning, but the TW clowns made it Java only and tied it to Eclipse (and I detest Eclipse).  Bob Martin just announced a similar sounding tool called “Slim” as an alternative to the Fit engine for FitNesse, but it doesn’t seem to address the issues I have with Fit and FitNesse.  There’s also a chance that the textual DSL tooling in Oslo could do what I want StoryTeller to do.  I’ll keep an eye on all of these just in case.

About Jeremy Miller

Jeremy is the Chief Software Architect at Dovetail Software, the coolest ISV in Austin. Jeremy began his IT career writing "Shadow IT" applications to automate his engineering documentation, then wandered into software development because it looked like more fun. Jeremy is the author of the open source StructureMap tool for Dependency Injection with .Net, StoryTeller for supercharged acceptance testing in .Net, and one of the principal developers behind FubuMVC. Jeremy's thoughts on all things software can be found at The Shade Tree Developer at http://codebetter.com/jeremymiller.
This entry was posted in Uncategorized. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • krahman@phacil.com

    Hello Jaremy,
    Since our application is in Java, is there a Storyteller Java/J2EE version? Can I also import Fitnesse tests in Storyteller?
    Thanks,
    Kaiser

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Mike,

    About 3 mos ago. I forgot the details, but I needed a way to add Fixture definitions to Fixture and the FitnesseDotNet code seemed hard coded to pull the Fixture registrations from FitNesse specific files. In StoryTeller “before,” I was pro grammatically adding in the Fixture registrations.

  • Mike Stockdale

    No disagreement with most of what you say, but one point had me wondering: “The FitnesseDotNet engine has become far too coupled to FitNesse. The last time I pulled a new build, I had to fork the code to get it to run outside of the FitNesse Wiki engine.”

    When did you last pull a new build? FitnesseDotNet has been running outside FitNesse since 2007-01-30…

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Bil,

    I’ll respectfully disagree with you on that one. No code-behind == doing way, way, way too much in Xaml and Xaml is teh suck.

    I’ll be using the ShadeTree WPF DSL stuff that Chad & I wrote this spring.

    +1 on Presentation Model though.

  • http://weblogs.asp.net/bsimser Bil Simser

    PresentationModel and no code-behind for the UI? It’s the only way to go with WPF.

  • http://weblogs.asp.net/astopford Andy Stopford

    Hi Jeremy,

    Picking up Howards point about Gallio, I see two options here.

    ST gets a Gallio plugin for tools automation.
    You base ST on top of Gallio.

    While your correct about the tools integration that Gallio enables and a plugin would be workable do also consider that Gallio is a test automation framework and as such has a framework that MbUnit 3 uses and NBehave are looking to use. It ranges from data integration (row, pair-wise through to external data formats), custom integration, types through to reporting and all the meta goodness in between. I’d encourage you to join the Gallio list, let us know your requirements and where Gallio might save you some time.

    Andy

  • Pascal Laurin

    For more than a year now, we’ve been working on our own Fit framework and frankly I think it is a success with acceptance from our BA.

    The thing is, in the financial world one app is used by every BA and its called Excel. They always work with Excel and sometimes we even get specifications in spread sheets with all the formulas and macros.

    So, we’ve created our own Fit framework reading .xls files and since then BA are writing tests on their own and asking for more. They want new aspects like more negative testing, new application feature covered, etc.

    We even get ask by the marketing guys if our client might participate in the Fit effort!!

    My point is that I think the first and most important factor to successful Fit acceptance is using and editor your clients/BAs are used to work with. Forget the Wiki and HTML format. I’m not even sure DSL would work either.

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Howard,

    I don’t see Gallio as being much of an aid here. Maybe something I could use to help create test runners for TD.Net or ReSharper.

  • http://blogs.conchango.com/howardvanrooijen Howard van Rooijen

    Hi Jeremy,

    I’m really excited to hear this news – I’ve been following your progress on ST for a while and I’m really interested to see the new direction you’re going to take this in.

    One question tho – are you going to build it from scratch or is there the possibility of basing it on the Gallio framework ( http://www.gallio.org/ ) whch would handle the test execution and give ST a common reporting infrastructure too?

    Regards,

    Howard

  • http://www.pksoftware.net/devblog/post/2008/03/Acceptance-Test-Framework.aspx Paul Kohler

    For a while I felt bad not liking the FIT approach, I even tried simplifying it (http://www.pksoftware.net/devblog/post/2008/03/Acceptance-Test-Framework.aspx) before getting busy with other stuff, you know, work!… At the end of the day I had similar issues especially in regards to getting the business/testing people embracing the FIT the approach. Change is hard!
    Look forward to seeing the new story teller evolve!

  • Sai

    Hoping to see the new story teller. As you said Twist is locked with Java and eclipse which leaves a lot of people using other platforms unable to use it. Take a look at Concordian. It is also based on Java and used to build textual DSL for testing.but not locked to any IDE

    Regards,
    Sai

  • Eric

    I’m really looking forward to observing how the new code base comes together. Reading blog posts about all this TDD/IOC/MVP/SOLID/blahblah alphabet soup is one thing, but reading *real* code is the best way to actually understand how it all works.

    Speaking of which, is the new code base missing some files in SVN (like a .csproj for the tests, and an .sln), or am I hopelessly primitive in understanding how to build and run the code?

  • http://www.bluespire.com Rob

    Let me know if you need any WPF assistance. I’d like to see this be a success.

  • http://www.bizcoder.com Darrel Miller

    “code generation is a border line code smell”

    Amen to that.