Is there a future for Fit testing?

I use Fit testing and I’m even building a tool for easier test creation and management of Fit tests.  I’ve used it to drive web UI’s, WinForms UI’s, logic in domain model classes, web services, and even a polling windows service.  Some things have gone well and others haven’t.  My dilemma today is this.  I have some logic that needs some automated acceptance tests.  It’s a little doubtful to me that anybody but me will ever look at these tests.  I’m strongly considering just writing integration tests in NUnit instead of Fit. 


Long term, all the testing gurus I know of seem to be driving towards scripting tests in Ruby and I’m thinking of moving in that direction in the future.  I’ll still use Fit tests for set-based logic where it excels, but otherwise it just seems, I don’t know, clunky.


What do you guys think?  Does Fit have a future, or is it just a little blip that’s never really going to take off?


 


I’m not ready to throw in the towel on executable requirements by any means, just considering a change in mechanism for expressing the tests.

About Jeremy Miller

Jeremy is the Chief Software Architect at Dovetail Software, the coolest ISV in Austin. Jeremy began his IT career writing "Shadow IT" applications to automate his engineering documentation, then wandered into software development because it looked like more fun. Jeremy is the author of the open source StructureMap tool for Dependency Injection with .Net, StoryTeller for supercharged acceptance testing in .Net, and one of the principal developers behind FubuMVC. Jeremy's thoughts on all things software can be found at The Shade Tree Developer at http://codebetter.com/jeremymiller.
This entry was posted in Uncategorized. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • Usman_kec

    Yes always

  • http://www.spiderlogic.com shobha

    How do I apply RowFixture with junit in RAD .. Please explain with a simple program.

  • http://weblogs.asp.net/bsimser Bil Simser

    I feel the pain. I’m dealing with helping guys (and girls) out on writing fit tests here and there’s a lot of perplexed looks and crazy fixtures happening. I still have some faith in fit and it really does a good job at some tasks (although if your users are writing fit tests that duplicate unit testing, they’re going down the wrong path).

    For me though I’m groking BDD right now and how different (easy/hard) it is to do using something like Behave# vs. Fit. Blog posts to come shortly on all of this as I’m playing in the sandbox this week.

  • http://iancooper.spaces.live.com/ Ian Cooper

    We have used FIT a few times, because we liked the idea of subcutaneous layer testing, but I can’t help but feel like Scott that the tool set produces tests that are sometimes fragile and sometimes over constrained by the tabular format.

    Unless you have a strong customer-authored test requirement it seems easier to drive layer testing with an xUnit tool and write tests in the same language that you code in.

    Because we tend to have a hexagonal architecture, when we need to be able to have variable test inputs so that we can test differing ‘acceptance’ criteria we have tended to find it easier to use a custom test harness, built in WinForms, as a front end to provide those inputs that performs the layer test. In other words we riff of the FIT testing idea, but use our own engine instead of using FIT. The cost here doesn’t tend to be material in a larger project, but I guess smaller projects could find it hard to justify the cost of a write your own layer testing tools approach.

    I have not seen the use of Ruby that you are specifically talking about, but I guess you must either be talking about driving the UI using Ruby or writing a DSL-like script in Ruby to define tests. I think those are possibilities; but if those tests drive the UI then they will tend to suffer from the fragility of UI tests and thereby miss on some of the benefits of layer testing as an approach.

    Of course there may be a sweet spot for early use if IronRuby in being able to combine the expressiveness of scripting languages with easy subcutaneous testing of .NET projects, and it would be a good way to get Ruby skills into the Enterprise..

  • http://designingcode.blogspot.com Keith Nicholas

    we have done fit off and on for a while now. But for the most, unit testing frameworks can do mostly what we want at an acceptance testing level.

    However Rick Mugridge is working on http://www.zibreve.com which might be interesting, and I know Rick has some interesting ways of making FIT do some cool things.

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Nat,

    I read your post last week and it helped move me closer to finally giving up on Fit. There is the Green Pepper engine as an alternative that is effectively a rewrite of the Fit engine:

    http://www.greenpeppersoftware.com/en/products/GreenPepperOpen/

  • http://nat.truemesh.com Nat

    I’ve written a few notes about my experience with FIT here: http://nat.truemesh.com/archives/000702.html.

    I think the *idea* behind FIT is excellent and one of the most promising directions that exists for improving the communication between non-technical users/customers and developers.

    However, the implementation causes a lot of problems. The framework design, especially how reflection is used, makes it very expensive to maintain FIT tests and maintain code that is specified by FIT documents. Its “cathedral” style of free software project management means that user feedback is not fed into the design of the framework fast enough.

    However, I don’t think that dynamic scripting languages are going to be the answer. They don’t really address communicating with non-technical users and they still require reflection or other glue code to link the script interpreter to the application under test, so there’s still maintenance overhead.

    I’d like to see a radical redesign of the FIT framework. Most of what’s in the FIT framework can be replaced by existing open source libraries: JUnit for running the tests, TagSoup and XOM for parsing and generating HTML, etc. FIT should provide a clean object model model for extracting test data from tables, lists and other HTML structures and annotating those structures with test results. It should reduce the use of reflection (the most baroque of which is in FITLibrary, admittedly) and ensure that the links between the HTML and Java better support refactoring and diagnosis of failing tests.

  • http://www.bellware.net ScottBellware

    The idea of Fit testing is great. The tools in practice drive up maintenance costs. In my experience, the increase in the effort to maintain Fit/Nesse tests increase exponentially in relational to the volume of tests and test code.

    I think that mata-programmable DSL’s are a good avenue to explore.

  • http://grabbagoft.blogspot.com/ Jimmy Bogard

    We’ve started to use more BDD-style requirements using stories and scenarios, with “Given…When…Then” syntax. In the future, I’m looking more towards RBehave, as it can closely match the original story conversation.

    In the meantime, I’m working with Joe Ocampo (http://www.agilejoe.com) to produce something similar to RBehave that we can use in an arbitrary test framework. We wound up creating very similar frameworks, and we’re now joining them into Behave# (http://www.codeplex.com/behavesharp).

    I still can’t get over the clunkiness of Fit personally. I think the problem is that code approaches were clunkier because the syntax of the code obscured the intention of the tests, especially to business owners trying to understand the tests.

    DSLs using more expressive languages, whether it’s Ruby or Boo or whatever, seem to obscure intent the least. Anything that gets us closer to the language of the business will win out eventually, I think.

  • Adam Geras

    I experimented using Powershell because it is a scripting (dynamic) language that has access to .NET libraries (2.0) – and it has some funky syntax that makes it simpler to work with than even NUnit in some cases.

    Then I added some testing primitives to make it easier for writing acceptance tests, tabular or otherwise.

    See more at http://www.codeplex.com/psexpect.

  • http://www.JeremyJarrell.com Jeremy

    @Jeremy

    Gotcha, I’ll be waiting :)

    Jeremy

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Other Jeremy

    Um, that’s kind of my Project X. I might have an answer in three months.

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @mcgurk,

    I started StoryTeller (http://storyteller.tigris.org) largely because I was dissatisfied with Fitnesse. When I say Fit I’m referring to the core engine, and that’s what I’m afraid needs to be challenged.

  • Eric Anderson

    You’ve hit the nail on the head. FIT tables seem to be great for testing… well, tabular data.

    We are still using FITnesse to drive conversations with the product owner as well. Just last week, we sketched something out in FIT tables, then had a conversation with the PO to clamp down what he wanted. After that, we set about making what was a sketch into a test that passes.

    When we get away from using FIT for sketching out ideas, it will most likely be in a move to use Groovy. This may happen as early as a couple of weeks from now. We’ll probably still us FIT for discussing things with the PO. But, you’re right, it’s pretty clunky.

  • http://youredoingitwrong.mee.nu mcgurk

    FITnesse?

  • http://www.JeremyJarrell.com Jeremy

    Can you explain more (or point me to a reference) about how you would drive the tests with Ruby, specifically in reference to WinForms app? I’m not familiar with this.

    Thanks!
    Jeremy

  • Jonathan Rasmusson

    I feel your pain Jeremy.
    I too have longed to use FIT effectively on projects, but find them clunky, and a chore to maintain.
    When you add in the job of selling it to customers (whom usually don’t seem to keen on maintaining or writing more) I too have taken the pragmatic route of just writing the unit tests myself and moving on.

    I don’t think that makes us bad people – just pragmatists ;)