I announced the StoryTeller reboot a couple weeks ago. I had a couple people email me about it, or volunteering to help. I didn’t respond (I’m sorry) because I didn’t know quite how things were going to go. I think I can finally say that I’m confident that StoryTeller is on a good path and start thinking about other contributions. At this point, I’m able to author tests against our system at work and I’ll be able to demo the testing engine at KaizenConf this weekend.
Here’s what I’ve got so far. It’s very rough, but it’s a start. I’m deviating a lot more from Fit than I had thought I would, but it’s still easiest to describe the functionality in terms of Fit analogs.
Terminology:
- Test – duh.
- ITestContext – the state of the currently executing test. My main vehicle for maintaining state during a test. One of the gripes about Fit is the clumsiness of maintaining state in a test across multiple Fixture classes. I think I’ve generally alleviated that problem with StoryTeller. ITestContext is also more or less a service container for the test infrastructure.
- Step – A test is composed of steps. The “Step” in StoryTeller is a logical operation in the test, or group of operations. A Step does not necessarily equate to a single table row in Fit. In the StoryTeller model, a single Step might equate to a half dozen lines in the rendered test output. For example, editing my application’s Address form requires 6-7 different actions for filling in different textboxes and selecting dropdown values, but you always have to do these 6-7 things at a time, so it’s a single Step. I think this is hugely important in making StoryTeller test authoring more mechanically efficient that Fit.
- Grammar – The fundamental unit of test execution and expression. Each Step is processed by a single Grammar in the test execution based on the Step’s “grammarKey.” It is possible to create a CompositeGrammar to group related Grammars into a coarse grained unit. A TableGrammar is a special kind of Grammar that allows you to recreate a ColumnFixture from Fit in StoryTeller for example driven tests.
- Fixture – A class that provides one or more Grammars. Unlike Fit, a Fixture in the StoryTeller architecture can mix table and flow driven grammars in a single class for easier coding and state sharing.
public interface IFixture
{
string Name { get; }
IEnumerable<IGrammar> Grammars { get; }
IGrammar FindGrammar(string key);
void ForEachGrammar(Action<string, IGrammar> action);
}
ColumnFixture Analog
The thing that Fit is best at is table driven “example” tests, or the Row style tests in MbUnit. Unlike Fit, the StoryTeller analog can just be a method instead of a completely separate class. Any simple Grammar can be exposed as a TableGrammar. Here’s an example of turning a normal method into a TableGrammar:
[ExposeAsTable("Add numbers", "Rows")]
[return: AliasAs("sum")]
public int AddNumbersTogether(int x, int y)
{
return x + y;
}
It’s limited to a single return value at the moment (a serious limitation), but that’s a lot less code than the Fit equivalent. The ExposeAsTable attribute just says “this method is a table grammar.” The AliasAs attribute is just helping to fine tune the test rendering. Don’t worry, it’s not crazy with attributes, but I am using some to control formatting.
DoFixture Analog
This was actually my very first use case for the new engine. The DoFixture allows you to do flow based testing. At its simplest, just write public methods on your Fixture class:
public class AddingFixture : Fixture
{
public AddingFixture()
{
// This creates a TableGrammar using the AddNumbersTogether2() method
Grammar("AddNumbersTable").IsTable("Add some numbers together and see what happens").TheInnerGrammar
.Is(FindGrammar("AddNumbersTogether2"));
}
[ExposeAsTable("Add numbers", "Rows")]
[return: AliasAs("sum")]
public int AddNumbersTogether(int x, int y)
{
return x + y;
}
[return: AliasAs("sum")]
public int AddNumbersTogether2(int x, int y)
{
return x + y;
}
}
Any public methods will be picked up as Grammars. In addition, there is the beginning of a DSL for programmatically defining Grammars. Here’s a real life example from our AddressFixture that tests our Address CRUD and Query web pages:
public AddressFixture(ApplicationDriver driver)
{
_driver = driver;
// More or less script how the Address screen is edited in one logical step
Grammar("EditAddress").IsScreenEditor<EditAddressViewModel>("Edit Address").WithPrefix(x => x.Address)
.Enter(x => x.Address.Address1)
.Enter(x => x.Address.Address2)
.Enter(x => x.Address.City)
.Select(x => x.Address.StateOrProvince)
.Enter(x => x.Address.PostalCode)
.Select(x => x.Address.Country)
.Select(x => x.Address.TimeZone);
// Check all the values saved to the database
Grammar("CheckFields").IsCompositeTitled("Verify Persisted Fields")
.LoadThisObject<Address>()
.VerifyPropertiesOf<Address>(x =>
{
x.Check(o => o.Address1);
x.Check(o => o.Address2);
x.Check(o => o.City);
x.Check(o => o.StateOrProvince);
x.Check(o => o.PostalCode);
x.Check(o => o.Country);
x.Check(o => o.TimeZone);
});
// Create a new Address object and persist it
Grammar("DefineAddress").IsCompositeTitled("Address in the Model Is")
.SaveNewObject<Address>(x =>
{
x.SetAllPrimitivePropertiesSpecificToThisType();
});
// Create and persist multiple Address objects
Grammar("AddressList").IsToSpecifyTheListOf<Address>();
}
I should point out that quite a bit of that stuff above is Dovetail-specific language for our application architecture.
RowFixture Analog
Very frequently, you need to verify a set or list of values against an expectation.
// This Grammar clicks the "Save" button expecting input validation failures
// and validates the displayed validation failures against an expected
// list, displayed as a table like the Fit RowFixture
Grammar("ClickSaveUnsuccessfully").IsVerifySetOf<ValidationError>("Clicking Save should fail with validation errors", x =>
{
x.Before((step, context) =>
{
clickSave(context.Retrieve<ScreenDriver>());
var notifyDriver = context.Retrieve<NotifyDriver>();
context.CurrentObject = notifyDriver.GetErrors();
});
})
.MatchOn(x => x.Field)
.MatchOn(x => x.Message);