Steve Hebert's Development Blog

Steve's Blog - From .Net to dotMath and everything in between.

Test Driven Development Article - Applying to a model

I've been playing around with TDD on our project and it works on some simplistic pieces, but I'm having a difficult time applying it beyond simple smoke tests due to our app. I came across a decent article in MSDN Magazine by Will Stott and James Newkirk last month, but I think the real world application is a little simplistic. I like the concept and the areas where I apply it are useful, but I'm struggling with some more complex situations.

A large part of our application business objects produce output based on the existing state of the running plant. What I submit on time(x) is very different than what is applied on time(y). There is a big value in keeping a known state and testing from that point, however running a series of tests modify the state of that plant. Therefore the test results aren't predictable due to the tests that ran before. I'm really limited to doing smoke tests and testing the structure of the data than testing the data itself (i.e. using XSL to determine structural validity).

I'm curious if anyone has run into these situations or seen any articles dealing with this type of environment. All the TDD articles assume, to some degree, a given knowable state that's easy to get back to over the course of testing.

Published May 13 2004, 11:58 AM by shebert

Comments

Steve said:

It sounds like your not starting from a blank state when your running your tests, is this correct? If the data (state) is modified every time you run the tests, and not set back to an initial starting point at the beginning of every run of the tests it will definitely be very difficult to validate the data (state) is correct. I'd recommend that before the tests are run (that test the data) you should get the data to a known state. Then you can check that the different peices of logic within your application bring the data to the "new state" that you expect. Is that possible/feasible in your particular application?

# May 13, 2004 8:40 AM

Steve Hebert said:

You're exactly right and that's where the trouble arises. Getting back to a known state in the course of testing - between tests - is not trivial (reloading a SQL database whose size is not trivial). This would mean I would drive datasets along with the tests, but that becomes extremely time intensive (setup, data maintenance across version changes, and even test runtime).

That said, I've been toying with the idea of stringing tests together to test operational consistency, but then I'm leaving the land of TDD as I understand it.

TDD is effective on some of our outlying and internal core functions, but applying a reproducable test pattern to our model is where we'd see the most benefit. I'm not aware if TDD can address this.

# May 14, 2004 2:05 AM

Steve said:

The scenario which you mention is one of the more difficult aspects of TDD as I see it. When you have really large amounts of data and lots of interdependencies it can be difficult to get everything to the state that is necessary.

Would it be possible to seperate the tests into smaller peices that tests smaller sets of functionality. This might make the ability to rollback to a known state for the data easier. This would cover some of the individual peices of the application, however, as you stated this won't test the entire application. What might be interesting for this is a seperate set of integration tests that string a set of the individual tests together. These tests would test the "stories" within the application. They'd probably take longer to run which is why you'd probably want to seperate the tests into a seperate "integration test suite."

Definitely an interesting problem, hopefully some others will chime in and discuss some better options :-)

# May 14, 2004 2:21 AM

Steve Hebert said:

That last note triggered some interesting thoughts. I wonder if I could trigger a begin transaction/rollback transaction before/after each test. I could drive that code into the testing framework to enforce it at a higher level. I'll have to try that out....

Thanks for the note, I like the stories idea as well- I'm a little concerned about the interdepencies these introduce in the test and how that affects maintainability. I'll have to give that a try and see how to minimize some of those problems. Thanks!

# May 14, 2004 2:48 AM

Steve said:

Finally making it back around to this post. I always forget to check back on the comments of posts I've replied to :)

Anyway, the transaction idea sounds like a good one. That's an approach that James Newkirk talked about in Test Driven Development in .NET. I've used it in a couple real small projects with good success.

Your concern regarding the interdependencies is definitely something I'd be hesitant of as well. One of the biggest pains I have with TDD is the dependencies of test classes and their associated data. I'll definitely look forward to hearing how you make out!

# May 21, 2004 3:37 AM