Jeremy D. Miller -- The Shade Tree Developer

July 2005 - Posts

• Start with a *correctly* failing test

  One of the tenets of test driven development I used to (ok, I still do sometimes) blow off is that you always start with a failing test.  You know the drill -- "red bar, green bar, refactor."  Write the test for the new little piece of code Run the test and verify that it fails for the right reason Implement the code and make the test pass Wash, rinse, and repeat The second part of step 2 in that list is pretty important too and it's easy to skip it.  An obvious, unspoken requirement of doing TDD is that your unit test code needs to be correct.  It's not enough to get the red bar (that's easy).  Make sure the unit test is failing correctly before you start writing the real code.  That little sanity check of running the test to see the failure is a good way to quickly validate your unit test.  For example, I routinely employ NMock to create dynamic mock objects in unit tests.  In your unit tests you're telling NMock which methods should be called in the test by passing in strings and setting up the argument expectations.  When you're using NMock won't get any fancy statement completion or handy compiler errors when you fat finger the name of the methods (NMock2 or Rhino Mocks should alleviate this problem.  More on that someday).  If you just run the unit test first, you can usually spot any kind of problems with the NMock expectations pretty quickly before that gets confused with the real code.  One of my favorite TDD idioms now in .Net is to write the test for a new method, then use ReSharper to generate the method stub to match the signature in the unit test.  Out of the box, ReSharper will generate the method with a single line of code "throw new NotImplementedException()."  When you run the failing test, you just look to see that you're breaking because of the NotImplementedException.  As soon as I see test output like the example below, I know my test setup is functioning and I can go ahead and implement the new method. [Test] public void CloseViewWhenViewIsDirtyAndUserRespondsOk() { // Define the expected interaction _msgBoxMock.ExpectAndReturn( "AskYesNoQuestion", true, Presenter.CLOSE_WARNING_TITLE, Presenter.DIRTY_CLOSE_WARNING); _viewMock.ExpectAndReturn("IsDirty", true); _viewMock.Expect("Close"); // Perform the unit of work _presenter.Close(); // Verify the interaction _msgBoxMock.Verify(); _viewMock.Verify(); } ------ Test started: Assembly: SampleCode.dll ------ TestCase 'SampleCode.HumbleDialogBox.PresenterTester.CloseViewWhenViewIsDirtyAndUserRespondsOk' failed: System.NotImplementedException : The method or operation is not implemented. c:\documents and settings\jeremy.miller\my documents\visual studio projects\samplecode\humbledialogbox2\presenter.cs(46,0): at SampleCode.HumbleDialogBox.Presenter.Close() c:\documents and settings\jeremy.miller\my documents\visual studio projects\samplecode\humbledialogbox2\presenter.cs(109,0): at SampleCode.HumbleDialogBox.PresenterTester.CloseViewWhenViewIsDirtyAndUserRespondsOk() 0 succeeded, 1 failed, 0 skipped, took 0.03 seconds. ---------------------- Done ----------------------   Posted Jul 29 2005, 01:29 AM by Jeremy D. Miller with 1 comment(s) Filed under: Test Driven Development

• A Stupid Way to do Error Management

  This is obviously a rant, so buckle up. I'm at the tail end of a little project adding some customizations to a legacy system.  There isn't anything resembling a service (or business/domain) layer other than T-SQL SPROC's, and the vast majority of the business and validation logic is in the stored procedures.  The clown that wrote a great deal of the original system (since departed) had a serious T-SQL fetish.  The guy apparently truly believed that T-SQL was simpler and easier to maintain than C#.  One of the results of this is that validation warnings like "Vendor does not exist" are created and generated by a stored procedure and passed back to the application by the message in the SqlException that's thrown by the command execution.  Whatever is in the body of the exception, or any exception thrown for that matter, is automatically put into the audit tables where it is displayed to the end users on a web page.  Mr. User, you ask why didn't your invoice go through?  Because of "Object reference not set to an instance of an object."  Duh.  I don't know if there is any easy way to look at the exception to see if it is a validation error versus a system sql failure, but that's not the point, it's a stupid strategy. A suggestion I'm making going forward is to handle system errors and validation errors separately.  I'm not an expert on user interaction design, but I'm pretty sure you're not supposed to be sending raw exception messages to users.  Plus it's just stupid to use exceptions for control logic.  Not to mention it's incredibly moronic to create user messages in stored procedures.  We know we'll need to provide some internationalization eventually.  The idea that I'll have to scour stored procedures to localize the web displays pisses me off.  Add in the fact that we need to support multiple database engines -- and I'm starting to get red-faced so I'll stop now.  Just a plea to my colleagues, let's please stop abusing stored procedures.  Put business logic and UI code in predictable places and let's start thinking of better ways to do exception management.  If you don't stop putting crap in sprocs, I'm gonna start getting angry and I'm bigger than you are.     Posted Jul 27 2005, 03:25 PM by Jeremy D. Miller with 5 comment(s) Filed under: Ranting

• A Taxonomy of Tests

  This was originally a series of posts at my old blog a few days after I recovered from the midnight showing of "Revenge of the Sith." I made the mistake of posting on writing good unit tests while suffering from "Jedi Flu" and Jeffrey Palermo took me to task for not defining a unit test first, so here's my attempt at describing the various types and approaches of tests I've seen or used on agile projects. Keep in mind that I'm a developer first and only an occasional architect, “BuildMaster”, or tester second. The first thing to keep in mind is that agile processes don't change the fundamentals of software development. It is still best practice to test the code at multiple levels of integration (the old V-curve idea). It's great to mock the database in your unit tests, but the database is still there and just itching to get out of synch with your middle tier code. A Taxonomy of Tests Roughly, I would divide tests into: Unit tests Acceptance tests Intermediate/System/Integration tests, written by developers Other: smoke tests, regression tests, performance tests, miscellaneous validation Testing Strategy The answers will vary from project to project due to manpower constraints and the nature of the development, but any and every project is going to have to answer these questions below. Who writes what tests? Who automates the tests, and how? Where and when do the tests run? How do we want to organize the test suites? How much testing is enough? Sleeping with the Enemy One thing I've learned that does not vary; it takes a great deal of cooperation and mutual respect between the testers and coders. Working in the same room is ideal. On projects where the tester and developers are separated by either organization or location, the development effort will suffer. For that matter, both developers and testers need quick and easy access to whoever holds the reins on requirements. At a bare minimum, don't look at testers as the enemy.   Unit Tests One of the truly great advantages agile development has over traditional waterfall projects is in team dynamics. There is far more interaction and communication between disciplines. Just being on the project at the same time is an advantage over “throw it over the wall” processes. Everybody’s common goal on an agile team is producing working software. Not code, tests, UML diagrams, or Gantt charts, but working code that meets the customer’s expectations. At the most granular level is the unit test. Unit tests are written by developers to verify, or in Test Driven Development specify, the functionality of a very small piece of code to the satisfaction of the developer. Usually a unit test works on one class or a small cluster of classes at a time. One of the key attributes of a unit test is the ability to test a small unit in isolation. We want to prove that the unit of code itself is working without corruption or interference from other classes. Purposely designing code in order to create isolated unit tests is part of a successful Test Driven Development effort. Mocks and stubs are an invaluable tool to create the isolation. Some people insist that all external dependencies (databases, Active Directory, web services, etc.) be mocked out in unit tests. My best advice is to consciously quarantine inconvenient things behind gateways with an abstracted interface. Use Inversion of Control to minimize dependencies from the majority of your .NET code to these gateways. When you can't eliminate a dependency, use Dependency Injection to provide a mechanism to replace the real thing with a mock or a stub. I have had very positive (well, mostly) experience with the Model-View-Presenter (MVP) pattern described by Martin Fowler as a strategy to maximize the testability of user interface code. Unfortunately, some units of code only exist to manipulate the database or interact with infrastructure services like messaging. Not to mention that you still have to unit test the UI screens. MVP is not enough in and of itself to guarantee a properly functioning user interface (trust me on this one). You can either marginalize this code by not writing tests and hope for the best (or let the acceptance tests take care of it), or bite the bullet and write a battery of tests that are connected. My advice is to put some ketchup on the bullet and just do it. I do prefer to separate connected and non-connected unit tests into separate test fixture assemblies. I generally lean on only the non-connected tests during refactoring or new development. I do think it is absolutely necessary to run the integrated unit tests in both your check-in procedure and the automated build. There is a school of thought that says if you really unit test well enough against the mocks, integrated testing is unnecessary. The NEO project is the best attempt I've seen anybody use to actually approach this ideal. I still think this philosophy is unattainable and possibly dangerous. I've been in some heated arguments on the merits of mocking ADO.NET when testing custom data access classes. I mostly vote no to mocking ADO.NET. In this case, the unit of valuable work is manipulating the data in the database. Who cares if the code sets the “expected” parameter values when the real stored procedure has been changed in the database behind your back? In this case, the effort to value ratio is all wrong. Of course, when you’re a consultant onsite at the client you work according to their goofy development standards and politely say “Thank you sir, may I have another?” Acceptance Tests   From Ward Cunningham's wiki: "A formal test conducted to determine whether or not a system satisfies its acceptance criteria and to enable the customer to determine whether or not to accept the system." The acceptance test is exactly what it sounds like, it verifies that the implementation of a user story works as the customer requires. In a literal interpretation of XP theory, the customer writes the acceptance test. In reality, the acceptance tests are going to be written by some combination of the business partner, testers, business analysts, or product managers. It really doesn't matter to me, as long as they are done in a timely manner with some degree of correctness. You'll never have the ideal XP customer, but it doesn't necessarily mean the project is hopeless. It is a huge advantage when acceptance tests are approved and ready during coding. User stories, use cases or huge paper System Requirements Specification documents -- the best and most effective requirement is a cut and dried acceptance test. When the coders can (and do) easily execute the acceptance tests themselves, the bug counts will go down. The development team as a whole can use the acceptance tests as the “Red Bar” metric to watch during an iteration. A user story is “done” only when all the finalized acceptance tests pass. James Shore has a good post on acceptance tests as requirements here. Developer/Tester Cooperation Testing features during an iteration as soon as the code dries is a tremendous advantage. It's much, much simpler to fix bugs that were just coded before the memory of the code fades and the bug is covered up my much more code. My current project has not had acceptance tests written during iterations, and it has clearly hurt us in terms of wasted effort and misunderstood requirements. Never again. Organizational issues between testing and development are not specific to agile development, but the desire for rapid iterations will expose any friction between requirements, code, and testing. Moving fast without sacrificing testing requires the ability to cycle code quickly between development and testing environments. Code that is easy to test helps quite a bit, too. Developers have to support the testers with the mechanics of testing. Both testing and development can benefit from a good Continuous Integration (CI) strategy. Testers should be involved in the CI effort. I feel very strongly that it is the developers’ responsibility to ensure the test environment is in a valid state. “BuildMaster” folks are a desirable luxury, but I always want visibility into the test server environment. Automating Acceptance Tests To paraphrase Ron Jeffries, "Unit tests are for coding right, acceptance tests are for writing the right code."  Acceptance tests are a good and necessary thing because they are effectively the requirements in an agile process.  Ideally, the acceptance test should be an unambiguous statement to the proper functioning of the software.  If the acceptance test passes, the requirement is satisfied.  As far as testing is concerned, an automated test that runs often and visibly is far more valuable than a slow manual test buried in a test plan. A common objection to iterative development is the need for so much regression testing. The best answer to this perfectly valid objection is to automate as much as you can.  Enterprise applications are rarely finished; they're abandoned when they are judged to be too expensive to change. Automating acceptance tests is a great way to extend an application's lifecycle by reducing the cost of regression testing. Also, to paraphrase a former colleague -- "If your tests aren't automated, how do you know they're passing?" Think on that one. Who writes the test is one decision. Who automates the tests, and how, is a different decision. Hopefully you're lucky enough to have testers with programming or scripting abilities. Even if you do have technical testers, plan on the developers assisting the testers with test automation. I'm not a big fan of the mechanics of FIT and its derivatives, but it is definitely a step in the right direction for acceptance testing (I love the concept, but I hate the implementation of NFit). The FIT model allows for writing acceptance tests in near English, with some non-trivial work behind the scenes to complete the wiring for the custom FIT fixtures. Conventional wisdom says that acceptance tests are black box tests that test the system end-to-end. This doesn't necessarily have to be the case. I’ve seen both Brian Marick and Brett Pettichord write about using white box tests for acceptance tests, especially for user interfaces. Using Service Orientation principles to create a cohesive service layer can be a great aid for automated acceptance testing. I think (N)FIT chokes on user interfaces, but it is perfect for driving an isolated service point (note I didn’t necessarily say web service) in an automated test. These white box tests can be much easier to setup and execute. Some flavors of MVC architecture allow for slipping in a screen proxy in place of a real UI view (I've had mixed results with this). Again, it is not a “perfect” black box test, but it goes a long way towards validating the behavior of the system. Good testers can work with developers to write these automated whitebox tests. It does take pretty good communication and cooperation between coders and testers to take advantage of these opportunities for test automation. It is definitely advantageous for the testers to be familiar with some of the inner workings of the system. I definitely think agile development starts to blur the line between developers and testers, and that's not an entirely bad thing.   Everything Else   While the majority of your focus on an agile project is on either unit tests or acceptance tests, there are quite a few other types of useful tests that fall through the cracks. All of these types of tests can and should be part of your continuous integration strategy. Developer Written Partial Integration Tests Bigger than a unit test, but smaller in scope than an acceptance test. That's a big mouthful for a category, but I've never seen any two development teams that use the same name for this broad category of tests. For multiple reasons, most of my team's unit tests are disconnected -- i.e. databases, web services, and the like are mocked or stubbed. I want to test each class in isolation, but they have to interact with the real services too. A secondary battery of developer tests that test partial code stacks are very advantageous in uncovering problems in the actual interaction between classes or subsystems. I've known some XP developers who actually emphasize these semi-integrated tests more than the disconnected unit tests. I like to keep the types of tests in separate test assemblies, but that is just preference. These are really hard to describe, so here's a couple of ideas I've used or heard of -- Testing a view and controller or presenter together. I would still try to mock out the backend business services from the controller in order to focus on testing the integration of view and controller. This is especially important in the new Model-View-Presenter mode of UI development. Testing from a UI controller down to the persistence layer. Test from the service layer down. SOA is surely overhyped and overused, but it presents some great opportunities for automated testing. Because the .Net SOAP serialization is a bit brittle I always put automated tests in for the web service proxy class communicating with a local web service. It's also good to test the service code without IIS being involved. Environment Tests Environment tests verify that a build or test environment is fully connected and correctly configured. I helped do a presentation on continuous integration to the Austin Agile group last week and we spent quite a bit of time talking about this very topic. The gist of the slide was preventing lost time by testing teams by uncovering disconnects or failures in the testing environment at code migration time. Nothing is more aggravating than reported defects that end up being merely a mistake in code migration. I spent a week one time getting grilled because a data interface wasn't working correctly. It turned out that the middleware team had shut down the webMethods server in the test environment and not told the development team. Putting aside the total incompetence of everybody involved (including me), had an automated environment test been in place to verify the connectivity to webMethods, a full week of testing would not have been wasted. I had some periodic trouble on a project last year with maintaining various copies of the system configuration (database connections, file paths, web service locations, etc.) for development, build and test environments. To combat this problem in the future I've actually added some support into my StructureMap tool for doing environment checks and configuration validation inside of an automated deployment. We sometimes move code a lot faster in agile development and it provides a lot of opportunity to screw things up. These little environment tests go a long way towards keeping project friction down. I got the name for this off Nat Pryce's post here. Smoke Tests Smoke tests are simple crude tests to exercise the system just to see if the code blows up. They don't provide that much benefit, but they're generally low effort as well. On my current project we're replacing a large VB6 component with a new C# equivalent. One of the things we did to create a safety "tripwire" effect was to run a huge amount of historical input XML messages and configurations through the new C# engine just to find exceptions. We found and fixed several problems from edge cases and some flatout bugs. It was definitely worth the effort. Regression Tests My last two projects have been more or less rewrites of existing legacy code. Oftentimes hairy legacy code isn't understood very well by the organization (hence the rewrite). Twice now I've had to jury-rig the legacy code to create repeatable automated tests that simply prove the new code creates the same output for known input. It's not entirely a good test because it has no connection to the intent of the code, but both times it found problems. The irritation level was high because the legacy code wasn't pretty or easy to get running, but otherwise unknown defects removed and fixed made it worth the effort. Automate the regression tests in the build if you can. If nothing else, there are always business processes and requirements buried in the code that aren't known or documented anywhere else. "Build Time" Validations There are always some aspects of the system that the compiler and maybe even the unit test library can't validate. Use the automated build as an opportunity to create "build time" checks for potential problems or enforce coding standards. Here's some ideas of things to enforce in the build: A compiler warning or "TODO" threshold. I've heard of people making a build fail for this. A limit on ignored unit tests SQL validator. Gotta keep the sql external to the code somehow, though. O/R Mapping. We're starting to adopt NHibernate for persistence. One of the first steps will be validating the mapping in the build Automated code coverage is kind of a test of your tests Static code analysis There's probably more, but that's all I can think of. I'd love to hear other ideas. Posted Jul 27 2005, 01:21 PM by Jeremy D. Miller with 12 comment(s) Filed under: Test Driven Development

• Using Continuous Integration? Better do the "Check In Dance"

  So you're making the move to install a Continuous Integration process on your project.  Most of your early attention is going to be spent on getting the CI server up and the build scripts put together and functioning.  That part of the process in the .Net world is getting easier as the tools have matured and the accumulated knowledge available online has multiplied.  My company is moving to full blown CI and TDD on our main systems soon.  Other than getting migrated off of VSS, I'm not too worried about the infrastructure setup.  What I am worried about are the developer practices.  One of the key practices in agile processes is Collective Code Ownership.  Collective ownership can make a project team much stronger, but there are some inconvenient side effects in coding construction.  Because agile iteration work is mostly scheduled by business need and not by what is convenient for us the developers, it's not unusual for two or more active coding threads to be simultaneously changing some subset of the code.  The idea of continuous design* implies that even big architectural changes can be happening on an ongoing process.  The key point is that things are changing and that can be risky.  CI is one of the enabling processes that supports both continuous design and collective ownership, but you've got to take advantage of it.  The CI build process is a mechanism to find and eliminate problems from code changes as early as possible.  The key for developers to get the most out of CI without being burned is to be disciplined about their "Check In Dance." Basic Guidelines Check in as often as you can.  Try to reach stopping points as often as you can.  This goes back to the basic agile philosophy of making small changes and immediately verifying the small change.  When you're doing Test Driven Development you strive to keep "Red Bar" periods as short as possible.  The same kind of thinking applies to code check-ins.  Make small changes and see the impact on the rest of the code immediately.  Merging code will be less painful the more frequently a team integrates their code. Avoid stale code.  If you have to keep code out for any length of time, make sure you are getting everyone else's changes.  Try really hard not to keep code out overnight.  If you're using shared developer workstations, put some sort of sign on the workstation that there is outstanding code on the box.  I've seen several XP zealots swear that they'll throw away any code left overnight.  Personally, I think that's just a silly case of "I'm more agile than thou," but it's still a bad idea to leave code out overnight if you can help it. Don't ever check into or out of a busted build.  Checking in might make it harder to fix the build because it will cloud the underlying reason for the build, and you can't really know if your changes are valid. Communicate and negotiate check in's to the rest of the team.  Frequently the complexity of a merge can be dependent upon who goes first.  Some teams will use some kind of toy as a "check in token" to ensure that there is never more than one set of updates in any CI build.  Pay attention to what the rest of the team is doing too. If you're working on fixing the build, let the rest of the team know. DON'T LEAVE THE BUILD BROKEN OVERNIGHT.  That's also an occasional excuse to your wife on why you're home late from work.  Use with caution though. Not every member of the team needs to be a full-fledged "BuildMaster," but every developer needs to know how to execute a build locally and troubleshoot a broken build.  If you're suckered into being the technical lead, make sure every team member is up to speed on the build.   The "Check In Dance" The best practice for effective CI is to perform the integration on a developer workstation before that code escapes into the build server wild.  It's okay to break the build once in awhile.  One of my former colleagues used to say that the CI build *should* break occasionally just to know it's actually working.  What's not okay is to leave the build in a broken state.  That slows down the rest of the team by preventing them from checking in or out.  Even worse, somebody might accidentally update their workstation with the broken build and get into an unknown state.  If you follow these dance steps, you can minimize build breaks and run more smoothly.  Besides, it's embarassing to have the "Shame Card" on your desk. Let the rest of the team know a change is coming if it's a significant update. Get the latest code from source control.  Do a merge on any conflicts. Run the build locally and fix any problems found. Commit the changes to source control. Stop coding until the build passes. If the build breaks, drop everything else and fix the build. Here's a tip on annoying the crap out of your teammates.  I found a MIDI file of the "Chicken Dance" song one time and commenced playing that out loud everytime I checked in.  It was funny for about a day or two.   * I detest the term "Emergent Design."  That term makes it sound like you just spit out a bunch of code like a new age painter randomly splattering paint on a canvass and voila -- a coherent design will spontaneously create itself.  When I worked with the High Priests of Agile, I used to frequently get chewed out and told to "just let the design emerge."  To this day I have no earthly idea what that means.  I think it means just write lots of "it/then/else" statements until the code practically screams for restructuring, then have a big, heated hullabaloo meeting to agonize about the proposed refactoring.  Agile processes can be just as stupid as any CMM-driven waterfall if you're not thinking straight. Posted Jul 25 2005, 03:03 PM by Jeremy D. Miller with 23 comment(s) Filed under: Continuous Integration, Test Driven Development

• Quick way to spot a Blob class

  I'm in an all day set of meetings today to talk about how we're going to make our legacy web framework testable in the next version of the application.  It's going to be a cool project to do.  One of the first things you notice about this codebase is the central class.  It's an obvious blob class.  How do I know it's a blob?  First off, I counted 31 "usings" statements at the top of the class file.  Then I see business logic, HTML, javascript, and database manipulation all happily jumbled together in an ASP.NET user control.  My very first recommendation is basic surgery.  "Extract Class" here I come.  I don't know what I'd say for a hard limit to the number of namespace references, but if you've got a bunch you probably better start thinking about splitting up the class. Let's all say it together -- "One class, one responsibility."  Someday I'm gonna make this my very first tattoo. Posted Jul 22 2005, 04:58 PM by Jeremy D. Miller with 2 comment(s) Filed under: Ranting

• TDD Design Starter Kit

  Here's a handful of articles on designing with or for TDD I had originally posted on my old blog.  I've been haphazardly trying to write about all the things I wished I'd known when I was starting to use TDD.  To be honest, I think my first TDD/CI project was a mild disaster.  There's a few tricks and principles to follow to keep out of TDD hell and I'll eventually get around to more content on the subject.  TDD Design Starter Kit - Static Methods and Singletons May Be Harmful TDD Design Starter Kit - State vs. Interaction Testing TDD Design Starter Kit - Dependency Inversion Principle TDD Design Starter Kit – Responsibilities, Cohesion, and Coupling A Simple Example of the "Humble Dialog Box" I'm still meaning to write about the topics below.  If you've got any suggestions for future posts, feel free to send them to me and I'll see what I can do. Assigning Responsibilities Inversion of Control Dependency Injection (with and without a "container") Gateway Pattern Mocks and Stubs - Best (and Worst) Practices  Untangling Code from Configuration (huge pet peeve of mine) Law of Demeter/Tell, Don't Ask     Posted Jul 21 2005, 08:00 PM by Jeremy D. Miller with 6 comment(s) Filed under: Test Driven Development

• What an Amazing (Code) Smell You've Discovered

  I'm naming a new code smell today, "Obsessive Tracing." You know exactly what I mean. If you see a long method or class with a *lot* of Debug.WriteLine("1") or Debug.WriteLine("I'm in here now!") methods sprinkled throughout the code, it's a good bet the code smells to high heaven. Those trace statements are in there because the code is prone to breaking and hard to understand. The point of a code smell is to recognize a problem so you can begin to move in a different direction. I'm not sure what you do with legacy code, but with new code the key in my opinion is a well factored solution for ease of understanding and strong unit testing. Excessive amounts of debugging often means your unit testing isn't granular and comprehensive enough. Excessive debug statements might also mean a developer could benefit from reading up on the capabilities of their debugger. Geek points for nailing the movie line in the title. No Chris Fields you don't count, that one's too easy for you.   The original post with comments is at http://jeremydmiller.blogspot.com/2005/06/what-amazing-code-smell-youve.html.  I think it took about 15 minutes for someone to give me the movie title. Posted Jul 20 2005, 06:19 PM by Jeremy D. Miller with 11 comment(s) Filed under: Ranting

• Qualities of a Good Unit Test

  Many developers, including me sorry to say, treat unit test code as a second class citizen. After all, NUnit test fixture classes aren't going into production, so why should you put a lot of time, effort, and thought into them? An unfortunate reality is that unit tests are rarely "write-only" code. Badly written unit tests or interdependent unit tests hamper refactoring or adding new code to the system. I've heard of teams that were afraid to add new code to a system because the changes would introduce test failures. One of the advantages of automated unit testing should be the ability to safely modify existing code. Bad unit tests turn this advantage on its head. I'm giving an internal presentation today on mock objects and I prepared a slide on the "Qualities of a Good Unit Test." I actually couldn't find much in the way of resources on the web for this, so I made up the list below. I'd love to hear from the rest of you what your best practices are for unit tests. Unit Tests Should Be Atomic A unit test should only test a small piece of functionality. This falls inline with the idea of getting into a rhythm of "write a little test, write a little code." A litmus test is to ask yourself if any part of the unit test could stand alone in a separate unit test. Another feedback loop for your unit testing quality is the amount of time you spend with the debugger. If your unit tests are coarse, a test failure is more difficult to find. If the unit test exercises a small amount of code, the test failure cause can usually be spotted very quickly. Order Independent and Isolated There should never be an order dependency, intentional or not, between unit tests. Problems arise when one unit test leaves some kind of dirty data laying around. Testing against a database or some sort of static cache is a common culprit. The theme of the day was mocking (actually Episode III), so put evil stateful things behind nice, deterministic mocks or stubs. Intention Revealing A unit test can, and should, be a valuable form of documentation. At best, a unit test should explain the intended usage and function of a class. At worst, the unit test should still be easy to debug in the case of a regression failure. Excessive data setup can obfuscate a unit test beyond any hope of comprehension. Easy to Setup How easily can I set up an environment to run the tests? If the answer is "not very," I might never get the tests running. As much as possible, decouple unit tests from external dependencies like databases and web services by using mocks or stubs. When you do test against an external dependency, the dependency better be setup correctly by the automated build. Runs Fast You're never going to run the test just once. For the sake of constant feedback, successful continuous integration, and your sanity, make sure your tests run quickly. Thirty minute build and unit test cycles are nothing but friction. Just to keep in the mock rut, mock things that make network calls to speed up your testing. I like to segregate connected tests into a seperate assembly. This way you can quickly run the majority of your unit tests on demand without the lag from network calls. There's surely some obvious things I'm missing, but I'm short on sleep (and a midnight movie awaits) at the moment so that's all I've got. Posted Jul 20 2005, 09:10 AM by Jeremy D. Miller with 18 comment(s) Filed under: Test Driven Development

• Succeed with TDD by designing with TDD

  After 2 years of using Test Driven Development on .NET projects I'm a believer. Effective TDD leads to cleaner code, fewer bugs, and superior architectural qualities. Systems written with TDD are easier and safer to modify and deploy, leading to a reduced cost of ownership over the lifetime of an application. Yes, TDD means a lot more code to write. Yes, TDD can make the activity of writing code take longer, but it can and does shorten the time to delivery of working, production-quality code. Using TDD dramatically changed the way I approach OO coding for the better, but I made a lot of mistakes along the way. The easiest way to learn TDD is to work closely with teammates that are already comfortable with TDD. Otherwise, you are on your own, and a lot of the writing I have seen on TDD falls into one of two categories -- "TDD is Cool" and "How to use NUnit." You've got to start somewhere, but when faced with the very real world of enterprise architecture (databases, messaging, user interfaces, Active Directory), the simple examples of create an object, pump in some inputs, and validate the results suddenly don't seem to be enough. In my experience, newcomers to TDD often struggle with the mechanical work of writing unit tests, negating most of the value of TDD by writing unit tests that are too coarse. A common complaint is that the tests are too hard to write, or just take too much time to write. Frequently, well-meaning developers will "just make it work" first, and retrofit tests around the new code later. I did this pretty often on my first TDD project -- with the result largely being a set of fragile tests that flat out sucked and broke the build anytime someone breathed too hard on the codebase. The usual culprit is the way that the code is written and structured -- god and blob classes, stovepipe architecture, poor separation of concerns, and an overuse of static methods. So how do we write code to maximize the efficacy of TDD? The most important thing is to write good OO code (I'm not trying to exclude folks doing procedural or functional coding, but this post is meant primarily for .NET developers). A couple years most of my team of System Architects (sic) were sent to an introductory(!) class on OOA/D and came back giggling and blathering about code being "Highly Cohesive" and "Loosely Coupled." All those fuzzy sounding OO qualities we've known about for years aren't just a way to impress other developers while stroking your beard, they are absolutely necessary to easily utilize TDD. It's time to take these concepts seriously in the .NET world. Read this from Scott Bellware on TDD in the Microsoft community versus the Java world. A bit more on cohesion and coupling later. Which Comes First, Chicken or Egg? A TDD developer follows a continuous cycle of writing a small, isolated unit test for a little bit of functionality, then writing a little code to make the test pass. Ideally, an individual unit test involves a discrete chunk of a single class (or a few classes). How do you know if your unit tests are small/isolated/discrete enough? Easy, VS.NET debugger sessions become uncommon and short and test failures are easily diagnosed and remedied. So if small unit tests are good, how do you design your code to maximize the TDD goodness? By designing with TDD in the first place. Using TDD as a Heuristic From Merriam Webster - Heuristic: "involving or serving as an aid to learning, discovery, or problem-solving by experimental and especially trial-and-error methods " Simply put, TDD is a technique for determining class structure by making testability a first class consideration in your design. Focusing on testing a unit of code at a time leads to creating cohesive classes with a distinct purpose and responsibility. The need and desire to quickly setup an isolated unit test on a class will lead to a loosely coupled design. Here's a couple of quick questions answer to test whether your class structure really exhibits desirable architectural qualities: Paraphrasing Michael Feathers, can someone fire up VS.NET on your code, take any arbitrary method, and write a unit test in a short amount of time? Can you completely remove your computer from the network and run 100% of your unit tests covering both user interface and business logic code? Using a local database instance or a wireless connection does not count. Can you write tests that only involve one or a very few classes? If you are writing a web application (or a Web Service for that matter), can you run unit tests on much of your user interface code with IIS turned off? Test Driven Development is an Alternative Design Process There is far more to TDD than automated unit testing. Much like UML modeling or CRC cards, Test Driven Development is a process to explore a design and arrive at a good solution. The difference, in my mind, is that TDD is a "bottom up" process, where other design techniques are "top down." The truly good practitioners focus on rapidly building discrete, working pieces of code, then arranging the coded classes into an emerging structure guided by a knowledge of good design principles and a strong working understanding of Design Patterns. Some of the impetus for using UML or CRC is the belief that it easier to explore design ideas by using abstract non-code artifacts because code itself is too difficult or inefficient to change once it is written. TDD can turn this assumption on its head by making code easier to change and restructure because the resulting code is cohesive, and safer to change because a good automated unit test suite can catch errors resulting from the changes. A good refactoring and code navigation tool like ReSharper (JetBrains rocks!) makes this type of malleable code assembly more efficient. One way to think about TDD is an analogy to Lego blocks. The Lego sets I had as a child were the very basic block shapes. Using a lot of little Lego pieces, you can build almost anything your imagination can create. If you buy a fancy Lego set that has a single large piece shaped like a pirate's ship, all you can make is a variation of a pirate ship. Over the next month or so, I'm going to blog a TDD Starter Kit of design concepts and strategies. Keep watching this space, the next posts will be shorter and actually contain code. Posted Jul 20 2005, 04:54 PM by Jeremy D. Miller with 16 comment(s) Filed under: Test Driven Development

• What's This Blog All About?

  Thanks to the fellows at CodeBetter for letting me set up shop here.  This has been consistently one of my favorite web destinations for quite a while and I'm excited to be joining the crew.  I'm just starting the process of moving my blog over from http://jeremydmiller.blogspot.com, but for now here's my introduction.  The original post is at http://jeremydmiller.blogspot.com/2005/03/whats-this-blog-all-about.html.    When I was growing up in a farm community in Missouri, there was a "special" breed of folks around called Shade Tree Mechanics. Usually they were not the most reputable people in the world, but they had a knack for fixing mechanical problems and a reckless fearlessness to tinker with anything. A shade tree mechanic can be spotted by finding the pair of legs sticking out from under a beater car on blocks, surrounded by other skeletal vehicles crowding the rest of his scrubby, junk-laden yard. The beaters you see abandoned all around him aren't useless, they're fodder. He takes bits and pieces and tweaks and tunes, and comes up with a creative solution to your needs. Reputation notwithstanding, a shade tree mechanic knows how to get things running. While I don't have any particular mechanical ability (despite a degree in mechanical engineering), I like to think that I am the developer equivalent of a shade tree mechanic . My hard drive is certainly littered with the detritus of abandoned open source projects (and one successful project here). I've had a short, but varied, path into and through the land of software development ... Shadow IT - Getting my feet wet writing rogue applications for my engineering team. Having a blast cowboy-coding and generating a lot of fodder for the DailyWTF. Oracle, ASP, VB, and DHTML. I moved on to be a "real" developer. Captive IT - Working in in-house IT for a Fortune 100 company designing enterprise applications. I actually got to be the technical lead and architect on a mission-critical system for my first real development project. No prior experience, no adult supervision, no problems. I described the architecture to a former colleague, and his only comment was, "You're going to spend some time in Purgatory for that one..." I also got to experience my first Death March Project, a lot of corporate politics, and secretive reorganizations. On the positive side, I learned about high availability, disaster recovery, and instrumentation, and I began to successfully apply OO design patterns and UML modeling. Our waterfall process changed often, and was enforced capriciously by a dizzying array of compliance bodies, laying in wait like 18th century highwaymen determined to ambush hapless travelers. Non-Coding Architect - The last reorganization left me a non-coding architect just as the Microsoft world was dumping the DLL Hell (think VB/COM/ASP) technologies for .NET. I got to sharpen my Powerpoint skills while decorating my cubicle walls with Dilbert cartoons. The organization simply could not resist any type of evil fad -- CMM, Six Sigma, "It's not a process, it's a framework," fuzzy-headed SOA strategies, offshoring, "programmers are a commodity," "Who Moved My Cheese?", the "Fish" book, and a matrix model (yay for dotted-line bosses). I made a half-hearted, quixotic attempt to introduce iterative and incremental processes, but no one in management cared. In order to regain some type of technical relevancy and avoid 80-some hours of mandatory CMM training, I jumped again to become a consultant. High Priests of Agile - I spent a mostly positive year and a half working in the .NET practice for an elite consulting company. Extreme Programming, Test Driven Development, Continuous Integration and .NET. Consulting is a great experience. The intellectual opportunities are abundant, and no matter how bad an assignment or client is, you can solace yourself that it's only a matter of time before you'll get to move on to a new venue. I had to learn software design all over again to incorporate TDD techniques. Besides too much StarBucks on the expense account, I got an absolute bellyful of XP theory and zealotry. If I hear, "In the white book, Kent Beck says ..." one more time, I'm going to puke. XP, and agile in general, works -- but software was written successfully before XP, and some of the older stuff (UML, Responsibility-Driven Design, Design Patterns) is more effective in certain situations. Besides, constant travel with an infant at home is a non-starter, so I left for a job close to home. Independent Software Vendor - I am now working for a product-development company doing .NET development with a Scrum process. I am finally getting to make a positive impact on an organization by helping introduce TDD, CI, and new architectural ideas. Being empowered in a pragmatic and stable organization to proactively make improvements has been a ray of light after a series of frustrating positions. A ten-minute commute via a scenic backroad doesn't hurt either. I would like to use this blog to explore my thoughts about software design, .NET development, and agile methodologies. TDD and XP practices are getting a lot of attention, but I think there is a huge gap in writing between agile theory and successful practice. In particular, I would like to share my experiences with TDD and the role of design within an agile methodology. My name is Jeremy Miller and I am a .Net Developer and Architect in Austin, TX. Posted Jul 18 2005, 10:00 AM by Jeremy D. Miller with 8 comment(s)

• Resources from the Continuous Integration talk last night

  If you were at the Continuous Integration talk last night at the Austin SPIN, here is the set of resources I mentioned. The slides from the presentation are here or at http://www.jwspro.net/aspin/. Thanks for coming out last night in the rain and putting up with me. Articles Continuous Integration by Martin Fowler and Matthew Foemmel Evolutionary Database Design by Martin Fowler and Promod Sadalage Integrate Often from extremeprogramming.org Driving on CruiseControl by Lasse Koskela. Setting up CI in the Java world. The Agile Difference for SCM by Brad Appleton from CM Crossroads Mike Roberts on multi-project continuous integration Tools I published my company's .Net tool usage with descriptions and links here. CruiseControl (Continuous Integration for Java) Books I know there has been a couple of books published on CI now, but I don't know any to recommend. I heartily recommend Michael Feather's Working Effectively with Legacy Code for retroactively applying Test Driven Development and CI. Posted Jul 15 2005, 07:16 PM by Jeremy D. Miller with 3 comment(s) Filed under: Continuous Integration, Test Driven Development

• Bunch of Good Links

  First, maybe the best blog post I've every read from Mike Spille. I've been guilty of the "guardrail to guardrail" effect a time or two. On a project last year we probably did some dumb stuff because of someone else's negative experience with a certain technology on his previous project. Read the comments too --> "BTW, you left out the obligatory insult to VB." In a similar vein, James Bach warns us against being absolutist about "Best Practices." One of my favorite quotes is "A maturity model is basically a gang of best practices hooked on crystal meth." Just to add my 2 cents worth, I think it's best to always keep an eye on the first causes. I've written before that a centralized architecture group handing down black and white mandatory best practices from Mount Olympus to the pitiful wretches in the trenches is a terrible organization anti-pattern. There is always some kind of exception to every rule and the guys doing the actual work should be capable of exercising their own judgment. Lastly, integration guru Gregor Hohpe asks what the real difference is between coding and configuration. Here you go Gregor, the difference is that I can write unit tests for code and use a debugger to troubleshoot code. I don't know why people keep inviting Gregor to these kinds of conferences because he just mocks the proceedings in his blog later. My StructureMap tool is very configuration intensive and I have an unnatural tolerance for external XML configuration. To compensate, I've tried to provide the usage of .Net attributes for some configuration to be closer to the code. I've also had to invest a lot of energy into creating better tools for troubleshooting and validating configuration files in StructureMap. Since I dogfood StructureMap, expect more and better troubleshooting tools (because my colleague gets irritated otherwise). Posted Jul 14 2005, 07:13 PM by Jeremy D. Miller with no comments

• Impressions from the VSTS talk last night

  I went to a talk on Visual Studio Team System last night given by Chris Menegay at the Austin DotNet Group. I tried to go into this with an open mind, but my bias is clearly with open source tools and agile processes. I'm still not convinced VSTS provides any value beyond our current tools and process, but it'll help somebody. Either way, I think VSTS is a valuable addition to the .Net world. If nothing else, it's helped introduce lifecycle practices to shops that have none and started some healthy conversations. I wrote this several months ago, and nothing I saw last night substantially changes my opinion. http://jeremydmiller.blogspot.com/2005/04/bypass-vsts-and-get-yourself-poor-mans.html Without further rambling, cue the "wah, wahhh, wahhhhhh" theme music and let the cliche begin... The Good I think VSTS could be great for project tracking. Being able to intelligently log checkins to a user story or a bug fix with the time spent on the task could be a great boon to project management. The Heisenberg Principle always applies to this kind of tracking. If I, the developer, have to go out of my way, my velocity is going to be slower. VSTS might change the cost/benefit equation somewhat. A lot of any Agile process is the ability to accurately forecast user stories during iteration planning. Using a good "yesterday's weather" report from past iterations could certainly help. We're still going to take a look at Trac though as an alternative. The code coverage visualization is awesome. It's not clear yet, but it might be possible to create test coverage reports from other tools outside of VSTS (NUnit, NFit, etc.) It is an integrated suite of tools that will be supported by MS. I personally don't buy the argument that assembling a stack of NUnit/NAnt/CC.NET/NDoc/Subversion is really that hard, but I've been immersed in the agile world for awhile and I'm comfortable doing that (I've also worked with some of the NUnit and CC.NET folks too, so I'm not really impartial). For a shop with no experience in these tools or with a hostile policy towards open source, I can see the value of buying the whole stack in one place. The check-in policies (the code must compile, all unit tests must pass, static analysis conditions must be satisfied, etc.) might be a pretty slick way of optimizing a continuous integration strategy. The Bad Call me close-minded, but I wouldn't touch MSF with a ten foot pole and a firewall in between. The process templates are editable, but it doesn't sound like there is a GUI yet for doing the editing. I'll be interested to see if anybody (with much more patience and ambition than me) tries to make an XP or Scrum template. At this point it doesn't look like you can use any other source control system than what comes with VSTS. As much a pain as it has been trying to move from VSS to Subversion, I'm not very excited about another migration. The source control is clearly better than VSS (how could it not be), but I didn't see any particular value over Subversion today. VSTS does not support Continuous Integration, only scheduled builds. You'll need to continue using CruiseControl.Net. I fully expect the CC.NET guys to have a plugin for VSTS and/or MSBuild for CC.NET by the end of the year. The web testing looks weak to me. I still think we're looking towards either Selenium or Watir. The Ugly From the initial impression, the MSF for Agile process looks like a heavyweight process. It is clearly meant as an iterative process. While that's certainly an improvement on the older MSF, iterative alone doesn't automatically translate to being agile. VSTS is clearly aimed at very large companies using laborious, high ceremony processes--everything that my employer is not. I think process automation is a great thing, but if your process is so complex that nobody can know the whole, I think you've got some serious issues. Pricing. When alternative tools are available and generally free, I have a hard time justifying the cost. The one piece of the presentation where I thought Menegay was completely off his rocker was the idea that VSTS became a directed workflow tool to tell everybody (expecially developers) exactly what they should be doing. As a tracking tool for project record keeping, I think VSTS is great. Using something like VSTS as your primary communication channel is absurd. Turning developers into mindless zombies doing exactly what the VSTS workflow tells them to do sounds like a recipe for disaster to me. Not to mention a steep drop off in developer retention. EDIT 8/1/2005:  Check out Menegay's response and my comment at http://weblogs.asp.net/cmenegay/archive/2005/07/15/419600.aspx Posted Jul 12 2005, 07:07 PM by Jeremy D. Miller with no comments

• Don't play with strange databases...

  ...you don't know where they've been, or who's been touching them. I've been burned pretty badly in the past by making assumptions about existing databases. Database documentation isn't always helpful either, because it is out of date or just wrong to begin with. Even if the documentation is "correct," the corresponding code may be using it differently than the database designer intended. Even worse, you often have to interpret the data, and that's dangerous when you don't know the database. The only accurate source of information is usually the subject matter expert for the system. What's the answer? I don't know. All I can say is to approach someone else's database with great caution. Kinda like going up to a big, mean looking dog and trying to make friends. I wouldn't make any kind of assumption about the meaning of any table or field. Put some effort into understanding the schema and expect to go back and forth a little bit. It's the year 2005. Isn't the existence of SOA supposed to eliminate the need to be doing integration directly against a database? I'm writing a little code to integrate our main application with another 3rd party system. The only way I have to do this right now is to just write SQL queries against their database schema. I feel dirty. Even worse is when shops start writing their own extensions and back door queries against a 3rd party application. That's a great idea. Take a database that you're not supposed to touch directly, and that you don't fully understand, and write all kinds of custom code into it. That'll really make it easy to upgrade the software package later. Don't Make It Worse At a previous employer they use an ancient inventory system that is written in a rare 3GL language. If you think of your IT infrastructure as the cardiovascular system of a manufacturer, the inventory system is the heart muscle. Everything else talks to the inventory system. The business logic of the system was bound up in the UI screens, so the only front door integration point was a screen emulation package from the vendor that was only certified for low volumes. At this point an intelligent IT organization in the SOA era would start looking for alternatives. My old employer beat the integration problem by writing hundreds of PL/SQL procedures to duplicate the business logic in the screens in lieu of a real service layer. To support other functions they added about 300 custom tables to the out of the box database. Just to make things worse, the database customizations were different region to region. At one point I needed to write an integration from the inventory system to my application. I went and asked the SME where I could find a certain piece of information. He told me to look in table A and I turned around to leave. He then said, "you could also get it out of table B, or table C come to think of it." Oh, ouch. When I left they were struggling with a batch job that was supposed to run nightly that was taking about 25 hours to run and locking all kinds of database tables. They've experienced some problems with scalability, who would have guessed? Even though the system is horribly obsolete and arguably an opportunity cost slowing down new development, they have no chance of upgrading to the newer J2EE version with web services because their database is way too wired. Idiots. Posted Jul 11 2005, 07:02 PM by Jeremy D. Miller with 3 comment(s) Filed under: Database and Persistence, Legacy Code, Ranting

• Harris Boyce warns us that O/R isn't a silver bullet

  Harris Boyce wrote a response to my ADO.NET contagion post: "Soul Vaccination" for Data Access Layers. I think that Harris is quite right in warning us about overusing Object/Relational mapping. While there are still plenty of scenarios where I would forgo an O/R approach for the classic data access layer, I still think that the "Persistence Ignorant" (POCO/PONO/POO whatever) approach for business classes is the way to go for the sake of testability. I think the decision hinges primarily on the amount of business logic in your application (and secondarily on the comfort level of the developers with OO coding in general). Even if I'm not using an O/R tool, I usually use some type of Inversion of Control to keep the rest of the code loosely-coupled from the data access. Before diving into O/R mapping, you might take a look at these resources. Understanding some of the underlying patterns and mechanisms of O/R mapping might help alleviate some of the pain or help in making design decisions. Patterns of Enterprise Application Architecture by Martin Fowler has a couple of good chapters about O/R patterns. The chapter on organizing business logic is a canonical read for deciding whether or not to use a Domain Model approach that would lead to using O/R for persistence. Applying UML and Patterns by Craig Larman has a great chapter on creating a small persistence framework. This book is my choice as the best introduction to OO programming around. This article from Scott Ambler introduces the basics of O/R My article on C# Today from a couple years ago, but it's derivative of the guys above. Posted Jul 11 2005, 07:01 PM by Jeremy D. Miller with no comments Filed under: Database and Persistence