Jeffrey Palermo (.com)

Sponsors

The Lounge

News

Recent Posts

Syndication

Tags

View more

Archives

Advertisement

Jeffrey Palermo (.com) » Rocky gets it all wrong about Test-Driven Development (TDD) - level 200
Images in this post missing? We recently lost them in a site migration. We're working to restore these as you read this. Should you need an image in an emergency, please contact us at imagehelp@codebetter.com
Rocky gets it all wrong about Test-Driven Development (TDD) - level 200
I'm listening to the latest .Net Rocks show with Rocky Lhotka, and I'm saddened that he gets it so wrong about TDD.  DNR's last episode ROCKED with Jean Paul Boodhoo, who talked about TDD and other Agile concepts.  Jean Paul is a practitioner of TDD, as am I, and he explained the process and its benefits very well.  Congrats, Jean Paul on a job well done.

Rocky, on the other hand, gets it all wrong.  I respect Rocky and his contributions to the developer community, but he's wrong on this issue.  Let me recap:

At the beginning of the dnr show, he comments on TDD.  He makes some invalid points.  His points are bulleted, and my responses are in green:
  • It comes from the premise that you don't have intellisense in the IDE.
    • I'm really not sure how he came up with this since most other good IDEs have intellisense. 
  • In the Microsoft space, you lose so much by going down the road to TDD.
    • This is a very vague statement, but I've gained so much by doing TDD.  My rate of development has sped up because my debugging time has gone almost down to zero!
  • Rocky misunderstands the process because he states that it involves writing a bunch of tests first before coding.
    • TDD involves writing a single test first and writing code to fulfill it.
  • He states that on large systems, you can't do it one test method at a time.
    • Can't?  Any large system is made up of smaller parts, down to actual classes and methods as the building blocks.  TDD helps define the classes and methods.  TDD forces you to start small, but helps build a solid foundation.
  • He says that developers don't think the way of "test-first".
    • This is actually a true statement but in no way is a weakness of TDD.  Test-first is a learned method just like programming as a whole.  Learning to think first is very similar to understanding the purpose of the code first.  If you can understand the purpose of the code before writing the code, you end up with less code churn and a clean class design.
  • Rocky relates that he has always made small test projects or console apps to be test harnesses to test code.
    • I used to do that also, until I graduated to NUnit.
    • These test harnesses are valuable, but as you accumulate them, they are hard to manage, and since they are valuable, you don't want to throw them away.  NUnit allows these test harnesses to be managed effectively.  Each test method is a test harness.
Another false point made in conversation is that once these test fixtures are created, developers will be hesitant to refactor code because it would cause them to have to refactor all the tests that use the code.  Let me say from experience that unit test coverage makes me want to refactore MORE!  I am so comfortable refactoring because I have unit test coverage that assures me that I haven't broken anything.  In fact, when I am in parts of the code that is older and doesn't have good test coverage, I am scared of refactoring because I have no way of being sure that I didn't break anything.  Note, that I also use Resharper, and I would not want to code at all without it or the DevExpress tools.

I love .Net Rocks, and I've listened to it from the very first show (and I've been a guest), and I'm glad all viewpoints are represented, but the above bullet points are incorrect, and I hope some of you will comment on them as well.

Posted 03-28-2006 9:30 PM by Jeffrey Palermo
Filed under: ,

[Advertisement]

Comments

Bjørn Reppen wrote re: Rocky gets it all wrong about Test-Driven Development - level 200
on 03-29-2006 4:42 AM
You have a couple of points.. but I have to say I share Lhotka's feelings about test-driven development.  So far I have been unsuccessful in getting used to the TDD idea.
Jason Haley wrote Interesting Finds
on 03-29-2006 6:14 AM
Jeffrey Palermo wrote re: Rocky gets it all wrong about Test-Driven Development - level 200
on 03-29-2006 7:43 AM
 
Bjørn, do you do any automated testing?  If the answer is "no", then it will be hard to just jump into TDD.  TDD requires unit testing.  I'd recommend starting to write some unit tests and integration tests.  Once you are comfortable with that, it may be easier to try TDD.  
karl wrote re: Rocky gets it all wrong about Test-Driven Development - level 200
on 03-29-2006 7:51 AM
I think there's a lot of education that still needs to go on about the unit testing and TDD development strategies. Whether that's only true of .NET developers, I don't know.  I can speak for myself and say that I still have a lot to learn about it, and I've seen other "blunders" from the .NET community (ie, that funny msdn article) about the topic.

It's clearly either very difficult, or very different and people just aren't understanding it or accepting it (or atleast a perception that it's one of those two things).
Sam Gentile wrote re: Rocky gets it all wrong about Test-Driven Development - level 200
on 03-29-2006 8:16 AM
Par the course
Jeffrey Palermo wrote re: Rocky gets it all wrong about Test-Driven Development - level 200
on 03-29-2006 8:42 AM
Karl,
It definitely is a shift in thinking.  I still talk with folks who still think in procedural code.  They have yet to make the shift to object-oriented thinking.
Brendan wrote re: Rocky gets it all wrong about Test-Driven Development - level 200
on 03-29-2006 8:49 AM
Anyone new to TDD ought to watch this video that Scott pointed out a while back.. Forget the fact that they're really talking about pairing, and look at how they construct their tests, one at a time and in very small steps... That's how it should be done!

http://codebetter.com/blogs/scott.bellware/archive/2006/01/05/136288.aspx
Tim Murphy wrote re: Rocky gets it all wrong about Test-Driven Development - level 200
on 03-29-2006 9:34 AM
I am somewhat spit on this one.  I am a big fan of Rocky.  I was shocked by his statements in this DNR episode.  I would really like to give Rocky an opportunity to expand on what he said.

Personally, I haven't done much in the way of TDD.  I can see many benefits to it, but there is still part of me that feels that if you are always focused on the micro you are going to miss the big picture and can go down a seriously wrong path.

Now if we say that you lay out the major highways of your design and then use TDD to do local streets perhaps I could buy that.  I guess I'm still trying to figure this out.
Bil Simser wrote re: Rocky gets it all wrong about Test-Driven Development - level 200
on 03-29-2006 12:11 PM
It is sad that Rocky has this view of TDD. I missed the episode but will listen to it on the plane tommorow. I respect Rocky and all his work on CLSA and in general, but when a leader comes out with statements like this I have to scratch my head.

The two main things I see are that having unit tests are your safety net. I can't stress how important it is to have them, and yet people just blindly go forward building safety or mission critical systems with 0 tests and get frustrated when they tell me it will take longer to add tests.

As for TDD, yes it's a mindset an a shift but once you get into the groove, it becomes second nature. Not only do you have a good set of solid tests but a) your business entities actually start getting cleaner as you're not adding bloat to them to "get stuff done" and b) you get your business partner involved as the requirements become clearer as you're thinking about testing from get go and how to structure requirements based on testing.

A good set of requirements are ones that you can look at and see the tests form in your head (or at least that's what I've found). You don't get these requirements right off the bat, just like your class design isn't always correct up front but they emerge through communication and collaboration with everyone.

TDD just works but everyone needs to be on board and the design/development team needs to be thinking a little differently with different goals. It's not hard, but it's not recipie driven software design either.
Adam D. wrote re: Rocky gets it all wrong about Test-Driven Development - level 200
on 03-29-2006 2:16 PM
There was a great message on the TDD group the other day. Someone asked why TDD was not allowed to be used in a shop where they were employed. The reply from an architect or some dev lead was said with a knowing smirk: "Is that the NUnit tests? We don't have time for those." lol
MB wrote re: Rocky gets it all wrong about Test-Driven Development - level 200
on 03-29-2006 4:53 PM
In general, I'd have to disagree with Rocky - except for the statement about developers not thinking test first, and the statement about refactoring tests.
IME, I think that what he says there has a grain of truth to it. Others <a href=http://butunclebob.com/ArticleS.TimOttinger.ReasonsToStayWrong>agree</a>. Since the tests depend on the code, they create a weight on the code that can hinder large scale changes. Of course, the TDD way is to say that you should only be making small changes that add up to a large change - refactoring tests as you go. Possibly valid, but not a natural inclination to most devs I know.
Rockford Lhotka wrote re: Rocky gets it all wrong about Test-Driven Development - level 200
on 03-29-2006 5:09 PM
It may be that I don't have a full appreciation for TDD - anyone claiming to know everything is a fool :)

But I do want to be clear that I was cut off in the dialog before I could spell out that I _absolutely_ use and favor unit tests (and in particular nunit). I wouldn't be caught without unit tests, and all those little tests that I used to write are (and have been for years) captured in unit test format.

What I object to is the idea that I should write tests before I write the code. I guess it is just an old habit that is hard to break - but I write my method first, then write the test(s). It works for me.

I used to write the tests as little apps, then as nunit tests. Either way, unit testing is critical and I surely wouldn't mean to suggest anything to the contrary.

Rocky
Greg Young wrote .NET rocks 3/28/2006 (Rocky)
on 03-29-2006 7:48 PM
Greg Young wrote .NET rocks 3/28/2006 (Rocky)
on 03-29-2006 8:13 PM
Fregas wrote re: Rocky gets it all wrong about Test-Driven Development - level 200
on 03-30-2006 9:56 AM
Rocky,

I think we all appreciate your response and the fact that maybe the "interview" didn't capture your full appreciation or understanding of TDD.  However, the fact that you are still writing your methods first, then your tests seems to suggest you have missed the biggest part of TDD--at least from the perspective of most of the TDD folks i've talked to.  TDD is a DESIGN METHODOLOGY, NOT a testing methodology.  It is meant to help you DESIGN better code, and automated testing is really just a side effect.  

If you are writing methods first, then you've already created a design and are just building a unit test for automated testing.  This is fine, but you miss probably the most crucial aspects of TDD which is writing better code by looking at objects or methods from the perspective of the client using them.  

Rockford Lhotka wrote re: Rocky gets it all wrong about Test-Driven Development - level 200
on 03-31-2006 10:09 AM
Yup, I freely admit that I disagree with TDD as a design methodology. I much prefer behavioral OOD using a modified CRC approach, hands down, to anything else I've tried.

But I am glad to hear that no one views TDD as a testing methodology, because as such it would be truly inadequate. The other part of what I was unable to complete in the interview (because this was just the lead-in banter part, not the real interview at all) is that developers are _terrible_ testers (with perhaps a few exceptions).

This is because developers test to make sure something works. Testers, on the other hand, test to find ways something doesn't work. Testers focus on the edge cases, the exceptional cases, the scenarios that a developer often (typically?) simply ignores.
Jeffrey Palermo wrote re: Rocky gets it all wrong about Test-Driven Development - level 200
on 03-31-2006 10:45 AM
If anything, TDD might be a micro-design methodology.  Personally, I can't create a non-trivial system completely unit-test first.  I have been using TDD for right at a year now, and I'm very pleased with how it has helped the whole process.  

I love TDD, but I'm not one who says "if you don't use TDD, you suck".  That's not rational.  

I don't use formal CRC cards, but I like that approach.  My team uses the whiteboard liberally.  In fact, every wall in our war room is whiteboard space, and most of the time, there is not an empty board.  Before we start something, we will sketch out a crude class diagram and talk about class responsibilities and each class's collaboraters.  Only after we have a high-level design to we begin to test-drive the creation of one of these classes.  It may be a prerequisite to create the interfaces that it will depend on.  And then the first steps are creating the class, and the test class that will test drive it (not in any strict order).   After we actually have those classes, then the TDD actually begins, and that's were a test method will drive a single class responsibility (that has been decided ahead of time).  

Let me be clear.  We don't design methods ahead of time, only the important classes.  Our TDD process fleshes out what methods we need bases on the responsiblities we have given the class.  If a class starts to get too big, we have to qualms about refactoring out some of it to a separate class.

TDD with bad design skills will still give you crap software, and you probably will end up with integration tests instead of unit tests.  There are a huge number of ways you can do it wrong.

I believe that my team is doing it right, and what we've ended up with is a VERY loosely-coupled system that is super easy to test.  I believe that because we emphasized testability through TDD, our subsequent integration tests and FIT tests were very easy to create.

My team doesn't stop at TDD.  We do the whole gamut of automated testing.  Because of these, we have high confidence that our software works.  

Thanks for clarifying, Rocky, and I hope to see you again at Tech Ed.  This year more people from the Austin .Net User Group are attending.
Jeremy D. Miller wrote re: Rocky gets it all wrong about Test-Driven Development - level 200
on 03-31-2006 10:47 AM
Rocky,

There's absolutely no reason you can't use any other design methodology *with* TDD.  I've found Responsibility Driven Design to work very well in conjunction with TDD.  TDD design is about breaking a design into little pieces and tasks, using a unit test to describe the pieces, and then assembling the pieces into the aggregate.  RDD with CRC cards is a very natural complement to TDD.

The one thing that TDD does better than any design tool or methodology I've ever used or seen is enforce loose coupling and a separation of concerns.  You simply can't easily unit test tightly coupled code.  Testability very frequently coincides with what we'd generally call a good design.

I wrote an article on this last year:  http://codebetter.com/blogs/jeremy.miller/articles/131726.aspx
Bil Simser wrote re: Rocky gets it all wrong about Test-Driven Development - level 200
on 03-31-2006 11:13 AM
Rocky,

Good to hear it from you on what the intent was. Sometimes in editing things are missed and even then verbal communications is just one part of expressing yourself and subtleties are sometimes lost.

I think everyone agrees that TDD as a testing methodolgy misses the point of TDD as it's a design methodology (or micro-design as Jeffrey said). At least having the unit tests there are the godsend. How you arrive at the end design is a matter of choice and I'll admit that 100% TDD is a little insane.

What I do find with TDD is my business classes tend to be leaner and more succint in what they need to do, but YMMV.
John Meyer wrote re: Rocky gets it all wrong about Test-Driven Development - level 200
on 03-31-2006 1:02 PM
I have to agree with Rocky on this one!  Getting a failed test to pass is debugging, not developing.  TDD is testing and debugging, and I'd much rather do developing in the software engineering part of my job, since I already get more than my share of testing and debugging in the maintenance programmer part of my job
Tim Murphy's .NET Software Architecture Blog wrote A TDD Holy War
on 03-31-2006 1:45 PM
DonXml's All Things Techie wrote Rocky Lhotka and Test Driven Development
on 03-31-2006 1:48 PM
Brian Sherwin wrote re: Rocky gets it all wrong about Test-Driven Development - level 200
on 03-31-2006 2:53 PM
The biggest problem here is that there are really three groups that don't get along real well with each other.
First, there is the group that says "ALL code has to have a test FIRST" (hardcord TDD).
The second group says, "We don't have time to write all of these tests before we write code--too much time is built running the tests just to see red when I know it's going to fail.  That's just STUPID." (TDD is for idiots).
The third group says, "Wow, unit testing gives me confidence to refactor.  I should have unit tests to cover all my code.  I should at least be thinking of how to write the test as I'm doing the development, and ideally, I write that test soon after I wrote the code.  TDD is too much, not TDD is too little."  (Both sides can work.)
I'll be honest--I don't TDD *ALL* of my code.  I do however TDD any refactorings.  Recently I was rewriting a module of a huge application I am working with, and the approach was--write a test that excercised the method I was replacing/rewrite code/verify new method returns the exact same results.
    Assert.AreEqual(Obj1.TestMethod, Obj2.TestMethod)
If my memory serves me, Microsoft took some heat for "trying to redefine TDD" with their "Right Click/Generate Unit Tests."  Probably the biggest thing is that automated testing is much easier and more visible (at the developers level) today than it was in say...the last release of VB.COM or C++
It doesn't have to be all or nothing to be an effective developer.
Brian Sherwin's Blog wrote ROCKY 8--The purists are fighting again...
on 03-31-2006 2:57 PM
Jeffrey Palermo wrote re: Rocky gets it all wrong about Test-Driven Development - level 200
on 03-31-2006 3:18 PM
Well said Brian!
It doesn't have to be all or nothing.  For instance, I don't test simple code that Resharper generates, like simple property accessors.

Everything has to have common sense applied to it.
Brian Sherwin wrote re: Rocky gets it all wrong about Test-Driven Development - level 200
on 03-31-2006 3:51 PM
Actually, if I lumped you into the purist camp, it's only when things like "all wrong" are thrown out.

You excused yourself from this camp when you say you don't hold to the thought of "if you don't use TDD you suck."

More so, the purist thoughts in me comes out when people comment and they haven't even heard the statements, read his followup, etc. and make naysaying comments.  Your site is not the only one to take this up by the way.
Jeremy D. Miller -- The Shade Tree Developer wrote TDD Misconceptions
on 03-31-2006 3:52 PM
Jeffrey's post on Rocky Lhotka's off the cuff remarks about TDD is spawning a good discussion and several...
Tim Gifford wrote Rocky's comments on Test Driven Development
on 04-02-2006 8:19 PM
Chris Donnan : Programming - Brooklyn Style - Rocky and Jeffrey battle on about Test Driven Development … and my 2 cents wrote Chris Donnan : Programming - Brooklyn Style - Rocky and Jeffrey battle on about Test Driven Development &#8230; and my 2 cents
on 04-03-2006 10:59 AM
Chris Donnan wrote re: Rocky gets it all wrong about Test-Driven Development (TDD) - level 200
on 04-03-2006 11:27 AM
I think that all too often people practice TDD without the other supporting practices. If people try to do TDD, but do not practice IoC for example - their tests will be much harder to write - and it will be more frustrating, less useful etc. It is like taking something out of the context it was intended to be used in. I have seen this happen. That being said - I believe that TDD has value without things like IoC, but when used WITH IoC, mocks, etc - the value increases greatly.

Thanks for always providing an interesting read :)

-Chris

(
ps - I elaborate here - http://chrisdonnan.com/blog/2006/04/03/rocky-and-jeffrey-battle-on-about-test-driven-development-and-my-2-cents/ )
Jeffrey Palermo wrote re: Rocky gets it all wrong about Test-Driven Development (TDD) - level 200
on 04-03-2006 11:35 AM
Chris,
Thanks for the comment and the related post.  I agree that other tools complement TDD and vise-versa.    TDD without other tools and practices (and solid OO design) _would_ be pretty weak. :)
Giddy Up! - Erik Lane's Blog wrote Test-Driven Development (TDD) - Good or Bad? I don't know.
on 04-03-2006 12:17 PM
Mike S. wrote re: Rocky gets it all wrong about Test-Driven Development (TDD) - level 200
on 04-04-2006 3:21 AM
TDD is both a testing and a design methodology.  The key word is "a".  It is never described as "the" methdology and so it isn't meant to be the only testing methodology used on a project.  But to not validate it as a testing methodology in my opinion misses one of its best benefits which helps keep development solidly on the rails.  I think of it as a controller of sorts and prevents code from getting ugly and unwieldy by never letting it stray too far.
John Wood wrote The Case for Unit Testing
on 04-04-2006 10:36 PM
Leaving aside the holy war of design methodologies, it's worth noting that regardless of what design...
John Williams wrote re: Rocky gets it all wrong about Test-Driven Development (TDD) - level 200
on 04-05-2006 5:14 PM
Look here... the "reality" of TDD, is that most developers are doing real TDD: http://secretgeek.net/TODO_driv_dev.asp.

All the TDD/XP jihadists need to get a grip. You're not getting a virgin when you complete your project: http://www.softwarereality.com/ExtremeProgrammingRefactored.jsp

In the "real" world, NOT the echo-chamber of the blabosphere, managers and developers with real deadlines are going to be a very hard sell for TDD, pair programming and other such "enlightened" practices. Because people don't agree with TDD/XP practices and may use other "passe" methodologies does NOT make them wrong. There are many ways to skin a cat and combining Unit Testing with other more established practices is how the "real" world is going to use "Agile" methodologies, NOT by following the Agile manifesto to their death. Come to think of it, didn't the unabomber have a manifesto? Oh ya, right. He was crazy too... ;-)
Raj's Blog wrote Test harnesses and TDD
on 04-10-2006 12:09 PM
Jeffrey Palermo [C# MVP] wrote TDD speeds up unit testing - level 300
on 04-12-2006 11:29 PM
Before reading this post, I recommend reading a previous post of mine along with the comments and trackbacks...
Sam Gentile wrote 80 People for Extreme Programming in .NET at Austin Texas!!
on 04-15-2006 8:32 PM
I have been in Austin Texas the last 2 days joining my CodeBetter mates Jeffery Palermo, Jeremy Miller...
Sam Gentile wrote 80 People for Extreme Programming in .NET at Austin Texas!!
on 04-16-2006 3:07 AM
I have been in Austin Texas the last 2 days joining my CodeBetter mates Jeffery Palermo, Jeremy Miller and the one and only Scott Bellaware last night for a bbq at Jeff's house in Leander, TX. Jeff talks about it here. Then I gave a 2 hour talk on Extreme
Sam Gentile wrote 80 People for Extreme Programming in .NET at Austin Texas!!
on 04-16-2006 3:20 AM
I have been in Austin Texas the last 2 days joining my CodeBetter mates Jeffery Palermo, Jeremy Miller and the one and only Scott Bellaware last night for a bbq at Jeff's house in Leander, TX. Jeff talks about it here. Then I gave a 2 hour talk on Extreme
Scott Bellware [MVP] wrote Mort or Elvis? A Question for a Bygone Era
on 04-25-2006 2:50 AM
I've spent a lot of time bloviating on the Mort vs. Elvis thing.  I got confused once, and thought I was an Elvis.  Not really understanding what an Elvis is, I accidentally identified with the Elvis persona and championed the Elvis cause.  I feel dirty.
Developer wrote re: Rocky gets it all wrong about Test-Driven Development (TDD) - level 200
on 04-25-2006 6:32 PM
I think you are being very arrogant...You speek like if your opinions were the only right thing in the world. There are lot of people out there working without TDD and they are successfull as there are a lot of people out there with TDD that are not (take a glance at the CHAOS reports, from the standish group).

Saying this, I am not advocating again TDD, which I think is great. What I would like to express is that there are other things that can lead to success or failure. I think communication

Remember: there is no silver bullet!

And remember too: you are not the owner of the truth!
tecosystems wrote links for 2006-04-26
on 04-26-2006 3:22 AM
GNULinuxupdatedw4.0.jpg (JPEG Image, 1024x849 pixels) very useful explanation of where some of the Linux distros derive from, and what communities they rely on (tags: desktop distribution distros Linux opensource community visualization image history)
james governor wrote re: Rocky gets it all wrong about Test-Driven Development (TDD) - level 200
on 04-27-2006 5:32 AM
We have been thinking about taking this debate into a closely related space- anyone for test-driven SOA?
http://www.redmonk.com/jgovernor/archives/001504.html

So Jeffrey - I hope you get a chance to hook up with Cote on agile issues in Austin soon. Agility and .NET and Chuys sounds good!
Jeffrey Palermo wrote re: Rocky gets it all wrong about Test-Driven Development (TDD) - level 200
on 04-27-2006 9:02 AM
James,
I did meet with Cote last week.  Nice guy.  We're talking about growing the Agile community in the Austin area.
Fear and Loathing wrote TDD isn't enough
on 04-27-2006 10:17 PM

There&amp;rsquo;s been a bit of a bru-ha-ha recently with Rocky Lhotka&amp;rsquo;s comments on TDD (which I&amp;rsquo;m...
Ayende @ Blog wrote DotNetRocks #169 and TDD
on 01-13-2007 2:24 PM

DotNetRocks #169 and TDD

Ayende @ Blog wrote Rocky Lhotka on TDD, Take #2
on 01-13-2007 2:24 PM

Rocky Lhotka on TDD, Take #2