Startups and TDD

Yesterday Uncle Bob put up a post on using TDD in start up environments¬†“The Startup Trap”¬†its a good read. Check it out.

Nate soon after posted:

then

I wanted to write a few comments about TDD in startups. Good code is the least of the risks in a startup. Sorry but worrying about technical debt making us go slower when we have a two month runway and likely will pivot four times to quote Bob.

Captain Sulu when the Klingon power moon of Praxis exploded and a young Lieutenant asked whether they should notify Star-Fleet: “Are you kidding?” ARE YOU KIDDING?

One of the biggest mistakes in my career was building something appropriate…

It was just after Hurricane Katrina. I was living in a hotel. An acquaintance asked me if we could hack together this business idea they had for a trading system. He had the knowledge but not the know how. I said sure, hell I was living in a hotel!

In less than two weeks we had an algorithmic trading system. It was a monstrosity of a source base. It was literally a winforms app connected directly to the stock market. UI interactions happened off events directly from the feed! Everything was in code behinds (including the algos!) Due to the nature of the protocol if anything failed during the day and crashed the app (say bad parsing of a string?) the day for the trader was over as they could not restart.

But after two weeks we put it in front of a trader who started using it. We made about 70-80k$ the first month. We had blundered into the pit of success. A few months later I moved up with the company. We decided that we were going to “do things right”. While keeping the original version running and limping along as stable as we could keep it while adding just a few features.

We ended up with a redundant multi-user architecture nine months or so later, it was really quite a beautiful system. If a client/server crashed, no big deal just sign it back on, multiple clients? no problem. We moved from a third party provider to a direct exchange link (faster and more information!). We had > 95% code coverage on our core stuff, integration suites including a fake stock exchange that actually sent packets over UDP so we could force various problems with retry reconnects etc/errors. We were very stable and had a proper clean architecture.

In fact you could say that we were dealing with what Bob describes in:

As time passes your estimates will grow. You’ll find it harder and harder to add new features. You will find more and more bugs accumulating. You’ll start to parse the bugs into critical and acceptable (as if any bug is acceptable!) You’ll create modules that are so fragile you won’t trust yourself, or anyone else, to modify them; so you’ll work around them. You’ll build a festering pile of code that, with every passing week, requires more and more effort just to keep running. Forward progress will slow and falter. It may even reverse as each release becomes buggier and buggier, and less and less stable. Catastrophes will become more and more common as errors, that should never have happened, create corruptions and damage that take huge traunches of time to repair.

We had built a production prototype and were suffering all the pain described by Bob. We were paying down our debt in an “intelligent” way much the way many companies that start with production prototypes do.

However this is still a naive viewpoint. What really mattered was that after our nine months of beautiful architecture and coding work we were making approximately 10k/month more than what our stupid production prototype made for all of its shortcomings.

We would have been better off making 30 new production prototypes of different strategies and “throwing shit at the wall” to see what worked than spending any time beyond a bit of stabilization of the first. How many new business opportunities would we have found?

There are some lessons here.

1) If we had started with a nine month project it never would have been done

2) A Production Prototype is common as a Minimum Viable Product. Yes testing, engineering, or properly architecting will likely slow you down on a production prototype.

3) Even if you succeed you are often better to stabilize your Production Prototype than to “build it right”. Be very careful about taking the “build it right” point of view.

4) Context is important!

Never underestimate the value of working software.

This entry was posted in Uncategorized. Bookmark the permalink. Follow any comments here with the RSS feed for this post.

13 Responses to Startups and TDD

  1. Pingback: Working outside the Disciplines | hansenfreddy.com

  2. Pablo says:

    Why does Uncle Bob always assume that, it you don’t do TDD then you are not doing tests at all? I personally like the “test last” approach. There was something about “TDDing all” that doesn’t feel right to me. Then I found this interesting article: http://theruntime.com/blogs/jacob/archive/2008/01/22/tdd-proven-effective-or-is-it.aspx

  3. I agree: the validation of the business idea is much more important than the validation of the code correctness in a startup context. That said, having a mess for core domain code is indeed slowing you down in the 9-months run; but building a whole new system is probably as risky as maintaining and cleaning up the current version.
    I’m currently experiencing a large refactoring of startup code, and I’m grateful that they didn’t use TDD or the product probably wouldn’t have existed. But maintaining the application for several years points out lots of technical debt that has to be fixed if we want to follow the market; if you are not required to add new feature just leave the mess alone, but if you’re going to launch in several new countries paying down debt is necessary to contain costs in the long run.

  4. gregyoung says:

    @chris the problems with the prototype were architectural. It was a complete rewrite to fix. No amount of maintaining it right could have fixed them. It was just built well wrong :)

  5. Wayne M says:

    I could see not doing TDD (although IMO you should still have some integration tests) in a startup, because your goal is to launch a product quickly and spending a long time making sure your code adheres to all of the SOLID principles and the like will reduce your time to market, but there has to be a happy median between just slogging out garbage and dealing with the fallout later and spending too long and missing the window for a sellable product.

    Even outside of startups, at least in the Tampa area where I live, most business know nothing of TDD and SOLID and craftsmanship, and don’t care. You talk about good design and you get “but we need this by the end of the week for a demo” or “We don’t have time for that” or worse of all “We don’t do that here”. I was fired from a developer job after a year and a half because I was trying to show the benefits of SOLID or testing or any kind of quality; a lot of companies and dev teams just don’t care.

  6. Dafydd Giddins says:

    I fully agree with the concept of getting working software out the door as fast as possible, but that doesn’t mean you can’t do TDD. In fact when you do TDD, you have working software much faster than if just hack and slash as fast as you can.

    Greg described a situation where in two weeks they built a Minimum Viable Product without using TDD. You can just as easily if not as easier build an MVP using TDD. You don’t have to factor your code beautifully when doing TDD, it might help but at least with Tests in place you can safely move forward with your MVP or Pivot your MVP without the risk of losing the investment you have already put in.

    I think Gregs point about creating 30 production prototypes in the time it took to mature they first prototype would have made more money is miss-leading and probably untrue (i actually find it kind of offensive). You think you could come up with 30 new ideas that had that much impact and revenue in that short space of time,… really? Instead you matured a product that was proven, increased the revenue stream and ensured that revenue stream would be stable for the foreseeable future. That sounds a much more sound strategy for success that throwing a new idea out every two weeks and hoping for success. Hope is not a strategy. Yes you did over engineer from your own admission but that is not a fault of the TDD style of work or of writing elegant well factored code, it was your fault for adding features that where not needed. A failing in management not software engineering approach.

    Your quote of claiming to build something appropriate as being a mistake is also miss-leading. You didn’t build something appropriate, that was your mistake (referring to your mature product that is). You did build something appropriate in your first prototype (not built well IMO) but then you made it in appropriate but adding features that where not needed.

  7. Raif says:

    I’ve been thinking about this post off and on today. It reminds me of a decision I had to make early in my career. I could make a lot of money and sleep with a lot of hot ( disease ridden ) women if I build websites for the porn industry. Both good things no doubt.
    Now bear in mind that I am a die hard nihilist. One will never fix the world. One will probably never even make it any better. But do I really want to actively contribute to it’s decrepit state? Do I really want to make it worse? Sadly I decided I did not.
    You could start 10 new start ups by writing heaps of crap code and make a million dollars, and leave a legacy of crap for some other sucker to have to clean up and yes you’ll have a million dollars. I have a hard time arguing with people for who having a million dollars is the be all and end all. It’s a compelling goal. But there is also something compelling about building something with integrity, that is worthwhile. Something that other people will come in to work on and say sweet! this isn’t a honkin piece of shit.
    Again, I’m not down at UT Business school advocating quality and a reason d’etre over crap for cash. Those people are lost souls anyway. But you don’t have to contribute to the decline of western civilization. Or at least you don’t have to choose to.
    Just my 2 cents. With adjustment for inflation.

  8. Pingback: Clean Architecture vs. Startups « IExtendable

  9. Pingback: The Morning Brew - Chris Alcock » The Morning Brew #1310

  10. I agree that you shouldn’t have spent a bunch of time building a new application alongside your prototype. You did the right thing in shoring it up and fixing the worst pain points. I personally do not believe in building a green-field app when you already have a working brown-field one.

    Your point 3 is off. You’ve built your prototype and decided to keep it. At this point you should “maintain it right,” not start over with a green-field app. Your point 4 is dead-on: “Context is important.”

    I’m curious, is your prototype app still in use? Did it survive?

  11. Excellent points Greg. The quote “fail fast” comes to mind. Good coding practices are great but only if you can output something useful/usable in a very short time frame otherwise you are out of time and money. I hate cutting corners when building a solution but if I only have a few weeks to prove an idea then something has to give. When you have a revenue stream then go back and do it right. One man, limited resources – key advantage: speed to market.

  12. Pingback: Geek Reading March 6, 2013 | Regular Geek

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>