Maybe there's more than one way to develop good software?

Posted by Luke Francl
on Sunday, February 08

I’ve been following the recent blow up on the topic of test driven development with interest (perhaps a vested interest).

Basically what happened is Joel Spolsky said requiring 100% test coverage was doctrinaire and maybe mangled some testing gurus’ ideas a little bit. Which made them mad and there was this big harrumph about testing on blogs, Hacker News, and elsewhere.

Other people jumped into the discussion. Jay Fields offered a very well thought out post with a key point: if it hurts, you’re doing it wrong. Paul Buchheit said “unit tests are 20% useful engineering, and 80% fad”; and then Bill Moorier, the VP of Justin.TV, posted a followup to that discussing his dislike of unit tests.

Some people are trying to rip the test skeptics a new one. Giles Bowkett says it’s proof that the mainstream will never catch up, because they’re so totally clueless.

I find this all bemusing.

Here’s the thing.

Joel Spolsky created the original version of FogBugz, a pretty sweet piece of software. Paul Buchheit created GMail, a product that millions of people use and love every day. Justin.TV is also a very innovative product that’s popularized a new form of expression. Justin.TV is so “mainstream” that a guy committed suicide on it.

The testing advocates have a similarly impressive list of great software.

Maybe there’s more than one way to develop good software?

I tried to make this point explicitly in my Testing is Overrated talk: one of the reasons why test-driven design works is that it makes you think deeply about the problem you’re trying to solve. But it’s not the only way to think deeply about a problem! And it’s certainly not a guarantee for building successful software.


Leave a response

  1. Mike WoodhouseFebruary 09, 2009 @ 03:41 AM

    The stupid thing is that everyone with half a brain (and the others are unreachable and unteachable) understands what the ultimate goal is: great software. As defined variously, but including such characteristics as delighted customers, clean design, flexible in all the right places, performant, scalable, revenue-generating and ripe as a mature Stilton. OK, the last one may not appear on all that many lists…

    I’d been writing software for (gulp) 20 years before I even heard of TDD, and I’d only recently discovered that what I’d been doing to other peoples’ code for much of that time would have been called “refactoring” if I’d only thought of test coverage.

    Because I’m old and paradigms shift that much slower than they used to, I’m still not as test-infected as I like. And I really would like to get more test-infected, because I can see how it makes me think more carefully about my design and the end-product is better. That means I produce software that meets its goals more successfully.

    What annoyed me when I listened to Joel & Jeff was that they clearly hadn’t even bothered to investigate what TDD was, so they were dismissing a technique that might (only “might”, mind you) have helped them to do what they say they want: make delighted customers. Of course, this is what makes me come back each week: no other podcast makes me want to climb inside and argue. If they follow through with last week’s promise to have Uncle Bob Martin join them this week, it should be unmissable.

    As for the “I never unit-test and look how great I am” side, that’s confirmation bias in action. Or survivorship bias. At least one of those. Without good statistics on failures that part of the debate is entirely anecdotal.

    What we need is more side-by-side academic studies, of the kind that they’re doing at the University of Sheffield

  2. Jon DahlFebruary 09, 2009 @ 09:38 AM

    I look at it from a slightly different angle: you can build great software without doing everything right. TDD is a good idea, but so are a lot of other things, and no one does them all.

    Wouldn’t it be great if the only people who weighed in on issues like this were people who had experience with both sides: who had spent years doing TDD right (not half-assed), and had also spent years writing good software without unit tests? Or if language/framework comparisons were only done by people with significant, and relatively equal, investments in both technologies?

  3. Barry HessFebruary 09, 2009 @ 09:53 AM

    I can hardly listen to Ruby on Rails podcasts anymore. The echo chamber is ringing my ears. Mostly I’ve just heard it all before. Thus, I’m finding the complete alternative that Stack Overflow provides to be quite interesting. I find the generated discussion to be extremely interesting as well.

    Clearly, I love quality. Likewise, I err on the side of quality tests to test important bits of my application over a quantity of tests to improve statistical ratios. I’m very self aware; I know I probably just don’t get it.

    My difficulty with TDD and testing in general is represented by the simple existence of Jay Fields’ post. What are the odds that a given team will have a team member that can truly analyze the appropriate testing tools for a given project? What I’m hearing over-and-over is that it requires someone with 20-40 years of software experience to complete this analysis successfully. A vast majority of teams do not have this person and 20-40 years of experience does not guarantee one can succeed in such analysis. So frankly, minimal testing keying on important bits of core code seems like the most likely route to success, given a team of skilled software developers.

    Great point, Luke. There are different ways to succeed (make a living) at building software. If you find minimizing your unit testing to lead to a great cost-benefit equation, do it. If you find TDD to be your best bet, do it. But also take Mr. Fields’ advice: the TDD toolset that is succeeding in Project A may be precisely wrong for Project B.

  4. Luke FranclFebruary 09, 2009 @ 10:03 AM

    Mike, I think the problem with many TDD/BDD advocates is that they believe they have The Only Way to create great software and if you’re not using TDD, or not using it “right” you’re doing it wrong (I like to say that XP can’t be said to have failed, because, like communism, it’s never been tried).

    Jeff and Joel should give TDD fair shake. But I think people like Joel (especially) who are successful without buying TDD lock, stock, and barrel represent a threat to the movement, precisely because they ARE successful.

    Jon, I think you’re wrong in that there really IS more than one way to build great software. Clean Room/formal methods and XP couldn’t be MORE different but both methodologies have resulted in good software.

    Now, I don’t want to excuse those who knock TDD without bothering to try it, but on the flip side TDDers should be open to the fact that smart people are building good software without using TDD. Maybe there’s a reason for that? (Hint: smart people. The same reason good software gets built with TDD!)

  5. Sam SmootFebruary 16, 2009 @ 06:21 PM

    You know what I like best about this conversation? A Ruby focused discussion about TDD that doesn’t mention BDD (well, until the last comment anyways).

    Yes, BDD bashing is my flavor-of-the-month.

    BDD is TDD+Encapsulation says BDD fans? Why not TDD+SoC? TDD+SRP? TDD+Composition? There is easily a “Top 10” list of all-time greats when it comes to the temple of OOAD, why does Encapsulation (which isn’t even necessarily limited to OOP) deserve to bastardize and dilute the conversation about Test Driven Design?

    So in short, BDD should just curl up and die. Useless.

    Anyways, about TDD… awesome comments. Especially John’s. Less guilt is good if you and your clients are happy with your products.

  6. lukixMarch 06, 2009 @ 02:31 PM

    I am successful without buying TDD lock, stock, and barrel represent too. I do my work precisely all the time.