Nate asked me this morning if I had read or heard about James Shore’s “The Problems With Acceptance Testing” post. I hadn’t, but, as with the things of an agile project, if it’s important, it will be talked about again. Indeed, by this afternoon, a few people in the agile testing community had responded. Here are a few “money quotes” from people I follow:
Clear examples and improved communication are the biggest benefits of the process, but using a tool brings some additional nice benefits as well. A tool gives us an impartial measure of progress. Ian Cooper said during the interview for my new book that “the tool keeps developers honest”, and I can certainly relate to that. With tests that are evaluated by an impartial tool, “done” is really “what everyone agreed on”, not “almost done with just a few things to fill in tomorrow”. I’m not sure whether an on-site review is enough to guard against this completely.
In my experience, teams that don’t do automated acceptance testing quickly get to a point where adding new features goes slower and slower, just because it takes longer and longer to test all of the functionality. Sometimes they start trying to figure out which functionality they need to retest, and which “couldn’t possibly be broken by this change.” This starts down a very slippery slope.
Bottom line, I’m concerned about this issue because I like the clarity that results from having concrete tests that are agreed to be “the definition of done”. At the same time, Jim is a smart and experienced person, and we need to pay attention to what he’s finding out there.
I think ATDD/AT gets bad rap cuz teams don’t know how / don’t try to design tests well.
As a tester, I feel somehow offended.
Roughly where I am, though I may be changing my mind.