Menu

TDD vs BDD. The giants of test go head-to-head (2)

Episode 63 - 23 Nov 1985

Today, we'll bring the TDD vs BDD battle to a bruising conclusion.

Will Test Driven Development be pinned to ropes? Or will Behavior Driven Development be out for the count?

  • 2:18 - Round 5: Code Coverage
  • 2:54 - Round 6: Maintainability
  • 4:07 - Round 7: Portability

This episode follows on from the previous four episodes. Here are the links in case you missed them:

And remember to grab your free TDD vs BDD Cheat Sheet

Previously...

TDD and BDD went head to head.

Round 1 [Focus] was a narrow win for BDD.

Round 2 [Writer/Reader] was a decisive win for BDD.

But then the tide turned:

Round 3 [Speed] went to TDD.

And Round 4 [Specificity] went the same way.

That's two rounds a piece. But this bout is far from over.

Go live?

Welcome to Development That Pays.

I'm Gary Straughan.

And welcome back to second and final part of our TDD vs BDD battle.

Before we get back to the fight, there's the small matter of

the question I left you with last time.

Here's a quick reminder:

"You're responsible for a major system.

It has excellent test coverage, with a full complement of Unit Tests and Behavioural Tests.

A major deadline is looming and you're gearing up for the release.

Vexingly, there's one failing test.

And the Development Team say they won't be able to fix it in time for the planned release

Question 1: Do you go ahead and release to live?

Question 2: Does your answer to question 1 depend on the TYPE of test that is failing."

So what do you think. Would you release?

Alistair, you were clear...

"If there is a failing test then the system is not production ready.

I would argue that it (almost) never depends."

Although you did go on to say that might be mitigating circumstances,

such as the value of the new features in the release

outweighing negative impact of a bug.

Andrew, you were more bullish.

And I liked that you expected the Product Owner to be part of the Go / No Go decision

My answer? Well, I'm not a brave man, so I probably wouldn't...

But let's dig just a little bit deeper.

If all of the Unit Tests are passing, and one Behavioural Tests is failing

This is bad news.

A failing Behavioural Test means that the "contract" has been broken.

That's a clear DON'T RELEASE situation in my book.

The other case - all the Behavioural Tests passing and one Unit Test failing - is more nuanced.

If all the behavioural test are passing, then the contract is intact*.

Everything that the system promises to do*, it does.

Whatever is failing is having no impact* on the external functionality.

One possibility is that application code covered by the failing test doesn't do anything.

So the question becomes: "Why NOT go live?"

Did you spot the asterisks?

These arguments only hold water ONLY if the level of code coverage is very high.

Which brings us right back to our bout.

Round 5: Code Coverage.

A high level of code coverage is a good thing.

The higher the level, the better you sleep at night.

But 100% code coverage is rare and difficult - especially where unit tests are concerned.

Levels of 70% and 80% would be considered by many to be good.

For BDD, the picture is rosier - the code coverage percentages tend to be higher.

For a microservice with a well-defined API, there's no reason NOT to have 100% code coverage.

Round 6: Maintainability

The only constant is change.

So a good question to ask is what will cause the tests to change.

Any change to the functionality of the code is going to require the unit tests to change.

And by the way, if the functionality Is to change, then the tests change first.

A change is still development, and the development is still test driven!

What about on the BDD side?

Not all functional changes impact the external behaviour.

Going back to the cash machine example for a moment:

  • I could exchange the card transport mechanism for another model.
  • I could exchange the screen for one with better resolution.
  • I could upgrade the keyboard

None of these substitutions would require a change to this Behavioural Test.

For the sake of fairness, I should say that the the inverse situation is possible:

If new and different behaviour can be achieved by rearranging exiting code building blocks,

then the only test to change would be a Behavioural Test.

But generally speaking,

The high level nature of BDD tests and - more than that -

their contractual nature

Mean that they tend to change Infrequently.

I'm calling this round for BDD.

It's time for the final round.

Round 7 - Portability

Unit Tests live side by side with the code that they cover.

They are highly specific to the way that the code is written.

They are intertwined.

If there's one thing that Unit Tests are NOT, it's portable.

Behavioural Tests are the polar opposite.

Remember my Behavioural Test of "Trying to start a motorbike"?

Was that test specific to my motorbike?

Was the test specific to the model of the motorbike?

Not at all.

That test could be applied to HUNDREDS of motorbikes.

Behavioural Tests aren't at all coupled to the code.

You could decide to completely rewrite your application in a differ programming language

and use exactly be same tests.

BDD wins. By a mile.

The Judges' Decision is final

The bout is over.

Before we crown the victor, lets remind ourselves of how the battle played out.

Round 1: Focus

TDD is inside out. BDD is outside in.

TDD is do the thing right BDD is do the right thing.

Both are noble aims.

But the judges came down on the side of BDD two to one.

Round 2: Writer/Reader

Code against (more or less) plain english.

The outcome of this one was never in doubt. Another round to BDD.

Round 3: Speed

Speed of execution is the super power of Unit Tests.

A round to TDD.

Round 4: Specificity

When something goes wrong, Unit Tests are your friend.

Round 5: Coverage

A controversial round to be sure.

100% coverage for BDD might mean dozens of tests.

For TDD it's likely to be 100's of tests for the same codebase.

But, hey, no one said this fight was going to be fair.

Round 6: Maintainability

Yes, Change is inevitable.

But Behavioural Tests will change less often than Unit Tests.

And Round 7: Portability

Another all-too-easy win for BDD.

And the winner is...

Could I have a drum roll please.

The winner of tonight's main event is...

STOP, STOP, STOP

Guys, guys, guys.

Why all the aggression?

There's no conflict here.

TDD and BDD aren't adversaries.

TDD and BDD complement each other just about PERFECTLY.

Build your system using both BDD and TDD and you'll be building a system that

does the Right Thing

AND does the Thing Right.

You get to have your cake and eat it.

Now That's what I call a glorious victory.

Have your say

I'd love to get your thoughts on TDD and BDD battle.

Especially on this list of rounds.

Are these the rounds that you would have chosen?

Did I miss one that you would have included?

Take a moment to let me know in the comments below.

I look forward to reading your comments,

and I look forward to seeing you next time.