Episode 122 - 18 Apr 2018

Get multiple estimates. Ideally, from different people

To err is human; but we can average out the “errs” by estimating in a team.

I think this makes sense intuitively: If we ask a group of 5 people to estimate the weight of a locomotive and get these results:

- 60
- 140
- 120
- 120
- 80

We could average them out (60 + 140 + 120 + 120 + 80)/5 = 104

Which isn’t far off.

But things don’t always work out so cleanly.

Now that this group is warmed up, let’s test them again:

“How long does it take to get from Paris to Brussels?”

(it’s a question we’ve asked before. And if you saw last week’s episode, you know that I am - quite literally - asking for trouble.)

Here are the estimates:

- 4 hours
- 1 hour
- 2 hours
- 2 hours
- 4 days

Whoa… that’s quite a range: 1 hour to 4 days!

4 days is 96 hours. That’s a range of very nearly 2 orders of magnitude.

Huge!

What’s wrong with these people? What’s going on inside their heads?

Let’s take a look:

- The person that said 1 hour was thinking of going by aeroplane.
- The person that said 4 days was thinking of going by bike
- 4 hours: driving.
- 2 hours: train. Makes sense
- 2 hours: aeroplane. Interesting: this person has factored-in the time to get to the airport which is sits north of Paris.

5 people estimating... 5 entirely different things!

Which brings us to the second key:

What if we asked them a different question:

“How FAR is it from Paris to Brussels?”

The results are in:

- 150 miles
- 200 miles
- 260 miles
- 180 miles
- 220 miles

This time the answers are in board agreement: they all of the same order of magnitude.

An estimate of 2,000 would be one order of magnitude different; 2 orders of magnitude would be 20,000 miles.

Which will get you most of the way around the earth!

If we look inside their heads...

Looks like everyone’s looking at the problem in the same way: everyone is using the same basis for their estimate.

It’s an OBJECTIVE measure.

That’s In contrast to the SUBJECTIVE measure we started with: travel time.

When we’re estimating in software development, we want to avoid “development time” - which is a SUBJECTIVE measure.

Instead, we want an OBJECTIVE measure.

And the unit of measure of this “objective measure” is the Story Point.

Here’s a definition from Mike Cohn:

“Story points are a unit of measure for expressing an estimate of the overall effort that will be required to fully implement a product backlog item or any other piece of work.”

So it’s a measure of the effort required to build, test and release the Feature or story.

Mike goes on to say:

“Because story points represent the effort to develop a story, a team’s estimate must include everything that can affect the effort. That could include:

- The amount of work to do
- The complexity of the work
- Any risk or uncertainty in doing the work

At the risk of stretching an analogy to breaking point…

we could say similar things about a map.

- The amount of work to do… is like distance
- The complexity… is like the terrain - which is another thing
- And uncertainty or risk… Okay… now I’m struggling… areas of political unrest.

Let’s move on…

We’ve still one more key to get to.

As we touched on last time, the originators of Story Points felt the need to bring a new unit of measure into the world because other non-time unit of measure candidates - lines of code, function count, number of commits - were too easy to fake.

But they’ve given us a measure that is ABSTRACT - you can’t look up the size of a Story Point.

However…

the absolute size of a Story Point doesn’t matter. The reason for that is a consequence of our third and final rule.

Sorry people; maps again!

The team we started with did a pretty good job of estimating the distance from Paris to Brussels.

But everyone on the the team was either from France or Belgium. And none of them have been to the USA

So this time I’m going to take them right out of their comfort zone:

“How far is it from San Francisco to Los Angeles?”

They are going to struggle.

Would it help to see a map? Yes, but only if they can figure out the scale.

We’re not going got get much joy out of these guys

Let’s give them one last try:

“The distance from San Francisco to Los Angeles is ______ the distance from San Francisco to Fresno.”

Chances are that they’ll all say “TWICE as FAR”

Do you think anyone here thought about miles or kilometres this time? There was no need!

Compared to any of the questions we’ve asked so far, this one is super-easy!

Let’s try another one:

If I asked you which of these is heavier, you wouldn’t hesitate!

And you’d also be able to have a stab at how many times this one is heavier than that one.

All you could do it without having a clue about the weight - in tons - of either of them.

Seems we’re pretty good at estimating the relative sizes of things.

And that’s why it’s not a problem that a Story Point doesn’t have an absolute size.

In practice when we’re estimating using story points, we’re interested in the RELATIVE size of Features.

And it’s the relative sizing that’s behind the choice of the fibonacci series...

But I’m getting well ahead of myself: that’s a tale for another day!

That’s wrap up with a reminder of the three keys to estimating

In summary, three keys to ANY kind of estimating:

- #1 - Get MULTIPLE estimates from different people
- #2 - Use OBJECTIVE measures
- #3 - Use RELATIVE estimates (if possible)

Watch "Agile Estimating & Planning [2018] - 3 Keys to Better Estimates " on YouTube.