Episode 92 - 05 Jul 2017
We're reached a very exciting stage in our Lean Startup Journey:
It's time to reveal the RESULTS of our Minimum Viable Product - our Minimum Viable Experiment.
Watch as "Agile Done Right", "Real Life Agile" and "Agile That Pays" go head to head!
And remember to grab a copy of the Lean Startup Cheat Sheet
Previously...
We pulled on to the “Lean Startup” runway, and pressed the starter button on our Minimum Viable EXPERIMENT.
Today...
We’ll look at the results. Will we be CLEAR to PERSEVERE? Or will this PILOT need to PIVOT?
Welcome back to a rather exciting stage in our Lean Startup journey.
Today I’ll reveal the results from our Minimum Viable Product / Minimum Viable Experiment.
To recap very quickly: this whole thing started with a plan to create a 5-day mini-course.
And to do so using principles from The Lean Startup - the book by Eric Ries.
We looked at the Build Measure Learn model, and its hidden counter-clockwise flow:
After some waffling I came up with this assumption:
Which implied this metric:
But what would be a good experiment? What might we use as our Minimum Viable Product?
We looked to Zappos, Dropbox and Buffer for inspiration.
Long story short: Buffer’s “fake door” approach seemed the most applicable, even though it would FAIL to provide the METRIC we were looking for: a direct measurement of email sign-ups.
It would, however give us a direct measurement of INTEREST and DESIRE - necessary precursors to the the ACTION of email sign-ups.
And in the absence of an alternative, we pressed on.
Buffer’s MVP delivers the old one-two by (a) promising something awesome, then (b) apologising that “it’s not quite ready”.
I stole it, painted it yellow and dubbed it the Mundane MVP.
All that remained was a mechanism to get people to the Landing Page. A Facebook advert would seem to fit the bill.
Set things up correctly, and we get two very useful metrics:
I know what you’re thinking: why run one experiment when you could run three? Excellent point.
You were good enough to provide 100+ possible names for the course of some kind, from which we selected a final three.
(Actually four. One didn’t make it past Facebook’s profanity filter.)
Three experiments - differing only in the name of the course.
That’s the theory. Wanna see how it looks in practice?
Here’s the ad in my Facebook News Feed.
Before I click the button, I should warn viewers of a nervous nature should move away from the screen
Ready?
This is the Landing Page. Not sure what was thinking about putting my mug shot in there.
Anywhere, there’s a nice big action button, a few bullet points, and another big button.
Both buttons lead to the “Thank you” Page - which should probably be called the “I’m very sorry page”.
One last thing before I show your the results: it’s all too easy when running an experiment to look at the result and convince yourself that the results confirm... whatever you want them to confirm.
We’re not going to fall into that trap, are we? After all. We have an important decision to make.
We need to decide whether to persevere, or pivot.
So before looking at the results, we should have a feeling for what good and bad looks like, starting with this metric:
According to Wordstream, the average across all industries is just 0.9%.
My targeting was rudimentary:
Not very sophisticated. So I’ll be surprised to get a result above 1%.
What about the second metric:
Well, I’d like to think that anyone who made it this far would be quite likely to click through.
Unless, of course, my ugly mug puts them off.
Perhaps somewhere in the region of 50%?
No more stalling: here are the results.
The adverts for "Agile Done Right", "Real Life Agile" and "Agile That Pays" were displayed:
And were clicked:
Giving us click-through rates of:
A very positive result: all are higher than I was expecting - and "Agile Done Right" did very well indeed.
That’s the Ad performance. What about the Landing Page performance?
Number of visits to the landing pages (for "Agile Done Right", "Real Life Agile" and "Agile That Pays") were:
Those numbers surprised me: I had expected them to agree with the ad click numbers above. A spot of Googling revealed that I should have expected a discrepancy between these numbers. But a discrepancy this large?
As for clicks on the orange buttons. They were:
Giving click-through rates of:
Ouch. Those rates are way lower than I was expecting.
Where does that leave us?
On the positive side, it Looks like there’s a demand for an Agile course - especially for an Agile course entitled “Agile Done Right”. Kudos to James Allen for coming up with the name.
On the negative side, click through numbers on the big orange buttons were way lower than I was expecting.
Was it really my face that made the difference?!?!
What do YOU think?
I’m really interested in your interpretation of these results.
Do they indicate that I should pivot? Or persevere?
Pivot means that we come up with a new idea - and test it with a new Minimum Viable Experiment.
Persevere means that I set to work and create “Agile Done Right. The mini-course” for real.
Let me know your thoughts in the comments below
Watch "The Lean Startup: Minimum Viable... results!" on YouTube.