Episode 109 - 06 Dec 2017
It’s five weeks since I launched an MVP - a Minimum Viable Product.
It happened in such a rush, that something important got left out. Something that is, arguably, the most important aspect of a Minimum Viable Product:
The Measurement!
Luckily, it’s never too late for a great Experiment. But when I came to think about the measurement side of things, I got stuck:
I’m going to need your help on this one.
So you've launched your MVP Your Minimum Viable Product.
Congratulations!
But did you forget something?
Did you forget that it's an EXPERIMENT?
Did you forget.. about the MEASUREMENT?
If you say last week’s episode, you'll know that I laid out FIVE keys to a successful Minimum Viable Product launch.
No 5 was the big one: Remember that it's an EXPERIMENT.
Experiment means MEASUREMENT.
I launched my Minimum Viable Product about 5 weeks ago.
I confess that I was more concerned with getting it out there than performing a great experiment.
But I believe it’s never too late for a great experiment.
Now that it’s out there, I’d like to improve the measurement side of things.
And I’d like to involve you in the process of deciding what should be measured. If you’d be up for it.
I guess we should start with a look at the system under test.
“Scrum vs Kanban, the mini-course”. In a three-lesson - three-video - mini-course.
The videos are delivered by email - sent on consecutive days.
To get the course, you have to first find your way to the course landing page, AND click the button, AND fill in the form.
That’s the core system.
Upstream, there are a bunch of potential paths to the landing page:
The socials (such as Facebook, LinkedIn and Twitter)
Search engines (such as Google)
YouTube
Downstream, there’s an email that goes out on the fourth day. (I say more about that in a moment.)
The person is then added to my mailing list - that’s the regular Wednesday email.
Think that’s the whole thing.
Plenty of moving parts.
Plenty of things that I could measure.
But what should I measure?
What would you measure?
There’s some low-hanging fruit here:
Conversion rates on the upstream part Watch figures for each of the lessons.
But the downstream area. That where things get a trickier.
That’s where I’m struggling. That where I need your help.
To tee things up, I’d like to play you a clip from episode 85.
It concerns the Lean Startup Build-Measure-Learn loop.
“... the flow here is in a clockwise direction. Of course it is. It would be weird if it wasn’t.
Anyway. It’s a flow of doing. A flow of action. What’s clear if you read the book, is that there’s also an anti-clockwise flow. It’s a flow that you’ll struggle to find in a Google Image search - although I did find one eventually. Check this out: Assumption, Metric, Experiment Notice that it’s a flow of “thinking” rather than a “doing”. A flow that comes - just case it’s not 100% obvious - before the “doing” The hidden anti-clockwise / counter-clockwise flow is, in my opinion, the key that unlocks the power of the Build Measure Learn model.
Thank you Gary. Wise words indeed.
To work out what to measure - the metric - we need to start with the assumption.
In the same episode, I took a stab at it:
“The Mini-course will result in more email opt-ins”
Definately measureable. And easy to measure. Excellnet. Job Done. The End.
Not so fast. There’s a more important assumption that’s hiding between
“The Mini-course will help to grow my audience”
“The Mini-course will result in more email opt-ins”
I don’t just want people to opt-in. I want them to stay opted-in.
Consider these two cases:
Person A that stumbles across dtp… sees a link to a cheat sheet.. Downloads it.
Person B stumbles across dtp… sees a link to the Mini-course.. Signs up. Goes through all three lessons.
QUESTION: When Wednesday rolls around and I send out an email to say “check out today’s episode”
Who is more likely to open the email Who is more likely to click to watch the episode.
And… who is more likely to click the unsubscribe button?
My assumption is that
The assumption that wraps this all together is something like.
“The Mini-course will result in more emails subscribers with high engagement.”
Easy to say.
Only problem is, I’m not sure how to go about measuring it…. is I’m turning it over to you.
How would you test this assumption?
What would you measure? How would you measure it?
Let me know in the comments below.
Quantitative meansure are good.
But qualitative works too. Perhaps a questionaire at the end of lesson three… or delivered by email No. 4.
Again - let me know in the comments below.
Watch "The Lean Startup - MVP without Measurements?" on YouTube.