About lean startup

From Me Pare Whanau
Jump to: navigation, search

overview

  • A systematic process for iterating your web application from Plan A to a plan that works
  • by Ash Maurya
  • US$19 from [Runningleanhq]
  • 278 pages, double spaced
  • for a no-punch pulled, simple outline of the essential steps to Lean, I recommend reading Running Lean [Runningleanhq]
  • the smallest thing you can do to learn from customers is talk to them


Essential points

  • a basic Lean Startup technique is doing the smallest thing possible to learn from customers


  • customers don’t care about your solution - they care about their problems


  • a problem worth solving boils down to three questions
  1. is it something customers want? (must-have)
  2. can it be solved? (feasible)
  3. will they pay for it? If not, who will? (viable)


  • the lean canvas is the tool for gathering evidence that your hypotheses on product/solution fit, and solution/market fit have been qualitatively validated, and quantitatively verified by your customers


  • building and testing a path to customers is not only a slow process, but it is imperative that you start doing it as early as possible


  • in a great market -- a market with lots of real potential customers -- the market pulls the product out of the startup, Marc Andreessen. Pmarca Guide to Startups


  • to maximise experimentation learning, pick bold over incremental changes


  • don’t be a feature pusher and/or feature creep can become an addiction, Ben Yoskovitz


  • simple products are simple to understand


  • don't spend a lot of effort acquiring customers and then just let them walk away, Gary Vaynerchuck


  • waste is any human activity which absorbs resources but creates no value, Womak/Jones Lean Thinking


Some more key points

  • distinguish between customers (those who pay) and users


  • while you might be aiming to build a mainstream product, you need to start with a specific customer in mind


  • a business should be run like an aquarium, where everybody can see what's going on


  • the 3 must-haves in your team are: development, design, marketing


  • building and testing a path to customers is not only a slow process, but it is imperative that you start doing it as early as possible


  • if you get no resistance to your pricing, consider testing a higer price.


  • once you have a MVP built replace your teaser page with a full-fledge marketing website to sell your product


  • actionable metrics are ones that tie specific and repeatable actions to observed results


  • don't expect your users to come to you - the burden of identifying problems and reaching out to your users is yours


  • before selling your MVP to strangers through your website, sell it face-to-face to friendly early adopters


  • if you can’t convert a warm prospect in a 20 min face-to-face interview, it will be much harder to convert a visitor in under 8 seconds on your landing page.


  • usability research shows you can uncover 85% of your product's problems with as few as five testers


  • you can exit the MVP interview process when your early adopters can consistently make it through your acquisition and activation flows eg
    • be able to clearly articulate your UVP
    • be primed to sign-on for your service
    • accept your pricing model
    • make it through your activation flow
    • know what to do next


  • the ideal starting goal is 100 unique visitors to your landing page per day or 5-10 sign-ups a day


  • you don't need a lot of users to support learning - just a few good customers


  • getting to Product/Market Fit is the first significant milestone for a startup


  • surveys are more effective at verification than learning


  • before you launch measure what people say, after you launch measure what people do


  • while continuous development helps your product development process for speed, be wary of simply cranking out more features faster
    • more features dilute your unique value proposition


  • when early users run into problems, they don't turn into testers, they leave


Acronyms and Definitions

  • UVP = unique value proposition
    • single, clear compelling message that states why you are different and worth buying
    • you are making a compelling promise


  • MVP = minimum viable product
    • should be like a great reduction sauce - concentrated, intense and flavoursome
    • delivering on the UVP promise


  • AAARR = pirate startup up metrics - acquisition, activation, retention, revenue, referral


  • Five Whys = for root cause analysis


  • 3A's of metrics = actionable, accessible, and auditable


  • a cohort is a group of people who share a common characteristic or experience within a defined period


  • product/market fit means being in a good market with a product that can satisfy that market, by Marc Andreessen


  • CHI - customer happiness index - use a formula to grade activity on a scale of 1-100, that is calculated using frequency, breadth, and depth of feature usage


  • Kanban Board - is a scheduling system that tells you what to produce, when to produce, and how much to produce


  • MMF - minimal marketable feature is the smallest portion of work that provides value to customers
  • if its too samll to announce to your customers its a bug fix


Roadmap

Useful points

  • Decouple the problem from the solution, and test both through customer interviews
  • Customer interviews allow you to derive the minimum feature set to address the right set of problems
  • This is the Minimum Viable Product (MVP)
  • Pivot experimentation allows you to correct your direction
  • Pivot experimentation allows you to validate the business model and find a plan that works
  • To maximise experimentation learning, pick bold over incremental changes


Part 1: problem/solution fit

The Lean Canvas

  • distinguish between customers and users
  • split broad customer segments into smaller ones
  • sketch a lean canvas for each customer segment
  • sketch them quickly - its ok to leave sections blank
  • sections of a learn canvas
  1. customer segments
  2. problems - top 3
  3. unique value proposition - single, clear compelling message that states why you are different and worth buying
  4. solutions - top 3
  5. channels - paths to customers
  6. cost structure - customer acquisition costs, distribution costs, hosting, people etc
  7. revenue streams - revenue model, life time value, revenue, gross margin
  8. unfair advantage - can't be easily copied or bought


The problem interview

  • aim for 10 to 15 people a week, sample of 40-60
  • identifying customer segments ie who has the pain?
  • what are you solving?
  • how do they solve these problems today?
  • prefer face-to-face; pick a neutral location; ask for sufficient time; stick to a script; document immediately afterwards


Useful points

  • customer discovery is about exploring what you don’t know you don’t know
  • the customer interview is a form of qualitative validation that is quite effective in uncovering strong signals for or against hypotheses using a “reasonably” small sample size.
  • prepare yourself to interview 30-60 people over a 4-6 week period which comes to talking to 2-3 customers a day with some time built in for iteration
  • set an expectation of learning over pitching
  • the Problem/Interview script outline is good
    • welcome - set the stage, 2mins
    • collect demographics - test customer segment, 2mins
    • tell a story - set problem context, 2mins
    • problem ranking - test problem, 4mins
    • explore customer's worldview - test problem, 15mins
    • wrapping up - the ask, 2mins
    • document results, 5mins


  • an early adopter is a customer who ranks one or more of the problems you’re solving as a must-have and will generally pay to have it solved
  • refine the problems as you go - drop problems with a strong 'don't need' signal, add new 'must-haves'
  • understand your early adopters existing alternatives is key to formulating the right product
  • pay attention to words customers use to uncover the 'key' words in your UVP


The solution interview

you have completed this stage when you can confidently

  • identify the demographics of an early adopter
  • have a must-have problem
  • can define the minimum features needed to solve this problem, and
  • have a price the customer is willing to pay
  • that you can build a business around (using a back-of-the-envelope calculation)


use your results from the problem interview to learn:

  • who are your early adopters - customers segments
  • what is the problem you are solving
  • how will you solve these problems
    • what is the minimum feature set needed to launch


  • what is the pricing model
    • will they pay
    • what price will they bear


  • build a mockup/demo that
    • looks real
    • is quick to iterate
    • can be re-used in the solution
    • uses real-looking data


  • an outline of the Soluton Interview script is
    • welcome - set the stage, 2mins
    • collect demographics - test customer segment, 2mins
    • tell a story - set problem context, 2mins
    • demo - test solution, 15mins
    • test pricing - revenue stream, 3mins
    • wrapping up - the ask, 2mins
    • document results, 5mins


Part 2: product/launch fit

Get to release 1.0

  • typical release cycle: requirements - development - QA - release
  • there is very little learning from customers that happens during development and QA
  • reduce MVP scope and shorten the cycle between requirements and release so that you get to learning parts faster by:
    • start with a clear slate and justify the addition of each feature
    • start with your number 1 problem
    • immediately eliminate don't-needs, and add nice to haves to your features backlog queue
    • consider other customer feature requests to make your product complete or usable
    • charge from day 1, but collect on day 30
    • focus on learning, not optimisation


Pirate Startup Metrics - AARRR

  • For launch focus on the first 2 metrics
  1. Acquisition - How do users find you?
  2. Activation - Do users have a great first experience?
  3. Retention - Do users come back?
  4. Revenue - How do you make money?
  5. Referral - Do users tell others


  • to reduce the cycle time to release
    • reduce the number of steps but not at the expense of learning
    • deliver on your unique value proposition
    • be prepared for when things go wrong


  • activation flow
    • how do i get started?
    • what are the additional steps required?
    • did they get to the UVP?


  • most of what goes wrong right after you launch happens here
  • structure your activation flow for learning over optimisation
  • reduce steps but not at the expense of learning


Get started deploying continuously

  • releasing software continuously throughout the day in minutes versus days, weeks, or months
  • controversial as concerns around quality coding
  • if implemented correctly, it offers strict testing and monitoring barriers to release
  • continuous deployment is itself a feedback loop for continuous learning and improvement
  • the continuous deployment cycle is: test | deploy | monitor | commit


  • commit - reduces waste by reducing work-in-progress undeployed code
    • code in smaller batches
    • always be trunk stable


  • test - eliminates manual testing QA
    • testing is everyone's responsibility
    • use a continuous integration server
    • do not tolerate any failing tests
    • prefer functional tests over unit tests
    • start with your activation flow


  • deploy - pushes your tested code into your production environment
    • outsource as much of your server infrastructure as possible
    • create a separate staging area if you are so inclined
    • one-click push and rollback
    • deploy manually first, then automate
    • implement a simple feature flipper system


  • monitor - to automatically detect, alert, and recover from unexpected errors
    • start with off-the shelf monitoring
    • tolerate unexpected problems only once


Get ready to sell

  • a great product eventually sells itself
  • beforehand, use a product website containing:
  1. your landing page
  2. your pricing page
  3. your tour page


  • replace your teaser page with a full-fledge marketing website to sell your product
  • the acquisition funnel includes
    • UVP - why should i care?
    • other pages - tell me more
    • pricing - how much does it cost?
    • click sign-up - how do I get started?


  • each page should have
    • a primary call to action
    • a secondary call to action


  • the landing page is by far the hardest of the 3
  • you have 8 seconds to engage the visitor


About the Landing Page

  • provide a compelling reason to buy your product
  • the essential elements are
    • UVP - the unique value proposition
    • supporting visual - a visual aid that resonates strongly with your target audience
    • a clear call to action - it should stand out
    • an invitation to learn more - including an 0800 number
    • social proof - customer testimonials, as seen on


  • make feedback easy
    • it shows you care
    • you don't have a scaling problem yet - you won't be bombarded by phone calls
    • tech support is a continual learning feedback loop
    • tech support is customer development
    • tech support is marketing
    • provide multiple feedback channels


  • the about page - provide a compelling reason to buy from your company
  • the terms of service and privacy policy - see existing examples on web
  • tour page - video/screenshots - if your customers expect more details, technical specs etc


Get ready to measure your user lifecycle

  • the objective before product/market fit is all about quickly identifying and troubleshooting hot spots in your user lifecycle
  • remember, metrics are people too ie
    • metrics can't explain themselves - you need to talk to people for that
    • don't expect your users to come to you - the burden of identifying problems and reaching out to your users is yours
    • not all metrics are equal


Funnel reporting and cohort analysis

  • the funnel report is a powerful analysis tool but raises issues for lifecycle usage in:
    • inaccurate conversion rates
    • dealing with traffic fluctuations
    • relating results to changes


  • cohort analysis
    • to overcome limitations with funnel reporting measure cohort events


How to build your conversion dashboard

  • a key design principle is decoupling data collection from data visualisation
  • this allows you to build your conversion dashboard incrementally


How to collect data

  • map metrics to events
    • acquisition = viewed home page + viewed pricing page + viewed sign-up page
    • activation = viewed sign-up page + step 1 + step 2 + step 3 + did key activity


  • map key events to other macro metrics
    • retention = logged in + did key activity
    • revenue = upgraded account
    • referral = invited collaborator


  • track raw events
  • log everything potentially interesting eg OS, browser type, time of day


How to visualise your conversion dashboard

  • build a weekly cohort report
  • be able to drill into your sub-funnels
  • be able to go behind the numbers - to get to a users that created them


The MVP interview

  • before selling your MVP to strangers through your website, sell it face-to-face to friendly early adopters
  • learn from them
  • refine your design, positioning, and pricing for launch
  • the MVP interview is less about pitching and more about learning
  • consider using screen recording software to record the testing session for others to watch later eg camtasia


  • what you need to learn
    • UVP positioning. Does it provide a compelling reason to buy. Does your landing page resonate with your early adopters?
    • Revenue streams. Is the pricing model right?
    • Activation flow. Do customers make it all the way through your activation flow? What the usability hot spots? Does your MVP demonstrate and deliver on your UVP?


  • generate hypothesis for each of the above to be tested in tme MVP interview
    • early adopter will be able to articulate UVP after 5 seconds test.
    • early adopter will be able to navigate to pricing page


  • the Solution/Interview script outline is good
    • welcome - set the stage, 2mins
    • show landing page - test unique value proposition, 2mins
    • sign-up and first user experience - test solution, 15mins
    • show pricing page - test pricing, 4mins
    • wrapping up - the ask, 2mins
    • document results, 5mins


Are you ready to launch

  • review your results every 5 interviews
    • usability research shows you can uncover 85% of your product's problems with as few as five testers


  • start with the most critical problems


  • do the smallest thing possible
  • resist the tempation of completely redesigning a new landing page


  • make sure things improve
  • validate that your fixes improve things in subsequent interviews


  • audit your conversion dashboard
    • make sure everything is working


  • what is the MVP interview exit criteria
    • you are done when your early adopters can consistently make it through your acquisition and activation flows


Part 3: product/market fit

  • getting to Product/Market Fit is the first significant milestone for a startup
  • this is when you start validating your complete product
  • step 1 define a metric to measure Product/Market fit
  • then systematically iterate towards achieving it
  • you can always feel when product/market fit isn't happening
    • customers aren't quite getting value out of the product
    • word of mouth isn't spreading
    • usage isn't growing that fast
    • sales cycle takes too long
    • deals never close


The Sean Ellis Test

  • to determine if a product has achieved product/market fit, ask a sample of customers:
  • how would you feel if you could no longer use [product]
    • very disappointed
    • somewhat disappointed
    • not disappointed
    • n/a - I no longer use the product
  • if over 40% of your users are saying that they would be 'very disappointed' without your product, there is a great change you can build sustainable, scalable customer acquisition growth on this 'must have' product


Measuring Product/Market fit

  • this can be reduced to building something people want
  • the most indicative measure is retention ie repeated use of your product
  • based on Sean Ellis test, the threshold is 40% retention
  • while revenue is the first form of validation, retention is the ultimate form of validation


  • focusing on building a scalable business model at this stage is a form of waste


Complete your conversion dashboard

  • extend your conversion dashboard with retention
  • establish a Customer Happiness Index eg CHI = [(Number of hours logged in)/{Desired number of login hours)*0.2 + (At least on key activity)*0.8]*100
  • define an active user as CHI > 80
  • add active user to your conversion dashboard


Don’t be a feature pusher

  • while continuous development helps your product development process for speed, be wary of simply cranking out more features faster
    • more features dilute your UVP
    • don't give up on your MVP too early - first troubleshoot and resolve issues with existing features
    • features always have hidden costs - start with no
    • you don't really know what customers really want


  • implement the 80/20 rule
    • 20% of effort on implementing new features
    • 80% of effort on existing features


  • constrain your features pipeline
    • limit the number of features that can be concurrently worked on
    • only work on new features after you've validated that the features you just deployed had a positive or negative impact


  • use a Kanban board to track features from backlog to in-progress to done
  • only track Minimal Marketable Features on the Kanban Board


  • a feature is only Done when it provides validated learning from customers


Build your feature backlog

  • if you offer a trial period, as it will be rich with learning from customers, use this to build up your feature backlog
  • your first objective during trials is reducing abandonment on your acquisition and activation paths
  • your next objective is increasing retention and engagement
  • you final objective is getting paid


  • acquisition and activation
  • priority - ensure you are driving enough traffic to support learning
    • explore acquisition and activation sub-funnels to see where users are dropping off
    • reach out to your users who failed a particular step
    • catch and report unexpected errors


  • retention
  • priority - get users to come back and use your product during trial
    • aim at 25% retention rate during a trial
    • consider drip marketing email pre-determined messages to your users during a trial
    • follow up with your interviewees


  • revenue
  • priority - get paid
    • implement payment system
    • track revenue on your conversion dashboard
    • track revenue over time in a detailed way
    • get paying customers to talk to you
    • get lost sales prospects to talk to you


  • process feature requests
    • first question - is it right action, right time? if no ignore
    • is it small feature or a bug? if no defer it - add to kanban board
    • is it an emergency? if yes - fix it; if no defer it - to task board


The Feature Lifecycle

Have you built something people want

  • review your conversion dashboard results weekly
  • prioritise your goals and features backlog
  • formulate bold hypotheses
  • add/kill features
  • monitor retention
  • run the Sean Ellis test