In our last post we looked at various management approaches to reduce risks. One of them pertained to business experiments. In this post, we’ll dig a bit deeper to better understand this approach.
You have most likely heard of business experiments. The Harvard Business Review, as well as many other business management publications, published multiple articles on the topic in the last few years. They were mostly in the context of innovation.
What is a business experiment?
If you have not, a business experiment is simply testing a concept (idea, program, process, design, product, strategy, etc.) with stakeholders (customers, suppliers, distributors, employees, etc.). The experiment’s goal is to provide pertinent data to assist with decision making (usually with go/no go decisions or sometimes fine-tuning).
So yes, you do know what a business experiment is. It’s been around for, well, pretty much ever. Now you’re asking yourself; Why this article is drawing my attention to it?
The reason is that business experiments are no longer used as an occasional management tool, they are transforming into the way businesses are managed.
Why are companies experimenting at an increasing rate?
As we mentioned, business experiments are used to collect data that assists management with decision-making.
Decision-making in the business world is becoming a more demanding task (to say the least).
- Companies are significantly leaner (i.e. less resources and more work for everyone, including managers).
- The amount of information available, from secondary sources, is enough to make any manager’s head spin.
- There is a higher supply than demand for managers hence making bad decisions is a riskier proposition for your longevity in a given company than ever.
- In many sectors, markets are moving at a faster pace than ever before with competitors coming in from everywhere and customers’ choices exploding.
Hence managers are turning to ways to reduce the time and the risk involved to make the best possible decisions.
Running a business experiment to gather pertinent and timely data quickly answers the needs of over-burdened managers to reduce the risks of the decisions they make.
This explains, in part, why organisations are experimenting. What explains the increasing rate at which they are doing it, and transforming it into a management approach, has more to do with the following reasons:
- It’s cheaper and faster than ever to run business experiments (many can be done in minutes with free tools)
- An increasing number of employees or outside consultants have the skills to run experiments, analyse and interpret the collected data
- The costs to store, process and communicate the data/result are very low (compared to a decade ago) and falling constantly.
Business experiments and the Lean Startup framework
So how does an organisation go about implementing business experiments as a managerial approach? There is no one way to do so. There are however experimentation frameworks out there that can structure your approach.
One of these is the Lean Startup framework.
The base of Lean Startup is the experiment. The Lean Startup experiment is based on the scientific experiment model. It follows a build, measure, learn process.
Build (designing your experiment)
Building your experiment is a 3 step process
1. Identify the critical assumption associated with the concept to be tested
When you stop to think about it, there are thousands of assumptions we could test to assist with decision making when managing a business. If we tested all of our assumptions there would be no time or other resources left to run the business. Hence, you only want to test the critical assumptions. The ones that, if not validated, pose a business risk that you (or your organisation) are not willing to take.
2. Transform your critical assumption into a hypothesis statement
Your critical assumption was a thought you put into words. Your hypothesis statement is one sentence that can be validated (or invalidated).
3. Design the experiment that will validate (or invalidate) your hypothesis
Designing your experiment is only limited by your (and/or your team’s) imagination. The way you choose to render the concept of your hypothesis (how you present it or illustrate it) is called your Minimum Viable Product (MVP).
An important part of the MVP is the minimum part of it. In the design of your experiment, you will be aiming at spending the least amount of resources in order to obtain the maximum amount of learnings from your experiment.
This doesn’t necessarily mean bootstrapping your experiment. It simply means that you will make sure that whatever learnings you need to get from your experiment, you get by using the least amount of resources possible.
This can mean it will cost you nothing but a few minutes of the time of one person or tens of thousands of dollars, if that’s the only way to go about getting the information you need. Of course, the cost of your experiment must be proportionate to the financial risk associated with it.
The reason you are making the experiment is to get data that will help you with your decision making. Hence, you want that data to be reliable.
This is where you need a bit of knowledge about primary data gathering. You need to make sure the data you collect is not biased. You won’t be looking to get statistical quality data. That would take too long, cost too much and be an over kill for your purpose.
You’ll simply need the clear direction that your data in headed for.
In business experimentation instead of gathering a lot of data once, you gather a small amount of data repetitively. Although not as precise as statistical data it does provide you with a sufficient amount of information to de-risk, to a great extent, your decision. It is also a more suited approach to an environment that changes rapidly.
When deciding on the data you will be capturing in your experiment, remember these three important rules.
Your data should be:
If it doesn’t go in the direction you thought it would then you can change something in your business strategy or product or program that will make the data go in the direction you want.
You need to be able to re-produce the exact same experiment, in similar conditions and get similar results. For example, if you are testing the design of a snow shovel and are doing it during the storm of the century, the data you collect won’t be accountable.
The more people who interpret the data you collect, the deeper and richer your learnings will be. So make sure that the data captured during your business experiments is shared throughout your organisation. Ensure that everyone knows they are welcome to share their interpretations on the data. Making your experiments, the data collected and the results accessible throughout your organisation will also accelerate business experiments process as some parts/resources of one experiment can be used for others.
The more experience one has at conducting experiments, the faster and more accurately they are done.
This part of business experiments, although it may seem like the easiest, is the hardest. Learning means you are either
- absorbing completely new information (rarely the case) or
- you are changing in some way (sometimes drastically) already stored information in your brain
The second type of learning is the hardest. The more contradictory the data you have is to the one you previously had, the more difficult it is to learn from it.
This stored information in your brain will create a filter that will impact how you interpret new data.
Also, the more resources that has already been invested in a project, the more difficult it is to pivot on a previous course of action.
The benefits of using business experiments to manage
The main benefit, as mentioned previously, is to reduce the risk of your business decisions. This in turn will minimise your losses on various projects.
Another benefit, that is not obvious, is the improvement in work relationships. This happens for multiple reasons.
First, employees get a sense of empowerment. If they submit ideas to upper management with supporting empirical data, they know their idea will be considered.
Also, managers don’t need to spend as much time justifying their decisions. They let the data speak for itself. They do however need to include questioning the quality of the data into their process.
Finally, as mentioned, organisations that use business experiments on a daily basis to manage usually encourage employees to share the results of their experiments. This not only improves internal communication and efficiency it also creates an environment where mistakes, that bring new learnings, are valued.
This type of environment is essential to not only foster innovation but pro-activity.
Getting started with business experiments
If you think that using business experiments may be a profitable management approach for your business, start with one project (which will be a meta experiment) during which all higher risk decisions will be taken with supporting data. The project you select should be one that has high inherent risk within it (like launching a product in a new and different market). It should also be a type of project that is somewhat recurrent in your organisation. This will help you have a baseline scenario in order to compare the results of the meta experiment. Make sure to identify the metrics you’ll be evaluating before starting your meta experiment.
Some metrics (there are many others) you may want to look at would be:
- How long the project took from start to finish
- Overall budget
- What % of the initial planed output was achieved
- Variables pertaining to team cohesion
- Variables pertaining to employee (the ones who worked on the project directly and indirectly) satisfaction
- ROI projected vs achieved on a timeline (3, 6, 12, 24 months)
Implementing any new management approach takes a while. There is no one-size-fits-all recipe. You need to…yes you got it…experiment and find the approach best suited for your organisation.
If you need coaching or help getting started with your first experiment Baker Marketing can definitely help.
 Although MVP most often refers to a prototype of a product, it also means the representation of your hypothesis you will present to participants of your experience.
You may have already heard about Lean Enterprise but, because it is a relatively new concept, you most likely haven’t yet.
What is Lean Enterprise?
In short, Lean Enterprise is the Lean Startup approach adapted to large organisations. It has already been adopted by many large organisations such as GE, Toyota, the White House (about 4 years ago), NPOs and many startups that became large such as Zappos.
How can Lean Enterprise be used?
Lean Enterprise can be used as the guide to an entirely new innovation program. However, since most large organisation already have an innovation program, Lean Enterprise can simply build on the existing program, improve it, make it more efficient and, over time, permeate the entire organisation with a culture of innovation.
Explaining Lean Enterprise
Explaining Lean Enterprise could be a very long process given that it touches all aspects of business and will take a very distinct flavor in each or the organisation that adopts it.
When entrepreneurs or intrapreneurs (folks responsible to make innovation happen in large organisations) want to explain their projects to me, I usually ask them to describe their business model.
Hence, I’m thinking that describing Lean Enterprise’s business model is will be a good way to explain this approach to manage innovation in large organisations succinctly.
Understanding Lean Startup first
If you are unfamiliar with the Lean Startup approach then I suggest you brush up on it. Lean Enterprise requires a very good understanding of the underlying principles as well as the capability to run Lean Startup-type experiences. There is an abundance of Lean Startup documentation online including the many Lean Startup related posts on this blog. Here is a sample of them that you can check out as a primer.
The Lean Enterprise Business Model
The image below (click to make it larger) is Baker Marketing’s version of what Lean Enterprise’s business model would look like. This version would not make for a good investors’ pitch but it’s also not its purpose. Hopefully, this business model canvas will help you to understand, at a glance, what Lean Enterprise is all about.
The Lean Enterprise Business Model Canvas
Click for full view
Although this canvas is pretty straight forward, I will detail the Customer Segments and Value Proposition sections of the Lean Enterprise business model for better comprehension.
Targeted Customer Segments
The Lean Enterprise innovation approach can be used is just about any large organisation that needs to innovate. The more disruptive the innovations it produces, the more gains it will get from the Lean Enterprise approach.
For profit organisations
This segment has the most straight forward application of Lean Enterprise for innovation purposes.
There are already dozens of large organisations incorporating Lean Enterprise in their existing innovation programs to make them more efficient. It is the case of such companies as Google, GE, Intuit, Toyota, Adobe, etc.
Non profit organisations
NPOs around the world all have a common pain; they are struggling to get enough resources to achieve their goals.
In many cases they are also faced with the task to innovate in order to keep being relevant to both their users and benefactors. Lean Enterprise helps them by ensuring that all the resources they do have are used as efficiently as possible when they innovate.
Lean Enterprise also helps large NPOs develop a more innovation-friendly culture.
Public and para-public organisations
Given that efficient use of resources is a basic principle in the application of Lean Enterprise, most public organisations could benefit from its implementation greatly. Government institutions are also in dire need of catching up to economies changing at the fastest pace ever in history.
The White House initiated the Healthcare.gov portal re-design in 2011 with a Lean Startup (not yet known as Lean Enterprise) team. The embryonic project was then taken over by CGI. When CGI was unable to deliver on time and on budget, the project was reverted to the Lean Startup team. The results they achieved were so spectacular that the White House eventually adopted Lean Enterprise for all of their innovative projects. The approach also spread to other American government agencies and departments.
Aneesh Chopra, who was appointed CTO of the United States by Obama in 2009 (until 2012) was one of the catalysts in propagating the Lean Enterprise approach through the US government. 
This is, in part, what a large organisation can expect to obtain with the implementation of a Lean Enterprise approach to innovation that their current program may not be bringing them.
Help innovate more efficiently
The efficiency is obtained in large part with the fact that an innovation project that is completed, within a Lean Enterprise-based innovation program, will necessarily answer the needs of its target customers/users. Hence a product/market fit or a service/user fit will be achieved every time.
Efficient use of resources (especially human) being an underlying principle of Lean Enterprise, it is therefore a constant preoccupation of the project participants.
Use existing resources more intensively
If you work in a large organisation, I am certain you are aware of either unused resources or resources not fully used that can be put to contribution in innovation projects. If not, I suggest you ask and snoop around. You will find some in no time. The infusion of entrepreneurial attitude, brought by Lean Enterprise, makes using these resources second nature.
Build on existing innovation programs/practices
Lean Enterprise is based on principles, tools and techniques borrowed from many existing and proven management theories such as:
- Lean production
- Customer development
- Agile development
- Design Thinking
Most innovation programs already make use of the principles, tools or techniques of some of these theories. Lean Startup implementation can ‘’surf’’ on these existing programs in an organisation and add to them. This greatly flattens the learning curve and is more easily adopted.
Implementing a Lean Enterprise approach to innovation is like any other change management program. It must be done incrementally, ensuring all are onboard.
Increase speed to market of innovations
Lean Enterprise is based on rapid iteration testing of the various parts of a business model until you reach a product/market fit or service/user fit.
The approach has a built-in control mechanism to ensure innovation teams don’t go astray, lose focus or momentum.
Pivot quickly on ideas with no positive ROI
All of the products or services that get to market with a Lean Enterprise approach are successful. This, however, doesn’t mean that these successful products or services look anything like what was imagined initially.
Lean Enterprise does not prevent or fix product or service ideas that would flop in the market. It does however quickly show which ideas need to be discarded thus avoiding the waste of resources to bringing them to market. Luckily, most ideas simply need a few pivots (okay many pivots) to achieve success with their markets.
Better manage innovation financing risks
The use of innovation accounting in Lean Enterprise innovation programs allows for incremental financing with a known risk coefficient of each of the projects increments.
It bridges the gap between corporate innovators and financial managers.
You can read more about innovation accounting in these previous posts.
Creates innovation culture in the long run
The real prize, at the end of the journey of implementing Lean Enterprise, is the creation of a true culture of innovation.
Some of the most important barriers today in achieving a true organisation-wide culture of innovation are the following:
- Mistakes are neither welcome nor tolerated
- Resources (human, financial, physical, etc.) are kept in silos
- Decision making power is diluted (with a heavy weight at the top of the pyramid)
- Over-abundant and rigid processes that result in slow reactions to market changes
The Lean Enterprise approach to innovation, intrinsically, removes those barriers.
As any other change management program, Lean Enterprise takes time, effort and commitment. Commitment from the innovation project champions but also from top management. Although still in its infancy, the Lean Enterprise approach shows promise of integrating innovation into large organisation’s main stream of business instead of treating it like a special cousin, as it is now.
If you want to learn more about Lean Enterprise and its application, you can pre-order Eric Ries’ new book due out in the fall, called TheStartup Way.
If you have a Lean Startup Circle community in your area, you can contact the organisers to locate experienced coaches or attend their meetups and see how entrepreneurs and intrapreneurs can help each other innovate more efficiently.
 This hour-long talk with Aneesh Chopra explains how Lean Enterprise came to be in the US governement
 Characterised by a sharp increase in the sales or adoption growth for a sustained period.
This will be the second to last post of a series of how to apply the Lean Startup approach to a new business.
Until now in this series on applying Lean Startup, we started with an introduction, then looked at the ideation phase, the discovery phase and the pre-sell phase (also known as the Death Valley).
Either because you have made it this far in reading this series or, even more important, you have successfully crossed the Death Valley (pre-sell phase) and came out of it with the holy grail of a product/market fit (when the hockey blade becomes the stick on your revenue chart).
You now find yourself in the concierge phase.
What is the concierge phase
The boxes below presents a very high level summary of usual situations your start-up can expect in the concierge phase.
- Your core features all work pretty well
- You created your first (official) road map to additional features
- If you are outsourcing, you are either looking at optimizing your suppliers or taking steps to bring production in-house
- If you are manufacturing in–house, you are looking at getting decent equipment to start producing at a larger scale
- You are looking at more efficient tools to learn about your customers, markets, and environments
- You are exploring new customer segments
- You are aggressively growing your initial markets
- You are constantly reassessing the total size of your markets
- If you are an innovator in your market; you are keeping an eye out for the chasm (1)
 The saturation of the early adopters market and passage to the early majority (re. The innovation adoption curve)
- You are feeling the need to put processes down on paper so your teams has a more homogenous approach
- You realise you need a lot of processes and procedures but don’t want to bog down your agility
- You are on boarding team members at a rapid rate
- Job definitions are getting more specialised
- Your core team is trying to find a fit with their new, more limited, roles in the company (spoiler alert – some won’t adjust)
- Your core team feels as though they spend more time coaching new resources than getting work done
- Money is coming in at a decent rate from sales
- Labour costs need to be controlled as they are growing faster than your sales at times
- Extra office space and equipment mean increasing your bank margin or taking out (mostly) short term loans
- Investors are now calling you and want to hear about your scaling strategy
Too busy for Lean Startup
You are now running a small business that is experiencing the fastest growth rate it ever will, short of an acquisition.
It is easy and oh so tempting to abandon the build, measure, learn approach. After all, you know your initial market’s needs very well by now and you are crazy busy fulfilling orders, fixing issues and well, running a company.
It is a trap most entrepreneurs will fall into. Until their growth rate slows down, stalls and starts to plummet. The dirty secret of the concierge phase is that most companies’ revenues during this period don’t look like a straight hockey stick handle. That straight upward slope is just the trend of your revenues. The revenues themselves go up and down regularly. If you want your slope average to be positive and steep, you need to minimise those downs. Most times these down periods will happen for the following reasons:
- Your customer needs are changing due to a shift in the market (often due to a new competitor)
- You experience process or production failures
- Your initial early adopters market is getting saturated and you didn’t react quickly enough to open new markets
- Your early adopters markets are saturated and you haven’t figured out how to sell to the early majority customers.
Continuing to apply the Lean Startup approach during your concierge phase will ensure that any new features or internal processes will answer the needs of your customers (external and internal). It will also ensure that market changes are captured and acted upon. This does mean that many of your processes must incoporate Lean Startup elements in them. In some cases, it can also mean that the you will need the build, measure and learn processes themselves to be written down and into job descriptions.
Incorporating Lean Startup in your processes is the key to keeping your company innovative and agile as it grows.
Your product, processes, marketing and overall strategy will adapt continuously. When you need to cross the chasm to reach your early adopters, the Lean Startup approach will be your natural bridge to the other side.
In our next and final post of this series, we’ll take a look at the tools that are most useful in the concierge phase.
 The saturation of the early adopters market and passage to the early majority (re. The innovation adoption curve)
It’s the end of the year already. It went by incredibly fast. This is the time to look back and identify what needs to be fixed. It’s also the time to be grateful for all that we were able to achieve.
One of the things I am always very grateful for is the knowledge I gain during the year. Important sources for the knowledge I pick up are the books and articles I read.
I have to admit that I didn’t have as much time to read this year as I got involved in many (maybe too many) projects. I did however manage to read some very good books on marketing, innovation and Lean Startup.
As I did last year, I am sharing with you some of the ones I especially liked.
Best Reads on Marketing
I didn’t keep up with all that is new in marketing this year. It’s nearly impossible to do so. I tuned in to a few webinars that helped me focus my readings.
I read mostly on mobile marketing (various aspects), influencer as well as community marketing. Community marketing is a strategy that isn’t as easy to implement as one might think. Here is a post I wrote on how to find a profitable community marketing partner.
Again this year, I found that my most interesting marketing reads came from blog posts on mainly two sites; eMarketer and HubSpot
Speaking of HubSpot it is one of the case study in Sean Ellis’s book; Growth Engines: Case Studies of How today’s Most Successful Startups Unlock Extraordinary Growth.
Through ten case studies, including Yelp, Uber, LinkedIn and HubSpot, Ellis explains the different types of growth engines and the contexts in which they worked best for those companies.
Although I read Growth Engines to better understand a concept that is integral to Lean Startup, this book offers some very valuable marketing lessons. It also touches on growth hacking, a term coined by Ellis. It inspired me to write a post on what growth hacking is and isn’t.
Another book that I thoroughly enjoyed was UX Strategy by Jamie Levy. Having been a Product Manager at a time where UX was but one part of the job description, it was great to delve into the depths of UX strategy.
Whether you are starting a new venture to create the killer app, or trying to innovate in an existing small, medium or large business, this book is a must read before you start. It can help you define a winning value proposition. It also guides yourr competitive analysis and helps you see which features you need to focus on.
The book is an easy read and doesn’t require any prior knowledge on UX design or app development.
Best Reads on Innovation
I was invited to a university workshop on blockchain earlier this year. Given I knew nothing on the topic I figured that it would be a great opportunity to learn. Montreal, where Baker Marketing is located, is a hotbed for blockchain research and development. The workshop did teach me the basics of blockchain but left me wanting to know more (a lot more, this is exciting stuff and definitely a game changer) about how this new technology could be used.
One of the researcher at the workshop suggested Blockchain Revolution: How the Technology behind Bitcoin is Changing Money, Business and the World, by Don and Alex Tapscott. It was exactly what this non scientific reader needed. The Tapscotts explain the concept in very simple terms. They also explore a large number of applications for blockchain. They clearly show how significant a game changer this technology could be.
In Spring I also contributed to the organisation of the Montreal edition of the Intrapreneurship Conference.
During the conference one of the keynote speakers was Guillaume Hervé. Hervé is a veteran practitioner of intrapreneurship. He contributed to several corporate spinoffs in the aeronautics and health sectors. In case you are not familiar with the term, intrapreneurship is entrepreneurship adapted to large enterprise.
Intrapreneurship is however not the same as entrepreneurship. These differences are the focus of Hervé’s book Winning at Intrapreneurship: 12 Labors to Overcome Corporate Culture and Achieve Startup Success.
Based on the 12 labors of Hercules, Winning at Intrapreneurship looks at the traps, pitfalls and myths of innovating in large businesses. Hervé saw them all in his career as an intrapreneur. He shares with us some tricks of the trade on how to avoid and debunk them. You can read more on this topic on the post I wrote titled Entrepreneurs as Corporate Innovators.
Best reads on Lean Startup
Continuing on the innovation in large business topic, Eric Ries published a second book this year. It’s titled the Leader’s Guide to Adopting Lean Startup at Scale.
First, Eric innovated in the way he published the book. He financed the book with a Kickstarter campaign. The backers were invited to join the Leader’s Guide community (managed by Mightybell). He used the community to test hypotheses about the content and cover of his book. Yep, he did it the Lean Startup way.
Unfortunately however, Eric only printed as many books as there were backers who pledged the sufficient amount. It isn’t available anywhere for purchase now that the Kickstarter campaign is over. You can however get a free digital copy if you know someone who invested in the campaign.
The Leader’s Guide is based on Eric’s experience (as well as that of some backers) on implementing Lean Startup in large corporations, like GE, and government organisations (like the White House).
I especially like the format of the book. Symbols are used in the margins throughout the chapters in order to quickly understand what the text pertains to. The coach’s Guide sections, for example, are about tips and subtleties in implementing the concepts.
It’s truly a guide that you will go to when implementing a Lean Startup approach to a large organisation.
Eric also announced that he will be publishing a third book next year. It’s tentative title is The Startup Way.
We were lucky to have another great Lean Startup practitioner write his second book this year. Ash Maurya penned Scaling Lean: Mastering the Key Metrics for Startup Growth.
As a follow up to his first book, Running Lean, Ash is now looking at how to use metrics to scale your business once you have found the elusive product/market fit.
His rigorous approach to using key metrics to track your progress and focus your efforts has shown great results in many successful startups.
Finally, I want to mention a website whose author consistently publishes great Lean Startup material. Tristan Kromer’s Grasshopper Herder is chalk full of Lean Startup ideas, tools and resources. Tristan was until recently one of the organisers of the San Francisco Lean Startup Circle.
If you are starting a new project and interested in putting Lean Startup into practice take a look at the series of posts on implementing Lean Startup. It is meant to guide you along your journey when you first start your project.
This concludes this year’s crop of my best reads on marketing, innovation and Lean Startup. Maybe some of them will become your favorites.
Thank you for taking the time to read Techno Marketing this year. I hope you take some time off during the holidays to rest and replenish, as we will.
Baker Marketing offers you its best wishes for the holidays. May 2017 be filled with health, serenity and lots of successful projects.
In this series of posts we have looked at how to use Lean Startup during the ideation and discovery phases. We also dove into the experimentation process and the tools to use in the discovery phase.
In this post, we’ll look at how to use Lean Startup in your pre-sell phase.
What is the pre-sell phase?
The pre-sell phase begins when you have sufficiently tested your leap-of-faith assumptions of your business model to be confident that you have a shot at a viable business. This not because you believe so (as you did in the ideation phase) but because most (or all) of your business’s stakeholders (potential customers, suppliers, distributors, etc.) have told you so.
Typically, during the pre-sell phase you will be:
- Finding a (or several) co-founder(s)
- On-boarding your first employees
- Developing your product/service
- Starting to put into place some key partnerships
This is also the phase at which you will start getting real feedback from paying customers (closer to the end of the pre-sell phase) and other stakeholders. It’s the reality test.
A test, unfortunately, many startup ideas fail. It’s when you will either:
- Make significant pivots,
- Change your mission entirely (go back to square one)
- Throw in the towel, or
- Confirm your business model
The end of this stage, when successful, is also often the time to go for seed funding.
Why continue experimenting?
So now that you have confirmed your business model, why should you continue experimenting? It would be so much faster to just go full steam ahead and develop.
Your ideation and discovery phases were mostly planning. As you know, plans usually don’t work out the way we think they will.
Also, during the time you take to put in place your business model, elements of your business’s eco-system as well as market realities will change. The higher the volatility of your business environment, the more important it is for you to test the elements of your business model while developing. Otherwise you may expand great efforts and resources to develop key partnerships, distribution networks, and develop your product/services for nothing.
This may not seem so bad from this end of the journey. Entrepreneurs who put every ounce of energy they had and every penny they owned (and some they don’t), for a few years, in a business that doesn’t lift off will tell you it’s as difficult as going through a death in the family. Some never fully recover.
Lean Startup experiments in the pre-sell stage
The mechanics of the experiments we saw in the discovery stage will be the same.
You will still start by asking yourself; Will the negative impact on my future business be significant if the assumption I am making (regarding whatever part of your startup you are currently working on) isn’t real? If the answer is yes, you turn it into a hypothesis you can test. You design your experiment using tools to minimize your efforts, chose your metric and standard and execute your experiment.
By this time, you will be getting very good at experimenting. Hence most of your experiments will take you less than a couple of days to run and often only a few hours or minutes.
Also, because you will have gotten out of the building often during your discovery phase experiments, you won’t need to talk to your stakeholder face to face as much. Many of your experiments will be done on the web or over the phone. You can also hire people to do them for you.
There will be times however when you aren’t sure of your next steps. This is usually the flag that means you need to go be around your potential customers. Talking or observing them. They will invariably show you the way.
At this stage, you will not only be testing elements of your product/service you will also be testing your distribution channels. The web will be your first one if you’re using it. Also, if you are thinking of launching in markets that are not physically close to you, now is the time to start testing with those markets.
Not only will you be familiarizing yourself with the tools to find those markets but also you will, hopefully, have developed the reflex to go to the Lean Startup community for assistance.
The tools of Lean Startup
Now that you will most likely have a, or several, co-founder(s) and employee(s), often not in the same city or working space, efficient communication will be a real need.
Slack, Trello and Dropbox may already be part of your daily routine. You may now want higher performance project management tools (such as Asana, Jira or Wrike) or to start taking notice of Slack management best practices.
If you’ve embraced the integrated multiplatform communication route of Slack, you may also want to look at some of its competitors such as Bitrix24 and HipChat.
If your product is IT application, you’ll want to use one of the thousands of productivity apps available, often for free, to help you develop and test it. Tools such as Balsamiq Mockups , Fluid UI or HotGloo for wireframing. GitHub, Zappier or LucidChart for developing your application. Test Fairy, Hiptest or SauceLabs for automated testing. Once your app is functional you can also put it up on Product Hunt and see what kind of reaction you get from a crowd of mostly innovators and early adopters.
If you are developing a non IT product, you can look for tools such as Ask Your Target Market (AYTM), Facebook or Survey Monkey and combine them with any potential customer email lists or profiles you have already collected.
You will also need to use your creativity to design experiments such as webinars or gatherings to have potential users interact with your product. Tools such as Meetup, Eventbrite, GotoMeeting will be very useful.
When you want to start testing packaging and pricing, A/B testing will be your friend. A/B testing is also very useful to test the conversion rates of different elements of your web pages. Tools such as Unbounce, Optimizely and LauchRock will be necessary unless you hold a degree in statistics.
If you are launching a new restaurant or brick and mortar business sites such as Potloc can help you find the best location and get feedback from potential customers.
Upping your game on metrics
Experimenting and collecting all this information is great. You’ll need however to store, crunch and make sense of all of this data.
You’ll also need to figure out what metrics will be the key ones for your business. If you haven’t read Lean Analytics (Croll & Yoskovitz) yet, you need to by now.
Analytics is one of the parts of Lean Startup that isn’t intuitive. There is knowledge involved here, quite a bit of it. The learning curve is steep. If you will be running an online business (fully online or parts of it) you definitely need some deep analytics knowledge in your company.
Zoho Reports, Segments and Tableau are all great options. Initially you may however simply use an Excel spreadsheet in conjunction with Google Analytics. By the end of this stage, you may find however that the home grown solution requires too much time to manage and you’ll see value in the previously mentioned apps.
As you can see applying the Lean Startup approach gets easier and easier as your startup grows.
The next and last post of this series will be on how to apply Lean Startup during the concierge phase of your startup.
 Although you may be selling at a discount or simply selling and reimbursing as your product is not yet ready.
 Doesn’t integrate with Slack
This is the fourth post of a series that will try to show how Lean Startup can be useful at various phases of an organisation’s life. This post is the continuation of Lean Startup in the Discovery Phase – Part 1 and will show you how to conduct a Lean Startup experiment in the discovery phase of your project
Your first Lean Startup experiment
In Lean Startup all ideas that impact your business model are to be considered as assumptions to be validated. Hence, experimentation is the hearth of Lean Startup. A Lean Startup experiment can be summarised by the Build, Measure and Learn loop.
In order to build a Lean Startup experiment, you first need to transform your assumption into a hypothesis. A hypothesis has the form of a simple, objective statement that can be validated (or invalidated). This may seem simple but it is often one of the most difficult parts of the experiment. It entails that you already have a significant understanding of your experiment participants’ lingo, environment and mindset. Hopefully, you obtained some of this knowledge during your ideation phase when you informally discussed your project.
Your hypothesis must also produce results that will be actionable whether you validate or invalidate it. If, for example, you state your hypothesis as: Customer segment A will always prefer my product to the other solutions presented, all you need is one potential customer to choose another solution to invalidate your hypothesis. What action will you take if you invalidate your hypothesis? None; as your business doesn’t require 100% of a customer segment to adopt your product to be viable.
Simply dropping the word ‘’always’’ in your hypothesis statement will allow you to take action if you validate your hypothesis (you’ll go forward with the segment) or invalidate your hypothesis (you will either pivot on your segment choice or adapt your product further to fill the needs of this segment).
It is also interesting to note that this hypothesis statement contains an unverified assumption. You assume that the ‘’other solutions presented’’ will be the most important substitutes to your product. The experiment, if well designed, will allow you to verify this as well. You may find out that your target segment uses other substitutes that you didn’t account for.
Once you have stated your hypothesis clearly, you need to design a Lean Startup experiment that will enable you to validate or invalidate it. As you now know, you will have a whole lot of these Lean Startup experiments to conduct during your start-up process. Therefore you will be thinking of designing not just any experiment, but the experiment that will require the least amount of resources to achieve your goal. If your product requires expensive machinery to produce, you certainly don’t want to build the entire factory to validate your assumption. You’ll need to be creative and find ways to get your machine’s concept across to the people in your experiment in the least costly way (in $, time and other resources) possible.
The way you will have found to get your product concept message across will be your Minimum Viable Product (MVP). Hence, in this particular example, your MVP could take the form of a drawing on a napkin, on a computer (2 or 3D), in a video, a physical model with or without moving pieces or using a similar product that you supplement with explanations of what your product will do differently. Whatever you feel is necessary and sufficient (no more, no less) to get the message across.
You also need to keep in mind that your Lean Startup experiment should be replicable. This means that in a year from now, if someone else needed to audit your results, they could replicate your experiment and get similar results. In other words, don’t choose a period, a setting or participant profiles that will skew your results.
Once you have identified your MVP, you will need to decide what metric (data) you will be collecting and well as the standard it will need to measure against. In the example cited above, you would be collecting how many people in your experiment chose your product over other solutions that were offered to them. The standard, by definition, would have to be above 50% of your sample given your hypothesis states a preference. Any results below 50% of participants in the experiment choosing an alternative solution would not show a preference. If your hypothesis statement did not imply a comparison but simply an interest in using your product, the standard (% of participants in this case) could be set at whatever you feel would be sufficient to demonstrate clear interest.
In order to reduce the objectivity of standard setting, you should try to get the input from someone who is very familiar with the stakeholders in your experiment in respect to your product/service.
When measuring, you aren’t looking for statistically significant results. This would be ideal but too costly in time and other resources. Instead, you will simply be looking for a clear direction in the data. If your experiment data doesn’t show a clear direction, you will need to include more participants. Occasionally, your experiment will not show a clear direction. You gather whatever learnings you can from it and move on to designing another experiment to validate your assumption.
A word of caution here, despite wanting to go as fast as possible in your Lean Startup experiments you still want to conduct them in a way that will get you to validate or invalidate your hypothesis. If your data collection methods are not done properly and your results are seriously skewed, you will have wasted your time and energy. Hence, you still need to have some basic knowledge of primary data collection techniques. Here are a few good sources of basic information on data collection that you should take a look at if your knowledge on the topic is nil or very low.
During your first Lean Startup experiments, which should be conducted face to face with your participants (this would be the Get out of the building part), you will have the greatest amount of learnings.
While you conduct your experiment, you should always leave room for the unplanned. If a participant reacts in an unanticipated way, run with it and dig into it.
Try to avoid justifying your product/service. Embrace all the comments you will have, especially the negative ones. Dig as much as you can into those. They will give you the best insights. Let the participants talk and listen to the words used, observe body language and notice expressions they use. If at all possible video or tape-record your interactions.
Keep an open mind. It won’t help your start-up if the participants understand your point of view or for you to tell them how you understand their problem. It will however help you tremendously to understand their point of view and what they perceive their problems are or your solution to be.
Your first Lean Startup experiments will help you learn what your stakeholders think about your business model assumptions, as well as all the assumptions you didn’t know you didn’t know. On top of this, you are also learning how to conduct an experiment.
Expect to make mistakes, a lot of them, in the entire build, measure and learn process. That’s ok. Learn from them as quickly as possible and always keep your ultimate goal in mind. If you start an experiment and realise you didn’t state your hypothesis correctly, design it right or choose to collect the right metrics, make immediate changes. Don’t persist with 10-20 other participants with a flawed experiment. Your ultimate goal is to validate a hypothesis, not run an experiment. Similarly, if you had planned to involve 50 participants and you have a very clear direction after 15, stop.
The experiment log
If at any point in time in your start-up process you will be going to venture capital firms for funds, you will want to record your experiments in a log. Your Lean Startup experiment log will document your progress as a start-up. It will show how you dealt with the unexpected, all your learnings and the progress made towards a product/market fit. The data collected in your experiment log will also serve as input in your innovation accounting. Innovation accounting, developed by David Binetti, is one of the more complex but exciting component of Lean Startup. You can learn more about innovation accounting in these previous posts on how innovation accounting is changing the rules, how it’s done and on the concept of innovation options.
Practice makes perfect
As you may have figured out by now, applying Lean Startup requires effort, lots of it, and discipline. Well, that is the reality of starting a business. Lean Startup experiments have one purpose; to bring you closer to having a profitable business or perennial organisation.
You may not be very good at the whole build, measure, learn thing at first. That’s ok. Just keep at it and practice, practice, practice. It gets easier, much easier, I promise. After a few Lean Startup experiments you will start getting the hang of it. Not only will you already have a better understanding of the mindset of your stakeholders, you will develop tools and processes that will make experimenting much faster. You will also develop skills that will enable you to identify critical assumptions quickly, as well as spot, absorb and act on useful information more efficiently.
After a few dozen Lean Startup experiments, the build, measure, learn process will become second nature to you and others, applying it, in your organisation.
Using Lean Startup is similar to buying an insurance policy. It does require more efforts initially than if you simply build your product and service and take it to market. The benefits are that you won’t be building a product or service that will never lead to a profitable business.
Our next post will present some of the tools to help you run your experiments faster, cheaper and more efficiently.
 As long as your sample is not completely biased. Your participants sample has to somewhat reflect the overall population of the stakeholders of your experiment.