When you start exploring ideas, uncertainty is at its highest. You don’t know yet what’s going to work or not. Your customers will play an important role when testing out your value proposition and business model. Experiments that involve customers will help to unearth insights to (in)validate critical hypotheses around your ideas. But it’s also important to be mindful of how, where, and when you’re trying to collect customer related data. First, the data you’ll get will differ based on what customers say, and what customers do. Second, the way customers behave will change based on whether or not you are present during the experiment.
Here’s how you can get around these issues when conducting customer experiments:
What customers say and what customers do
Start experimenting with techniques that can unearth verbal evidence from customers. Don’t fall in the trap of asking for opinions, but seek to get input on experiences and motivations instead. You need to formulate questions that trigger facts. For example, don’t ask ‘would you...?’ to someone, but ask ‘when is the last time you have …?’. This is a quick and low cost approach. But remember that the reliability of the data you get when speaking with customers won’t be enough. Customers might tell you one thing and then act differently (consciously or not). You should always ‘double-verify’ your most critical assumptions by making customers take actions. Get your customers to do something to generate much harder and more reliable data than verbal evidence because it requires more effort from your participants. Observe how your customers react to things, or how they interact with a prototype to discover insights you won’t find when only verbally interacting with them. e.g. if customers complete the purchase process on your website and proceed with payment.
How customers behave when you are here and when you aren’t
Your presence around customers will give you an opportunity to ask questions and dig deeper into behaviors during the experiment. It’s your chance to directly learn and improve your idea, just don’t over do it--your presence might lead customers to act differently than how they would if you weren’t observing them. In an indirect observation of how customers behave, you’re more likely to get reliable data that is close to how customers would act in reality e.g. how long they actually stay on a website. Although It reduces your opportunity to learn about other things, staying away enables you to generate stronger evidence to a specific hypothesis.
We’ve compiled and categorized a library of testing techniques based these two factors.
Customers know better than you when it comes to value propositions they care about and are willing to pay for. You can reduce the uncertainty around your ideas by leveraging that collective intelligence and testing your assumptions with customers. As Steve Blank, pioneer of the Customer Development methodology, says, “You may be smart, but you are not smarter than the collective intelligence of your potential customers.”
What types of customer experiments have you conducted, and what did they teach you?
