If you want to run a good customer test, ask these quesitons
When I’m planning a test with my teams these are the questions that we ask ourselves. It really helps us to focus our efforts in planning, recruiting and creating a a prototype or other artifact to test.
What do we want to learn?
This is the best place to start and should drive the answers to all the questions that come behind it.
Get with your team and list out all of the things you want to learn or what you need to know to move forward. This should include learning about the customer, product and business.
There will be more you want to learn that you can reasonably fit into a test. Start with the biggest and most important questions you have to answer right away.
Who do we want to learn it from?
What customers and/or users do you need to get in front of. In this context I refer to a customer as someone who pays and the user as the person interacting with the product. They might be the same person or not depending on the product.
It’s helpful to think about the characteristics of the customers you want in your test. In B2B situations I often want to test with people of different level so sophistication so I can narrow my target or see if a feature is relevant.
Try to make sure you have three to five people from each group in a test. If you have less than three it’s possible that one person will send you on a wild goose chase.
What do we need to show them to learn it?
With this context in place we can focus on what we need to show the test subject. These can be screens, notifications, landing pages, emails, flows, physical displays….
If you skip ahead to this step you will likely get lower quality feedback, which is annoying since recruiting is a pain in the ass, or you will end up doing a lot of rework on the materials for your test.
When you are creating materials do as much as you need to get good feedback and no more. Anything more you do might be wasted effort. This can be tough for designers and engineers who aren’t comfortable showing rough work.
How should we do the test?
Choose the method that is the best mix of speed and quality. I would rather have good feedback and multiple iterations than amazing feedback and few iterations.
Early on I like to do one on one sessions remotely and in-person. This gives me a lot of rich information so I know what changes in direction to make. As I get further along I might use methods like surveys, landing pages or prototypes users can explore on their own to get feedback from a larger sample size.
