DIY Market Research, Done Right

In the past, marketing leaders would have needed the help of an external firm—and a budget running into the tens of thousands of dollars—to conduct a solid market research survey. Today, they can use tools like SurveyMonkey and Qualtrics to do much of this work themselves. But, much like me trying (and failing) to renovate my kitchen, these marketers will need more than just the right tools to do the job. They’ll also need an essential set of basic skills.

These six steps can help marketers write survey questions that yield actionable data:

Go Beyond Yes/No: When I’m working with students in my MIT Professional Education classes on survey design and market research, they will sometimes write questions that ask point-blank whether a customer is satisfied with their product or company. The impulse is understandable, since satisfaction level is the thing many of these professionals most want to measure. However, a yes/no question like “Are you satisfied?” is a blunt instrument, and one that doesn’t allow for any nuance in responses. It’s much better to give respondents a larger range of responses to choose from. You might end up finding the answers you’re looking for based on the portion of respondents who say they’re “somewhat satisfied”—but not “mostly satisfied.”

Think Like Your Customer: Often, marketers are a bit too close to their own product, leading them to ask about small details using technical terms that may mean nothing to the average respondent. It’s important for survey designers to put themselves inside the minds of their customers, asking questions that will resonate with them. Remember, a survey is essentially a conversation. Consider how you would speak to a customer one-on-one, and then use similar language in your survey questions.

Leverage Conjoint Analysis: It turns out that most people are pretty bad at thinking in hypotheticals. Ask them whether they want their living space to be larger, less expensive, or better located, and they’re likely to answer: Yes. But by asking questions that force people to think in terms of tradeoffs, companies can quickly get a better sense of what their customers truly value. Conjoint analysis is a survey technique in which respondents are asked to choose between multiple options. For instance, a smartphone maker might present several different feature sets at the same price point and ask customers which build they would be most likely to buy.

Ask About Actual Behavior: While survey questions about customers’ desires can yield valuable data, it is sometimes also helpful to ask about the purchasing decisions they’ve actually made in their real lives. Again, this goes back to the limited utility of hypotheticals. If you ask someone how much they’d be willing to spend on a certain type of product, they may imagine their best, most frugal selves sticking to a strict budget. But if you ask about how much they’ve spent on similar products, you may learn that they’re actually willing to spend double or triple that hypothetical number.

Test Your Questions: Before spending time and money surveying thousands of people, it’s important to make sure your questions yield  the sort of information you’re looking for. The best way to do this is to pre-test your survey design with a small batch of customers. If you’ve written a question about tech support, but respondents misinterpret the question as asking about more general customer service support, it’s important to learn this early on so you can revise the question. Companies might also validate the results of their surveys by running A/B tests of online ads, to see if real-world customers respond to the same offerings as survey respondents.

Know Your Sample: Even when companies insource survey design, they sometimes outsource the task of gathering a representative sample of existing or prospective customers. But typically, marketers don’t have the expertise to evaluate sample quality for your purposes. It’s important to watch out for low-quality responses from people who are filling out surveys in bulk. One way to do this is to sprinkle in a couple of simple factual questions to see whether respondents are even reading the survey (rather than filling it out randomly). It’s also important to take steps that can flag potential bots, which have become a real problem in the world of market research.

The emergence of online surveying tools gives companies the ability to conduct more—and more thorough—surveys than ever before, but the use of these tools can also create unintended problems. On top of that, the field is constantly evolving. But by arming themselves with a bit of ongoing formal training, marketers can get professional results from their DIY efforts.

Source: MTLC