I confess to sometimes pouring (mostly private) scorn on banal, ill-conceived or contrived research. I'd apologise, though I'm afraid I also have a low tolerance for mediocrity and perhaps through therapy a penchant for sarcasm would also quickly bubble to the surface.
I save my harshest judgement for (mis)'leading' research which delivers lofty descriptions of products or services that respondents could never say no to and then asking them at what price point they would pay for their product or service. These studies rarely deliver very little differentiation in preference but are often interpreted as affirming the price proposition desired. Surprised?!
It was refreshing, therefore to read the recent Harvard Business Review article - Ask Your Customers for Predictions, Not Preferences.
As Julie Wittes Schlack points out; "Purchase intent is notoriously overstated in survey reposes, showing little correlation with actual sales performance…In contrast, when you ask a diverse group of people not 'What are you going to do?' but rather, 'What is going to happen?' the results tend to be far more accurate."
Prediction market testing returns a greater degree of accuracy and has the ability to provide more discernment between concepts. Comparisons with prediction market results to the results from other forms of tests and real-world outcomes are promising. What I love about prediction markets are the rich and unique insights it also delivers and the potential to easily sight 'step-change' ideas.
As HBR points out; "Prediction markets...provide a far more nuanced read on consumers’ thinking and passions than do traditional methods. They let us immediately see which concepts are polarizing, because they attract significant positive and negative investment. We can also discern which ideas are most differentiated and attract the greatest passion based on the number of investors they attract and the average number of points invested."
Similarly, Choice modelling in market research (as described by Decision Analyst is an 'analytical method used to simulate real-world consumer purchasing behaviour') is used to guide product positioning, pricing, concept testing as well as to predict changes in demand and market share.
Neither prediction market testing nor choice modelling are new concepts but both are under-utilised in the Australian market. A simple survey or an internally conducted set of focus groups (for preference testing) are often employed to validate conventional thinking and generally seen as more cost effective options. Results might convince those not familiar with the rigour required to deliver robust results but they are 'safe' and where more accurate methods are available, dare I say, naive.
Trialling new research methodologies, especially those which engage real customers requires bravery and vision. Knowing when to use them requires deep research and insights knowledge and leadership. As HBR puts it - "The most visionary companies not only explore new research methodologies, but also engage their customers' passion and expertise in the design and testing of the products and services they'll eventually be asked to buy."
Be curious, always - never be afraid to learn something new. Explore new methodologies which deliver better results - neither you nor your business/client will regret the insight you will garner not just from the results but also from the implementation of the process and the mere learning of a new skill.
Your comments are welcome and you are encouraged to express your views in a positive and productive manner in line with our Code of Conduct.