I confess to sometimes pouring (mostly private) scorn on banal, ill-conceived or contrived research. I'd apologise, though I'm afraid I also have a low tolerance for mediocrity and perhaps through therapy a penchant for sarcasm would also quickly bubble to the surface. 

I save my harshest judgement for (mis)'leading' research which delivers lofty descriptions of products or services that respondents could never say no to and then asking them at what price point they would pay for their product or service. These studies rarely deliver very little differentiation in preference but are often interpreted as affirming the price proposition desired. Surprised?!

It was refreshing, therefore to read the recent Harvard Business Review article - Ask Your Customers for Predictions, Not Preferences.

As Julie Wittes Schlack points out; "Purchase intent is notoriously overstated in survey reposes, showing little correlation with actual sales performance…In contrast, when you ask a diverse group of people not 'What are you going to do?' but rather, 'What is going to happen?' the results tend to be far more accurate."

Prediction market testing returns a greater degree of accuracy and has the ability to provide more discernment between concepts. Comparisons with prediction market results to the results from other forms of tests and real-world outcomes are promising. What I love about prediction markets are the rich and unique insights it also delivers and the potential to easily sight 'step-change' ideas. 

As HBR points out; "Prediction markets...provide a far more nuanced read on consumers’ thinking and passions than do traditional methods. They let us immediately see which concepts are polarizing, because they attract significant positive and negative investment. We can also discern which ideas are most differentiated and attract the greatest passion based on the number of investors they attract and the average number of points invested."

Similarly, Choice modelling in market research (as described by Decision Analyst is an 'analytical method used to simulate real-world consumer purchasing behaviour') is used to guide product positioning, pricing, concept testing as well as to predict changes in demand and market share.

Neither prediction market testing nor choice modelling are new concepts but both are under-utilised in the Australian market. A simple survey or an internally conducted set of focus groups (for preference testing) are often employed to validate conventional thinking and generally seen as more cost effective options. Results might convince those not familiar with the rigour required to deliver robust results but they are 'safe' and where more accurate methods are available, dare I say, naive. 

Trialling new research methodologies, especially those which engage real customers requires bravery and vision. Knowing when to use them requires deep research and insights knowledge and leadership. As HBR puts it - "The most visionary companies not only explore new research methodologies, but also engage their customers' passion and expertise in the design and testing of the products and services they'll eventually be asked to buy."

Be curious, always - never be afraid to learn something new. Explore new methodologies which deliver better results - neither you nor your business/client will regret the insight you will garner not just from the results but also from the implementation of the process and the mere learning of a new skill.

Operating in retail and online commerce, finance, telecoms, media, any subscription based or CRM gathering service you should be currently engaged in predictive analytics to ensure better understanding and retaining customers (or readers) and acquiring new ones more effectively.

Based on TDWI Research, the key reasons companies utilise predictive analytics are; to predict trends, understand customers, improve business performance, drive strategic decision-making, and predict behaviour. Predictive analytics is a must for retention, optimisation and acquisition strategies.

It's a bit of a (data) minefield getting your head around not just understanding and best practice predictive analytics (if you're like me and neither a data scientist or a mathematician) but also best understanding the appropriate product solutions for your personal and corporate use.

I'm loving the site Predictive Analytics Today. It's full of valuable advice and insights covering Predictive Analysis (of course), BI, Big Data and Text Analytics. There's news, reviews and tips plus plenty more to really geek out on. If you're just getting into predictive analysis, want to compare products or get the headlines of the offerings for confidence in decision making on supplier products and negotiations, there's a cheats guide to the best predictive analytics software. Incredibly useful and comprehensive.

Check out The Top 23 Predictive Analytics Software, as outlined by Predictive Analytics Today, which shows the usual suspects and arguably best providers in the likes of SAS, SAP and IBM with functions generally covering data mining, statistics, modelling,visualisation, econometrics, optimisation and forecasting but also summarises some lesser cited players like DSS (Data Science Studio) which enable correlation and significant variable data discovery and allows for testing of best fitting models.

There are also many free software solutions. Take a peek at the Top 12 Predictive Analytics Freeware Software. Everything from R and Orange to RapidMiner. Personally, I'm a little hooked on R at the moment. It's relatively straight forward and easy to self-teach. Certainly a great way to start building your confidence in PA software before investing in more intricate solutions.

Are there any on these lists you're using? Which ones so you rate?

How is your predictive analysis journey going?

Technology is enabling DIY in all business departments with key benefits being greater efficiency and ownership. Research is no exception. Some neat products to enable DIY, in-house management and consultancy-based operations for research vary from analytics support to real-time surveying. Check out GutCheck, 20|20 Research and Qualtrics as a few suggestions.

A word of caution - good operators never blame their tools! Always seek professional guidance on implementation, methodologies and best-practice.