InsightaaS: Harvard Business Review (HBR) is recognized worldwide as a leading source of management insight and innovation. In this post, Kaiser Fung, a statistician for Vimeo and author of blogs and books on visualization and big data, dissects the reasons why websites can’t accurately predict what you will want to see on the web. Fung believes that at least in part, the problem arises because companies view “as a tool for upselling—they want to push us out of our comfort zone, to buy new things, and to buy more things.” He believes that instead, companies should focus on “easy wins” – on pre-populating searches with information gleaned from repeated requests (for example, for a specific shoe size). While Fung thinks that embedding these kinds of insights into personalization systems might not be “sexy enough for data scientists,” he argues that for consumers, “simple is the new sexy.”
It should be a golden age for personalization online, and the predictive algorithms that drive it. Data scientists, supported by the stunning growth in the gathering and processing of so-called big data, can extract patterns from massive stores of browsing and sales data in order to predict our likes and dislikes and tailor marketing experiences to us.
The predictive algorithms are often built to impress, designed over years by teams of PhDs from top universities. In 2006, Netflix tapped these brainiacs by offering a million dollars to anyone who improved its home-grown algorithm to predict movie ratings. It took three years before a combined team of seven researchers reached the target of ten-percent prediction improvement. The winning entry was not one algorithm but an ensemble of over one hundred algorithms, bearing exotic names such as restricted Boltzmann machines, and singular value decomposition. Big data flexed its muscles.
But there is a problem. Personalization still isn’t that good. Consumers still talk about it mostly when it’s laughably bad…
Read the entire post: http://blogs.hbr.org/2014/05/why-websites-still-cant-predict-exactly-what-you-want/
