Bad Stats are Holding Back Web 2.0
I use “stats” generically here because, at its simplest meaning, what is holding back web 2.0 is bad statistical planning. You have to ask the right questions with the right controls to get the right answers so you can make the right recommendations. The napkin business plan generation apparently has not learned its lesson in the web 2.0 sphere, with new sites and communities launching by the dozens. Unfortunately, web 2.0 communities commonly make the following assumption: massive data, regardless of its quality, is sufficient.
What we need is a more complete measurements (questions) and common, controlled subjects (wines).
To illustrate this point, I would like to identify 3 wine Web 2.0 communities (Corkd – BottleNotes – TasteVine), 2 of which do it the “wrong way” and one of which does it the “right way”. To my knowledge, no other industry or web 2.0 site is using the same methodology as the 3rd site, and it will make all the difference in the world. Full Disclosure: Virante worked on the development of TasteVine, so, while I believe my statements to be accurate and worth looking at, you should investigate them yourself.
1. Corkd.com: (the Netflix model)
This is the most common form of web 2.0 data gathering and recommendation generating that occurs on the web. It works on this logic…
- Get a ton of people to rank stuff 1-100 (formerly 1-5)
- Find people with similar rankings
- Create recommendations based on that data
The Netflix model presents the obvious long-term consequences: needing to run a $1,000,000 prize to get better recommendations from their data. The problem is not bad algorithms, it’s bad data.
In the case of wines, one individual may like sweet wines and give it a 100 while another likes dry wines and give it a 100. It would take massive numbers of duplicate tastings and rankings to build a decent recommendation based on these poor measurements.
The NetFlix/Corkd model compares apples to apples, but one person is looking at smell, another at taste, and yet another at color. Unless you can tease out WHY a person feels one way or another, you can’t use their opinions to make recommendations for others.
2. Bottlenotes : (The EHarmony Model)
This is a growing model of match-making methods but presents a similar set of problems. The gist is simple: give everyone a test at the beginning and join people together.
For the sake of argument, lets say you ask a person whether or not they like sweet wines as part of the profiling system. They say yes, but their tastes are actually skewed greatly because they consider only the very sweetest of wines to be sweet.
With the EHarmony model, the problem is you are not comparing apples-to-apples. You are asking questions about the individual’s subjective reality, with no fixed subject which we can all agree upon. Do you like tart, do you like sweet, do you like crisp, do you like ripe? This measurements are more complete, but the subjects vary.
3. TasteVine: The, uhhh, TasteVine model.
Imagine how good E-Harmony would be if everyone who signed up dated one of 10 people for a day and then rated them. Instead of obtuse, ambiguous questions about values, we would have real world tests. Unfortunately, this just isn’t possible in the realm of online dating. However, for wines, this is definitely a possibility.
The TasteVine model fixes the majority of these basic statistical problems from the onset…
- Taste the exact same wines everyone else does (1 from each varietal)
- Asks the same questions about the wine (is it sweet, is it smooth, do you like it)
- Create a TasteID based on this data to match with other users (your TasteBudds)
- Deliver recommendations based on reviews of other wines by your TasteBudds
It does require that 1 extra step, that you try some of the 12 wines to get better recommendations. However, this is correct method to gather and use data.
It will be interesting to see how this community fairs in the long run. It definitely has the best recommendation algorithm, but the user does have to taste a few wines to get that value out of it. However, even in competitor sites like Cork’d, the user would have to try a large number of wines to start building up an accurate profile from which the system can make recommendations. We shall see.