Are Recommendation Engines a Threat to the Long Tail?
Two Wharton academics released an interesting paper last week that asks whether online recommendation services are a threat to the aggregate diversity of items discovered by their users. The study is titled "Blockbuster Culture's Next Rise or Fall: The Impact of Recommender Systems on Sales Diversity" and I found it via a good summary article at PaidContent this weekend.
All indications point towards a rise in importance by recommendation engines, so this argument deserves examination. From eBay's acquisition of StumbleUpon to the CBS acquisition of Last.fm to this weekend's MSNBC acquisition of Newsvine - recommendation engines are big money. We've covered quite a few startups in this space and I'm sure it will continue to grow in prominence.
Perhaps more importantly, the "Long Tail" of diverse discovery is an important part of the meritocratic and democratic promise of the new web.
Good recommendation engines are also just plain fun.
After just a little consideration, the Wharton study seems more meaningful as a cautionary tale than as a critique of the inherent nature of recommendation engines. In discussing this with others I've found that most people swing quickly from believing the study is either obviously wrong or obviously correct. It's a more complex question than it might seem.
Recommendation engines should strive to be smarter than simply finding that "there is a high correlation between people who liked X and people who liked Y." I would argue, for example, that recommending other users of a system and highlighting their less popular discoveries could be a good way to solve the problem. Getting it right is probably easier said than done, but it seems there's still plenty of potential for recommendation engines to expand the long tail. The study's arguments are important to consider, though.
What the Study Says
A Wharton summary of the paper excerpts the following to explain the study's conclusion: "Because common recommenders recommend products based on sales and [consumer] ratings, they cannot recommend products with limited historical data, even if they would be rated favorably," the authors write. "This can create rich-get-richer effects for popular products and vice-versa for unpopular ones, which results in less diversity."
There's also some discussion of the Facebook app landscape, arguably an environment where the long tail doesn't hold up. See also this related discussion at TechCrunch.
The authors argue that individual users may consistently be exposed to items that are new to them, but we're all exposed to the same new items - resulting in greater individual diversity but less aggregate diversity.
Counter Arguments
The study includes a counter argument from Greg Linden, who helped develop Amazon's recommendation engine. Linden says "recommendation algorithms easily can be tuned to favor the back catalog -- the long tail -- as Netflix does."
The role played by early adopters, "cool hunters", taste makers and advertisers relative to recommendation engines would also be interesting to look at.
My personal fantasy for recommendation engines is this: I want del.icio.us to look at my bookmarks and recommend not just other URLs I might be interested in, but also other users whose tastes are similar to mine. I'd also like to see which of those recommended users tend to find items of interest earliest, so I can prioritize following them.
Repetition, perhaps another way to describe popularity, will probably always drive consumption - but if I can see all of the things that are discovered by people recommended to me then I can use their less popular picks as guidance.
If other metrics are considered, and surely they are in any sophisticated recommendation engine, then what's called "Attention Data" can help augment recommendations beyond merely what's most popular among people with similar interests. (Need an intro to Attention Data? Here's one that could work for you.)
It would be ill advised to reject recommendation engines as dumb popularity machines based on this study, but it is also important to take its arguments into consideration.
Share
Comments
Subscribe to comments for this post OR Subscribe to comments for all ReadWriteWeb posts
i tend to agree more with linden's analysis. recommendation engines will be used by intermediaries to create as custom of an experience for the user as possible. with the end objective being customization, i think recommendation engines will be used to empower the long tail -- not reinforce the herd mentality.
Hi Marshall,
First of all, great post! As you know, I am a co-founder and VP of Product Management and Business Development at mSpoke one of the leading players in next generation of recommendation and personalization engines. Therefore, I have a strong opinion on this issue. Specifically, this is an issue we take very seriously at mSpoke.
We have chosen to address it by taking a two step approach:
First, we actually use lots of different reasons for recommending content to users. You can see more here -- http://www.mspoke.com/blog/?p=6 You'll see a lot of this is a mix of your attention data in addition to your behavior.
If you don't do this, you have to rely on one single approach exclusively. (Obviously) Typically, this means techniques like leveraging the Wisdom of Crowds (eg collaborative filtering). While that works well in some situations (commerce especially -- ex: Amazon Recommendation), in many situations such as broad content areas (esp news & blog content) where there is less of a defined taxonomy it becomes a very difficult to deliver relevance especially as you first get started. This is called the cold start problem in the field of information retrieval.
Second, in addition to taking into account a variety of different streams of information about a user, we create an individual algorithm for each user. This is important, because it avoids certain algorithms/techniques for recommending content. The issue is that if you have one single monolithic algorithm, then the content that matches well will dominate the system.
Also, I should note we show the user the different components (we call them memes) that are part of their individual algorithm. They can manually adjust these and remove specific memes if they choose.
Finally, a shameless plug -- if you're interested in experiencing our engine in action, check out our new product FeedHub (www.feedhub.com) that recommends content from within the RSS feeds you are subscribed to.
Thanks,
Sean
Feed Each Other (http://feedeachother.com) does just the sort of "users of similar taste" recommendation you described. Check out the left column of any user profile page.
There is an upcoming conference on Recommenders next week that covers this topic and many other related ones. Check it out at: http://recsys.acm.org/program.html
MyStrands will also be covering the event at http://blog.mystrands.com.
Part of this phenomenon has to do with recommendation engines. Anything that purports to separate the wheat from the chaff is going to do so imperfectly if it's going to be economically viable. Those things recommended as "wheat" will have a certain sameness unless the recommendation engine works hard to over come that. Anyone that's looked at TechMeme and the leaderboard doesn't see 100 vibrantly different web sites. They see 100 sites all struggling to tell nearly the same story in their own way. Hence you can subscribe to maybe 1/5 of those and see all the interesting stories very easily.
So the issue here is where are the recommendation engines looking and are they trying to promote diversity? The promotion of diversity may or may not be compatible with the commercial objectives of the recommender. I love it, personally, because I always find the most interesting stuff just a little off the beaten path. NetFlix loves it because it reduces their inventory costs. Movie hits shoot up like skyrockets and then come down rapidly as everyone sees the movie. If Netflix can get you to look at something besides the hits, they have fewer wasted copies of those hits.
Here is the other piece though, and it has to do with the web itself, not the recommenders. It moves in "punctuated equilibrium", a term I borrowed from evolutionary biologists:
http://smoothspan.wordpress.com/2007/10/01/the-internet-first-breeds-diversity-then-conformity-punctuated-equilibrium/
What this means is we move in fits and starts. First there is a big evolutionary change. Call it a sudden mutation that works. MySpace is an example. Then, as everyone sees how great the new new thing is, there are hundreds of variations on it. Often the variations are more successful: think Facebook (more successful) or Ning (a variation the jury is still out on).
This means that the apparent plethora of choices consists of many surface level choices and just a few deeply different avenues. A recommender can't create meaningful choice where there isn't any yet, so again, their results can seem homogenized.
Cheers,
BW
Last.fm is another recommendation service where the rich-gets-richer phenomenon may not be entirely true; it tends to recommend a lot of obscure songs/bands -- in fact, the recommendation page allows you to select how obscure you want your recommendations should be.
Bob Warfield is right on. Most people that are trying to build sites around automated recommendation fail (see findory), not because their technology isn't great, but because they're biting off more than they can chew. I wrote about this a while back:
http://breasy.com/blog/2007/04/23/the-limits-of-personalization-technology/
The right approach is to rely on interpersonal recommendations. If a friend who usually recommends great stuff to you gives you one bad recommendation, then you blame your friend and not the system. That scales...
"there is a high correlation between people who liked X and people who liked Y."
That is exactly what Social Suggester does
http://www.socialsuggester.com
Even tho recommendation sites are a dime a dozen nowadays, this site allows users to focus their search for less known results across music / movies / books / etc.
This Wharton study reinforces what most of us already implicitly feel about many existing recommendation systems: they generate mediocre results (ex. "You bought Die Hard, you might also like Die Hard II."). Most recommendation systems rarely surprise us with new, great recommendations for items we've never heard of before. As the study points out, long tail items get lost because, by their very nature, they have few ratings or prior purchases.
As co-founder of Trusted Opinion, this is a problem I've thought about a lot. My opinion is that any black-box, algorithmic approach to recommendations misses the boat, because it ignores the person-to-person "trust" element that makes recommendations valuable to us in real-life. If you want to drive a consumer's confidence in the recommendations she receives online, you need to provide transparency and visibility into how the recommendation was produced.
Marshall, I think you hit the nail on the head in your summary when you discussed "users whose tastes are similar to mine", i.e. affinity. We built a system at TrustedOpinion.com based on the premise that the opinions of your social circle are more relevant. We weigh the opinions of your social network highest, giving the highest weight to people who you're directly connected to. The result is each user gets a personlized score for each item, and sees a 3-d graphic that displays the individual opinions of friends their social network (out to the 3rd degree).
It always surprises me that most recommendation systems are based solely on mathematics and ignore psychology. The measure of a "good" recommendation is tied intrinsically to the recipient's confidence in it. If people can't understand how a recommendation is generated, they'll ignore it even when it is a good suggestion.
Thanks,
-- Todd
Great post, Marshall. Personalized recommendation sytems often underperform because they confuse "popular" for "interesting." It's a tough problem to solve because "interesting" means different things to different people depending upon what they know already.
For instance, if I'm new to the web 2.0 space, Read/WriteWeb is a very interesting (and popular)recommendation for me. However if I'm an expert on web 2.0, all of the popular web 2.0 sites are not interesting at all to me because I already know about them.
Meme: Popular != Interesting. Recommendation systems should focus on interesting.
Shameless plug: We're incorporating interestingness and attention data (optional) into the next version of Youlicit to deliver more interesting personalized recommendations. It will also identify "experts" in the areas you're interested in, so you can subscribe to their recommendations. Have a look. Would love to hear your thoughts. Thanks.
Toufique
Founder, Youlicit