Last April, we held a contest asking readers for their web 3.0 definitions. Our favorite came from Robert O'Brien, who defined Web 3.0 as a "decentralized asynchronous me."

"Web 1.0: Centralized Them. Web 2.0: Distributed Us. Web 3.0: Decentralized Me," he wrote. "[Web 3.0 is] about me when I don't want to participate in the world. It's about me when I want to have more control of my environment particularly who I let in. When my attention is stretched who/what do I pay attention to and who do I let pay attention to me. It is more effective communication for me!"

What O'Brien was getting at is basically what Kiss was getting at: personalization and recommendation. And that's the promise of the Semantic Web. The easiest way to sell the Semantic Web vision to consumers is to talk about how it can make their lives easier. When machines understand things in human terms, and can apply that knowledge to your attention data, we'll have a web that knows what we want and when we want it.

ReadWriteWeb contributor Sramana Mitra put it another way on this blog last February, when she said that web 3.0 will be about adding context to personalization. "Personalization has remained limited to some unsatisfactory efforts by the MyYahoo team, their primary disadvantage being the lack of a starting Context," she wrote. "In Web 3.0, I predict, we are going to start seeing roll-ups. We will see a trunk that emerges from the Context, be it film (Netflix), music (iTunes), cooking / food, working women, single parents, ... and assembles the Web 3.0 formula that addresses the whole set of needs of a consumer in that Context." Or in other words, web 3.0 will be about feeding you the information that you want, when you want it (in the proper context).

Of course, the versioning of the Internet is kind of silly, and probably shouldn't keep going, but it is a fun way to look to the future and predict what we might be coming our way. What do you think of Kiss's idea about web 3.0 being about recommendation and personalization?

Some of the topics we expect to discuss in the show:

• Measuring effectiveness of recommendation systems
• Draw backs / problems (referring to our recent post 5 Problems of Recommender Systems)
• Opportunities our guests see for innovation in the space

We welcome your suggestions for discussion points, either in the comments here or by tuning in LIVE to the show - via Calliflower or Facebook - and participating in the chat room.

UPDATE: the audio is available now.

Download MP3

In a world at risk of information overload, where the line between content producers and consumers is no longer clear and where the pace of everyday life is increasing rapidly - I'd say the recommendation engine business is a very smart one to be in. There is ample precedent and this startup is moving into a relatively established field. Richard MacManus lauded the company's previous multi-million dollar investment and our enthusiasm here for this project continues.

The company announced this morning that it has raised a $24 million B round to take its recommendation system far beyond music. The company says it intends to "lead the social recommendation industry." The round was lead by Spanish bank BBVA and with the participating of existing investors from their June round of $25m. That's an insane amount of money and some people are bewildered why MyStrands has been given it. I'm not one of those people.

The Initial Product and Sales

The company started with an iTunes plug-in that recommends songs similar to what you're listening to and quickly expanded its offerings to include a full multi-media juke-box platform that lets people personalize public playlists with their mobile phones and profiles from home. With offices in Oregon, New York and Barcellona, Spain - this is a web 2.0 company with brisk sales already. The company says it will bank $12 million in 2007. That's no mean feat.

Why This is Very Smart

Music, however, is just the first of many areas of engagement for MyStrands. A huge part of the Amazon.com story is its product recommendation process. Netflix has some of the world's top scientists racing each other to outdo its in-house recommendation engine for a million dollar prize. Last.fm's music recommendation community went to CBS for $280 million. StumbleUpon built web page recommendation into a tasty morsel for eBay to scoop up.

Now MyStrands has a war chest to hire top scientists and bring try and take recommendation to the next level for any type of data. If you can't see the value in that, then you're probably not paying attention.

MyStrands would be a great place to see attention data made real, be it in APML or some other open data standard. This is the kind of company that could create piles of that data or make great use of inbound Attention Data for superior recommendations. MyStrands' recent hire of Scott Kveton, Chair of the OpenID Foundation, to be the company's Director of Open Platforms makes me think that something like that is probably in the works. Kveton says the company is "looking closely at APML, as well as working on some other 'open formats' for describing user taste data. The gist is, the users own this data and we want to give them as much control over it as possible."

When I first saw MyStrands several years ago I thought it looked like a trivial and akward little iTunes plug-in. Like so many startups, though, this company had a much bigger vision all along. Now that vision has $55m in backing, is making big hires and is will record $12m in sales already this year. I'd say this is one company you've got to keep an eye on.

The feature O'Neil wrote about appears to be nothing more than the latest FriendFeed rip-off: truncating repetitive activities. (Ex-Googler Paul Buchheit's FriendFeed is like a Facebook R&D lab without stock options.) Whether Facebook is doing more than that publicly or not, you know they have to be working on recommendation behind closed doors.

O'Neill wondered whether this feature might give the user an option to view other items from the same or other users that Facebook deemed similar to the original post. I say I'm drinking coffee and Facebook shows me a movie, a picture and another message about coffee from one of my friends this morning. That would be pretty awesome for us as users and it would increase ad impressions for Facebook.

More likely is the explanation offered by AllFacebook readers, that this new feature is just a way to scrunch up items that are basically the same so users can't spam their friends' newsfeeds and so that newsfeeds are more pleasing to scan down. In other words, in the image above, the two-headed person on top probably posted about "besplatno-ing" like four times in a row. Facebook decided to show just one of those messages and add a link to view the rest.

That's how FriendFeed does it and it works really well. This seems like a plausible explanation of this screenshot, but it's also a real lost opportunity. Facebook's corny "your friend is a fan of this advertiser's stuff" may be more creepy than compelling - but automated recommendations of all types of items could be great.

We're big fans of recommendation technologies here at ReadWriteWeb, from relatively simple "people who like X also like Y" to more complicated algorithms. The systems are fun to learn about, but the fact of the matter is that recommendation doesn't have to be hard. The hard part is amassing enough data and interested people to be able to make recommendations. Facebook has plenty of data and people, though its labyrinth of privacy restrictions might complicate things a little.

So if this isn't it, and we suspect it is not, we sure do hope that Facebook will soon surface the recommendation technology we assume they are working on behind the scenes.

A couple of years ago, Alex Iskold outlined what he saw as the 4 main approaches to recommendations:

• Personalized recommendation - recommend things based on the individual's past behavior
• Social recommendation - recommend things based on the past behavior of similar users
• Item recommendation - recommend things based on the item itself
• A combination of the three approaches above

Amazon: King of Recommendations

In that post, Alex analyzed what Amazon.com - probably the canonical example of recommendations technology on the Web - used to power it's recommendations. Unsurprisingly, he found that Amazon used all 3 approaches (personalized, social and item). Amazon's system is very sophisticated, but at heart all of its recommendations "are based on individual behavior, plus either the item itself or behavior of other people on Amazon." What's more, the aim of it all is to get you to add more things to your shopping cart.

As Xavier identified, other newer Internet companies have tended to focus on specific methods of recommendation. For Pandora, it is a deeper analysis of the item (using its "gene" theory); Strands has taken a boatload of VC money to try and become the number 1 social recommendations provider on the planet; and Aggregate Knowledge is taking more of a behavioral approach.

Google: Focus on Personalized Recommendations

The most successful Internet company of this era has without a doubt been Google. It too has been using recommendation technologies to improve its core search product. There are two ways that Google does this:

1. Google customizes your search results "when possible" based on your location and/or recent search activity;
2. When you're signed in to your Google Account, you "may see even more relevant, useful results based on your web history."

So Google is using both your location and your personal search history to make its search results supposedly stronger. This is very much the 'personalized recommendation' approach - and indeed personalization has been a buzzword for Google in recent years. However, the two other types of recommendation are also present in Google's core search product:

1. Google's search algorithm PageRank is basically dependent on social recommendations - i.e. who links to a webpage;
2. Google also does item recommendations with its "Did you mean" feature.

There are surely other ways recommendations technologies are being deployed in Google search - not to mention the range of other products Google has. Google News, its start page iGoogle, and its ecommerce site Froogle all have recommendation features.

ReadWriteWeb Resources for Recommendation Technologies

Let us know what types of recommendation technologies other companies are using. We also invite you to explore using our custom ReadWriteWeb Resources:

• RWW Recommendation Industry Feed Favorites OPML file (save link)
• RWW Recommendation Industry Feeds - Best of Feed (copy and paste to your reader)
• Click to preview the above feeds before subscribing (pop-up window)
• RWW Recommendation Site Search (Visit and Bookmark)

These nanogenres are generated by Nanocrowd's analysis of thousands of movie reviews and commentary from many different data sources. In fact, the three words in each nanogenre are chosen directly from the review data. We liked how easy it was to get in to this application, and its addictive ability to keep us clicking on movie after movie, creating new lists and discovering new movies. In fact, the last movie site that had us so distracted was the Internet Movie Database.

Of course, we couldn't end this post without mentioning some of Nanocrowd's competitors, applications like Clerk Dogs (Ars Technica review), Jinni (another Ars Technica review) and the big players, Amazon and IMDB (yes, they have a recommendation engine too!)

Each of these has some merits. For example, Clerk Dogs will recommend movies based on your movie suggestion in only one step. And they also have some really cool graphs showing match criteria. Jinni requires you to create a login before you can get started, but then you can search for just about anything, such as a genre, plot, mood, or actor name.

How does Nanocrowd stack up against the other guys?  Time will tell, and the site is still being actively developed (in fact logging in to save movie lists isn't supported quite yet), but so far, we like that we can dive in and have a list to take with us to Blockbuster (they have a recommendation engine too, by the way) in five minutes or less.

Sponsored Lyrics Pages?

Interestingly, Last.fm also announced that it will allow sponsors to advertise on these lyrics pages, including the ability to skin the entire page. We cannot help but wonder if this is a prelude to similar advertising options on other parts of Last.fm's site.

800,000 Lyrics

Even though 800,000 is a large number, this still leaves the majority of Last.fm's catalog without lyrics. The music industry has always been highly protective about lyrics and has been playing a cat-and-mouse game with many of the independent (and often user generated) lyrics sites.

LyricFind, too, started as a rogue lyrics site in 2000, but has licensed content from over 1,700 music publishers since then, including EMI, Sony, and Universal BMG.

Overall, adding lyrics to song pages is a smart move by Last.fm, as it turns Last.fm into even more of a one-stop music experience and also gives the service yet another feature that sets it apart from other music recommendation and discovery sites.

All indications point towards a rise in importance by recommendation engines, so this argument deserves examination. From eBay's acquisition of StumbleUpon to the CBS acquisition of Last.fm to this weekend's MSNBC acquisition of Newsvine - recommendation engines are big money. We've covered quite a few startups in this space and I'm sure it will continue to grow in prominence.

Perhaps more importantly, the "Long Tail" of diverse discovery is an important part of the meritocratic and democratic promise of the new web.

Good recommendation engines are also just plain fun.

After just a little consideration, the Wharton study seems more meaningful as a cautionary tale than as a critique of the inherent nature of recommendation engines. In discussing this with others I've found that most people swing quickly from believing the study is either obviously wrong or obviously correct. It's a more complex question than it might seem.

Recommendation engines should strive to be smarter than simply finding that "there is a high correlation between people who liked X and people who liked Y." I would argue, for example, that recommending other users of a system and highlighting their less popular discoveries could be a good way to solve the problem. Getting it right is probably easier said than done, but it seems there's still plenty of potential for recommendation engines to expand the long tail. The study's arguments are important to consider, though.

What the Study Says

A Wharton summary of the paper excerpts the following to explain the study's conclusion: "Because common recommenders recommend products based on sales and [consumer] ratings, they cannot recommend products with limited historical data, even if they would be rated favorably," the authors write. "This can create rich-get-richer effects for popular products and vice-versa for unpopular ones, which results in less diversity."

There's also some discussion of the Facebook app landscape, arguably an environment where the long tail doesn't hold up. See also this related discussion at TechCrunch.

The authors argue that individual users may consistently be exposed to items that are new to them, but we're all exposed to the same new items - resulting in greater individual diversity but less aggregate diversity.

Counter Arguments

The study includes a counter argument from Greg Linden, who helped develop Amazon's recommendation engine. Linden says "recommendation algorithms easily can be tuned to favor the back catalog -- the long tail -- as Netflix does."

The role played by early adopters, "cool hunters", taste makers and advertisers relative to recommendation engines would also be interesting to look at.

My personal fantasy for recommendation engines is this: I want del.icio.us to look at my bookmarks and recommend not just other URLs I might be interested in, but also other users whose tastes are similar to mine. I'd also like to see which of those recommended users tend to find items of interest earliest, so I can prioritize following them.

Repetition, perhaps another way to describe popularity, will probably always drive consumption - but if I can see all of the things that are discovered by people recommended to me then I can use their less popular picks as guidance.

If other metrics are considered, and surely they are in any sophisticated recommendation engine, then what's called "Attention Data" can help augment recommendations beyond merely what's most popular among people with similar interests. (Need an intro to Attention Data? Here's one that could work for you.)

It would be ill advised to reject recommendation engines as dumb popularity machines based on this study, but it is also important to take its arguments into consideration.

Customers of Spanish bank BBVA will now be offered recommended products and services, individual and anonymized aggregate analytics and personalized goal setting and alert services, all based on their banking activities.

What's Interesting About This Deal?

Using the Strands Social Recommender technology, BBVA will be able to offer intelligent observations and suggestions for personal finance. A demo of the product shows, for example, that users of the system might be given interesting statistics about the financial activities of people in a particular demographic group, then asked whether they belong to that group. It's like having a private, personal, math-powered financial adviser available for your use on demand.

With interfaces for the iPhone, BlackBerry and Nokia phones - analytics and recommendations will also be available outside of the desktop web browser. This is the kind of heavyweight application to see coming from online recommendation services.

Privacy Concerns

How will bank customers feel about having their personal and financial details thrown into the collective pot for analysis of recommendations to other customers? We think it may take some getting used to, but that kind of information is undoubtedly being aggregated inside of banks already. The prospect of allowing users to benefit directly from their collective data is an appealing one.

Will the recommendations offered all point crudely toward buying more services from the bank? Given the huge war chest that Strands commands and the caliber of hires they've made over the last year, we hope that the company's banking recommendations and observations will prove truly useful and engaging for customers and not just for the bank's bottom line.

Only time will tell, but we've said for some time that in a world drowning in data - powerful recommendation technologies that help point towards personally meaningful information have huge potential. Financial services are the next frontier for these experimental technologies and we hope that Strands will disclose statistics in time demonstrating the impact their service had on the financial lives of users around Europe.

Disclosure: Strands is a RWW sponsor.

Slate.reddit is populated automatically via Slate.com's RSS feeds - so there are no manual submissions, as on the original reddit.com. All that Slate readers need to do is vote and comment.

I asked Alexis Ohanian what led to the deal. Alexis told me that "the relationship with Slate began when their articles started popping up on reddit shortly after we launched last June." That was followed by numerous other front page stories on reddit, at which point emails were exchanged between Reddit management and Slate.

Reddit's Recommendation Engine

What I like about Reddit is that it aims to be a 'recommendation engine'. As it states in the Help section:

"reddit is a source for what's new and popular on the web -- personalized for you. Your votes train a filter, so let reddit know what you liked and disliked, because you'll begin to be recommended links filtered to your tastes."

Personalization is of course the holy grail for Web apps, which we've established before on R/WW is a difficult thing to achieve. But it does seem to be a point of differentiation for reddit, from digg and Netscape - both of which focus more on community recommendations rather than reddit's personal recommendations. And reddit does have some smart people working on this. Aaron Swartz announced today the new version of reddit's recommendation system:

"One major improvement is that it's faster than ever before -- it's practically live. Head to your recommended page and vote on something and the recommender should whisk it away and give you a new recommendation within seconds."

I'll have to test that out! In any case, it strikes me that at the very least Slate.com will get to understand what articles on their site appeal to readers the most (provided the new reddit tool gets sufficient take-up over time). That extra feedback loop, via reddit, will be a valuable source of data for Slate.

Alexis from Reddit told me there are some other similar white label projects in the works, but this is their first announcement. Personally I think it's great to see these community news apps being white labeled to media organizations, where I've always felt they belonged.

If that team withstands the month long period of scrutiny that begins now, it will not only mean fame and (some) fortune for them and a big boost for Netflix - it could signal a key turning point for recommendation technology on the web.

In January of this year, we took an in-depth at the Netflix Prize, asking if 2009 could be the year that the goal would be met. In that post we discussed a New York Times profile of the contest as well, where we learned that the company's existing recommendation engine called Cinematch is credited with driving 60% of Netflix's rentals. That system is especially good at predicting "long tail" movies, older more obscure titles that are less well known but make up 70% of what Netflix customers pick. Improvements in Cinematch's effectiveness plateaued in 2006 and the move to offer a big cash prize for outside innovators has captured the imagination of thousands of engineers and their fans.

How Does it Work?

How do you judge improvements on recommendations? Netflix provides contest participants with huge piles of anonymous data about what movies certain customers rated highly, then the teams built algorithms to predict which movies other customer profiles would rate highly based on past patterns. BellKor's Pragmatic Chaos says it can now guess what people will like with a 10% improvement over Cinematch's success rate.

That gets difficult when movies like Napoleon Dynamite, which some people loved and other people hated, get thrown into the mix. It's nearly impossible to predict whether a person will like films like that.

Most of the predictive recommendation systems entered in the Netflix Prize are reported to be quite similar - so we asked in January whether it was going to take a radical breakthrough to top 10% instead of just continued iteration.

That breakthrough may have come when the four teams put their heads together, or it may have been an iterative victory. Time and science will tell.

Some people believe that recommendation as a technology has the potential to be even bigger than search. In our favorite article on the subject, written eight-teen months ago now, Dr. Rick Hangartner, Chief Scientist at recommendation engine Strands, puts it like this:

In the near term, search engines will increasingly incorporate simple recommender technologies to handle approximate queries (e.g. "You asked for this, and based on similar queries/behavior by others, you might be looking for this."). But in the long term, the recommender industry will be larger, and recommender technologies will be more pervasive than the search industry and search technology as we know it. [Because there will be recommendation going on all over the web.]

When you also consider that the wisdom of the crowds is often dominated by small, powerful groups, then the validity of user ratings is further called into question. So why not just get rid of explicit user ratings and use implicit recommendations instead?

YouTube wants to know if a thumbs up/thumbs down system would be be more effective (two options), or even just favoriting (one explicit action to say you like an item).

However possibly a better option is to remove explicit ratings altogether. Does YouTube even need to ask its users for ratings, given the wealth of user interaction data it has?

Earlier this year, ReadWriteWeb profiled some sophisticated recommendation technologies which rely on implicit user data. Many of these systems track user data and, with a set of (usually proprietary) algorithms, come up with recommendations for users. This type of system could well replace ratings altogether in YouTube. While YouTube probably already makes use of the ratings data in its recommendations, as noted above such data is typically unreliable and not very valuable.

As an example of how this could work on YouTube, here is our description of Baynote's recommendation system:

"Baynote observes real-time user behavior on a site and looks for implicit, emergent patterns. It uses collective intelligence and an affinity engine to analyze the data. Common behaviors which it tracks include page refers, queries, mouse movement, time spent on a page, peer behavior (see note about communities below)."

Other similar recommendation technologies we've profiled include MyBuys, ATG and richrelevance.

Are explicit user ratings still valid in consumer apps such as YouTube and Amazon? While we're arguing that implicit recommendations data could enable YouTube to scrap user ratings altogether, on the other hand products like RateItAll are still built around the star system. Let us know your thoughts in the comments.

According to coverage today in Boston.com (via PaidContent), MatchMine starts by asking for demographic information about a user and asking us to rate a variety of sample media. A desktop application the company calls a "gumball machine", probably built in Adobe's AIR if MyMovieMatch is any indication, then lets users flip through recommended media items and learns from their ratings of each. The company aims to let users port their media preference profiles, called their MatchKeys, to a variety of sites around the web.

If AIR is at the center of the company's products, I presume this investment will go down as one of the first substantial AIR-centric investments to date. Though recommendation engines are one of the things I'm most excited about for the future, that seems like a big bet on a run-time that's so far no where near as ubiquitous as Adobe's other products. The AIR product is probably just one version of MatchKey, the company was at Seattle's 360Flex conference last month getting feedback on their Flex SDK.

DEMO events have seen media recommendation engines before and there are still a number of viable players in this space despite the struggles they've all faced. The startups with major financial backing, including Oregon based MyStrands ($25m) and now MatchMine with $10m in their coffer, are probably best positioned to work on the science and reach out to a world that has only begun to recognize the value of their services.

Other interesting recommendation startups recently launched include Scouta (video and podcasts, including an iTunes agent), Seeqpod (music, iPhone) and Thoof (all kinds of things).

For more on this topic, see Alex Iskold's excellent articles The Art, Science and Business of Recommendation Engines and The Attention Economy: An Overview.]]>Discuss]]> http://www.readwriteweb.com/archives/matchmine_raises_10m_for_ria_r.php http://www.readwriteweb.com/archives/matchmine_raises_10m_for_ria_r.php Startups Fri, 21 Sep 2007 09:44:26 -0800 Marshall Kirkpatrick YouTube has begun experimenting with a visualization of related videos that's a poor knock-off of the Digg Swarm visualization tool.

NewTeeVee says it's more fun than Digg Swarm but I think it's less useful and actually a bit nauseating to watch. (Update: By the end of the day, it's actually much improved.)

Here's a quick screenshot of recommended videos based on the Zombies in Plain English video linked to above. An interesting algorithm, is it not?

My Take on This Tool

There's a couple of things that I think are notable here.

First, the videos are different than the "related videos" in the sidebar of the YouTube page. Second, this visualization is really half-baked. What's the biggest take-away for me here, though? Recommendation and visualization are going to be major issues in the near-term future. In a world of information overload - effective systems of visualization will be small gold-mines and effective recommendation engines will be very large gold mines. That's why the company that made the "Plain English" videos, Common Craft, got hired to do product intro videos for Google - because visual explanation is a rare skill with big payoffs. That's also why it's not at all crazy that recommendation engine MyStrands has now raised more than $55 million. Recommendation and visualization are going to be key challenges for the future. More than just a neat little experiment, today's YouTube visualization is a peak into the future. The fact that they released something so half-baked into the wild just goes to show that visualizing recommendations is easier said than done.

Building a recommendation engine is a complex endeavor, which we discussed here a year ago. But in addition to being a technical challenge, there are also fundamental psychological questions: do people want recommendations and if so, then when are they open to them? Perhaps an even bigger question is: what happens when the user receives one or more bad recommendations? How tolerant will they be?

Genetics of Recommendation Engines

All recommendation engines are trying to solve the following problem: given a set of ratings for a particular user, along with those of the whole user base, come up with new items that this user will like. There are many algorithms that can be applied to the problem, but all of them focus on three elements: personal, social and fundamental:

• Personalized recommendation - recommend things based on the individual's past behavior
• Social recommendation - recommend things based on the past behavior of similar users
• Item recommendation - recommend things based on the item itself
• A combination of the three approaches above

A social recommendation is also known as collaborative filtering - people who liked X also like Y. For example, people who liked Lord of The Rings are likely to enjoy Eragon and The Chronicles of Narnia. The problem with this approach is that peoples tastes do not in reality fall into simple categories. If two people share the same taste in fantasy movies, it does not mean that they will also both like dramas or mysteries. A good way to think about this problem comes from genetics. Many times we meet people who have features that we recognize and have seen in others. For example, eyes might look familiar, or lips, but it is a totally different person.

The other kind of recommendation is an item-based recommendation. The best example of this system is the Pandora music recommendation service. It works by ranking each musical piece by more than 400 different characteristic - musical genes. It then automatically matches the pieces based on these characteristics. There are challenges with tuning the algorithm to work well, but it is also challenging to apply it to other verticals. For movies, for example, you'd need to come up with ranking each movie along many scales, starting from director, cast, plot; and then obscure things like musical score, locations, light, camera work, etc. It certainly can be done, but this is complicated.

The Guy In The Garage

The complexity of the recommendation problem is due to its vast space of possibilities. Much like it's hard to figure out which exact gene is responsible for a particular human trait, it is hard to figure out which bits of the movie or music make us rate it as 5 stars. Reverse engineering human thinking is hard. Which is exactly why one of the contestants highlighted in the Wired article is relying on a very different trick to make his algorithm work.

Nicknamed Guy In The Garage, Gavin Potter from London is relying on human inertia. Apparently, the rating of the movie depends on the ratings of previous movies that we just saw. For example, if you watch three movies in a row and rate them with 4 stars, and then watch the next one which is slightly better, you will rate it 5. Conversely, if you rate three movies in a row with 1 star, then the same movie that you would otherwise rate as 5 would only get 4 stars from you.

Just when you think that this is not true, you will discover that this algorithm now sits in the 5th place and still is making progress, while other algorithms are spinning. Enhancing formulas with a bit of human psychology is a really good idea and this is where we turn next.

Replacing Recommendations with Filters

How many times has this happened to you: a friend recommended you a movie or a restaurant, so you went there all excited - but ended up disappointed? A lot! It is obvious that hype sets the bar high, increasing the chances of a miss. In math speak, this kind of miss is known as a false positive. Consider now what would happen if instead of recommending a movie, a friend tells that you are not going to like certain movie, so do not bother renting it.

What bad can come of that? Not much, because likely you are not going to watch it. But even if you do and you like it, you are not going to be experience negative feelings. This example demonstrates the difference between our reaction to a false negative and a false positive. False positives upset us, but false negatives do not. The idea of respinning recommendations as filters is about leveraging this phenomenon.

When Netflix makes recommendations, it sets itself up for a sure failure. Sooner rather than later it is going to miss and recommend you a movie that you are not going to like. What if instead of doing that, it would show you new releases and have a button: filter the ones I am not going to like. The algorithm is the same, but perception is different.

Filters in Real-Time Culture

And this idea becomes increasingly important and powerful in the age of real-time news. We are increasingly oriented towards continuously filtering new information. We do this with our RSS Readers everyday. We think of the world in terms of streams of news, where things of the past are not relevant. We do not need recommendations, because we are already over subscribed. We need noise filters. An algorithm that says: 'hey, you are definitely not going to like that' and hide it.

If the machines can do the work of aggressively throwing information out for us, then we can deal with the rest on our own. Borrowing from the spam box in emails, if all the tools around us had a button that said 'filter this for me', and maybe even had a mode where such a filter is on by default, we'd all to get more things done.

Conclusion

Building a perfect recommendation engine is a very complex task. Regardless of the method, collaborative filtering or inherent properties of things - recommendations are an unforgiving business, where false positives quickly turn users off. Perhaps applying psychology to the problem can make people appreciate what these complex algorithms are doing. If instead of recommending things, machines would filter things we definitely won't like, we might be more forgiving and understanding.

Now tell us please about your experiences with recommendation engines. Were there ones that worked really well? Would you be open to filtering instead of recommendation? Besides movies and news, where would you like to have these filters?

See also our follow-up post 10 Recommended Recommendation Engines.