Now the app joins a handful of others (really, just a handful) on the new Palm Pre. But before you go and download this one, there's something you need to consider about LikeMe: their reviews may be compromised.

LikeMe on Pre

It's a shame, to be sure, that the quality of this app's content still remains under question as they launch their latest offering on the Pre, a device that certainly needs as many apps as it can get. The LikeMe app even has some great features that take advantage of exclusive Pre functionality, like the ability to send recommendations via MMS straight from the app to friends in your phone's contact list.

The webOS application also uses Pre's GPS service to identify nearby places that have been recommended by your friends and people like you. You can use the GPS feature to share your location with friends, too, turning LikeMe into a combo of a Yelp-like service and a mobile social network of sorts.

Six Months Later, the Questionable "Reviews" Remain

Unfortunately, the accusations about the reviews (or perhaps we should call them "ads") comprise the integrity of the site and make us question the quality of its content. Although we're sure nearly all review sites that rely on user ratings have some outside manipulation going on thanks to business owners who want to counter negative reviews, in this case the manipulation is more of an inside job. And the reviews that were called out in January as being suspect are still on the site today, so obviously the company either thinks they've done nothing wrong or they don't think anyone will know.

Given the small number of apps available for Pre, LikeMe has an opportunity to gain a foothold there due to a lack of competition. Perhaps that's really why they decided to launch exclusively on the Pre - not because of "its ability to multitask and unique points of integration," as their press release says.

But at this point, we think maybe Pre users would be better off opting for the mobile Yelp site instead.

UPDATE: Response from LikeMe:

"Here's the deal...in the beginning we used friends and family to start populating the community. That included VVM personnel and some people from the ad side.

You can also interact with the items being tracked via the Glue plugin which features a "like" button and another "2 Cents" button which lets you leave a comment about whatever it is you're viewing.

As with the previously released Firefox plugin, the Glue IE plugin also delivers the same type of interactions as you would expect: the connected conversations around everyday things, recommendations, and web-wide "top lists" that include the top items across the entire Glue network.

You can grab the Glue IE plugin from the main page of the Glue web site here. Note: the "Download" button still features the Firefox logo only at this time, but clicking the button reveals the IE download is available as well.

Disclosure: Alex Iskold (@alexiskold) is the founder of AdaptiveBlue, the company behind Glue, and occasional RWW feature writer.

Rating Items to Train the App

The first thing you'll notice when launching the app is that it features multi-colored news items just like the internet site. However, on the iPhone, these items don't automatically scroll as they do online - you scroll through them with your finger...and frankly, we prefer interacting with the content this way, if we have to be honest.

OhPan on the Web

OhPan on the iPhone

The individual news items feature the same "star" and "lightning bolt" buttons which are key to the recommendation system. On the iPhone, the buttons are even colored (the star is green, the lightning bolt is red), so it's even more obvious which one means "like" and "dislike." We wonder what they have against the traditional checkmark and "x" though? Oh well.

Logging In: Where's my Facebook Connect?

When you log into the app, it automatically creates an account for you so you don't have to go through a set up process to starting using it. That's a nice feature but we would have at least liked the option to log in via Facebook Connect or Gmail, like their web site offers.

Instead, if you want to create an account of your own, you can click on "Setup" at the top and fill in your info under "Ohpan Account." Unfortunately, this feature didn't work for us - buttons didn't respond in some cases and when they did (after clicking "New Account" for example and filling in name and email - twice, ugh, they want you to confirm it) the info didn't seem to be saved. Trying the "Log in with Existing Account" was also disappointing. Because we had always used Facebook Connect to login online and that was not an option on the iPhone, all our previous ratings with which we had already trained the app were no longer available - we had to start from scratch.

Localization Could be Better

The localization features of the app have potential, but are not ready for primetime yet, mainly because of the odd selection of locales. We could understand if the app only offered major metropolitan areas (NYC, LA, San Francisco, etc.) to start with, but their list of cities/locales to choose from is downright bizarre. NYC and LA are present, but other areas include Arizona, Baltimore, Buffalo, Calgary, Canada, Carolina (which one?), UK, Vancouver, Washington (State or D.C.?), Toronto, and a few others. We're still scratching our heads to figure out how they came up with that list. We're hoping that it's just a work in progress - the app is, after all, only days old. Perhaps the list will be updated as new versions are released.

There's also a setting that allows you to send your location to Ohpan, which is confusing since it doesn't seem to do anything. Do they want you to pick from a list of places or do they want to geo-locate you? It's unclear what this setting is for at the moment.

Communities are Cool and Quirky

Still, even with the annoying and broken login and localization features, we enjoyed using the app once we delved into the "Communities" feature under "Views." This lets you pick a more specific topic (Tech News, Sports, Entertainment, etc.) and view and rate the items within that one area. Again, there are some bizarre choices made here. Some of the topics include things like "Spaceship Earth," "Change of Course," "Party," and more...not your typical fare and not really self-explanatory. But in a way, it was kind of fun exploring these non-traditional categories, each of which you can filter by "latest," "today," "this week," or "All Stars."

You can also choose to view "Charts" from within the "Views" area which show you the top publishers, images, and items of today, this week, and all-time.

Of course, you can ignore this area altogether if you wish and just scroll through the stream on the main screen of the app where all the different type of content is mixed together, ready for your rating. When you find an article you want to read in more detail, clicking the "Source" button opens up the full webpage within the app. When you're done, click "Stream" to return to the list.

You can also forward the article via email or publish it to Twitter ("Publish Item to Twitter"). Another button simply reads "Publish Item." We pushed it and maybe something happened, but we have no idea where it went. We never gave the app any of our social networking credentials, so what was it doing?

Conclusion: Needs Work, but We're Keeping Our Eye on It

In the end, we felt like we were playing around with an unfinished application, but one that could be great. We wish the developers had taken more time to fix bugs, complete the features, and add finesse to the UI before launching. These days, people are going to judge the app in its current state, somewhat poorly, and then move on. There are so many apps in the App Store, our screens are getting full. If it doesn't work well from the get-go, it could easily be deleted from our phones within days if not hours.

Still, the idea of news stream based on your own preferences is one that's innovative and unique and would be a great alternative to an RSS reader for those of us who are tired of being reminded how many unread news items we have. For that feature alone, we'll keep our eye on this app and hope that over time we'll see improvements.

OhPan is available now in the iTunes App Store for $1.99.

The feature O'Neil wrote about appears to be nothing more than the latest FriendFeed rip-off: truncating repetitive activities. (Ex-Googler Paul Buchheit's FriendFeed is like a Facebook R&D lab without stock options.) Whether Facebook is doing more than that publicly or not, you know they have to be working on recommendation behind closed doors.

O'Neill wondered whether this feature might give the user an option to view other items from the same or other users that Facebook deemed similar to the original post. I say I'm drinking coffee and Facebook shows me a movie, a picture and another message about coffee from one of my friends this morning. That would be pretty awesome for us as users and it would increase ad impressions for Facebook.

More likely is the explanation offered by AllFacebook readers, that this new feature is just a way to scrunch up items that are basically the same so users can't spam their friends' newsfeeds and so that newsfeeds are more pleasing to scan down. In other words, in the image above, the two-headed person on top probably posted about "besplatno-ing" like four times in a row. Facebook decided to show just one of those messages and add a link to view the rest.

That's how FriendFeed does it and it works really well. This seems like a plausible explanation of this screenshot, but it's also a real lost opportunity. Facebook's corny "your friend is a fan of this advertiser's stuff" may be more creepy than compelling - but automated recommendations of all types of items could be great.

We're big fans of recommendation technologies here at ReadWriteWeb, from relatively simple "people who like X also like Y" to more complicated algorithms. The systems are fun to learn about, but the fact of the matter is that recommendation doesn't have to be hard. The hard part is amassing enough data and interested people to be able to make recommendations. Facebook has plenty of data and people, though its labyrinth of privacy restrictions might complicate things a little.

So if this isn't it, and we suspect it is not, we sure do hope that Facebook will soon surface the recommendation technology we assume they are working on behind the scenes.

Even more significant, few sites today are constructed solely from internal site resources. Hosted metrics, recommendations, news, store locators, stock tickers, friend followers, and so on and so on are rapidly deconstructing the whole notion of "place" through the active participation of the "web-fabric" layer of the web community.

From this perspective, most recommendation services are still stuck in the feudal worldview: the black box recommender knows you (whether "you" are a visitor or place-owner) better than you know yourself and determines, in its infinite wisdom and authority, what content should be presented to you. The place-owner may have some input into presentation and even, though less so, content, but only in a very limited way.

While this situation is useful in certain cases because of the total passivity afforded the place-owner and visitor, it severely limits the potential contribution of recommendation technology.

Personal, Real-Time Conversation

There is a broader view of recommenders, though. The business value of recommendations is that they bring the place-owner into a one-on-one, real-time, conversation with the visitor. As such, a recommender must be able to accommodate the active participation of both the place-owner and visitor. Recommenders play the role of the salesperson, the agent in the company who has one-on-one contact with each shopper. This is in contrast to the site designer, who is more akin to the display designer in a bricks-and-mortar store and who can only target segments of the population who are expects to pass the display, not individual shoppers. Recommendations are also narrower in concept than personalization tools, which are analogous to store greeters: they may personally greet you when you arrive, but they generally don't follow you through the store as you shop or interact with you in real time.

Okay, but why a conversation? Consider the typical interaction between a sales agent and shopper in a bricks-and-mortar store. The shopper enters the store and starts looking around. At some point, the sales agent asks, "Can I help you?" "No thanks, I'm just browsing," By this point, the sales agent has probably already observed the shopper and made some inferences about the shopper's intentions and receptivity and about associated sales opportunities. The shopper, in turn, has been assessing the store's inventory and pricing.

Like these sales agent, place-owners have a tremendous amount of knowledge about shoppers, sales tactics (like cross-selling, upselling), and their own business objectives, both short- and long-term. Much of this knowledge is unavailable to automated recommendation engines, no matter how much data they gather (and the ultimate prize for optimizing discounted infinite-horizon shopper value is computationally intractable even if we had the data). So, the recommender is better tasked to take advantage of the wisdom of the place-owner "in the moment." Of course, an uninformed recommender is just a degenerate case and may still be useful.

One advantage of the web is that transaction costs are low. Most place-owners can't afford to have human representatives in sessions. Most explicit communication by the place-owner must be in the form of policy or strategy, rather than actual real-time communication. (Notwithstanding this, interaction with a live sales agent may well be an appropriate option for a recommender to trigger in certain situations.)

Situation/Response

One way to think about this is like "situation/response". The situation description might cover visitor location, web page visited, catalog, date (e.g. if it is a holiday), place-owner internal item information (e.g. from a supplier catalog or internal access and sales statistics), visitor community information (e.g. sales ranking, review ranking), or even external information (e.g. Google search ranking, Amazon ranking). The response should be a specification over recommender behavior, as well as resulting recommendation content (e.g. show a pair of Nike's under $50), and presentation, both style and modality (e.g. use an animated GIF showing all available colors). Perhaps, as mentioned above, modalities even extend to bringing a live sales agent into the real-time conversation.

While limited work has been done on place-owner participation in recommendation-system content and presentation, the situation is far more dismal for the visitor. A broad array of modalities are available for visitor interaction, but few if any are available in most recommendation systems. A simple "No, that's not what I'm looking for" (e.g. a thumbs-up or thumbs-down icon on a recommendation thumbnail) might go a long way to making the shopper feel noticed and appreciated. I can say to a human store clerk, "I'm looking for a pair of Nike's under $50" -- why can't I tell the average recommendation system the same thing? Notice that this starts to overlap with the expressivity needed on the place-owner side. The main difference is that the visitor is always in the moment, so there is (usually) no need to specify context.

The above sketch is intended to crack open the door on the enormous range of possible capabilities, modes, and time-scales of participation by place-owner and visitor. Once we've opened this door, there is no reason not to open it to the visitor community and the web-fabric community as well. There are three primary points:

1. A place is no longer a feudal domain; all stakeholders now demand a voice.
2. A recommendation engine is the locus where understanding of content and understanding of the visitor-in-the-moment come together.
3. As a result, recommendations are the logical ground for crucial real-time conversations between place-owner and visitor.

Given our initial definition of place, we might also ask about the role of and opportunity for participation among other stakeholders. For example, can the interaction between the site designer and the visitor or web-fabric community also be viewed as an ongoing conversation, rather than an episodic, one-way information flow at the time of site design? The answer is yes, but that is a topic for another time.

Conclusion

Recommenders need to open up to allow increased place-owner, visitor, and community participation in both content and presentation. This is best done with the assumption that a recommender is meant to facilitate situated, in-the-moment conversation between the place-owner and visitor.

This was a guest post by Bruce D'Ambrosio, VP and Chief Architect, OnDemand Personalization at ATG, Inc. He was the founder of CleverSet, which was acquired by ATG. He is also a former Oregon State University computer science professor.

Why We Need This

Sites like Yelp, Amazon, the iTunes store, and others have been coming under fire for not having trustworthy reviews. Thanks to anonymous user IDs on some sites, reviewers can be anyone with any agenda. Often they are. On Lunch, however, those drive-by reviews contributed by someone associated with the company or product being reviewed (or with an axe to grind) will not be prominently featured. The reason? Lunch.com's Similarity Network.

The Similarity Network

The Similarity Network is probably the most important feature of this new community - without it, Lunch would just be just another Yelp. After signing up, you kick start the matching engine by playing "ExhilaRATE." Although that name is somewhat unintuitive, clicking the link takes you to a section of the site where you can - guess what? - rate things like movies, books, food, sports, politics, animals...whatever. The experience of rating items here is a lot like that of Amazon's recommendation engine. If you've ever killed a few minutes on Amazon training it to get to know you better, you'll find Lunch.com's engine fairly similar.

The difference is that Lunch.com's engine groups things to rate into categories with titles that sound a lot like Facebook Apps (Top Movies of 2009, What's your Favorite Wine?). The Facebook flavor to these "games" makes sense because in the future, Lunch.com will launch a Facebook connected-experience, perhaps even a standalone app. In the meantime, however, you must go to the site to rate items.

The more you rate on Lunch, the better your matches become. You can see your matches and the percentage of compatibility between you and those like you. There are also tag cloud displays that show what items you both like and which ones you don't.

With Lunch, You Can Rate Anything

If you're still wondering why you would migrate away from more mainstream sites to something like Lunch.com, there's another reason this particular community holds appeal: it allows you to make anything ratable. Again unlike Yelp, ratings don't have to focus on products, services, places, etc. They could also be opinion pieces - like what you thought of Michelle Obama's new outfit for example. That opens the door for a much wider range of recommendations and - since you're matched with those like you - those recommendations will be relevant to your interests.

Lunch.com is in private beta, but you can try it now with the invite code "ReadWriteWeb." To use it, just click the link on the right-hand side of the screen that says "Have an invite code?"

]]> Discuss]]>

Since the site looks at a number of variables in the voting process, not just the specific story that's voted on, the system can adjust in a number of dimensions behind-the-scenes to get you the type of stories you want to read on a particular subject.  For example, if you consistently vote up stories from a certain source (either one of their generic 'watcher' sources or a particular user), you will end up seeing more stories from that user in the future. If you consistently vote a particular story type up (say stories with a conspiracy theory slant), you will end up seeing more of those types of stories.

From what we can tell, the system Veritocracy is going for will benefit overall with the input of as many people as possible. And, since it is a destination site with its voting features (as opposed to a news site that you can grab a feed from and never visit again), active participation is key. We found the process of looking and voting for news overall smooth and enjoyable, the product is very polished and we definitely recommend you go and check it out. It may just become the first place you visit on the web for news.

Loomia is similar to Sphere and another app we reviewed recently, Apture. These services all aim to serve up more clickable content options on media websites - which means more user engagement and time spent on site for publishers.

We spoke to Loomia CEO David Marks and asked him how Loomia compares to Sphere, which at first glance appears to have much in common with Loomia. Marks said that Sphere is trying to do "semantic classification", i.e. analyzing the content of an article and recommending further content based on the findings. However Loomia focuses more on the user and so it does behavioral type recommendations. This can result in a more diverse set of topics, because users typically have a range of content preferences. It depends on the article though, said Marks.

Loomia currently has 2 types of deployment:

• Content (e.g. WSJ)
• Video (e.g. Brightcove)

Marks told ReadWriteWeb that video advertising is currently selling well for big media publishers. Accordingly these publishers typically now want to drive users to their videos - and Loomia has a widget to do that.

Marks told us that a lot of their publishers are "dollar focused" this year, therefore recommendations have become more than just an interesting feature on a website - they can drive more advertising dollars. As an example, Marks told us that a media website's Finance section may sell out with ads, but its Politics section may not (fairly common in big media websites). But the Politics section tends to get bigger page views, so to address the imbalance Loomia's recommendations widgets can drive users from Politics to Finance.

We've been looking at how recommendations are being used in the retail sector a lot, and Loomia is a neat example of how the same technology can have real value for the media segment. Let us know in the comments what other recommendation technologies have caught your eye in publishing.

MyBuys' Technology

The services approach means that MyBuys deploys a variety of algorithms and doesn't favor one approach - unlike, for example, ATG, which uses a method it calls "Statistical Relational Learning" (SRL). This is really the crux of the difference between MyBuys and the other companies we've profiled so far. The likes of richrelevance, ATG and Baynote all have a defining technology (usually patented) which for each is the foundation of its recommendations approach.

MyBuys has no specific algorithmic approach. Rather it appears to license technologies from companies such as Blue Martini, BroadVision, MarketLive and Microsoft. However MyBuys does still have a patent on the technology which brings all these disparate algorithms together - it calls it a "patented portfolio of algorithms".

MyBuys' recommendations are a javascript include for their clients' websites - i.e. the heavy lifting is done on MyBuys' servers. Their clients can see their stats in a MyBuys portal, and also summary stats are emailed to the clients.

Understanding Consumers

MyBuys has a team of people that focuses on site performance for its retail clients. This team - which works across all of MyBuys' client base - focuses on driving performance using a variety of tools and processes. They also do experiments for clients to find out what works best. Rosenblum noted that MyBuys is almost always paid on performance.

On its website, MyBuys says that it "creates deep consumer profiles based on both explicit information we collect from you and from shoppers when they sign up for alerts and implicit information we collect as shoppers interact with your site." Rosenblum claims that MyBuys "understands consumers at a deep level", whereas he said that its competitors don't necessarily do. He told us that the Baynote approach is "strange" because they don't focus on the individual, but rather the 'wisdom of crowds' (which he said is a 'lowest common denominator' approach). Further, Rosenblum claimed that many of MyBuys' competitors don't understand the product catalog, that they suffer from the "cold start" problem - i.e. with a new product there is no place to start, unless you know about consumer retail behavior.

Side note: we're sure that MyBuys' competitors would disagree with some of the above assertions, so we welcome feedback from them in the comments below. One thing we've found in this series is that each company in this space is very willing to talk down their competitors! A sign of a very competitive market.

Examples

On to examples of MyBuys' approach. One is World Market, a retailer of furniture and other goods from around the world. It has a 'May we recommend' section on its homepage, which Rosenblum told us is based on MyBuys' algorithms and what other people have done on the site. After the user hits the homepage, MyBuys tracks that user - they know where they came from, they pay attention to what the user clicks on next, and so on. On product pages, there are a variety of different recommendations on the right of the page under the heading 'More great finds'. The categories under this heading can differ (e.g. for some products there may be no 'featured' recommendation).

Another example is Golf Galaxy, a web retailer of golf gear. This has recommendations such as "Other great ideas" and "People also bought". It also serves up recommendations in the shopping cart: "You may also like".

MyBuys doesn't just do website recommendations, it uses email a lot. If they know the email address of the customer, they will send follow-up emails (e.g. if a user abandons the shopping cart). Rosenblum told us that this works very well, however he assured us that emails are 100% opt-in. He said that for every dollar MyBuys drives through the site, another dollar comes through the email channel.

Conclusion

So how effective are MyBuys' recommendations? According to the company, when recommendations engage consumers (i.e. a user clicks on a recommendation), they're 5 times more likely to convert than when there are no recommendations. Rosenblum told us that its clients see an increase of overall site revenue between 5-20%, which is a similar figure to that which other recommendations vendors have given us. The addition of email usually results in even higher conversions, the company claims.

As to how MyBuys compares to its competitors, as we've noted in previous reviews it's very difficult to make a judgment on that. However we're interested to note that a recommendations vendor can compete well in this market without having its own unique patented algorithm. MyBuys pushes the 'services' approach much more than the other vendors. We're sure that some of MyBuys' claims about the competition would be challenged, nevertheless it appears to be a successful business in the retail recommendations market.

There's a certain seductive attraction to the idea that collaborative filtering and artificial intelligence could hand us our heart's desire before our hearts even think of it. Think of the time and effort I could save if I didn't have to make decisions about what to eat, buy, wear, listen to, watch or read. When it comes right down to it, free will is a genuine time suck, and seriously cuts into my blogging schedule.

More Noise to Signal

As we noted in our previous post, our series on recommendation engines has shown that every company in this market - including those which create their own platform, like Amazon and Netflix - have differing approaches and ideas on what makes a good recommendation engine. The key to richrelevance's approach, Selinger told ReadWriteWeb, is that people don't shop the same and so different recommendation types will be used for each shopper. This is markedly different from Baynote's approach, which specifically excludes a user's past shopping behavior and instead focuses on real-time community patterns.

In the worldview of richrelevance, shoppers at Amazon are different to the ones at Sears - one of the companies using richrelevance's technology. Furthermore, a person who has a shopping history at a store is different from someone who is totally new to that site. So, unlike Baynote, richrelevance takes into account a user's purchase history - if known.

If we look at an example from Sears, on this item page the richrelevance recommendations display in two places: on the left there is a 'People Who Viewed x Also Viewed' box, and at the bottom of the page there is a 'Top Sellers' section. Selinger told us that if a Sears user has a long purchase history, then they will see recommendations in Sears based on that. We asked if they need to be logged in to Sears as a registered user, but Selinger told us that it is cookie-based and so doesn't take into account their registered Sears user profile.

richrelevance's Technology: Is it Better Than Baynote's?

We were curious about why richrelevance thinks its approach is superior to those that exclude personal user bahavior, like purchase history (such as Baynote). Selinger told us that richrelevance constantly runs A/B tests, just as Amazon does, in order to find out what the most effective methods of recommendation are for any given customer (e.g. Sears) or individual user. This approach leads to using a mix of 'wisdom of the crowds' and 'personalization'.

The theory is that the consumer will tell you what kind of recommendations they like - e.g. at Sears users may like item-based recommendations in certain products, but personalization in other products. Selinger used the analogy of the 'personal shopper'; richrelevance tries different ways to help users shop, finding the best way by trial and error. There are different types of interaction for each customer, said Selinger.

We asked how this approach works for a brand new customer, because presumably there will be no shopper history to use. Selinger replied that richrelevance "works good straight away, but takes a while to get great recommendations". So they may start out with a new customer using existing data that richrelevance owns (e.g. from a similar vendor), and then gather and test data about the new customer.

As for the technology behind richrelevance, David Selinger has termed it "ensemble learning". In a recent blog post, in response to ReadWriteWeb's Guide to Recommender Systems post, David Selinger wrote that "no 'single algorithmic' approach can hope to keep up with today's ever-changing consumer mindset", so richrelevance doesn't try to force retailers and consumers "into a single bucket". Instead Selinger says that richrelevance has "built a system that adapts to the retailer and to each customer in real-time", which is done via "an adaptive type of artificial intelligence called Bayesian Ensemble Learning."

In a comment on a recent RWW post, Selinger claimed that "algorithms like collaborative filtering are a thing of the past" and that ensemble learning is the next generation beyond that.

Conclusion

Ultimately, only the customers of richrelevance and Baynote know if their recommendations are working. Both companies claim that their technology results in higher sales for their e-commerce customers - richrelevance says it results in a "5%-30% sustained sales lift" for its customers. It's difficult for ReadWriteWeb to corroborate those kinds of figures. What we do know is that Baynote is more focused on community behavior, whereas richrelevance takes both community and personal data into account - including purchasing history, which Baynote excludes.

We get the impression that richrelevance's approach is very broad - perhaps too broad? In an email thread with David Selinger, he told us about some of the different forms of recommendations:

"...there can be basic contextual (you're looking at Adidas, here's more Adidas) or social contextual (you're looking at these Adidas, people who looked at it eventually bought this); you can have basic behavioral (yesterday you looked at Adidas, here's more Adidas), or social behavioral (yesterday you looked at these 10 things, people who looked at those eventually ended up buying one of these 3 things); or basic profile (here's something from your wishlist) to social profile (you seem to like rock music, here's some new rock music)."

That's a lot of data that richrelevance is trying to process, in real time. Let us know in the comments whether you like richrelevance's adaptive approach to recommendations, or whether you think Baynote's more focused approach is better.

We had the opportunity to speak with Satnam Alag, author of the recently published Collective Intelligence in Action, about what makes for a good recommendation system, where the technology is heading, and why Netflix is finding it so hard to improve its own system.

ReadWriteWeb: In our recent post about Netflix, we identified four main approaches to recommendations: Personalized recommendation: based on prior behavior of the user; Social recommendation: based on prior behavior of similar users; Item recommendation: based on the item itself; And a combination of all three. Do you agree with the four approaches we laid out in our article?

Satnam: Those four categories are pretty comprehensive. I present an alternate classification of recommendation systems in my book. I lay out two fundamental approaches. The first approach, item-based analysis, determines items that are related to a particular item. When a user likes a particular item, related ones are recommended. The second approach, user-based analysis, first determines users who are similar to that user.

Further, there are two main approaches to finding similar items and similar users. For the first, content-based analysis, content associated with the item, especially text, is used to compute similarity. In the second, the collaborative approach, actions such as ratings, bookmarking, and so forth are used to find similar items. For the second, user-based analysis, a number of approaches have been taken, including ones based on profile information, user actions, and lists of the user's friends or contacts. Of course, you can combine any these item/user and content/collaborative approaches to build a recommendation system.

The dimensions of the particular item and user space are helpful in deciding whether to use an item-based or user-based approach. Typically, an item-based approach is used to bootstrap one's application when the number of users is small. As the user base grows, the item-based approach is augmented by a user-based approach.

ReadWriteWeb: Other than Amazon and Netflix, which Internet companies have most impressed you in their implementation of recommendation systems?

Satnam: Other than Amazon and Nextflix, Google News' personalization is my personal favorite. Google News is a good example of building a scalable recommendation system for a large number of users (several million unique visitors per month) and a large number of items (several million new stories every two months), with constant item churn. This is different from Amazon's, whose rate of item churn is much lower. Google decided to use collaborative filtering for its recommendation system mainly because of its access to the data of its large user base and because this same approach could be applied to other applications, countries, and languages. A content-based recommendation system perhaps could have worked just as well, but may have required language- or location-specific tweaking. Google also wanted to leverage the same collaborative filtering technology to be able to recommend images, videos, and music, for which it's more difficult to analyze the underlying content.

Among start-ups, my personal favorite is the one we are developing at my current company, NextBio. It's not available yet but should be next month. The key point about this particular recommendation engine is its strong use of an ontology, similar in concept to tags, to develop a common vocabulary for items and users. The system then makes use of profile information and user interactions, both short- and long-term, to provide recommendations. The system leverages both item- and user-based approaches.

ReadWriteWeb: What commercial opportunities do you forsee with recommendation systems over the next few years?

Satnam: A good personalized recommendation system can mean the difference between a successful and a failed website. Given that most applications now invite users to interact and to leverage user-generated content, new content is being generated at a phenomenal rate. Showing the right content to the right user at the right time is key to creating a sticky application. I would be surprised if most successful websites did not leverage recommendation systems to provide personalized experiences to their users.

ReadWriteWeb: Your book includes a discussion of collaborative filtering. Can you tell us a bit about how this fits into the overall picture of recommendation systems?

Satnam: In recent years, an increasing amount of user interaction has provided applications with a large amount of information that can be converted into intelligence. This interaction may be in the form of ratings, blog entries, item tagging, user connections, or shared items of interest. This has led to the problem of information overload. What we need is a system that can recommend items based on the user's interests and interactions. This is where personalization and recommendation engines come in.

In my book, I take a holistic view of adding intelligence to one's application, a recommendation engine being one way to do it. The book focuses on both content-based and collaborative approaches to building recommendation systems. It focuses on capturing relevant information about the user, information from both within and outside one's application, and converting it into recommendations. One of the things you mentioned in your write-up on recommendation systems is that you would like to apply such a system to your website to recommend things to users. Someone reading my book should be able to create such a system using the techniques I demonstrate.

Next Page: Satnam's thoughts on the Netflix Prize and whether the 10% mark will ever be reached.

ReadWriteWeb: Netflix is offering $1 million to the team that can improve its recommendation algorithm by 10%. It's been over 2 years now, with the leading company at 9.63%. There is some skepticism, though, that 10% will be reached anytime soon, because now the contestants are making only incremental progress. Do you expect the 10% mark to be reached soon?

Satnam: Netflix's recommendation engine, Cinematch, uses an item-to-item algorithm (similar to Amazon's) with a number of heuristics. Given that Netflix' recommendation system has been very successful in the real world, it is pretty impressive that teams have been able to improve on it by as much as 9.63%. Of course, the Netflix competition doesn't take into account speed of implementation or the scalability of the approach. It simply focuses on the quality of recommendations in terms of closing the gap between user rating and predicted rating. So, it isn't clear whether Netflix will be able to leverage all of the innovation coming out of this competition. Also, the Netflix data doesn't contain much information to allow for a content-based approach; it's for this reason that teams are focusing on collaborative-based techniques.

The challenges to reaching the 10% mark are:

Skewed data: The data set for the competition consists of more than 100 million anonymous movie ratings, using a scale of one to five stars, made by 480,000 users for 17,770 movies. Note that the user-item data set for this problem is sparsely populated, with nearly 99% of user-item entries being zero. The distribution of movies per user is skewed. The median number of ratings per user is 93. About 10% of users rated 16 or fewer movies, while 25% of users rated 36 or fewer. Two users rated as many as 17,000 movies. Similarly, the ratings per movie are also skewed: almost half the user base rated one popular movie (Miss Congeniality); about 25% of movies had 190 or fewer ratings; and a handful of movies were rated fewer than 10 times.

The approach: The winning team, BellKor, spent more than 2,000 combined hours poring over data to find the winning solution. The winning solution was a linear combination of 107 sets of predictions. Many of the algorithms involved either the nearest-neighbor method (k-NN) or latent factor models, such as SVD/factorization and Restricted Boltzmann Machines (RBMs).

The winning solution uses k-NN to predict the rating for a user, using both the Pearson-r correlation and cosine methods to compute the similarities, with corrections to remove item-specific and user-specific biases. Latent semantic models are also widely used in the winning solution.

The BellKor team found it important to use a variety of models that compensated for each other's shortcomings. No one model alone could have gotten the BellKor team to the top of the competition. The combined set of models achieved an improvement of 8.43% over Cinematch, while the best model -- a hybrid of k-NN applied to output from RBMs -- improved the result by 6.43%. The biggest improvement by LSI methods was 5.1%, with the best pure k-NN model scoring below that. (K for the k-NN methods was in the range of 20 to 50.) The BellKor team also applied a number of heuristics to further improve the results.

The BellKor team demonstrates a number of guidelines for building a winning solution to this kind of competition:

• Combining complementary models helps improve the overall solution. Note that a linear combination of three models, one each for k-NN, LSI, and RBM, would have yielded fairly good results, an improvement of 7.58%.
• A principled approach is needed to optimize the solution.
• The key to winning is building models that can accurately predict when there is sufficient data, without over-applying in the absence of adequate data.

The final solution will be along the same lines, combining multiple models with heuristics. Contestants will probably reach the magic 10% mark in the next year or two.

ReadWriteWeb: Some people think the 10% mark can't be reached with algorithms alone, but that the "human" element is required. For example, ClerkDogs is a service that hires actual former video-store clerks to "create a database that is much richer and deeper than the collaborative filtering engines." It's a similar approach to that of Pandora, which has 50 employees who listen to and tag songs. How far do you think algorithms can go in making recommendations?

Satnam: Recommendation systems are not perfect. A number of elements go into making successful ones, including approach, the speed of computing results, heuristics, the exploration and exploitation of coefficients, and so on. But it has been shown in the real world that the more personalized you can make recommendations, the higher the click-through rate, the stickier the application, and the lower the bounce rate.

Using humans to form a rich database for recommendations may work for small applications, but it would probably be too expensive to scale. I don't see them competing against each other, human versus machine. Even with human/expert recommendations, one first needs to find a human/expert with tastes similar to those of the user, especially if you want to go after the long tail.