To achieve this, we must first stop arguing over the different flavors of search.

Because the promise of social network integration with search is a current favorite topic, we'll focus in this post on that: a class of social search. This is also a response to the ideas brought up by Alex Iskold in his post on the future of search.

Alex proposes that we rank search results by a kind of Social Relevancy Rank, first displaying results from friends and people whom we follow and later displaying results from "taste neighbors" and influencers, etc. FriendFeed already filters results by your friends' content first. Twitter's Trending Topics, by contrast, shows the crowd's perspective. While one's personal social circle could improve the relevance of some search results (and I noted some months back that this is a promising model), this type of filtering is more challenging than it sounds.

First, as Alex points out, "trusted opinions are scarce." Our friends couldn't possibly know everything we're interested in, and the smaller our social circle, the worse the problem becomes. Even with large social graphs, sooner or later we will undoubtedly search for a topic that hasn't been indexed in our friends' activity streams, and then we'll get few to no results and suffer an inferior user experience. We'd be better off turning to good ol' Google... the very thing we're trying to best!

Secondly, getting Social Relevancy Rank right involves a lot of insight into what users care about. Alex comments that, "This is not difficult for FriendFeed to do because... it knows who you care about." But does it? On FriendFeed, I follow only a limited number of the people I actually care about. Do those people alone account for the things I care about? And when I perform a search, does the engine know what I'm caring about at that moment? True, we have to start somewhere -- as PageRank did -- and tweak the algorithm over time. But suggesting that even a smart Social Relevancy Ranking is clued in to what we care about at any given moment is presumptuous at best given the state of the art.

Yet, having different levels of social relevance is a good theory, and Alex's demarcations are sound, in essence. But each level more likely indicates degrees of social proximity than relevance per se; although in some cases closer proximity may very well indicate greater relevance. The problem is that relevance is highly contextual. It depends on many factors, such as your profession, your search query, your friends, your friends' knowledge about those topics, and the information that is publicly recorded in their activity streams.

For example, a financial analyst (i.e. an expert) wouldn't care if her closest circle of friends was Twittering about how complicated a new tax code is. As an expert, she'd rather know exactly how the new policies affect an edge-case client of hers. Filtering search results by "friends and following" at one end and "the crowd in aggregate" at the other may fail equally in uncovering the right piece of information for her.

For general users, the "it depends" factor may be the urgency with which information is needed. When the need is urgent, people will actively search for the information (in any number of ways); other times, information may be welcome but only encountered serendipitously or consumed passively. Browsing feeds, Twitter posts, and Facebook streams are all passive ways of discovering information. Putting these activities on a continuum in which information search is active but information discovery is passive could look like this:

But to actually achieve a "Social Relevancy Rank," we have to consider how layers of social proximity map onto this search-discovery continuum.

When people actively look for a piece of information (e.g. the best Barbary Coast Trail guide for tomorrow's hike), they likely require trustworthy, high-quality information that could at least inform their decision. "Friends and following" could serve as a reliable social filter at this stage. But as the urgency subsides (e.g. just poking around for a mint julep recipe a week before a get-together), we relax our requirements and even welcome a wider set of results. At this stage, filtering results by friends of friends, influencers, experts, and even crowds in aggregate is appropriate.

Of course, serendipitously discovering information from "friends and following" would be welcome in other instances. So, to actually improve social relevancy in search engines and discovery services, there would have to be a distribution of acceptable social filters whose levels depend on how active the user is and what the user is searching for:

What this still fails to address, though, is how to assess the urgency of a user's needs or how to derive that level of urgency from the user's known behavior. This is a problem that engineers, designers, and HCI researchers have been struggling to solve for a long time (and a million dollars will get you only so far).

The problem of effective search runs deep. You can have all the flavors you want -- social, real-time, semantic -- and tomorrow's flavor will be merely another riff on the same tune. Yes, social networks and the social graph have the potential to meaningfully filter millions of otherwise undifferentiated pages of results. But words like "meaningful" and "relevance" are so contextualized -- varying as they do from user to user and usage case to usage case -- that they can't be expected to mean anything unless they are anchored by context. Mapping social proximity to users' active and passive information consumption could help us create more contextualized user experiences on the social Web, resulting in less time spent naming the latest flavor of search and more time spent actually improving search.

Guest author: Brynn Evans is a PhD student in Cognitive Science at UC San Diego who uses digital anthropology to study and better understand social search.

Real-time Web search (of streams of activities) is a hot topic right now. Everyone, including Google and Microsoft, recognizes the value of using trusted contacts as filters. What was once called social search is now called real-time search, but this time it will really happen. First, it will be applied to streams and then to the Web in general.

What we are about to get is a Social Relevancy Rank. Whenever you search streams of activity, the results will be ordered not chronologically but by how relevant each is to you based on your social graph. That is, people who matter more to you will bubble up. How does this work? Well, there will be a formula, just as there is a formula for Page Rank.

Here is an idea so obvious that it is surprising Twitter has not implemented it already: front-load search results with people you follow. When you search for, say, "Wilco" on Twitter today, the results are in the chronological order. That is not really relevant because you do not know who most of these people are. But if instead you could see people you follow, the search results would be much more useful.

This is not possible on Twitter today, but it already works great on FriendFeed. There, results are filtered or ranked based your social graph. This is not difficult for FriendFeed to do because, on the one hand, it knows who you care about and, on the other, it applies its advanced feed search technology to your social graph:

This sounds awesome, but there is a problem. "Wilco" works well as a query because the band has just released a new album, but many other queries would return no results. Simply put, your friends on Facebook and people you follow on Twitter can't possibly have an opinion on every topic you may be interested in. This is a problem of sparse data: trusted opinions are scarce.

Small Worlds and Taste Neighbors

To solve the problem of sparse data, we need more data... obviously. One possible solution is to incorporate other sources that you trust (i.e. broaden your social graph). As a next step, search results could rank people you may not be directly following but who are being followed by people you follow. Or in Facebook-speak, friends of friends. You could argue that you are not familiar with their opinions and so cannot yet trust them, but given the small world phenomenon, their contributions are often just as valuable.

Another step could be to include people with similar tastes, so-called taste neighbors. This approach is common among vertical social networks such as Last.fm, Flixster, and Goodreads. These networks have ideas about which people, other than your friends, are like you. However, this is a costly calculation and takes time. In order for Twitter to do something like this, it would have to compare people based on links or perform semantic analyses of tweets over time. Yet even though this is a difficult problem, it will be solved in time.

The Influencers and the Crowd

Aside from using the "second degree" of your social graph or taste neighbors, a Social Relevancy Rank could front-load influencers. In the absence of any other metric, someone who is followed by hundreds of thousands of users is likely more relevant to you than someone you don't know at all. Using number of followers as a weight might be a good way to order the rest of the activity stream.

In general, combing through countless tweets from strangers is not terribly useful anyway. Just as people have stopped looking at anything beyond the first page of results on Google, sifting through pages of tweets in chronological order gets tedious quickly. What needs to be incorporated into the Social Relevancy Rank is the aggregate sentiment of the crowd: a score that tells you yay or nay and gives you an opportunity to drill into more results if you choose.

The Quest for the Perfect Filter

There is no such thing as a perfect formula. Even Page Rank isn't perfect. Yet we all use it and find it useful. Much as Page Rank has been adapted and tuned to search the web, Social Relevancy Rank will evolve over time to help us make sense of endless streams of activity. This ranking will have a profound impact on how we tap into our friends' opinions.

It will change the face of general Web searches in time, too. Today, results are automatically ranked by relevancy and freshness. Once Social Relevancy Rank is factored in, search results will be re-ordered based on social relevancy.

And now, as always, please tell us what you think? What would you expect from a search engine with Social Relevancy Rank built in?