Social Relevancy Rank: What's Missing?
The future of search almost certainly involves social networks, social graphs, or social filtering in some capacity. Companies will live or die by whether they get the "social" part right: creating the right level of intimacy, trust, reliability, social connectedness, and accuracy in their results listings. Of course, this specifically means that their user experience must at least meet or, preferably, exceed that of Google's.
To achieve this, we must first stop arguing over the different flavors of search.
Real-time search. Social search. Semantic search. These distinctions are essentially meaningless, especially when we can't even agree on definitions and when each of their boundaries remain undefined. Instead, we should recognize that they're all part and parcel of personalizing and contextualizing search for individual users. Let's stop playing the "name game" and start thinking holistically about how each (and all!) affects and improves what we think of today as "search."
Because the promise of social network integration with search is a current favorite topic, we'll focus in this post on that: a class of social search. This is also a response to the ideas brought up by Alex Iskold in his post on the future of search.
Alex proposes that we rank search results by a kind of Social Relevancy Rank, first displaying results from friends and people whom we follow and later displaying results from "taste neighbors" and influencers, etc. FriendFeed already filters results by your friends' content first. Twitter's Trending Topics, by contrast, shows the crowd's perspective. While one's personal social circle could improve the relevance of some search results (and I noted some months back that this is a promising model), this type of filtering is more challenging than it sounds.
First, as Alex points out, "trusted opinions are scarce." Our friends couldn't possibly know everything we're interested in, and the smaller our social circle, the worse the problem becomes. Even with large social graphs, sooner or later we will undoubtedly search for a topic that hasn't been indexed in our friends' activity streams, and then we'll get few to no results and suffer an inferior user experience. We'd be better off turning to good ol' Google... the very thing we're trying to best!
Secondly, getting Social Relevancy Rank right involves a lot of insight into what users care about. Alex comments that, "This is not difficult for FriendFeed to do because... it knows who you care about." But does it? On FriendFeed, I follow only a limited number of the people I actually care about. Do those people alone account for the things I care about? And when I perform a search, does the engine know what I'm caring about at that moment? True, we have to start somewhere -- as PageRank did -- and tweak the algorithm over time. But suggesting that even a smart Social Relevancy Ranking is clued in to what we care about at any given moment is presumptuous at best given the state of the art.
Yet, having different levels of social relevance is a good theory, and Alex's demarcations are sound, in essence. But each level more likely indicates degrees of social proximity than relevance per se; although in some cases closer proximity may very well indicate greater relevance. The problem is that relevance is highly contextual. It depends on many factors, such as your profession, your search query, your friends, your friends' knowledge about those topics, and the information that is publicly recorded in their activity streams.
For example, a financial analyst (i.e. an expert) wouldn't care if her closest circle of friends was Twittering about how complicated a new tax code is. As an expert, she'd rather know exactly how the new policies affect an edge-case client of hers. Filtering search results by "friends and following" at one end and "the crowd in aggregate" at the other may fail equally in uncovering the right piece of information for her.
For general users, the "it depends" factor may be the urgency with which information is needed. When the need is urgent, people will actively search for the information (in any number of ways); other times, information may be welcome but only encountered serendipitously or consumed passively. Browsing feeds, Twitter posts, and Facebook streams are all passive ways of discovering information. Putting these activities on a continuum in which information search is active but information discovery is passive could look like this:
But to actually achieve a "Social Relevancy Rank," we have to consider how layers of social proximity map onto this search-discovery continuum.
When people actively look for a piece of information (e.g. the best Barbary Coast Trail guide for tomorrow's hike), they likely require trustworthy, high-quality information that could at least inform their decision. "Friends and following" could serve as a reliable social filter at this stage. But as the urgency subsides (e.g. just poking around for a mint julep recipe a week before a get-together), we relax our requirements and even welcome a wider set of results. At this stage, filtering results by friends of friends, influencers, experts, and even crowds in aggregate is appropriate.
Of course, serendipitously discovering information from "friends and following" would be welcome in other instances. So, to actually improve social relevancy in search engines and discovery services, there would have to be a distribution of acceptable social filters whose levels depend on how active the user is and what the user is searching for:
What this still fails to address, though, is how to assess the urgency of a user's needs or how to derive that level of urgency from the user's known behavior. This is a problem that engineers, designers, and HCI researchers have been struggling to solve for a long time (and a million dollars will get you only so far).
The problem of effective search runs deep. You can have all the flavors you want -- social, real-time, semantic -- and tomorrow's flavor will be merely another riff on the same tune. Yes, social networks and the social graph have the potential to meaningfully filter millions of otherwise undifferentiated pages of results. But words like "meaningful" and "relevance" are so contextualized -- varying as they do from user to user and usage case to usage case -- that they can't be expected to mean anything unless they are anchored by context. Mapping social proximity to users' active and passive information consumption could help us create more contextualized user experiences on the social Web, resulting in less time spent naming the latest flavor of search and more time spent actually improving search.
Guest author: Brynn Evans is a PhD student in Cognitive Science at UC San Diego who uses digital anthropology to study and better understand social search.
Facebook
Comments
Subscribe to comments for this post OR Subscribe to comments for all ReadWriteWeb posts
A little bit dense, but definitely good information. I would ask, however, what it really means to drop the "name game" in terms of search models. I'm rather doubtful that any search service could be all things to all people; isn't it better for a service to specialize in a certain "type" of search?
I'm wondering why we need a ranking at all? The whole point of relevance is to find an audience of people who share or are interested in your ideas, area of interest, region, like-dislikses etc. So who cares what your rank is? If your site/blog is optimized for your area of interest, they will find you via search engines or WOM. This whole business of being #1 is overrated, over dramatized on the Web and reality TV, and frankly, it's time to get over it.
Intuitively, this seems backwards to me.
For example, if I'm actively looking for information via search, it's probably going to be from the broadest and least personally connected source (i.e. "The Crowd" in this case). For if I knew a friend had first-hand knowledge of the information, I would be contacting them directly instead of searching.
Conversely, I'm more likely to passively find information from close personal connections. They mention that they like X, and if it catches my interest, I ask them or do more research on X.
Basically, the closer the connection, the more likely they are to bring new information we may not actively be looking for, and the more specific the information we are looking for, the more likely we are to take into consideration information from less-connected sources.
Greg - but what if you didn't know that a friend was also into (X)? Or that a friend of theirs was, in fact, an authority on (X)? The idea of search is to find things without needing to know precisely where the answers are. Making search better is pretty much a matter of finding the best answers. In that formulation, deciding on what 'best' means and how to surface it is the issue.
One thing the post didn't touch on that I'd like to hear about is that some search results are basically equivalent - the mint julep recipe for example. There are only a few variations on that and the difference between them is minimal so you can rely on the Crowd segment pretty easily. The difference between some random person's recipe and that of your best friend isn't going to be significant.
However, a recommendation on the best hotel to stay at in some city probably will benefit from closer social proximity. If a friend of mine tells me that Hotel N is awesome, in a cool area of town and has great breakfasts, I can judge that far better than s Yelp review that simply tells me a lot of people liked it. I might end up with a good hotel either way, but the friend's recommendation is more likely to match my idiosyncrasies. The question then is how many degrees of friendship are relevant? A close friend's recommendation might be significantly better than the crowd's, but will that be true of a friend of a friend who doesn't know me?
It's interesting to see discussions on the future of search coming to the point where what we're essentially talking about is user-curated data, or people-powered search.
It's also interesting for another reason, search as an activity will most likely become rare and funtionally meaningless.
I certainly wouldn't go as far as saying semantic search is meaningless, because that's an excellent way of adding much needed specificity to a web page.
Nearly two years ago, I talked about what I called the Found Engine, which is where all of this is heading — we announce our needs, schedules, lists of friends and tasks to the various web applications we're now wedded to and they go off and find the stuff we're going to need to do all of those things, wherever those things may be lurking.
So we no longer search for things because those things we need are found for us and then surround us in a contextual cloud that's further refined by having this cloud of data & information (two totally different things) spread across a time line.
It's gratifying to see my ideas being played out, even if only conceptually.
Ultimately, those that rely on this way of surrounding themselves with relevant (maybe even mission critical) knowledge will go back to their various social networks and begin the process of un-friending / un-following those people they think are diluting the quality of the shared knowledge they'll be relying on so much.
Brynn,
The only part of your post I agree with is that the rank that I wrote about is not a universal thing and, like any other rank, is just an approximation.
Your example of an expert not caring about friend's tweet is a corner case. We don't need to solve search for experts, we first need to solve it for general public.
And as for the last part of your refinement, I can tell you that its plain wrong. You are suggesting that when I am in discovery mode that I am okay to discover via public instead of my friends and taste neighbors. If this was true than social networking as it exist today, would not exist. Today's social networks are based on discovery, not search.
What is true, however, is that exact same ranking that I wrote about applies to both search and discovery. No matter what I am doing, I want to filter the information using trusted sources first and general popularity last.
This article completely avoids the topic of privacy. In order for this social search model to work, ever more information is needed about each person and about their network of associates. Bad for privacy, good for big scary government or crackers. Isn't Google good enough the way it is?
What you are putting forth seems to be all about perception since true relevance can certainly be delivered (e.g. a kick-ass Barbary Coast trail guide) without the need for proximity. Do we know for certain that Social Proximity can influence the perception of relevance? I'm not so sure.
We do know for certain people's actions online and their perception of relevance are influenced by domains and rank strangers. Amazon's recommendations and TripAdvisor's reviews are testament to that as well as numerous tests and observations I've conducted on conversion optimization.
Many queries can be both bucketed into both recovery and discovery even though the intent of the user is one or the other. This presents a host of challenges some of which to some degree Google solves blending ads with the natural results sets on SERP. However this issue is one I've yet to see addressed in any dialog around social search though it may be an area best suited for it.
I think that the filtering takes place at different points depending on whether we are talking about active searches or passive discoveries. While active searches could be enhanced through a filtering system such as the social relevancy ranking described by Alex, passive discoveries are filtered the moment we build our social network. By deciding who to follow and who to friend we are essentially building these filters into our passive discoveries.
Which brings me to another point. You suggest that one of the dangers of social search is to run into a topic "that hasn't been indexed in our friends' activity streams" thereby leading to inferior search results. What about the equally problematic scenario where we are casting that network so narrowly that we are limiting the information to the point of only getting a one-sided view of the world around us? I posed this question in a blog post on active vs. passive serendipitous social search earlier this week and would love to get some input.
Brynn, good insights. I think in particular the emphasis on context is much needed.
In his original post, Alex declares in reference to general web search "...Once Social Relevancy Rank is factored in, search results will be re-ordered based on social relevancy." The point is that not all searches are fit for such ranking, some are and some aren't, and it doesn't depend solely on the searcher but rather on the specific query and other context aspects.
If I search for a factoid such as the boiling temperature of lead or the home page of an airline, social relevancy is only detrimental. If I search for trusted opinion on a topic, social relevancy is extremely relevant. Search engines attempt to discern your intent already today, Google's OneBox scenarios are good examples.
I don't agree, though, on your terminology plea :-) Semantic search is a totally different ballgame, one that may have zero relatedness to social networks, and it's often being mixed in just for the sake of using a buzzword. Real-time search, as someone commented on Alex's post, can just as well relate to searching authority news sources, whereas Alex assumed that social is always relevant there (which is obviously not always true). So we should talk social search where social graph and networks are leveraged, and remove the other distracting buzzwords out of the debate...
there is an underlying assumption in the post and comments that we can pre-compute the best results. this can never work in the general case simply because "what we are looking for" is often unknown BEFORE our first query(ies). the answer to the first question leads to more and better questions, which require conversational exchanges to emerge.
in other words, to presume that our needs can be inferred in advance will always limit our scope. while improving ranking and results is a useful quest, it's not the only one. what is not limited, and what takes advantage of our innate biology and recognizes the evolving, personal nature behind why we search, is to provide an interface for effective conversation—to "converse" with search results, with the interactive process of refining our interests as we refine our queries, with others in real-time and otherwise. if we design for conversation, limitations of computation are overcome.
InverSearch already addresses everything in this thread. Why do you need others to filter information for you? http://inversearch.blogspot.com
Stumpedia allows you to rank and personalize your own search results. The relevance of search results are unique to you and are determined by you, your social network of friends, your followers, and friends-of-friends.
http://profitableprofit.blogspot.com/
Mr. Evans,
Great article, but you left out the principle challenge with using social data to rank the web. Whatever the distribution the space of queries plugged into search engines might be, this much we can say for sure: it is a heavy (long) tailed distribution, which is to say that most of the stuff is not found in the most common 1000 or 10,000 or so queries.
Why does this matter? Every social ranking algorithm requires a certain density of people, but in a heavy talied distribution, all but unusually active topics or the central hump in the distribution (the most popular sites) will be indistinguishable from noise when compared to a social graph, unless you restrict it to a very controlled subset.
As every search engine already has information about the stuff in the bubble and social graphs will add little to this. As such we need to view the social aspects of search for what it is: the best way of returning close to real time results in RAPIDLY changing information that is of popular interest (so called real-time search). Real time search is extremely valuable in its own right, and I dont want to downplay it; but I think that applicaitons of social structures to reference searches will be of limited value.
You could argue that by restricting the social graph to some set of experts in a field would return high value results -- and I think you would be right. In fact, there are a handful of people who have done just that, with some success. But a search engine is automated, and the number of such expert networks over a heavy tailed distribution is large, and all such attempts to do this in an automated way run into the heavy tailed distribution problem, or if you perfer, a variant of the set cover problem. Both problems are intractible.
JSL
http://daylogames.blogspot.com/
Very good article I must say, I think if most of you check http://www.ClubDistrict.com Nightlife Simplified which I feel is the the new way of social network, brand new. I would really check it out
Trusted networks are more important than friend networks, when it comes to finding personalised relevent content and recommendations. That's what we're doing at http://www.rummble.com .
Excellent article Brynn!
That's a good point, Andrew! But how people assess trust in their friend networks? Is there implicit trust with some friends that could be learned from user behavior? Or does it need to be marked explicitly between friends?
I could also imagine that trust matters less for certain types of search problems. Something to think about...
Posted by: brynnevans.com
|
November 14, 2009 2:54 PM