Comments for How Effective is Google Personalized Search? 57% Say There's No Difference, or it's Worse!

How Effective is Google Personalized Search? 57% Say There's No Difference, or it's Worse!

2007-08-07T21:16:38Z

In Greg Linden's guest post defining Web personalization, he notes that Google Personalized Search uses technology acquired in 2003 from a small startup named Kaltix. He goes on to say that "the current version of Google Personalized Search learns from your search queries. Searchers do not have to do anything explicitly to use it; it is all implicit. The current Google Personalized Search likely is using the same Kaltix technology, building a high-level profile of you, then biasing all of your search results based on your long-term behavior."

With that in mind, it's curious that just under half (48%) of respondents in this week's R/WW poll haven't noticed any difference in their Google search results. Only 12% have seen an improvement, but perhaps of more concern is that 9% say their search results have gotten worse! You can test this yourself by going to googlonymous, which as the name suggests allows you to do an anonymous search on Google (hat tip Mind Booster Noori for the link).

]]> Mind Booster Noori also explained why personalized search results are inferior to him than anonymous results: "Google tends to give me the websites I already visit and know, but when I'm searching for something, I usually want to find something I don't know of, and Google fails to give me that. For me, usually, the results of a query to Google are better if I'm not logged in."

Let us know your thoughts on the effectiveness of Google Personalized Search. Don't forget also to contribute to our poll on this topic:

Comment from Marshall Kirkpatrick on 2007-08-07

2007-08-07T21:56:14Z

Sounds awful. Google Custom Search on the other hand, makes my world go 'round!

Comment from John M on 2007-08-07

2007-08-07T22:50:49Z

The thing is google has become the single interface to the web for alot of non-technical people. They don't use bookmarks, they don't use the address bar. If they want to go to Amazon.com, they type Amazon in google - so for this use case biasing search results to visited sites would be helpful. Unfortunately as the quotes in the article suggest, this behaviour actually detrimental to people who use a search engine to actually search.

Comment from Martyn on 2007-08-08

2007-08-08T11:51:15Z

I have confidence that as google builds a better picture of my interests and the types of site that I find useful, personalised results will diverge more and more from plain vanilla. I'm generally quite impressed at the diversity but relevance of the sites recommended by the Interesting Items tab in my search history, and I see that as a sign of things to come.

Comment from Andy on 2007-08-08

2007-08-08T14:55:54Z

That still leaves 43% !

Comment from Falafulu Fisi on 2007-08-08

2007-08-08T16:06:09Z

Richard said...
[The current Google Personalized Search likely is using the same Kaltix technology, building a high-level profile of you, then biasing all of your search results based on your long-term behavior.]

I pretty much doubt this. I think Google has gone on to develop other (advanced) machine learning algorithms into its Personalized search. Just watch the following video presentation by Google head of Research, Prof. Peter Norvig, about their technology and also they have developed new machine learning algorithms for their technology.

http://media.citris.berkeley.edu/Future_Search07_Peter_Norvig

So, there is no doubt that Kalix's technology has been bypassed by new stuff. You can't get stuck in a technology with algorithms from 2003. There is always new or improved algorithms that are being published in information retrieval, machine learning, data-mining journals, etc, from time to time. Therefore it is important to implement the latest one that is available from the literatures.

Here is a paper from Google, where one of the algorithms used in their personalization, PLSA (probabilistic latent semantic analysis) is used in tandem with others. So, I think that Google has moved pass Kaltix technology, since I doubt that Kalix had developed PLSA into their technology.

"Google News Personalization: Scalable Online Collaborative Filtering"
http://www2007.org/papers/paper570.pdf

Comment from Richard De St Croix on 2007-08-12

2007-08-12T15:41:21Z

My take on this is that accuracy may not be the only criteria for use - what it does is allow you to take a deep look into a selection of data resources that you've already selected for relevance to a specific task.

In a teaching context I can see this being useful where a course is based around a class blog for instance. At the beginning of the course the instructor sets up a blog, "primed" with a blogroll and links to recommended resources - the "custom search on the fly" feature makes it trivial to include a Google engine that just searches these resources. Students can be invited as contributors and the resource can build over the length of the course.

It also gives you a way of retrospectively searching and cross comparing feeds that you've only just picked up on - a kind of aggregated review across a particular infoset that I think can be genuinely useful in a research context, (effectively it gives me a "superset" of all the Search facilities on the blogs I subscribe to).

I set up one based on my blog-roll called SoapDish2 and it's instructive to try some of the, shall we say, more "common" terms students have been known to run through google and compare the custom search and the web view :) So I think, a useful tool for focus and example as well.