Yahoo Search continues its steady decline. Yahoo Search lost another 1% market share last month and has now lost a total of 5% since September 2008 when it still owned 18.8% of the market according to Compete. The total search volume on Yahoo was down 8% and Yahoo served 100 million less queries in September than August.

Google, Ask and AOL Hold Steady

Google's market share grew slightly from 72.3% to 72.6%, while Ask and AOL remained stable. Based on this data, Bing seems to be eating into Yahoo's market share, but isn't growing at Google's expense.

Searches Per Day

Compete's Marko Madjarac points out that Bing's numbers are even more impressive when we take into account that Bing's users tend to perform fewer searches on the service (5 searches per user per day) than Google's users (5.6 searches). Bing apparently lives up to its promise to get users to relevant answers faster than any other search engine. Yahoo users performed an average of 7.8 searches per day.

Update: According to Google, "the removal appears to be an internal error and not part of a DMCA request."

According to Chilling Effects, a clearinghouse for DMCA takedown notices, the complaint was apparently sent by an adult entertainment company, Gwen Media's Destined Enterprises, which has filed similar complaints with Google before. According to TorrentFreak, the complaint was sent by RemoveYourContent, a company that specializes in sending out DMCA complaints for the adult entertainment industry.

For the time being, the effect of this takedown notice is that PirateBay.com, a scam site that really shouldn't appear in Google's index, now sits at the top of the search results.

It's important to note, though, that this is far from the first time that Google has deleted search results after receiving takedown notices. To the best of our knowledge, however, this is the first time a well-known site like the Pirate Bay has been affected by this.

Fresher Search Results

Until now, if you wanted to see the freshest search results, you had to apply a little URL-hack, but now, this feature has become default in the Search Options panel.

Another interesting new feature is the ability to filter results by sites you have already visited and by sites you actually haven't visited yet. This feature only works when you are signed in to your Google account and have your Web History enabled.

Books, Blogs, and News

Google already introduced the ability to just search for books a few weeks ago, so this isn't really a new feature, but the company now also allows users to filter by blogs and news. While you could obviously always use Google News or Blog Search for this, you can now easily toggle back and forth between these sources and stay within the same search interface.

Shopping

Given how hard it has become to do product research on Google, as the search results are often cluttered with shopping sites, the ability to see more or fewer shopping sites is probably the most important new feature here. As Google points out, if you are doing research and aren't ready to buy just yet, being able to tone down the number of shopping sites will be extremely helpful.

Breaking the Habit

According to Weitz, this is why Microsoft decided to brand Bing as a 'decision engine' and not just as a search engine. Microsoft wants to get consumers into the habit of demanding more from a search engine than Google can currently deliver. This explains Bing's current focus on shopping and travel, for example, where Bing generally delivers superior results.

In order to gain more traction in a marketplace that is driven by habit, however, Microsoft also partnered with a number of hardware manufacturers to make Bing the default search engine on new computers, for example. Of course, the ubiquitous marketing campaign for Bing has quickly made the new search engine a household name, though as these survey results show, that alone won't be enough to drive a substantial number of users away from Google if the search results on Bing don't live up to users' expectations.

Bing and Yahoo

So far, this campaign seems to be working well enough. According to comScore, Bing is now close to a 9% share of the search market, up from Live Search's 8% in May. Once Bing also takes over Yahoo's search, these numbers will probably top 25%. It is also important to note that most users already use more than one search engine and are now becoming more and more familiar with Bing.

Another positive sign that the Bing team has noticed is that a growing number of users have started to recommend Bing to friends and family, and when asked to name a search engine, the number of users who bring up Bing unaided has also risen dramatically. As we pointed out last week, Bing Shopping is also growing rapidly.

Demand More

Of course, the Bing team is also not just sitting at home twiddling their thumbs and resting on its laurels. Microsoft will continue to update Bing over the next few months and continue to differentiate itself from its closest competitors.

We agree with Weitz that consumers should probably expect more from search engines than just a simple list of links. We will have to see if Bing will succeed to break most users' habit of just going to Google as their default search engine, but, if anything, it will drive Bing's competitors to stay on their toes and continue to innovate as well.

Great for Cities - But not for Sights

While this worked quite well for the examples Google gives in its blog post, a few of our other searches ('Yosemite National Park' for example), did not bring up any photos. Any search for a city, however, always gave us related images from the photo layer in Google Maps. Searches for local restaurants and sites typically don't bring up any images and searches for local sights already brought up search results from Google Images anyway, and from what we can see, nothing has changed there.

It's nice to see, however, that Google now surfaces more images. It is also worth pointing out that yesterday, Google also announced that users can now easily filter Google Images by Creative Commons license, which, until now, was really the domain of Yahoo's Flickr, which hosts the world's largest repository of CC-licensed images.

Competing With Bing

Of course, we also can't help but think that the fact that Google is facing renewed competition from Bing, which also does a nice job of mixing up images with local search results - though not quite as nicely as Google now does. Travel is one of the areas where Bing excels, and it is good to see that the competition in this market is driving all players to innovate and improve their services.

Their latest release, Google@Omgili, features a sweet mashup with Google search, giving users a well-rounded look at the fair-and-balanced web alongside social buzz from and about sources such as blogs, newsgroups, video-sharing sites, forums, discussion boards, Q&A sites, and review sites.

The translation function isn't amazing, but for trending or hot-button topics, the results can be quite interesting.

The basic Google@Omgili search yields traditional web results with discussions that link to each result:

Once a user clicks on the "discussion bubble" next to the link, he can preview some of the social chatter linking back to that particular result:

Users can also choose to refine the search to show results from, say, forums and discussions only:

This product also incorporates the True Knowledge API and integrates Snap.com's service to give users a snapshot and video/RSS/Wikipedia previews. And of course, users can choose any number of social sites to share Google@Omgili with their networks (but not to share specific search results, which is disappointing) and can choose to add Google@Omgili to their search bar.

[UPDATE: The day after this post was published, Geva emailed us to report his team had added sharing of the results page on social networks. "We love feedback!" he wrote.]

Although the original Omgili search offering calls to mind many, many social/forum search engines - most of which have financially struggled or failed, see Twing and Delver - the Google mash gives this engine a much more interesting and useful product.

Overall, it's a nice way to see which links are generating (or supporting) subjective online conversations and might be a good tool for tracking down topic experts or sources. On the other hand, it might also be a way to get mired down in the chatter from forum crazies. Six of one, a half dozen of the other - after all, whether you're brilliantly insane or insanely brilliant, you've got to post your theories and rants somewhere, right?

Even more significant, few sites today are constructed solely from internal site resources. Hosted metrics, recommendations, news, store locators, stock tickers, friend followers, and so on and so on are rapidly deconstructing the whole notion of "place" through the active participation of the "web-fabric" layer of the web community.

From this perspective, most recommendation services are still stuck in the feudal worldview: the black box recommender knows you (whether "you" are a visitor or place-owner) better than you know yourself and determines, in its infinite wisdom and authority, what content should be presented to you. The place-owner may have some input into presentation and even, though less so, content, but only in a very limited way.

While this situation is useful in certain cases because of the total passivity afforded the place-owner and visitor, it severely limits the potential contribution of recommendation technology.

Personal, Real-Time Conversation

There is a broader view of recommenders, though. The business value of recommendations is that they bring the place-owner into a one-on-one, real-time, conversation with the visitor. As such, a recommender must be able to accommodate the active participation of both the place-owner and visitor. Recommenders play the role of the salesperson, the agent in the company who has one-on-one contact with each shopper. This is in contrast to the site designer, who is more akin to the display designer in a bricks-and-mortar store and who can only target segments of the population who are expects to pass the display, not individual shoppers. Recommendations are also narrower in concept than personalization tools, which are analogous to store greeters: they may personally greet you when you arrive, but they generally don't follow you through the store as you shop or interact with you in real time.

Okay, but why a conversation? Consider the typical interaction between a sales agent and shopper in a bricks-and-mortar store. The shopper enters the store and starts looking around. At some point, the sales agent asks, "Can I help you?" "No thanks, I'm just browsing," By this point, the sales agent has probably already observed the shopper and made some inferences about the shopper's intentions and receptivity and about associated sales opportunities. The shopper, in turn, has been assessing the store's inventory and pricing.

Like these sales agent, place-owners have a tremendous amount of knowledge about shoppers, sales tactics (like cross-selling, upselling), and their own business objectives, both short- and long-term. Much of this knowledge is unavailable to automated recommendation engines, no matter how much data they gather (and the ultimate prize for optimizing discounted infinite-horizon shopper value is computationally intractable even if we had the data). So, the recommender is better tasked to take advantage of the wisdom of the place-owner "in the moment." Of course, an uninformed recommender is just a degenerate case and may still be useful.

One advantage of the web is that transaction costs are low. Most place-owners can't afford to have human representatives in sessions. Most explicit communication by the place-owner must be in the form of policy or strategy, rather than actual real-time communication. (Notwithstanding this, interaction with a live sales agent may well be an appropriate option for a recommender to trigger in certain situations.)

Situation/Response

One way to think about this is like "situation/response". The situation description might cover visitor location, web page visited, catalog, date (e.g. if it is a holiday), place-owner internal item information (e.g. from a supplier catalog or internal access and sales statistics), visitor community information (e.g. sales ranking, review ranking), or even external information (e.g. Google search ranking, Amazon ranking). The response should be a specification over recommender behavior, as well as resulting recommendation content (e.g. show a pair of Nike's under $50), and presentation, both style and modality (e.g. use an animated GIF showing all available colors). Perhaps, as mentioned above, modalities even extend to bringing a live sales agent into the real-time conversation.

While limited work has been done on place-owner participation in recommendation-system content and presentation, the situation is far more dismal for the visitor. A broad array of modalities are available for visitor interaction, but few if any are available in most recommendation systems. A simple "No, that's not what I'm looking for" (e.g. a thumbs-up or thumbs-down icon on a recommendation thumbnail) might go a long way to making the shopper feel noticed and appreciated. I can say to a human store clerk, "I'm looking for a pair of Nike's under $50" -- why can't I tell the average recommendation system the same thing? Notice that this starts to overlap with the expressivity needed on the place-owner side. The main difference is that the visitor is always in the moment, so there is (usually) no need to specify context.

The above sketch is intended to crack open the door on the enormous range of possible capabilities, modes, and time-scales of participation by place-owner and visitor. Once we've opened this door, there is no reason not to open it to the visitor community and the web-fabric community as well. There are three primary points:

1. A place is no longer a feudal domain; all stakeholders now demand a voice.
2. A recommendation engine is the locus where understanding of content and understanding of the visitor-in-the-moment come together.
3. As a result, recommendations are the logical ground for crucial real-time conversations between place-owner and visitor.

Given our initial definition of place, we might also ask about the role of and opportunity for participation among other stakeholders. For example, can the interaction between the site designer and the visitor or web-fabric community also be viewed as an ongoing conversation, rather than an episodic, one-way information flow at the time of site design? The answer is yes, but that is a topic for another time.

Conclusion

Recommenders need to open up to allow increased place-owner, visitor, and community participation in both content and presentation. This is best done with the assumption that a recommender is meant to facilitate situated, in-the-moment conversation between the place-owner and visitor.

This was a guest post by Bruce D'Ambrosio, VP and Chief Architect, OnDemand Personalization at ATG, Inc. He was the founder of CleverSet, which was acquired by ATG. He is also a former Oregon State University computer science professor.

Hitwise's latest data also confirms that Google's market share in the search business is continuing to grow at a steady clip (9% year-over-year). Year-over-year, all of Google's larger competitors lost ground, though at least between December and January, both Yahoo and Ask.com saw a very minor increase in their market share.

Longer Search Queries

Year-over-year, using one and two-word search engine queries became slightly less popular, while the number of three-word queries remained flat. Instead, a growing number of users are now opting to use longer queries. Overall, longer search queries have increased ten percent over the last year.

This is an interesting trend, and it could be interpreted in a variety of way. This could mean that a growing number of users is finding less value in the search results they get from relatively unspecific, short queries. It could also indicate that users are becoming more sophisticated in how they structure their queries when they are looking for very specific answers.

Do you have a theory why more users are turning to longer search queries? Feel free to let us know in the comments.

As Google also points out, scanning the search results page and deciding where to look at and what to click on seems to be an almost completely subconscious process.

Overall, users on Google's search page follow the familar F-shaped pattern that was also apparent in earlier eye-tracking studies .

What About Google's Ads?

Google's researchers don't mention advertising in this post at all. Neither the video, nor most of the images in the post include any advertising. Given how important these ads are for Google, we can only asume that Google regularly uses these studies to test how its users perceive the advertising on the search results pages.

Judging from the only images that show ads, most users only look at them after they have scanned through at least the first three or four search results.

Chances are that Google didn't want to give away too much of its secret sauce, though it would have been nice to see a bit more information about how users perceive and react to these ads.

Custom Search Engines for the Masses

It is worth noting, too, that Google has started to make more changes to its core search product lately, with a clear focus on customization. SearchWiki, which lets users re-organize search results, already represented a major change to Google's approach to search. The 'preferred sites' feature, when and if it is rolled out to all users, will give users another option to customize their search results. In some ways, this new feature will bring Google's relatively unknown Custom Search Engines to the majority of Google's users.

As Ed Oswald notes, it will be interesting to see if Google will use this data in some form or another to tweak its regular search results. However, given that Google isn't typically very open about sharing this information, we don't expect to hear from them anytime soon.

Portable Identity

Identity and portable identity/data remain hot topics. Luckily, the conference wasn't just another pointless "200x is the year of OpenID" love-fest. There were sessions on OAuth, MySpaceID, and Facebook Connect. At one of the Facebook Garage sessions, Dave Morin dug into the nitty gritty of Facebook Connect and really impressed. OpenID and OAuth remain medium-term goals for many web apps, while Connect has jumped immediately to the front on their development road map. This goes to the core vibe of Le Web 2008, which was all about business benefit.

Improved Search

Despite having nothing earth-shattering to say, Marissa Meyer of Google still captured the attention of the audience. Led by some excellent questions, she covered topics such as temporal search (which gives only results from a certain time period), personal search (if the search engine knows what you searched for and liked before, it can give you better results next time), and local search. Meyer noted that she thinks local search in particular will be a hot feature for 2009. There are many start-ups trying to crack this particular nut, but one can see from Google Maps, and more specifically the latest versions of Google Maps Mobile, that these start-ups should be very afraid and need to create something far better than a Google Maps mashup to succeed.

Cloud Storage

Cloud computing and storage is becoming more mainstream every day. There was nothing particularly new from Werner Vogels at Le Web apart from news that EC2 is now fully available in Europe. Amazon's offerings remain geared more to infrastructure, relying on others, such as PutPlace, JungleDisk, and RightScale, to deliver storage that is easily accessible. This is why people have been paying attention to Microsoft's current direction. Every Live product put out by Microsoft in the past two years has been bettered by the competition. There is one clear exception to this: Live Mesh. This is the first consumer-facing Microsoft web app that nails it. At my home we have it installed on four machines and have a directory tree on each syncing seamlessly no matter where we are. Who needs a home server or home network-attached storage (NAS) when Live Mesh does it all without you having to think?

Video Search

Didier Lombard, the head of France Telecom/Orange, said that the next big thing would be "video search," which initially sounded laughable. Then Google confirmed that it has been studying it for a long time, and then we found out that the start-up competition winner was Viewdle, which does exactly that. Viewdle's reps gave a fantastic overview of the technology, which of course spooked many when it was pointed out that it was originally developed for military applications. Being able to parse faces in video and associate them with specific people is mind-blowing. With $500 dual-core boxes, they can parse in nearly real-time, and with massively parallel NVidia GPUs, they can go far beyond that. Expect an exit here in 2009.

Business as usual

Much of the rest of Le Web was about business (and love!), with many presentations by successful web companies that aren't particularly technically innovative. From travel review websites to website builders to mobile IM, we've seen these before. The difference now is that many of these businesses are making money. Let's hope that 2009 shows more progress in both business and technology.

This was a guest post by Conor O'Neill, CEO, LouderVoice; conference photos courtesy of Flickr.

Social Networks Dominate

Interestingly, while Google's global list of rising searches is dominated by relatively time sensitive searches like 'Sarah Palin,' 'Jonas Brothers,' or 'Heath Ledger,' social networks like Facebook, Wer-Kennt-Wen, and tuenti gained enough worldwide popularity to break into this list.

Looking at the results by country, this trend only becomes stronger, with Facebook appearing on almost every country's list of fastest rising search terms, though it hardly ever appears on the list of actual top search terms for these countries. Otherwise, these country specific lists tend to be comprised of searches for local services and events, though YouTube, Yahoo, eBay, and the Wikipedia make regular appearances as well.

Also, the trend of using search instead of bookmarks clearly continues, as quite a few lists of top search terms actually feature 'google,' 'yahoo mail,' 'hotmail,' myspace,' and 'facebook.'

The Lists

Here is Google's top 10 list of fastest rising search terms:

For Lycos, the results were slightly less interesting, though Facebook makes an appearance on this list as well, together with 'poker,' 'golf,' and Clay Aiken. Given Lycos' minuscule market share in the U.S. search market makes this list slightly less representative of the overall U.S. market, however.

Current keyboard shortcuts include:

j or DOWN  -  Selects the next item.
k or UP       -  Selects the previous item.
l or RIGHT   -  Moves to the next category (results, sponsored links, refinements).
h or LEFT    -  Moves to the previous category (results, sponsored links, refinements).
<Enter>       -  Opens the selected result.
/                 -  Puts the cursor in the search box.
n                 -  Moves to the next result, and fetches more results if necessary.
p                 -  Moves to the previous result, reloading earlier results if necessary.
=                 -  Magnifies current item
-                 -  Shrinks current item
A                -  Switches to Accessible Search Results
W               -  Switches to regular Web Search Results

The "A" switches you Accessible Search Results, which identifies and prioritizes search results that are more easily usable by blind and visually impaired users. The "W" switches you back to regular results.

Keyboard Shortcuts For Everyday Use?

For now, the only way to use this search experiment is to use the Firefox 3 browser. If you have "find as you type" turned on in your options you may run into trouble navigating through results using the keyboard. Instead of moving you to the next item, Firefox starts looking for the letter "j" on the current page. However, with that feature shut off, navigation was a breeze - you could j / k back and forth through the list of results with ease.

This isn't really a feature that would work that well for everyday searches where you need to scroll through multiple pages of results in search of relevant articles. However, for informational searches such as those where you want to quickly learn about a subject well-documented on Wikipedia and other highly-ranked and respectable sites, navigating through the results with keyboard shortcuts could be a handy option.

The Accessible View Experiment is something that we could see using more often if there was a way to easily go from the regular Google.com homepage to the experimental Google.com site (perhaps via a keyboard shortcut?). They would also need to work on the find-and-you-type bug because that's a feature not worth giving up for a bit of navigation ease.

Keyboard shortcuts aren't for everyone, but for those of us who really enjoy the option, adding shortcuts to Google is something we could get excited about if the kinks were worked out.

That, after all, was the real advance that Google brought to the search engine market. Early search engines like Altavista, Excite, or HotBot simply weren't able to return the most useful results to users.

However, we are also getting close to reaching a new crossroad again, where even Google's results are often polluted by spam. Yet, at the same time, the great promise of semantic search engines is still just a promise for now. Given the latest data about the search engine market and the end of the Microsoft/Yahoo negotiations over acquiring the Yahoo search business, Google is pretty much becoming the de-facto standard search engine for most people.

Chances are that anybody who wants to enter this market and compete with Google is simply going to be bought by the search giant, so if anything, Google's strong position is going to get even stronger in the foreseeable future.

For now, though, the real question about Google's search index is when it will reach a size of 1 googol...

Photo by Flickr user Mykl Roventine.