11 Search Trends That May Disrupt Google
My first post for ReadWriteWeb (nearly a year ago) started with the premise that search was "game over", that Google had won and the only opportunity left was (re)search - i.e. what one does after the basic search. Unfortunately, none of the search start-ups since then has made a dent in Google's relentless march towards search market dominance. In this article, we outline 11 search trends that may change that.
The proposition that launched countless search start-ups was: "If we can get just 1% of the search market, we will have a very valuable business". That may be true, but getting 1% has proved elusive. It has been an all or nothing game.
That may be about to change.
It is possible that Google will not be beaten by one big competitor. It is possible that they will be pecked at by thousands of tiny start-ups using a new outsourced infrastructure.
But before getting to that punchline, here is my 11 point recap of the search market:
1. Disambiguation is (still) not enough motivation to switch. All those learned PhDs with backgrounds in natural language search and AI explaining that the words "paris" and "apple" have multiple meanings that Google cannot parse from a single search, massively miss the point. The average user has figured that out and either enters multiple words or refines the search based on the first search. Using natural language search - which is complex to code and expensive to process - is a classic "hammer to crack a nut" solution.
2. Webmaster push-back and basic economics will accelerate the trend towards an outsourced crawler market. Webmasters won't accept a proliferation of crawlers as some of them maybe malicious and all of them impact performance to some degree. Google Yahoo Microsoft (GYM) will always be accepted as they drive enough SEO, but marginal crawlers will struggle. Basic economics mean that only a very small number of players will be able to afford the giant server farms needed to index the whole Web. The YM parts of GYM (as well as Amazon) will increasingly offer their infrastructure to anybody who can build value on top.
3. Yahoo Search Monkey may have arisen from desperation, but we may also be witnessing a "Linus moment". SearchMonkey is the most well-defined entry into the outsourced crawler market. It comes from their recognition that it is too late to beat Google in a head to head battle, so it could be dismissed as a sign of desperation. However I prefer to see it as a "Linus moment", that point in time when Linus Torvalds simply said "here is what I have done so far, anybody who can take it to the next step is welcome to try". To be truly disruptive, Yahoo may need to open this up even more than they have to date.
4. There will be many more attempts to monetize Wikipedia. Well-funded search ventures such as Powerset have retreated to the much narrower goal of searching Wikipedkia. Freebase also uses Wikipedia as the their core data. Walking around the RPI Web Science Research Initiative, I could see many interesting R&D experiments coming out of Academia all of which used Wikipedia as a base. Wikipedia has just enough structure and normalization to be useful. Above all, the History feature makes "data provenance" possible and that is critical for trust.
5. Core search is still getting funded. This is not what one would expect in what is by any definition a consolidated market with one mighty big gorilla sitting on top. Look at Blekko getting $2m without even a prototype to show the world. Are the investor's nuts? Possibly, but they include some pretty smart guys like Marc Andreessen and the founder Rich Skrenta is clearly a smart guy (his Blog is a good read). Or look at Cuill, which got $25m as recently as April. Maybe they are idealists tilting at windmills. Maybe they know something that the rest of us don't. Only time will tell. These new entrants will eschew any hype, which they know has not one single point of value in adoption.
6. Image search is another "hammer to crack a nut". Searching images, video and audio is one of those "non-trivial" computer science projects that great engineers love to tackle. However great investors should steer clear. It is hard to code and incredibly expensive to process. The competition is tagging (see next point) which is classic "just good enough and improving all the time at virtually no cost" that is impossible to beat.
7. Tagging is quietly but massively disruptive. The fact that thousands of webmasters and bloggers tag their content so that they can be found by Google is Google's secret weapon. But it could get turned against them. A small incentive to be found by other search engines will change tagging behavior. This is likely to play out in lots of vertical niches, where a small change in tagging behavior can make a huge difference in findability and that can make a big difference to both buyers and sellers. Whether people use RDF or Microformats or some other defacto vertical standard will continue to be the subject of much debate, but the format itself is not the issue. The human drive to tag (to order one's world) is deep and strong and has financial motivations as well.
8. Whitelist is a good way to kill spam. Spam is the big problem for search as well as email and whitelists work well for both. In search this is done by a site that uses something like Google Custom Search Engine (or Search Monkey) to define what sites to search within a defined domain. Even if that means defining 1,000 sites and adding new ones every day, that is well within the range that a single human curator can do within a single market domain. The human curator deletes any spam sites manually.
9. P2P search could still be a long-term disrupter and Microsoft's route back to relevance. The only way to do search without putting all the Web's pages into one server farm is via P2P. I have written about Faroo's attempt here. It relies on .Net and this maybe Microsoft's card to play but only if Vista gets real traction. This is a real long shot, but an intriguing one.
10. There is tons of great data inside relational databases that is quite easy to search. It is the HTML layer that is getting in the way. As more sites learn how to expose their structured, relational databases as Web Services APIs, a lot more data will be available that does not rely on word search on HTML pages.
11. It's the Adwords, stupid! All the search wizardry don't matter a hoot if the monetization is not done right. There is plenty of motivation out there. Sellers want cheaper search words to buy. Publishers want a bigger piece of the cake. Buyers/searchers may even want cash back (we will see if Microsoft's crude tactic, lambasted in the Blogosphere, makes it in the real world).
Conclusion
Most of these trends point in the direction of search as infrastructure feeding thousands of innovators in niche markets - a long tail approach, in other words. Google will play in this infrastructure game - they already do with Google Custom Search - but it is vendors such as Yahoo, Microsoft and Amazon with equally deep pockets and much more to lose from total Google dominance, who will be the disrupting innovators in this next phase of the search market.
Image credit: davemc500hats
Share
Comments
Subscribe to comments for this post OR Subscribe to comments for all ReadWriteWeb posts
"Most of these trends point in the direction of search as infrastructure feeding thousands of innovators in niche markets..."
Am I correct in taking this to mean the hundreds of Alternative Search Engines that we cover on AltSearchEngines.com?
I had a feeling your ears would prick up at that line Charles ;-)
I think you're missing a big one that's going to wreak havoc on the market... It's not in the core search space.
With at least 50% of the places people go on the web being places they've already been, and at least 30% of queries fitting into this revisitation mode, better support for client side remembrance of these locations is going to take a huge bite out of search. It'll be most obvious in branded search, where engines make huge $s from brands buying their names.
The Awesomebar in Firefox3 uses the best of cognitive science (http://surfmind.com/muzings/?p=150) to predict where a user is trying to go. I'm betting on a route around the engines for a large number of use cases.
You have missed one more trend - search is a utility now and a winner is that one who can establish an emotional connection between search and user; create a new search environment... guess who is doing that :)
You forgot one.
Community driven vertical search portals, particularly ones driven by CPA business models, will erode search market segments away from Google. I have it on good authority that the guys Google fear most at the moment are not Yahoo, Powerset or any of the 11 you mention ... its the price comparison sites.
Why do you think MS effectively turned their search engine in a third place position globally into a price comparison site, or why Google is soon releasing merchant search?
The smart ppl are building CDVSPs :-)
So. Much. Fail.
I thought I read the whole thing, and I didn't see you mention mobile. Microsoft just released their NavOS which ties into live search. It's still unclear whether manufacturers will use the tech. It should be interesting.
A good insight, but I still think that all these may not disrupt, even if we add the mobile platform part!
I'm going to make a not-very-risky bet - none of these trends will disrupt Google. Not a one. Particularly the "human curators" - Yahoo tried that, didn't work.
I still think for anyone to disrupt Google they'll have to have a name that can be used as a verb. You're not going to say 'I cuill'd something' like so many say 'I googled something'.
I would think that just the scope of the database is a major impediment to any new company. An open-source crawler would be an interesting approach. The data could then be accessed by anyone with an idea for a new search method (perhaps a bright college student somewhere). In effect, separating the backend data collection from the search application itself. You could have any number of new search services created off of the same dataset. (Great idea for Mozilla if they weren't in the pocket of Google).
Also, what if you could do the crawling on a distributed basis, sort of like Seti or Folding@home. Creating a botnet for good, not evil. Lessens the need for a massive server farm.
Friendfeed once they build a big enough database. The draw would be search within your community of friends.
While the 11 points mentioned will somewhat cause a dent to Google's dominance, I think the single largest contributor could be display-text integration. Display Search Marketing adds the quintessential emotional value to a search results. Users today are looking for more compelling reasons to click on an ad than just text relevance. This may be esp. true in emerging markets where 'feel' is as important as the content. So whoever wins the display battle, will eventually race ahead. Have to admit that un'googling' is still far away!
I was thinking of specialty search. Back in the day before there was the web (and it's search engines), the best search tool out there was lexis for lawyers (legal database) which later spawned nexis (journal and newspaper database). These were specialty databases that were defined and closed. You not only were searching a small defined dataset but could limit your search by certain libraries within that data set. I think back to the future might be a viable play here. Why not a closed, moderated yet very large medical database open for free public search? Why is there not a free lexis/westlaw search database out there? If the goal is 1%, you string together a bunch of narrow specialty search engines and I think that goal becomes quite doable.
One thing that might be considered as well might be called "Distributed Local Search".
With the adoption of the OpenSearch Format into major browsers like FireFox and Internet Explorer, it provides users with a way to circumvent the major search engines all together and search directly from the source. This feature is still in it’s infancy, so it probably isn’t on the radar screens at this time. But, as more browsers begin to include the OpenSearch format in their feature list and more site owners realize the potential it brings, the trend will begin to get more recognition from the major players.
I foresee some kind of 3 dimensional search interface, that looks like a mind map but more intense. As new items are found, the 3-D environment changes dynamically. News is happening all the time, and if you search on top keywords, perhaps you should see the volatility of it happening in real time.
google has much to say yet
Targeting Google has either got to be a huge motivating factor for a team of developers, or extremely depressing. Still plenty of room for "innovation" though, as you point out in the 11 trends.
However - "search" isn't the only way to go about solving the problem of "finding things." Dynamic clustering of results, navigation/discovery via semantics/linguistics, social referrals, agents/alerts. I'd have to say that our global dependence on fairly "stoopid" (thanks for that, TIME) keyword search has stalled some of the progress in the "search market."
We're currently finishing up our next research piece, the Market IQ on Findability - and in the "timing is everything" mode, would be curious on your (and the readers here) feedback on a post I made earlier today, in response to a comment "Enterprise Search Still Sucks."
See:
http://www.biztechtalk.com/2008/06/re-enterprise-s.html
Mostly talking about the enterprise, but when trying to find something (content, people, data, etc.), search is only one method of many, regardless of where you're at.
I think custom search will be huge, because it's perfect for creating structured vertical search engines. I'm experimenting with one that lets you search for videos containing soccer (football) highlights. It's in beta for the moment focusing on the european football championship, but I plan to expand it to other events and sports. I use custom search, rss feeds and structured data (tournament schedules) to create a mashup (http://europeanfootballchampionship.googlepages.com/) that lets you read the latest news on the euro 2008 tournament while browsing videos containing the highlights of the games. You can write search queries in the classic search box or click on items in the structured data and the query will be executed automatically. It still needs a lot of work because I'm still missing the social media features like tagging, voting, sharing ecc. Waiting for my google appengine account to activate to get that done. If I compare searches done on plain old google to searches done on my custom engine the difference is huge. The first page of results in Google just contains one or two links on the subject I'm looking for while the custom engine gets them right at a 80% rate. I think that Google has a good strategy giving away for free lots of infrastructure and tools. There will be people who create content on top of this infrastructure. If a person is expert on a topic, who better then him can implement a vertical search engine that filters all the rubbish and provides you with quality structured data that will save time to the end users. Ad money is shared between infrastructure and content provider, the user gets quality stuff for free.
Nice post, keep it up!
Thanks..
Microformats can help with disambiguation. My proposal (http://microformats.org/discuss/mail/microformats-discuss/2007-December/011169.html) for using optional address components with the required "formatted name", would mean that if "Paris" is wrapped with an element having class="fn" it's a person's name, if it's wrapped with class="fn locality" it's a town or city.
Oddly, the cabal which runs the microformats "community" have yet to respond to that proposal, other than to say that two days was too little time to consider it. Six months on, they have made no further comment.
Fundamentally, I agree that unseating Google as the king of search at this point seems outside the realm of possiblility. But many believed the same thing of IBM, AT&T and others back in the day. Government, revolutionary change and other outside influences can and often do have a way of facilitating the impossible. Local, mobile, vertical and other sub-segments of search offer fertile ground for the emergence of many 1% opportunities.
Your insightful list is most thought-provoking.
Vertical search (i.e. trusted search) has a great future. Google struggles to keep spam out of the results. Good vertical engines with editorial supervision would be a wonderful innovation.
Baidu should also be mentioned. It has nearly 70% search market share in China and has been growing over the past few years. China is a huge and growing market for advertisers, so the revenue is there for growth and... I wonder ... how much it would take to expand into other countries. Remember how fast users adopted the uncluttered interface and natural search results of Google when it launched - so we know movement to a new search engine can be fast if it comes up with a unique value - switching costs for users are almost nil.
Consolidated online advertising market in Google has increased the cost for advertisers significantly. Our keywords now cost 200% more than a year ago for the same number of clicks.