Google to Index Twitter & Enter Real-Time Search Market: Implications Abound for Startups & Users
In the immediate wake of the announcement of Bing's indexing Twitter updates, Google has announced it will be doing the same.
Taking the wind out of the sails of many a real-time search engine, Google's and Microsoft's announcements further put to rest a maelstrom of rumors swirling around the startup's possible acquisition and partnerships. Marissa Mayer wrote today on the official Google blog, "We believe that our search results and user experience will greatly benefit from the inclusion of this up-to-the-minute data, and we look forward to having a product that showcases how tweets can make search better in the coming months."
The Deal Per Se
Also of note is the fact that Google's agreement to index Twitter posts is, in itself, newsworthy. The deal has apparently been brokered recently enough that no examples or screenshots are yet available.
No less newsworthy is the fact that both Google and Microsoft have announced Twitter search deals on the same day. With the rash of real-time search startups over the past two years and the heavy-hitters' focus on real-time search integration over the past several months, the terms of this particular game become increasingly clear. The search market money remains in advertising, and who controls search by serving the best, most relevant results, will also control search revenue.
The Deal As Dollars
If readers will direct their attention to the heady days of the dotcom boom, they will recall that Google was able to effectively kill Yahoo! search - and capture and monetize that traffic - when Yahoo! began serving search results from Google. Shortly thereafter, Google grew to dominate the search landscape in its own right.
When considering real-time search, one begins to see a similar pattern emerging. Twitter is but one of the many real-time properties that serve valuable backlinks with keyword-rich content. If Twitter or any of its ilk were to be acquired by a search giant, real-time search would be effectively weighted to one site or another; and with that weight of traffic would come a weight of ad dollars. It would be very unlike Google to let such an opportunity pass; however, indexing Twitter is a difficult, time-consuming task that has absorbed many teams in the sphere of startups in real-time search.
The Deal As Product
The sheer amount of data available is mind-boggling. Wowd has resorted to distributed computing (à la SETI@Home) to solve the problem. OneRiot has put a team of 25 engineers to work for two years to solve the problem of effectively serving relevant results from the real-time web. And Collecta has used IM-affiliated technologies to bring streaming data to the space, shaving off even the most minute delays between machine indexing and human discovery.
In short, these back-to-back announcements could spell either a mass acquisition of real-time search engines or a mass extinction. The search giants have enough market- and mindshare to make any startup obsolete; regardless of how late they are to this game, they have the power to bring real-time search to the mainstream market.
The Deal for Twitter & Startups
Also, both Google and Microsoft must consider when serving Twitter results whether the startup can handle a firehose of search giant-level traffic. The notorious Outages of the Summer of 2008 called the company's scalability into question. But when one considers the fact that 1 out of 5 Internet users is on Twitter (or a similar service) and almost all Internet users are searching via Google or Microsoft, one wonders whether Twitter's back end can handle a fivefold onslaught of traffic.
Will Twitter require a data center for handling Google-fed traffic? How will Twitter deal with serving results on topics to newer or less experienced Internet users? Were these questions part of the company's conversations with Google?
To many in the real-time search-o-sphere, these announcements resound as both validation of long years of research and development and as apocalyptic overtones of longtime giants muscling their way onto the frontier. Who will survive, who is in acquisition talks, and who will be designated for obsolescence? More ReadWriteWeb coverage is coming soon; please stay tuned and leave your comments below.
Share
Facebook
Comments
Subscribe to comments for this post OR Subscribe to comments for all ReadWriteWeb posts
This is great news both Bing and Google in a heated competition for the best real-time results. There are 3 critical factors to success on any front http://ow.ly/vNGJ
I'm wondering what the details of the deals were around either SE placing ads on results? It's logical that they either a) have asked that ads not be placed on the results and are still trying to figure out how they want to handle ads or b) Just outsourced one of their biggest revenue opportunities (which could be a very smart move). Or maybe I'm missing something.
Interesting assumption at the end that Twitter must jump when Google asks. The potential symbiosis between the two may be close enough that Twitter will instead push the feed data to the search partner.
Push could save N search partners all asking the same question 1000s of times/sec., the problem of the searchers having to follow millions, etc. It lets Twitter determine, meter and optimize the load on its system, perhaps with a dedicated pathway, multicasting, etc.
I agree it's hard to see how search startups can match this, if Google and Bing are committed to moving fast despite their size.
Woah there... a big assumption is being made in this post. This story implies that google search traffic itself will be sent to search.twitter.com ("Will Twitter require a data center for handling Google-fed traffic?") and google will have to wait on twitter for the search results!
Google has access to the twitter fire-hose. It seems most likely that twitter will send every tweet possible to some google service setup to handle the reception of the tweetstream. That the extent of twitter's support to microsoft and google - every tweet gets shoved down a pipe, one labeled google, one labeled bing. The other end of the pipe will start building its own history of tweets that google/bing will search using its own infrastructure.
Posted by: donpark.org
|
October 21, 2009 4:45 PM
Don Dodge (@dondodge) from Msoft tweeted earlier that "no money exchanged hands" between Bing and Twitter for this deal. Anyone else a little stunned by this? I believe him, but why then would Twitter do this deal if not for revenue?
MarkJ, If Twitter can execute this arrangement with minimal overhead, it should be worth the millions of additional daily appearances of their name on the Web.
Hey, all.
I personally believe the mutual back-scratching here is the crux of the deal.
Google gets crowd-sourced, real-time, user-generated data that's incredibly valuable for determining context and serving ads. Twitter gets an un-freaking-precedented traffic spike from very mainstream Internet users, many of whom will become Twitter adopters.
And yes. Eventually, the Google search users will be on the Twitter web interface. It's inevitable. Did Google video search stop users from getting to YouTube, DailyMotion, etc.? I rather think not. And I also rather think that Twitter as it stands does not possess the back end to support mainstream levels of search traffic.
Hence my speculation that part of the Google deal was one that involves handling traffic. Just my two cents as someone who loses sleep and increases blood pressure over this kind of thing.
yikes. Google's entering dangerous waters. I've already seen this in the searches.. as well as some Google Talk voicemail's showing up (??WTF, REALLY DANGEROUS WATERS THERE!). Hope they can somehow weed out the BS and keep their awesome lead in relevancy for organic results. BING is TRYING this too.. but I'm sure ultimately WILL FAIL and Google will prevail as always!
interesting, possibly reactionary announcement from google today. then again, it may have just coincided with marissa's speech at the web 2.0 summit today.
i saw a tweet from @ev tonight saying that he was celebrating the bing deal. no mention of google. that leads me to believe that twitter isn't generating any meaningful revenue from this deal. sure, traffic is nice, but at some point twitter needs to monetize that traffic. google should be able to help them do that. but, now that google's getting the proverbial milk for free....
since you lose sleep over this sorta thing, jolie, i'm curious as to your thoughts on whether search (even in its real-time incarnation) is still a relevant paradigm for the social web. i touched on the subject here http://bit.ly/YmmOO and hope to write more about it in the future.
@markjeffrey Hey Mark. I think when he (Don Dodge) says that no money changed hands he means that MSFT didn't write Twitter a check today for an upfront fee. Fairly certain that the Bing deal was structured as a rev share. However,the Google deal might have been a blend of rev share & upfront depending on the extent and duration of their (Google's) access to the complete firehose.
Im a bit on the fence about this.Just how I dont care if facebook pages get crawled outside of their teaser info, the “hey whats happening now” just doesnt appeal to me when its outside its universe, mostly because it doesnt make sense to anyone who isnt on twitter.
Jeff Pester,
Interesting ... a rev share deal would make the most sense. Twitter needs to prove revenue, like, now. That would accomplish it with no cash out of pocket.
Tim Wood and Jolie,
I believe SOME kind of value exchanged hands. I'm just not sure what. Possibly it was traffic ... but that does not seem like enough to me. I'm leaning towards Jeff's theory of revshare.
- M
Mark, My Tweet was realted to Facebook, not Twitter. There have been no comments from Twitter or Bing about the details of the deal.
My Tweet related to Sheryl Sandberg's comment, COO of Facebook. In responding to a question from John Battelle she said "No money changed hands. We are not looking to make money on our data feeds."
Hey that will be great if the twitter updates will be there on both Bing as well as google.
The consumers will have a great time because of the war between google & Microsoft also both will try their best efforts for pulling more people to their sites and applications which inturn would yeild more options & ways for the users.
Thanks for the information.
Thanks
Deepak
Don: Thanks for the clarification, apologies if I contributed to confusion by mis-interpreting your tweet!
So: we have zero information on what the Twitter / Bing actually was.