When the number of publishers and media outlets, from the casual to the professional, exploded in the mid 2000's, many people were sure that this body of content needed to be searched differently than Google searched old fashioned web pages. Scores of blogsearch startups emerged and now almost all of them have faded away into obscurity or advertising. One stayed vibrant and innovating, though, after all these years:IceRocket. This week IceRocket found a new home at a larger company, social media monitoring firm Meltwater. Icerocket meets Meltwater seems like a story of liquefaction, but time will tell how the technology transitions into a new post-search era.

Right: A survey of the fast-collapsing blog search market, circa January 2009.

There were no News Feeds or Walls, only what was called Really Simply Syndication readers (like Google Reader). We're talking ancient history here, people, more than five, maybe six years ago. (Decades at least in the dog years of fast-evolving social web technology.)

User experience and interface design were big challenges for all those companies that knew they had tapped into something incredible, but couldn't yet convince most of the world to get onboard. It was, and is still, a life changing experience to open up an RSS reader and consume a roaring river of news and opinion from the publishers you've hand selected to check for new content from automatically, all in one centralized place.

But for reasons that nerd historians will perhaps only someday understand, that stuff never caught on at scale.

Since then, IceRocket continued innovating. It learned to ingest granular fields of data in the blog posts it found, it expanded to include the faster-growing worlds of Facebook and Twitter, it looked the Social Media Spam Monster right in the eye and it started throwing punches. (Or at least, it did a pretty good job of spam control in a very spammy space.)

Where does a last-decade blog search engine go now? Into social media monitoring, that's where.

Even back in the blog search days, people knew that real power lay in subscribing to persistent searches. The tools to do that were, apparently, too bare metal for marketers, though.

I once did a project for Sun Microsystems where I put together 10 search queries each with at least 5 Boolean operators for Google Blogsearch, then scraped the results pages to remove HTML cruft and pushed the results into one feed delivered straight and another feed run through a buzz filter, then sent both of those into a Planet aggregator and put them up on the Sun site for the company's annual developer conference. It was awesome and in addition to paying me nicely for it, the company flew me to the event to interview singer Neil Young. It was RSS rock and roll.

These days that kind of stuff just isn't going to fly, though. The social web has exploded beyond cat gifs and jokes about how is babby formed and is now full of everyday people posting little updates about sex and TV and corporate brands.

Those brands want tools that are half magnets and half divining rods; technology that will find any badmouthing about their crappy products, decide whose opinions really matter and then place a button that can be pushed to neutralize the negativity in the laps of PR agents, with a pretty red ribbon tied around it. They are willing to pay a lot of money for that, too. (Other use cases include facilitated, real-time, authentic community engagement at scale.)

One of the firms selling social media monitoring quite successfully is the now ten year old Norwegian firm Meltwater. Meltwater was one of the first firms in Europe, if not the world, to bring press clippings services into the web world.

Meltwater booked $100 million in business last year and is now scooping up tech startups. In February the company acquired social media monitoring startup BuzzGain and in March it acquired social CRM startup JitterJam. Now it's swallowed up IceRocket and will integrate its technology, mathematicians and other staff into the Meltwater suite of products.

What will that look like? It's hard to say, but it's clear that the big dreams of blog and social media search haven't panned out as stand-alone businesses. Meltwater is an example of the red hot world of automated, algorithm driven pull-based media monitoring and display. The company has several different products and the one I saw demonstrating did not impress me very much at all. The others are said to be more powerful but the demand is clear for anything that will help the rest of the world manage the torrent of Tweets and Facebookings and Plusses.

And so what was once a promising startup in the past gets subsumed by a future that outdid what seemed like a deafening storm of new publishing, not so long ago. If you'd told the blog lovers of the last decade that the medium would be dwarfed by a much, much larger and arguably more disruptive movement of self publishing in the form of social networking, I don't know what people would have said. That seems to be what happened though.

In order to join a conversation, you've got to be able to find it first. Three years ago "blog search" was expected to be a booming industry, startups left and right developed different technologies and more than a few raised millions of dollars to help users search the part of the web made up of blogs. These days no one thinks consumer-market blog search is a serious business, but many of us still have a need to limit searches to blogs. What should we do? ReadWriteWeb offers some recommendations and an assessment of the state of the industry below.

Different circumstances call for different search engines. We've made a chart below illustrating our different recommendations to fill different needs. When, for example, we're looking to see if anyone else has written about a breaking news story yet - we use Google Blogsearch because it's the fastest. When we're putting a live search feed on a public web page, though, we use Technorati and crank up the spam-control it offers. Many businesses use profesional blog tracking services for some of their search needs, but we're not convinced those services are as useful as grabbing some of these worn old tools and doing it yourself.

Where These Services Stand Today

Technorati is the old stand-by, the blog search engine that the smartest blog lovers used to use. These days it's a sad shadow of what it used to be. The company leadership is focused on building an advertising network and search features have been shed like there's no tomorrow. The company's developers say that features will be returning, just in a more accessible form, but we're not holding our breath.

The service is slow, misses a lot of search results (perhaps in the name of spam prevention) and is so loaded down with cruft and extraneous page loads that it makes us want to scream.

That said, the fundamental value proposition of Technorati remains - it counts inbound links to every blog it has indexed and it will let you sort by that metric of "authority." More advanced RSS-heads will appreciate the fact that Technorati delivers "authority" numbers in its RSS feeds and those numbers can be used to fine tune spam filtering in Yahoo Pipes.

Google Blogsearch is the fastest in the industry but has gone almost untouched since the day it launched, except for a recent dabble with memetracking on the front page. Google Blogsearch spam control is not good and recently the search engine started bringing back search results from places like blog sidebars. It thinks that content is new, too, every time a new blog post (the content we really care about) is published. It's painful to look at Google Blogsearch results pages, but if you've got a need for speed or want to make use of the relative heft of the Google search input box for things like complex queries - then it's a good option.

IceRocket is Mark Cuban's baby and has improved more in recent years than anyone else on this list. It's quite a sophisticated tool for searching blogs. It's got trend analysis, author awareness and a number of other cool features. Unfortunately it only lets you organize search results by data and sometimes other needs arise.

IceRocket also misses some search results that even Technorati catches, though it catches some that Technorati misses as well.

Ask.com Blogsearch has become an unexpected favorite of ours over the years. It's nice. Spam control is pretty good, speed is pretty good, the size of the index is pretty good. It's a pretty good blog search engine. The best thing about it is that it's very easy to sort results by relevance, date or "popularity" of the source, as defined by the number of subscribers the source feed has in Ask's formerly market dominant feed reader Bloglines. Want to find out who the biggest blogs are that have written about Chihuahuas lately? (We'll just tell you, it's Jalopnik, Celebrity Baby Blog and Fark.)

If there's a downside here, it's that Ask does index a fair number of feeds that aren't really blogs. And it doesn't do anything else that's particularly fabulous. None the less, we find ourselves going back to it every day.

FriendFeed is a lot of things, but it's also a blog search engine of sorts. It's a cross-network, real time social site originally built by a team of ex-Google employees. It's pretty awesome and once you've got an account there you can search blog posts, Twitter messages, YouTube videos, SlideShare powerpoint presentations and much more. The down side is of course, it only lets you search the content that other users have synced with their FriendFeed account. That content has a whole lot of conversation going on around it though! Several members of the ReadWriteWeb team use the newly launched FriendDeck to do real-time tracking of FriendFeed. You can meet our whole team on FriendFeed here or join us in the RWW room (open to anyone) here.

That's How We See it - What's Blog Search Like for You These Days?

We'd love to hear about your favorite blog search tools these days. What do you use and in what circumstances do you use it? Is blog search itself old news in a new era of real-time microblogging? We welcome other perspectives on this field that may have lost some of its luster but remains useful and important several years after it was so hyped.