Back when there was hope for Needlebase, the Social Graph API and for Postrank, those services represented hope for the web making the world a better place. Of course people can still use stupid Facebook to organize a protest, or Twitter to speak without hinderance to the world, but with the demise of these three efforts, some important things are lost from the web. These are the kinds of things that a benevolent organization would have invested a lot of support in, for the sake of the world.

Rest in Peace, Needlebase

Needlebase, which came to Google in the acquisition of travel data giant ITA Software, was (is) like a magic wand; you could touch a part of the marbled, veiny web with it - and its magic would flow through every crease and crevice until the web's swirls and pockets were traced and could be illuminated in a flash of data visualization. Specifically, you could point and click to train Needlebase to recognize the various parts of a web page, then jump from page to page of structured data, extracting information and placing it in a database, map or other visual experience.

If, for example, you were preparing to attend a big conference, you could point Needlebase at the conference speakers' biographic entries, show it where the home page links are, where the "next page" links are, and then set it free. Like a cross between a bloodhound, a sheep dog and a magic unicorn, Needle would gather all those links up into a bundle. Set them inside a custom search engine and what have you got? Instant access to the collective published knowledge of every speaker's organization at a conference, usable to better understand what any other speaker says in context. In minutes.

My favorite story about using Needlebase is this one. One day here at ReadWriteWeb we caught wind of a local Salt Lake City newspaper that ran a story about a big new data center opening in town with a mystery anchor tenant. The paper believed that the tenant was Twitter, opening its first data center outside of San Francisco - as the company said it would, in a location undisclosed. We used the (now Google-acquired) web app called Needlebase to investigate.

We grabbed the URL of the Twitter List of the staff of Twitter Inc. and we trained Needlebase's point-and-click screen scraping tool to recognize what a user name, Tweet text and location field (when there was one) looked like on the page of staff Tweets. Then I clicked a button and said "go!"

In just a few minutes, the most recent 1125 Tweets from staff were pulled into Needlebase and we said "show 'em on a map!" Sure enough, one Twitter network engineer had posted a Tweet with a location attached to it right across the highway from the alleged mystery data center. He'd just left San Francisco, he had Tweeted, and arrived in Salt Lake City ready to get to work.

That Tweet was quickly deleted after we reported on it.

Needlebase was one of the most accessible of a class of tools that made data magic available to non developers. Magic.

There is too much information on the web for the human mind to understand it all, of course. The ability to draw sets of it together, to extract and sort it, and thus to discover new qualities about that which is described with the data, is humbling, it is a thing of contemporary existential beauty.

"It's not simply that there are too many brickfacts [datapoints] and not enough edifice-theories," writes author David Weinberger in his new book, Too Big to Know. "Rather, the creation of data galaxies has led us to science that sometimes is too rich and complex for reduction into theories. As science has gotten too big to know, we've adopted different ideas about what it means to know at all."

At least some of us have begun to adopt new ideas about what it means to know at all; there are not millions of happy people playing with DIY data extraction tools as a little part of that experience. And since they have not scaled in adoption, Google has decided to dismantle these instruments of ecstasy, just as the curtain began to rise from in front of the stage where the real story of life was to be seen.

Social Graph API, Great if You Care About Other People

The Google Social Graph API was an API that indexed all the rel="me" links connecting social media profiles around the web. You could use it to search for a person and discover all the places they had profiles. If you cared, that is. Google apparently doesn't, because now there's Google Plus. Presumably, not enough other people cared either. I cared though.

Here at ReadWriteWeb we like to use Martin Atkins' AJAX interface for the Google Social Graph API to find all the places that people post things around the web, just by searching for their names. It's a crude use of the tool, so much more could have been done with it.

Right: a search for my name surfaces my blog, Facebook, Twitter, LinkedIn, Quora and other profiles. Who really wants to learn new things about other people though? What a bore!

How else can you programmatically discover, from one hub of a person's identity, all the arms of their star of activity online? Being unable to know that, or being told to go instead to one single social network to learn more about real dynamic people, feels like a throw-back to the Dark Ages.

PostRank...

The worst loss to humanity at the hands of Google's startup eating monster of late remains PostRank, which Google acquired this Summer. I can't bear to write about that again, but the gist of the story is this: Postrank ingested RSS feeds and then let you filter for just the hottest content coming from any source over time; enabling you to subscribe to a much larger number of voices, with the knowledge you'd be less likely to miss anything really important. Google ended that part of the service and made it all about publishers tracking their own social media accolades, though.

The original version of Postrank was like a magic horn that a woodland fairy called Learning and Empathy could hold to its ear to hear the tiniest caterpillar stretch and yawn in the morning, along with the rest of the whole concert of forest noises (blogs), from all around the world. It was captured by Google and refashioned as a mirror for the fairy's hideous ogre sister Naked Self Interest, which the ogre (a publisher using Google Analytics) thought made her more beautiful and rich with pageviews, but which really only made her uglier and more vacuous every day.

I can't believe they are killing Needlebase and the Social Graph API. I can believe it, of course, but I'm thankful that my cynicism is still thin enough that it hurts every time something like this happens again. There are only so many more tools like this on the web left to kill, though.

Here's what Postrank does: you plug in any RSS feed to the system and it scores each post in that feed by the relative number of comments, inbound links, mentions on Twitter, saves on Delicious and other social media metrics. Then you can subscribe to a filtered feed of just the 10% most-discussed items in any feed. It's magic, it's gold and it's all too often unappreciated. Unfortunately, the company hardly focuses on that aspect of its business anymore. This deal could go one of two ways, very good or very bad, not just for Postrank but for its users and users of the entire social Web.

I once helped a recruiter search for her client company's name in Google Blogsearch. Then, we put the results through Postrank to see who wrote about the company and which items got the most traction. Those were potential recruiting targets.

I have more than twenty art and design blogs run through Postrank and then through an RSS-to-IM service, when one of their articles gets particularly hot. Those are great to read and fun to share on Twitter.

I once built a blog search aggregator for Sun Microsystems' annual conference using the now Yahoo-acquired Dapper to clean up blog search feed output and Postrank to populate a "hottest posts" widget next to the "newest posts" widgets for each of 15 topics. Sun liked that project so much they flew me down to the event and let me meet musical hero and surprise event guest Neil Young. Postrank helped me meet Neil Young - whose music I've listened to during some very trying times.

I built a mobile Web app for designers a few months ago and used Postrank-filtered blog feeds to populate a topical widget of all the hottest articles in the field, along with Dribble screenshots and Tweets from top designers.

I'm going to have a conversation with someone about a topic I don't know a lot about this afternoon, and I'm going to study up quickly by reading the most-discussed blog posts from the most-engaged-with blogs covering that topic.

I once had a dream, I don't remember if I was awake or asleep, of building OPML files of the top blogs in every country on earth, running them through Postrank to filter for the hottest topics being discussed, and giving those files to the Obama administration as a tool for international diplomacy. That was just a dream, but it wouldn't have been hard to create.

These are a few of the incredible possibilities that a service like this makes possible, a service that focuses on listening to other people. Not just listening to what other people are saying about you.

If social media is reduced from a world where anyone can speak and anyone can be heard to a world where we only listen to what people are saying about us or our companies, with each voice ranked for influence score and ignored if it doesn't score high enough, then I think some of humanity's lowest instincts will have triumphed over one of our most potent opportunities to use technology to better human relations.

Please don't kill the part of Postrank, Google, that is focused on tuning our attention to the incredible Web around us.

To check out this app yourself, navigate your phone to the URL designnews.isites.us. Read on for screenshots and a description of the geeky fun behind this little creation.

I began with a service called Widgetbox, which is very attractive and easy to use but in the end got replaced by the iSites site linked to above.

First, I bopped around the Web and found about 30 or 40 of the top blogs about design and typography. I uploaded them to the wonderful service Postrank, which let me filter out all but the top 10% of posts from those blogs that had received the most comments, inbound links, shares on Twitter, bookmarks on Delicious, etc.

I took those 40 or so filtered feeds and I put them into Yahoo Pipes. There I spliced them all together and put the resulting feed of hot blog posts from the design world into my mobile Web app as the first of three sections.

Then, I went to Listorious and I found a good-looking Twitter list of power-designers curated by Paul Olyslager titled UX-VIP. Nice list, Paul! Thanks for building it! I put that List URL into my mobile Web app and now Twitter updates from those 42 UX VIPs make up the second section of content.

Finally, I visited the design community Dribbble, where users upload screenshots of their works in progress and other users like and comment on them. It's an awesome site. One of the sections of that site is a Leader Board of the designers whose work has been Liked the most. I grabbed the RSS feeds of the top 35 peoples' images, put those feeds into Pipes, spliced them together into one feed and then fed them into the mobile Web app as well.

Then I clicked publish! (Well, I put together a few little graphic assets for Widgetbox, iSites whipped up some for me.)

Both of these sites charge about $25 per month to host the apps their services are used to create. (Once a publisher pays for it, readers can access it for free.) That meant, I found out (after doing all the above, like a dummy), that my birthday present for my friend was going to cost $300 for a year. My wife and I really like Brianne a lot, but I thought that would be a bit extravagant. I told Widgetbox that their $25 per month for up to 50,000 impressions might make a lot of sense for many people, but for a one-off app likely to get 100 impressions a month it sure would be nice if I could pay an affordable fee to turn my up-and-coming designer friend on to their fabulous platform!

In the end Widgetbox didn't work for me anyway because there was so much feed splicing insanity going on behind the scenes that the app would time out almost every time I loaded it.

Enter iSites, a similar if so-far less polished service which works similarly but caches the contents on the back end for maximum user experience!

I gave iSites the same feeds, they generously shared an app's worth of hosting at no cost (you can add it too, go ahead!) and they said they'd consider one-off pricing. (Widgetbox also donated an app, it just didn't work as well for me.) iSites also offers a 30-day free trial. The iSites apps need some more work, and the company says that's underway, but they're really pretty good already. Liking and Commenting on content is very cool - sharing on Twitter is very much not, yet.

A person could create mobile Web apps like this on any kind of topic and they sure are fun. Maybe I'll start a Kickstarter campaign to see who wants to pitch in on apps like this for other topics.

Either way, fun with feeds - fun with mobile Web app publishing platforms!

See also: How to Track the Future of the Music Industry

Yesterday Postrank announced the expansion of its browser plug-in to include Google search results, so that select Google results will now be appended with data about their social media engagement. (See screenshot below.) Today, Postrank launched a new data blog: tracking variable social media engagement between players in various industries, like auto manufacturers or newspaper websites. This is super smart and should be a great read.

For Postrank to start its own blog highlighting content engagement trends across industries, funneling readers into more in-depth reports which funnel them further towards becoming paying customers, is very smart. It should be fun for us as readers, too.

Stories told based on analysis of interlinking URLs produced by social media: I think there's a future in that.

If you're not familiar with Really Simple Syndication (RSS) - it is, in the simplest of terms, a powerfully simple technology that delivers new content from multiple websites to one single place you've subscribed to RSS feeds from. RSS has not changed the world in the ways its early adherents hoped it would, but it continues to change dramatically the lives of some of us unafraid to play around with it a little. Below are the 10 most exciting RSS and syndication technologies of the past year.

Flipboard

Selected coverage: Flipboard, New "Social" iPad Magazine will be Powered by Semantic Data

Flipboard is a well-funded iPad app that turns Twitter and other streams of content into a beautiful "customized magazine." Many people have tried to go deep on the visual impact of feed reading on the iPad, but none have embraced the possibilities as gracefully as Flipboard.

You know how I use Flipboard? I read my usual Twitter and Facebook streams through it sometimes, but it's the curated topical Twitter lists that work best on this service. I've got a Twitter list of hundreds of geotechnology pros that serve up incredible topical links. The Twitter list of anthropologists I grabbed from Tlists? What a great magazine they make every Sunday morning!

Web page pre-loading in the background, integrated social media sharing and commenting, video, image collages - the user experience is really hard to beat and it's only getting better. OPML import is the only thing that the 15 of us in the world that like to play with OPML files could ask for more.

Not Dead Yet Factor: Some people have the audacity to complain that this magical creature that turns links to their website into shining, seductive, glossy magazine pages for iPad using readers to slide right down into their websites... is violating their copyrights! That's the dumbest thing I've heard since someone told me that the tens of thousands of readers a Huffington Post link to our site sends are somehow a case of that site stealing from us, too.

Postrank
Selected coverage: How to Build a Social Media Cheat Sheet on Any Topic

RSS overload getting you down? Give Postrank a feed and it will give you back a brighter day. This service, which has been on our best of list every year we've written one about feeds, is invaluable. You plug in a feed and Postrank will score every item in it based on the relative social media engagement that item has seen (comments, inbound links, mentions on Twitter and a lot more). Then, you can subscribe to a filtered feed of just the most-discussed items on any feed.

We use Postrank about 15 different ways here at ReadWriteWeb. It's awesomeness cannot be surpassed. Watch this space, you'll see us use it some more ways in the coming weeks and months.

Not Dead Yet Factor: Postrank's main home page is now a publisher analytics fancy service. If you want to run other peoples' feeds through it, like a sophisticated strategic thinker able to defer immediate gratification for one technology step in exchange for far greater opportunities, then visit http://postrank.com/main.

Notify.me
Selected coverage: Real Time as a Service? Check Out What Notify.me Is Working On

The battlefield of RSS to IM/SMS/email delivery and alert services is littered with bodies - the field of battle between those services and the cold reality of monetization, that is. There are a small number of people who appreciate the delivery of a substantial number of RSS feeds within minutes of their publication, but it's not an insignificant number. It's services like this that keep all the tech blogs you read feeling fresh, readers. Other people in other fields are learning to appreciate them as well.

Notify.me remains alive, despite its own determination to die this summer. The company is now focused on selling advanced services to large, paying customers; it's expensive shooting RSS feeds all over the web by IM and SMS for free.

In July, 2010, the Notify.me team threw up its hands and said it was shutting down its free consumer service. A minor cry for help arose and thankfully, the company changed its mind. It said it was going to start charging people a small amount of money. It doesn't appear to have done so and the messages are still coming.

Let me tell you what a service like this is good for, outside a journalist's immediate interest: I once led a workshop for non-profit organizations where one participant worked in communications at a local women's advocacy organization. In that workshop, we grabbed the RSS feed for the New York Times and we ran it through a filter, filtering for keywords related to the field she worked in. We then took that filtered feed and we put it through Notify.me, setting it for multiple forms of delivery.

The plan, then, was for her to get an SMS whenever the New York Times wrote a story related to women's issues. She could take a look at it, and if appropriate, could call the local newspaper people she knew. "I don't know if you've heard," she could say (they probably hadn't, so soon), "but there's this story breaking on a national level. If you're interested in a local angle, our Executive Director is an excellent source and would be happy to get on the phone with you if you like." The reporter has been looking for something to write about all day and you lay a timely, high-quality interviewee in their lap. Boom! Now repeat a few times and what have you got? You've got an organization that people in your area associate with the issue because you're regularly cited as a source in the local media - because you were the first to know.

Not Dead Yet Factor: It's not dead yet. Someday it probably will be. Another service will have to take its place, or we'll all have to learn how to roll our own.

SuperFeedr
Selected coverage: The Dream Team Quietly Gathering Behind Real-Time Service SuperFeedr

You've got online content and you want it in real time. You want it in different formats. You want it marked up with geolocation data that corresponds to place names occurring in free text. You want it all and you want it for a fair price. What does it mean? Maybe you want SuperFeedr. It's like FeedBurner was for bloggers, but much more developer-focused. The company adds features all the time and founder Julien Genestoux is one of the most agile technologists you'll find online.

Not Dead Yet Factor: Barely born yet, but backed by BetaWorks and Mark Cuban, that's good for something. Plus Genestoux builds features so fast that he'll likely fit whatever need real-time feed geeks find they have, well into the future.

Google Reader
Selected coverage: Facebook Could Become World's Leading News Reader (Sorry Google)

If you read RSS feeds and you know it, you probably use Google Reader. It's ok. It's pretty good, even. It's not that exciting, but it serves a whole lot of people very reliably and capably. It has survived while everyone else has not. This year we saw former market-leader Bloglines and former innovation leader Newsgator Online close up their RSS readers and send everyone to Google Reader instead. Other services use Google Reader as a place to sync up.

Not Dead Yet Factor: Google almost never kills anything, and there have to be a lot of people internally at the company who depend on Google Reader, too. Unless they've all given it up for Twitter.

My6Sense
Selected coverage: My6Sense & The Geek Who Rode His Blog to the Edge of the World

You're on your phone and you want something good to read? They say that small screens lend to high-quality recommendations of well-targeted content - so why would we read Twitter and Facebook?

My6Sense is a mobile RSS reader that syncs with your Google Reader account (all of it, not just the first one thousand feeds like so many imitations!) and then watches how you interact with the items. It knows when you are reading, it knows when you've shared a link. It then offers two views of all your subscriptions: their most recent posts and the My6Sense recommended posts. The service learns from your behavior over time and offers a quality mobile feed reading experience.

Not Dead Yet Factor: It's probably a slow burn, the company is focusing on monetizing a commercial API. That's a good business to be in.

Blekko
Selected coverage: How to Use Blekko to Rock at Your Job

Blekko calls itself a search spam killer but it's got a whole lot more potential for the power user.

Blekko is a platform for collaboratively edited vertical custom search engines. It eats OPML files, among other things, and its outputs include RSS feeds. You want a feed of updates from 10 key medical sites whenever any news about a particular issue is written about? Blekko can do that. You want to track a collection of blogs that cover a particular topic and get a ping when they write about one company, one concept or one keyword across all their blogs? No problem. It's great.

A custom search engine creation service with RSS feeds. That deserves a place on this list.

Not Dead Yet Factor: It just launched. When it launched, I said it was too beautiful to live long, but its CEO has been around the block many times and tells me he knows what he's doing.

Facebook
All RWW coverage of Facebook

Facebook's syndicated updates from friends, families and media organizations are the single most important way that hundreds of millions of people around the world relate to the power of the feed. The company tried to do a lot this year, but it's hard to know how drastic the users' experience will end up being. None the less: Facebook Places alone represents the introduction of a radical new type of knowledge into many peoples' lives (where the people you know are right now) - and it's coming to them by feed.

OStatus
Selected coverage: Run Your Own Twitter Clone: Status.net Launches Public Beta

When you hear about Diaspora, when you hear about Status.net, OStatus is what's under the hood. This open-source amalgamation of communication technology standards is like Twitter for networks that are disconnected, but interoperable. "People on Different Networks Following Each Other" is the OStatus slogan.

What does it mean? Interoperability means social networks compete on features, not control over your friends, because switching costs are removed. You lose nothing if you switch networks.

OStatus didn't take off like a Tweeting rocket ship this year, but it saw some continued growth, development and attention. Someday, maybe someday, the asynchronous micro-messaging that so many of us find so much value in will break out of the clutches of one single company (wonderful as you are, dear Twitter) and become a real communication platform like the telephone. That's probably as crazy as imagining a time when AT&T customers can call Verizon customers though, isn't it?

Not Dead Yet Factor: It's not dead yet.

Dapper
Selected coverage: How Yahoo's Latest Acquisition Stole & Broke My Heart

Point and click on almost any field on almost any Web page and Dapper will give you an RSS URL you can use to subscribe to updates from that field, if and when the content there changes. It sounds like a simple thing, but it's incredibly powerful.

Dapper has been one of my favorite services for years and was joined by Needlebase in the DIY data extraction world that has so much potential.

In recent years, the devil bought Dapper's soul, turning it into a semantic advertising platform in order to monetize its core technology. Then Yahoo bought the whole company this Fall, which will allow the core feed-extraction tool to remain open, at least for a while longer. To use this incredible tool, you've just got to sneak in through the back door at Open.Dapper.net.

Not Dead Yet Factor: It's not dead yet. Maybe more alive than it's been in years, in fact.

Honorable mentions:

Yahoo Pipes - definitely not dead yet. The company released an experimental 2.0 version of this wonderful spaghetti pipes tool for RSS magic this year, but few people noticed and the company itself says its products aren't production ready. YSQL is a better bet, if you're comfortable working with that. If not, well - Pipes isn't dead yet.

Twitter - One of these days! Annotations! Meaningful location as a platform! This year had high hopes for Twitter's technology. The year ended up being about better up-time, a prettier Web site and the company's nascent ad sales efforts.

Ogre translates spatial files into GeoJSON using a command line tool for use in JavaScript Web apps. Awesome. Some people are using this for sure, to set proprietary geodata free. Too few people, though.

OneSpot - This content recommendation engine does a lot of things, but my favorite thing it does is look at any set of feeds you give it and then suggest thousands of other feeds it believes are related. It's easy to curate a few hundred top blogs in any field that way.

That's our list - how does it compare to yours? What's coming down the line that you think might shake things up in RSS and syndication in 2011? Let us know in comments.

PostRank keeps track of how content gets passed around on sites like Twitter, Delicious, Digg, Reddit and many more. The service's ability to track reader sharing and discussion on Facebook has been hampered, however, by Facebook's being closed off to outside data collection. This new solution, a Facebook app that publishers authenticate with, is a smart way to make progress towards solving that problem.

I personally am of the belief that the most valuable way to use PostRank is to use it to track other peoples' content, to discover emerging news from the hinterland of niche blogs, but despite the rise of world-changing publishing and subscription technologies - most publishers remain interested primarily in broadcasting and navel gazing. Content marketing optimization is a much better business to be in than the systematic illumination of well-blogged glimmers of insight into the human condition or any other topic. (If you are interested in using PostRank to better understand and discover highlights from other peoples' content, visit postrank.com/main.)

Details on the offering will be announced on the Postrank Blog this morning and the app can be seen live on Facebook.

Postrank co-founder Ilya Grigorik added another metric to Sherk's analysis: engagement per unique visitor. Can you guess which major media outlet scored the highest? It was the Guardian, in the UK. Next in line for most engagement per unique visitor were Slate, The New York Times, the BBC and The Economist. See below for a chart displaying the top 30.

You can see the raw data on this Google Spreadsheet. What do you think helps these leading sites rise to the top and transition so effectively to a social media world?

Note: A question has been raised in comments about MSNBC's numbers, because of the way its traffic resolves to an MSN.com domain. Short of recalculating, we'll suggest readers take into account that the organization's unique visitors per month may be substantially undercounted.

Is experimentation with social media by a media organization itself a factor in how much its content gets shared? The Guardian has what's arguably the media world's best iPhone app. Or is it most important to simply produce great content? Clearly social media traction per visitor doesn't lead directly to business success - Newsweek is beating out MSNBC and Forbes in this chart, but it was recently bought for $1! Does social media traction matter, from a business perspective?

This kind of an analysis is interesting to see run on big, general interest publications - but it can also be applied on very niche topics. See our new series The Top 10 Buzziest Blogs in Geolocation This Week, for example. I don't know how many of those could sell for more than the price of Newsweek - but they make great reading and knowing the top sites in that field offers all kinds of other types of business and technical value.

Now it's easier to use than ever if you're a Google Reader fan and on the Chrome browser. The company just released a Postrank extension for Chrome. It's not perfect, but it works much better than the previous Firefox extension did. You should try it.

The performance of this extension is much faster than the Firefox extension. It won't slow down your browser and it tracks conversation metrics in real time.

Unfortunately, the scale of relative conversation is "contextual" which means if you have a folder of feeds from tech blogs, and you select the folder view, the relative conversation levels will be tracked across all posts from all sources. In other words, the most popular blogs will have written the most popular posts. That's not very helpful. Postrank is most useful when it shows you unusually popular posts per source. So don't show me every post that ReadWriteWeb has in my tech blog folder just because it's an unusually popular blog, show me unusually popular posts for each of the blogs in my tech blog folder. I hope the company will change that.

None the less - this is a great way to see what's most popular on a blog-by-blog basis, with that blog's regular audience of readers. It's also a good time saver for Google Reader users. Google Reader's own Sort by Magic feature is a cool way to see universally popular content, but if you're into blogs about real-time geofencing APIs with natural language processing, or the ceramists of Lake County, Florida, or whatever other obscure topic you might be interested in - now Postrank plus Google Reader will show you the hottest posts in those sectors. That's very nice.

Now, however, with the Google Buzz firehose, PostRank subscribes to one real-time PubSubHubbub feed of all publicly-available Google info. PostRank's users, the company says, will notice a big difference in the amount and depth of Buzz content.

"If you're an Analytics subscriber, then you will see greatly improved coverage of Google Buzz starting today: more users, more meta-data, and a more complete activity stream on your dashboard. And if you're a user of our Data Services, RSS feeds, or API's, then you will also undoubtedly benefit from the improved coverage of Buzz activity: more accurate engagement scores, and more accurate PostRank scores."

PostRank measures the level of public engagement with online content. It determines how interesting and relevant a piece of content (blog post, article, vides, etc.) is "by analyzing the types and frequency of an audience's interaction" it. So the ability to completely represent a popular service like Buzz is integral to maintaining PostRank's efficacy.

Read more ReadWriteWeb coverage of PostRank and Google Buzz.

Read on for graphs of engagement below. The same analysis performed here can be run on any sets of top-level domains using the newly released Postrank Domain Activity API.

Postrank measures engagement by comments to posts, mentions on Twitter, inbound links, votes on Digg and many other quantifiable metrics. All the numbers below are over the last 90 days. Blogspot, the domain for Google's Blogger blogging tool, still reigns supreme almost 11 years after it launched.

What's the takeaway here? I look at these numbers and think a couple of things. First, it appears that the world-changing democratization of publishing by the first wave of blogging tools has had some sticking power. Second, just because our early adopters' scene and the mainstream media are talking about Facebook and Twitter instead of "the bloggers" nowadays, doesn't mean that people have abandoned blogging. Finally, even though curation services like Tumblr and Posterous are pretty awesome, they haven't gotten as much mainstream mindshare yet as the old classics have.

I think these engagement numbers are a good snapshot of the state of the technology over time. I find it encouraging that there is such a thriving blogging scene still today.

What do you take away from these numbers?

Postrank is generally very useful and accurate, but the maze of social media links can sometimes be challenging to penetrate. Running Facebook.com and Twitter.com through the API turns up an unrealistically low amount of engagement with those domains. Both are reported to be roughly 3 times as big as ReadWriteWeb. I suspect the ratio is a little bigger than that.

Other Interesting Observations

What other domains are being talked about across social media conversations? Postrank made some interesting observations today, including:

Twitter users talk about Google, Facebook users talk about Yahoo

Honda gets talked about far more than Ford or Toyota.

The geo map works similarly, but instead of extracting proper names, places and things, this algorithm just focuses on places. Thanks to this, a PostRank customer can easily see which cities, countries and regions are currently being mentioned online.

The demos are now available in PostRank's new Labs section - which opened earlier this month.

So does that mean that nobody participates in Buzz? It's hard to imagine more premium placement for a service than inside every Gmail inbox, so why hasn't Buzz caught on? To be fair, it's hard for any service to compete with the volume of imported Tweets and easily added RSS feeds. The fact that 10% of content published is added manually might even be seen as an early success... maybe.

Of course the best part of Buzz is the conversations in comments. In my stream at least, I see some amount of conversation but it's dominated by a few uber-geeks: people who loved FriendFeed before it was acquired by Facebook.

The whole Buzz model looks a lot like Facebook does these days, in fact. It doesn't do much else for users, and there are fewer people being social there. Why use Buzz when your friends are on Facebook? Perhaps that's the question and why Buzz hasn't caught on.

We're excited in principle about Buzz because of its potentially disruptive support for open data standards. Apparently it's mostly robots who get excited about such things, though, as they are mostly the ones coming to the party so far.

There are some hardcore Buzz users discussing this...over on Buzz, too. If you do use Buzz, you can be our friend here. We never post automated content there.

You can see last year's list here and most of those remain important services. Only one service makes a repeat appearance this year. It was a very big year for this class of technologies, after a long, sleepy period the Real-Time Web began to cause substantial disruptions over the last 12 months. Check out our list below and let us know if we've missed anything important or who your picks might be for next year.

Facebook has 350 million users today. Just 12 months ago there were a mere 140 million Facebook users. A syndicated stream is the default view in Facebook, meaning that 210 million more people have been introduced to this paradigm by Facebook in 2009. That's a powerful cultural change.

Twitter may not be anywhere near the size of Facebook, nor growing as fast, but for tens of millions of people, 2009 was a year they got comfortable with streams, lists (just like cute little OPML files!) and soon geolocation data - thanks to Twitter.

Echo, from JS-Kit is a reverse syndication service for distributed social media conversations. It brings back tweets and other mentions to the page they refer to. The service is growing fast and becoming more sophisticated every week. New features come so fast and furious that it's overwhelming but the end result is an experience that brings the dispersed social web back together again.

Fever is a gorgeous new RSS reader that costs $30 and lives on your own server. It's got a very interesting system for ranking hot stories by your own criteria - we just wish we could change the timeframe so that ranking was for every 2 or 3 hours, not per day. Fever looks great and works wonderfully on the iPhone. If people ask you what good web-based alternatives there are to Google Reader, Fever is a good place to start looking.

PubSubHubbub (and RSSCloud) are two feed formats for the real-time web. PubSubHubbub is method for pushing real-time updates from a publisher, to a hub and then to all subscribed parties - immediately. RSSCloud is a similar technology that originated years ago as a part of the RSS spec. These are the protocols that a whole new era of user and developer experience on the web will be built on.

Superfeedr is a new service powering millions of real-time feeds. It's a transformer, from lots of different formats into real-time feeds in PubSubHubbub or XMPP. It's like FeedBurner for the real-time web.

Tweetdeck (and Seesmic) are the market's leading stream readers. They are tools for reading and writing to Twitter, Facebook, MySpace, LinkedIn and someday other social network streams. There are lots of innovative stream readers on the market, from the beautiful Skimmer to the Inspector-Gadget style Favit, but Tweetdeck is the clear market leader. It's in a perpetual back-and-forth battle of the sweet features with Seesmic. Both are dramatically changing the way users experience the flowing social web.

ActivityStreams is a proposed standard way to markup user activity data in social networks. If everyone adopted the standard, then streams of data would be interoperable, we could see what friends on other networks were doing and we wouldn't be locked-in to the big networks because little innovators could provide tools for conversation. So far Facebook, MySpace, Netflix, Sun Microsystems and more are working hard at making this a reality. 2009 was a big year for ActivityStreams, right down to last week's announcement that a feed normalization API was released by startup Cliqset.

The Breaking News Online iPhone App is the best remnant of a fabulous story that's changed dramatically in recent weeks. BNO is a news organization that's so fast in breaking news from around the world that the Red Cross watches them for disaster news and MSNBC syndicates their stories. Unfortunately, the company owned by now 19 year old Michael van Poppel sold control over its wildly popular Twitter account to MSNBC this Winter, but the iPhone app remains a very valuable resource. BNO's research and original reporting is definitely one of the biggest stories in syndication of 2009 and its iPhone app is a must-have.

The numbers paint a stark picture: blogging has changed, but the blogging scene is in some ways in better shape than it was three years ago.

Highlights from the report include the following:

• Total reader engagement has grown 30% year over year or 69% total for the top 1,000 feeds, which includes blogs and mainstream news sites.

For 100,000 randomly selected blog posts in each of 2007, 2008 and 2009...

• Engagement on-site has grown in absolute terms but the share of total engagment that happens on-site vs. off-site has dropped 50%.


• Trackbacks have fallen from 19% of engagement to 3% of engagement.


• Engagement on social networks like but not limited to Twitter, FriendFeed and Facebook has grown from 1% to over 29% of total engagement. The Postrank staff admitted that this was a surprisingly low number but said that in aggregate there is still a whole lot of activity going on outside social networks.



• Segmenting from the last amount of effort required up to the most, reader engagement now looks like this: 29% is link-sharing on social networks, 29% is bookmarking or voting on sites like Delicious, Digg and Reddit, 38.5% is comments on or off-site and trackbacks are now 3% of engagement. "Trackbacks are taking a nose dive," Postrank CTO Ilya Grigorik told us by phone, "bookmarking sites have consistently gone down over the last 3 years, but voting on sites like Digg or Reddit has grown."


• Perhaps most significantly, blog posts now have a longer life span. In 2007 tracked posts saw 94% of engagement within the first day and 98% of that first day's engagement happened within the first hour. In 2008 that number fell to 83% within the first day and in 2009 it was a mere 64%. Thus Postrank concludes that 36% of reader engagement in the top blogs happens after 1 day. "While the real-time web is all about lowering the latency," Grigorik says, "the pervasive nature and number of people engaged in their communities and conversations (the Social Web) is helping with information discovery. People are worried that the real-time web will destroy their readership as everyone just gets distracted by the newest shiny thing on Twitter, but the numbers show something very different. It's so easy to spread information now that it lasts longer and finds more niches - this trend is helping content travel further."

If you can't make it to Mountain View, California in eight days, get ready to watch selected sessions streamed live online (thanks to Justin.tv). This isn't going to be talking-heads pushing their products on stage, this is going to be a high-value brainstorming, networking and collaborative learning. Check out the companies below; they are bringing financial support to this important gathering to talk together about the future of the internet.

Now check out these fabulous sponsors.

Enterprise

TIBCO

TIBCO's technology digitized Wall Street in the '80s with its event-driven "Information Bus" software, which helped make real-time business a strategic differentiator in the '90s. Today, TIBCO's infrastructure software gives customers the ability to constantly innovate by connecting applications and data in a service-oriented architecture, streamlining activities through business process management, and giving people the information and intelligence tools they need to make faster and smarter decisions.

Filtering

PostRank

PostRank offers a website and developer tools that make sense of social engagement data on the web. Want to discover the bloggers with the most reader-engagement or the blog posts that are hottest, in any niche? PostRank will do that for you, in real time. Whether you're a very large company or an individual blogger, PostRank can do things for you that no one else can. We at ReadWriteWeb like and use PostRank a lot.

Search

Faroo

Faroo is a P2P real-time search engine that combines explicit and implicit data to power its indexing and ranking technologies. The company specializes in difficult real-time analysis of international content, like breaking up long strings of Chinese characters for text analysis.

Publishing

WordPress

WordPress is one of the world's leading blog publishing services and software. WordPress made free real-time updates from millions of blogs available last month.

Aggregation

Nomee

Nomee is an application to manage the most important parts of your digital life. The Nomee desktop client aggregates activity updates from more than 120 different sites that your friends and favorite public figures are using, filters those streams and lets you view media in a beautiful interface.

These sponsors have made an important move in supporting the Real-Time Web Summit so please check out their products. If you're interested in sponsoring the event as well, it will be worth your while. Contact sales@readwriteweb.com for info.

Please join us for this important conversation on October 15th in Mountain View, California and streaming live online.