Writes Marquez, "Tom Curley did not clarify how a product that provided some news earlier would work or specify the target customers for the potential new service." However, more than anyone, we know the value of this service.

Independent bloggers continually struggle to scoop rival outlets to the point that embargoed releases are often broken. Some outlets have even chosen to forgo embargoes all together; however, ReadWriteWeb makes every effort to honor them. To a tech blogger, a 30 minute lead may mean the difference between the front pages of Digg, Techmeme and major traditional outlets, or appearing like a pack journalist. The groups that purchase a 30 minute lead on tech stories will solidify a huge influx in feed subscribers and mobile app downloads and as a result, gain more favor with advertisers.

Photo Credit: Marina Campos Vinhal

AOL's PlaySavvy: PlaySavvy is another site that offers parents a chance to check out age-appropriate kids games. Similar to What They Play, this site lists the ESRB ratings next to games and information is categorized by topic and gaming system. Products for console favorites like Xbox 360, PlayStation 3 and Wii are often featured on the site. AOL also offers parents daily articles and a section to view the most popular games.

Parents' Choice: Established in 1978, this site is run by the Parents' Choice Foundation - a nonprofit organization that offers awards to the top children's media and toys. Some of the award categories include websites, video games, DVDs, software, television programming and toys. This site definitely skews towards narrative-based educational materials. If your kid has a shorter attention span, you might want to visit PBS Kids instead.

Common Sense Media: Common Sense media is a nonprofit organizations dedicated to offering parents reviews and suggestions on appropriate content. The group produces movie, website, TV, game, book and music ratings as well as a number of educational tools for classroom use. The site can be personalized for kids between the ages of 2-17 years old and it's available in English and Spanish. This site tends to weigh the benefits of mainstream media sites.

PBS KIDS Island: This site offers parents reading resources, videos and games in English and Spanish. While the majority of the content is best queued up by a parent or teacher, the videos are of particularly high quality and completely commercial free.

National Geographic for Kids: This site offers kid-friendly games, videos, stories and activities for nature lovers. The same photography that has made National Geographic a hit with grownups is likely to please kids. Not surprisingly, the video content is equally compelling. Discovery Kids offers a similar experience and is a great site for visual learners.

Totlol: Totlol is a parent-moderated version of YouTube. The site offers video options specifically designed for children and even offers "age optimized interfaces" to ensure that kids of different ages get the most appropriate content. In 2008, ReadWriteWeb covered Totlol's beta launch. As of today the site charges a $3 per month subscription fee or $54 until your child grows up.

Photo Credit:Swaminathan

The Timeline

According to details on the newly revamped Silentale web site, the startup has three main features: the "Timeline," the "People Book," and "Connectors." In the timeline, you're presented with a view of all your messages from around the web and even from your mobile phone. Silentale is able to connect to Facebook, Google Contacts, most POP3/IMAP email accounts (such as those you might access in Outlook), Gmail, AOL Mail, Yahoo! Mail, Twitter, and it can pull in your SMS messages from your mobile phone. All these are presented in the scrollable timeline view with icons indicating their source along with the date, subject, sender, recipient, and of course, the message itself.

Connectors

The "Connectors" page is where you set up the various connections to the platforms whose messages you want archived. The list of connectors (see above) is short right now, but they promise more will be added in the future. These connectors crawl through your messages and contacts and archive them on the service once you've authorized Silentale to access those accounts. Not only will the service pull in the messages from that point forward, it will also work backwards in time to retrieve older messages too. Attached documents, including Office documents, photos, videos, and links will also be archived. Although at first you won't be able to search within these documents, that functionality will be "introduced shortly," reads the Silentale FAQ.

As for the SMS messages, they'll be archived using special mobile applications. At the moment, the company has developed an iPhone app and an Android app which both use your data connection to archive each text received to the Silentale service. Our only concern with this feature is in regards to those of us who choose to receive either Facebook or Twitter messages via SMS. Since that would be a large number of updates, it would be nice to exclude certain SMS short codes from the archiving process in order to save our precious battery life, which no doubt, the SMS archiving apps would eat up. Besides, since both Facebook and Twitter are available "connectors" on the Silentale service, those particular SMS texts would be redundant.

The People Book

Finally, there is the "People Book" view which is essentially an aggregated address book. Silentale finds the duplicate contacts from across your networks and combines their information together, merging their email, phone numbers, addresses, profiles, etc. into one single contact. This list, like the other views, is searchable, but it can be filtered by network as well to help you find your contacts with ease.

When you click on one of these contacts, all your conversations from across the various supported platforms are displayed. A message timeline at the top of the page lets you hop around from month to month and year to year, too.

Get Your Invite Now!

Of course, since Silentale hasn't launched yet, it's too soon to praise the service. We have no idea how well it will work. For all we know, it could be buggy and slow. But on paper, the service looks useful, promising, and - dare we say it? - exciting. How incredible would it be to have a master copy of all your communication from everywhere in one searchable resource in the cloud? We think it would be great. It's especially exciting since there aren't good ways to archive and search through your communications yet on some of these supported platforms - like Facebook and Twitter, for example.

Still, there may be some concerns about security when it comes to this service. How will Silentale access these networks? Hopefully they'll tap into Twitter via OAuth and into Facebook using Facebook Connect, but will they ask for our email passwords? That always makes people a little uneasy.

During the beta period, Silentale will be free, but when they publicly launch, it will be offered as a "freemium" type service. The basic (free) plan will allow 5 connections, unlimited contacts, but only 8 weeks of message history and a total storage space of 2 GB. The unlimited plan, which looks to be $50/year, will offer unlimited everything.

As we mentioned earlier, Silentale isn't open for business yet, but you can go ahead and sign up for your beta invite right here on the Silentale homepage. Just click the link on the upper-right.

With this new project, BBN hopes to "develop techniques that can generalize across the linguistic structure and content of diverse documents to extract relations and axioms directly from text rather than relying on a knowledge engineer to encode such information." Here's how it will work:

Although it is not immediately clear when (or if) this new machine reader will be available to civilians, we are certainly looking forward to trying something like this out. Some paranoid types will believe this is nothing more than "the man" trying to spy on us, but those people need to realize everything we do online is being watched by someone. If you are really concerned about your online privacy you should secure important data on your computer, call your government leaders and try to change privacy laws, or stay off the Web altogether.

Researchers, medical professionals, consumers, students and others are all likely to benefit from such an application. Not having to spend unnecessary time searching through mountains of information on the Web for something relevant makes life easier and allows us to be more productive.

That's where the creation of groups of sources comes in. Various services have different ways for users to separate their "friends" into different groups, viewable by topic, category or type of connection. Facebook is making changes today to make it easier to break your Facebook Newsfeed into groups. That's going to be very important. The best Twitter applications offer group functionality that the site itself doesn't. MySpace offers no such feature, yet. The Facebook news prompted us to try to articulate the value of group creation online. By better understanding the value that groups can deliver, we can better strategize our creation of groups.

Forgive me if this is all obvious to you; I know it's not to everyone. Even if it is, I think there's value in discussing fundamental qualities of emerging methods of communicating. The assumption in discussing these values is that you're an ambitious knowledge worker. If that's not the case then this logic may or may not apply.

The Value of Groups

Prioritization

Pulling high-priority sources out of the full stream and putting them in a special place enables you to catch more of the high-value information those sources publish. Why lose valuable messages in the whole sea of marginally valuable information that we all have access to? High value sources don't always publish high value information, but the increased likelihood of their doing so warrants putting them in a special place so that the unusually high signal-to-noise ratio they offer is maintained.

Below: I follow thousands of people on FriendFeed but have about 100 people who often discover or make news early in their own group.

Context

Different words and links have different meanings in different contexts. When subscribing to a large set of sources it's sometimes easy to forget who or what certain sources are when their content comes barreling down a full stream of information. Placing sources into contextual groups helps put messages in context, adding meaning and offering insight into the significance of some content.

Right: I've got a list of 300 tech industry analysts on Twitter pulled into a separate group in Tweetdeck. It's good to know that when I read these messages, they are coming from professional analysts.

Intimacy

We've all got far more connections online than it's realistic to maintain closely. You may be familiar with the concept of the Dunbar number. Researcher Robin Dunbar argues that 100 to 150 is the approximate natural group size in which everyone can really know everyone else.

Serious users of social media often maintain far, far more connections than that, though. What can you do? Strategic creation of groups facilitates social contact disproportionately frequently relative to contact with the entire list of our social connections. That disproportionately frequent contact lends itself well to greater intimacy.

Left: The ReadWriteWeb writers' chat, an invaluable resource for us in a world swimming with social connections.

Speed

Some sources of information are more important to be up-to-the-minute with than others. Strategic creation of time-sensitive groups allows you to have those groups alone delivered in a way that fits their time-sensitive nature. You don't want to be interrupted by updates from every source of information you have any interest in - but some sources are worth being interrupted by.

Right: High-priority RSS feeds delivered by IM.

The Value of the Full Stream

Many people are tempted to solve information overload by cutting back on the number of connections and subscriptions they are signed up for online. We argue that this is a mistake; group creation can help capture some of the same benefits of cut-back without incurring the loss of benefits felt by restricting a well-populated stream of information.

Serendipitous Discovery

Do you cancel your cable TV subscription just because you end up not watching most of the shows that are on at a given time? (Maybe with Hulu you do now.) Probably not. Channel surfing is a way to discover new things.

So too with the web; it's better to have too many options than not enough. Subscribing to a source of information substantially increases the likelihood that you'll see something good from that source.

Just don't worry about reading everything. Scan what you can and let fate bring you value from a bulk of undifferentiated information. Problem solved.

Weak Connections

All of us have some social connections that are stronger than others. There's value in those weak connections, too. Welcoming people into your full stream of social information is how weak connections are built and maintained.

Reciprocity

Some people say that social media makes almost everyone famous, at least to a small group of people. One definition of fame is a circumstance where the number of people who care about what you're doing is more than you are capable of paying attention to yourself. Online, though, we can all pay a little passing attention to the people who are paying attention to us. People appreciate RSS subscribers; they like friends on Facebook and Twitter. If someone follows you, it only makes sense to follow them back. (I need to follow my own advice better on Twitter.)

If a person isn't terribly important to you, just don't include them in a high-priority group. Interact with them when you get the chance. They'll appreciate the reciprocal connection, though

That's one take on the strategic value of groups and the full stream of information. What are your thoughts on this topic? Have you come up with any other super-useful ways to build, manage, or find value in groups online? We'd love to hear about it!

Title photo: Your Days - December test Group : 31 Décembre 2006 by Nawal_ CC on Flickr

McCarthy tends the gears of Techmeme, makes sure the content on the site remains of high quality and helps ensure the inclusion of new and important voices. It sounds like an awesome job and one that has probably never existed before - a half woman, half robot, news gathering machine. How can you get your blog on Techmeme? What's in the future for the site? We asked Megan in the following interview.

Marshall Kirkpatrick: What do you do all day? I imagine you standing next to one of the most awesome news discovery machines available, tending it, making sure it keeps running smoothly, and looking out beyond its reaches to feed it things it hasn't gotten to yet itself. Is that an accurate picture?

Megan McCarthy: That is fairly accurate, actually. I make sure that the news on Techmeme represents an accurate, current, and full overview of what's happening in technology right now. So, that's trimming back stories that aren't relevant, adding in viewpoints that ought to be heard, etc.

Marshall: Can you tell us a little bit about your personal background?

Megan: My personal background is a little varied. Prior to [writing for] Valleywag, I bounced around a few different jobs and places and never really found a niche. I lived in Hawaii for a few years, had various office drone jobs and other gigs to pay the bills (Nanny, bartender, coffee server). But I loved following technology and reading about what was happening in silicon valley - and I've been a news junkie since I was young.

News Selection and Twitter Tips on Techmeme

Marshall: So, did your coming on board "break" the "objectivity" of the site?

Megan: Techmeme is biased and has been so for a while. If you read Gabe's post announcing the addition of an editor, he makes that point.

What do you think, though? What changes have you noticed since I joined?

Marshall: I have noticed no changes to story selection, perhaps less wonky stuff. I've always considered Techmeme a very reliable source of news and I think you're doing a good job continuing that tradition - but there were certainly some people who grumbled about the human touch being formally introduced, an editor.

Megan: I think some of those people might grumble about anything.

Marshall: How can new bloggers get indexed on Techmeme?

Megan: We just introduced a program where people can tip relevant posts to us through Twitter. Anyone can tip any post they think is relevant to us.

Marshall: How is the new Twitter tips program working out? I see a lot of stories go up with thanks to Twitter, quite a lot - is it changing the face of the site substantially? Changing the content?

I see a handful of people getting thanks over and again, I imagine there's limited participation so far but how does the algorithm determine whose tips to accept and whose not to?

Also, a lot of people are sending tips regarding their own stories - is that ok? Even mainstream media outlets.

Megan: I don't think it's changing the content overall. Many of the stories that are tipped are ones which are worthy of a Techmeme headline. Not everything that gets tipped to us gets on the site. There are two situations that I can think of where the tip program does affect the content: It can help surface breaking stories faster, and if there are two similar stories from different outlets and someone cares enough to tip a certain one, that will probably effect which one ends up as a headline on Techmeme.

As for people tipping their own stories... personally I'm not completely opposed to it. If a writer has a breaking story that he or she wants to let us know right away, that's a good way to do it. But, they should keep in mind that their twitter handle will be credited with tipping us to the story. If "Thanks: Marshall" showed up next to every Techmeme headline you get, people might put two and two together and think that you really like your work.

To my knowledge, the identity of the person tipping the story has no effect on whether or not it will show up on the page. It's about the post itself.

Marshall: Well, if shame and loads of people saying "you're an f*ing jackass" was sufficient deterrent to anti-social behavior in social media, then...[indecipherable, record of this part of the conversation lost forever.]

Megan: Ha. Is he though?

Marshall: Oh I'm sure he is. ANYWAY. Is accuracy taken into account on Techmeme?

Megan: Accuracy is absolutely taken into account on Techmeme. That's one of my goals, anyway. If there's a post which has a lot of buzz around it, which turns out not to be true...

Marshall: What does that look like? Are you like "Steve Jobs is NOT out at Apple, I don't believe those reports! Story...gone!"

Megan: Or, a story that says "Steve Jobs NOT out at Apple" gets published next to the earlier, erroneous rumor.

Marshall: Then you yank the false story?

Megan: Either yank it or surround it with stories pointing out *why* it's false. Sometimes the false rumor becomes a story itself and yanking it can be jarring. We want our readers to be able to visit the site and know what's going on in technology - to know what people are talking about. The earlier rumor would probably be replaced as the top story by one with the correct information, but yanking it without giving our readers full context of the overall arc might be a bit jarring.

Marshall: You have to be reading a lot of these stories in great detail. What time does your work day start and end?

Megan: I start around 7:30ish and end later than that. News never stops!

The Future of Techmeme and Other Aggregators

Marshall: So, everyone wants to be an aggregator these days. All the young kids are like "mommy, I'm going to grow up to find recommended stories for an online news publisher."

What kinds of things do you foresee becoming points of leverage for content aggregators and news discovers in the future?

Megan: I think a reliable real-time web is going to have the greatest impact on aggregation services. I'd love to be able to see stories from sites as they're published, without a lag.

I hope that quality, accurate, and speedy stories get rewarded by receiving more attention - and that new voices are discovered and make the media chorus sound fuller and stronger.

You were asking me about my electric sheep dreams.

Marshall: Are you a cyborg?

Megan: Depends on my mood.

Marshall: At least between 7am and 7pm?

Megan: That sounds about right. This is super-nerdy, but reading an overwhelming amount of news is something that I rather enjoy doing.

Thanks to Megan McCarthy and Techmeme for doing this interview and doing the things they do each day - help us find the hottest conversation in technology. We appreciate it. You can find Megan on Twitter as well. Photo at top by Scott Beale

We had the opportunity to speak with Satnam Alag, author of the recently published Collective Intelligence in Action, about what makes for a good recommendation system, where the technology is heading, and why Netflix is finding it so hard to improve its own system.

ReadWriteWeb: In our recent post about Netflix, we identified four main approaches to recommendations: Personalized recommendation: based on prior behavior of the user; Social recommendation: based on prior behavior of similar users; Item recommendation: based on the item itself; And a combination of all three. Do you agree with the four approaches we laid out in our article?

Satnam: Those four categories are pretty comprehensive. I present an alternate classification of recommendation systems in my book. I lay out two fundamental approaches. The first approach, item-based analysis, determines items that are related to a particular item. When a user likes a particular item, related ones are recommended. The second approach, user-based analysis, first determines users who are similar to that user.

Further, there are two main approaches to finding similar items and similar users. For the first, content-based analysis, content associated with the item, especially text, is used to compute similarity. In the second, the collaborative approach, actions such as ratings, bookmarking, and so forth are used to find similar items. For the second, user-based analysis, a number of approaches have been taken, including ones based on profile information, user actions, and lists of the user's friends or contacts. Of course, you can combine any these item/user and content/collaborative approaches to build a recommendation system.

The dimensions of the particular item and user space are helpful in deciding whether to use an item-based or user-based approach. Typically, an item-based approach is used to bootstrap one's application when the number of users is small. As the user base grows, the item-based approach is augmented by a user-based approach.

ReadWriteWeb: Other than Amazon and Netflix, which Internet companies have most impressed you in their implementation of recommendation systems?

Satnam: Other than Amazon and Nextflix, Google News' personalization is my personal favorite. Google News is a good example of building a scalable recommendation system for a large number of users (several million unique visitors per month) and a large number of items (several million new stories every two months), with constant item churn. This is different from Amazon's, whose rate of item churn is much lower. Google decided to use collaborative filtering for its recommendation system mainly because of its access to the data of its large user base and because this same approach could be applied to other applications, countries, and languages. A content-based recommendation system perhaps could have worked just as well, but may have required language- or location-specific tweaking. Google also wanted to leverage the same collaborative filtering technology to be able to recommend images, videos, and music, for which it's more difficult to analyze the underlying content.

Among start-ups, my personal favorite is the one we are developing at my current company, NextBio. It's not available yet but should be next month. The key point about this particular recommendation engine is its strong use of an ontology, similar in concept to tags, to develop a common vocabulary for items and users. The system then makes use of profile information and user interactions, both short- and long-term, to provide recommendations. The system leverages both item- and user-based approaches.

ReadWriteWeb: What commercial opportunities do you forsee with recommendation systems over the next few years?

Satnam: A good personalized recommendation system can mean the difference between a successful and a failed website. Given that most applications now invite users to interact and to leverage user-generated content, new content is being generated at a phenomenal rate. Showing the right content to the right user at the right time is key to creating a sticky application. I would be surprised if most successful websites did not leverage recommendation systems to provide personalized experiences to their users.

ReadWriteWeb: Your book includes a discussion of collaborative filtering. Can you tell us a bit about how this fits into the overall picture of recommendation systems?

Satnam: In recent years, an increasing amount of user interaction has provided applications with a large amount of information that can be converted into intelligence. This interaction may be in the form of ratings, blog entries, item tagging, user connections, or shared items of interest. This has led to the problem of information overload. What we need is a system that can recommend items based on the user's interests and interactions. This is where personalization and recommendation engines come in.

In my book, I take a holistic view of adding intelligence to one's application, a recommendation engine being one way to do it. The book focuses on both content-based and collaborative approaches to building recommendation systems. It focuses on capturing relevant information about the user, information from both within and outside one's application, and converting it into recommendations. One of the things you mentioned in your write-up on recommendation systems is that you would like to apply such a system to your website to recommend things to users. Someone reading my book should be able to create such a system using the techniques I demonstrate.

Next Page: Satnam's thoughts on the Netflix Prize and whether the 10% mark will ever be reached.

ReadWriteWeb: Netflix is offering $1 million to the team that can improve its recommendation algorithm by 10%. It's been over 2 years now, with the leading company at 9.63%. There is some skepticism, though, that 10% will be reached anytime soon, because now the contestants are making only incremental progress. Do you expect the 10% mark to be reached soon?

Satnam: Netflix's recommendation engine, Cinematch, uses an item-to-item algorithm (similar to Amazon's) with a number of heuristics. Given that Netflix' recommendation system has been very successful in the real world, it is pretty impressive that teams have been able to improve on it by as much as 9.63%. Of course, the Netflix competition doesn't take into account speed of implementation or the scalability of the approach. It simply focuses on the quality of recommendations in terms of closing the gap between user rating and predicted rating. So, it isn't clear whether Netflix will be able to leverage all of the innovation coming out of this competition. Also, the Netflix data doesn't contain much information to allow for a content-based approach; it's for this reason that teams are focusing on collaborative-based techniques.

The challenges to reaching the 10% mark are:

Skewed data: The data set for the competition consists of more than 100 million anonymous movie ratings, using a scale of one to five stars, made by 480,000 users for 17,770 movies. Note that the user-item data set for this problem is sparsely populated, with nearly 99% of user-item entries being zero. The distribution of movies per user is skewed. The median number of ratings per user is 93. About 10% of users rated 16 or fewer movies, while 25% of users rated 36 or fewer. Two users rated as many as 17,000 movies. Similarly, the ratings per movie are also skewed: almost half the user base rated one popular movie (Miss Congeniality); about 25% of movies had 190 or fewer ratings; and a handful of movies were rated fewer than 10 times.

The approach: The winning team, BellKor, spent more than 2,000 combined hours poring over data to find the winning solution. The winning solution was a linear combination of 107 sets of predictions. Many of the algorithms involved either the nearest-neighbor method (k-NN) or latent factor models, such as SVD/factorization and Restricted Boltzmann Machines (RBMs).

The winning solution uses k-NN to predict the rating for a user, using both the Pearson-r correlation and cosine methods to compute the similarities, with corrections to remove item-specific and user-specific biases. Latent semantic models are also widely used in the winning solution.

The BellKor team found it important to use a variety of models that compensated for each other's shortcomings. No one model alone could have gotten the BellKor team to the top of the competition. The combined set of models achieved an improvement of 8.43% over Cinematch, while the best model -- a hybrid of k-NN applied to output from RBMs -- improved the result by 6.43%. The biggest improvement by LSI methods was 5.1%, with the best pure k-NN model scoring below that. (K for the k-NN methods was in the range of 20 to 50.) The BellKor team also applied a number of heuristics to further improve the results.

The BellKor team demonstrates a number of guidelines for building a winning solution to this kind of competition:

• Combining complementary models helps improve the overall solution. Note that a linear combination of three models, one each for k-NN, LSI, and RBM, would have yielded fairly good results, an improvement of 7.58%.
• A principled approach is needed to optimize the solution.
• The key to winning is building models that can accurately predict when there is sufficient data, without over-applying in the absence of adequate data.

The final solution will be along the same lines, combining multiple models with heuristics. Contestants will probably reach the magic 10% mark in the next year or two.

ReadWriteWeb: Some people think the 10% mark can't be reached with algorithms alone, but that the "human" element is required. For example, ClerkDogs is a service that hires actual former video-store clerks to "create a database that is much richer and deeper than the collaborative filtering engines." It's a similar approach to that of Pandora, which has 50 employees who listen to and tag songs. How far do you think algorithms can go in making recommendations?

Satnam: Recommendation systems are not perfect. A number of elements go into making successful ones, including approach, the speed of computing results, heuristics, the exploration and exploitation of coefficients, and so on. But it has been shown in the real world that the more personalized you can make recommendations, the higher the click-through rate, the stickier the application, and the lower the bounce rate.

Using humans to form a rich database for recommendations may work for small applications, but it would probably be too expensive to scale. I don't see them competing against each other, human versus machine. Even with human/expert recommendations, one first needs to find a human/expert with tastes similar to those of the user, especially if you want to go after the long tail.

Recently, FriendDeck developer Paul Kinlan released an Adobe AIR application of his FriendFeed tracking tool. Although still rough around the edges, this app has potential to become a viable alternative interface to FriendFeed for the service's heaviest users.

One of the best features of the new AIR application is how it syncs with the online version of FriendDeck. That means whatever changes you make within FriendDeck on the web show up in the desktop application and vice versa.(Side note: how we wish TweetDeck did this!)

But let's be clear. The Adobe AIR version of FriendDeck still needs a lot of work. The slider bars are hard to see, the columns can't be moved around, "un-like" didn't work in the first version we tested, pictures don't show along with posts, and performing searches requires the use of specific query syntax, like "friends:username." That last bit practically requires you to use a cheat sheet when building your columns. (For search query syntax, see the list provided here.)

Yet despite these problems - all of which are being addressed according to the developer - I still can't get enough of FriendDeck. Why? It's simple: FriendDeck gives me an eagle-eye view of the topics I care about on FriendFeed. Using the app, I can see a lot more information with a glance than when I use the FriendFeed web site itself.

Use FriendDeck to Follow Your Lists, Rooms, and Searches 

There are a couple of ways you can use FriendDeck. For me, a compulsion to organize things into groups has led to the creation of over 30 lists for tracking specific topical areas outside of my "home" feed. I don't check each list daily, but it's nice to have them there. I doubt that anyone else has embarked on such madness, but even if you have only a few lists, like "Favorites" or "Personal", you can add them to the FriendDeck application by typing in "list:list_name" (where, obviously, "list_name" is the name of your list).

Similarly, if you're less inclined to use lists and more inclined to track items by keyword, you can instead just type in the word or words you want to track in the search box. For rooms, you would just type "room:room_name."

Use FriendDeck as a More Advanced Feed Reader

Another great way to use FriendDeck has to do with one of the biggest secrets surrounding FriendFeed itself: you don't have to participate to use the service. Although community members will tell you this goes against what FriendFeed is all about, it's true. If you're the kind of person who could care less about "liking" items or leaving comments on the latest internet meme, you can alternately use FriendFeed as an aggregator on topics you want to track, no participation required. Instead, FriendFeed can simply become the framework you use to build the feed reader of the future: an aggregator which lets you follow much more than RSS feeds alone.

For example, take any subject about which you're passionate. You could build a private FriendFeed room that you fill with blogs, news sites, Twitter searches, Google searches, well-known Twitter users who post on this subject, and more. Within one interface, you can track it all. This is a million times better than just following blogs in an RSS reader or just tracking Twitter posts using Twitter's search engine because you're not limited to RSS alone.

You can share these rooms with others either by making them public or by inviting select users - like your colleagues - to join. Alternately, you can keep them completely private and only for your personal use.

You can then add these personalized rooms into FriendDeck by typing in "room:room_name" and soon you'll have one single application that tracks everything you care about on the internet - from blogs posts to tweets and so much more. And it's all in one window.

Too Complicated?

In explaining how FriendDeck works, it occurs to us that it may sound a bit complicated for the average user. That may be true, but then again, FriendFeed itself is an application that requires a lot of explanation, too. It's also quite possible that FriendFeed and FriendDeck will never cross over into "regular user" territory. That's OK, though. For anyone who has learned how to use FriendFeed - really use FriendFeed - applications like FriendDeck will hold appeal, despite their complications.

It's still far too early to call FriendDeck a complete success or failure, but it's not too early to use it. If you're geeky enough to learn its quirks, you may find the FriendFeed desktop application you've been waiting for. If not, then feel free to return to your browser.

But we're now at the end of November and really, to be frank, there hasn't been much progress made in the RSS filtering space.

First a bit of background about why information filtering is so important. Marshall Kirkpatrick wrote a post this week explaining how he monitors RSS feeds to get the latest news of interest to him. Like Marshall, I'm an Information Omnivore (I'll devour anything and everything!), but it's important for my business that I not get overwhelmed with information. As it is, I'm guilty of not replying to too many emails - let alone keeping up with the hundreds of RSS feeds I subscribe to. And OK, both Marshall and I are RSS-obsessed bloggers - but in reality nearly everyone is overloaded with information these days, RSS or not.

"I haven't included any discussion about small things like filtering feeds, scraping feeds or using RSS and email together but there's a lot more that can be done with RSS for research than I feel like writing about this morning."

Blastfeed is one of those new kinds of filtering tools. Developed by Paris company 2or3things, Blastfeed enables people to filter individual RSS feeds or groups of feeds. Patrick Ferran from 2or3things told me they're aiming the product at "info-consumers (like yourself) and for the corporate world."

As of now Blastfeed does pretty much the same things as FeedRinse, the RSS filter service I profiled back in April [disclosure: I did a very short bit of consulting to FeedRinse in May]. Both services allow you to input a set of feeds and do filtering on them - essentially creating new feeds, which you then add to your regular RSS Reader. Blastfeed also lets you create email alerts, or a widget for your blog - and IM/SMS notifications are coming soon. In that sense, Blastfeed also has similarities with ZapTxt, which recently introduced new features such as publisher widgets and custom RSS feeds.

One problem with all of these specialist filtering tools is that you can't filter and read in the same place. In my view, RSS filter services are crying out to be full-blown RSS Readers too. Although granted, that's not the case for IM/SMS based services such as ZapTxt. But if you're going to offer a filtering service with RSS output, why not integrate a full-featured RSS Reader?

In other words, I want to do my filtering and reading from the same service. Some of the popular RSS Readers already allow you to do this - think FeedDemon, Blogbridge and Google Reader. So while Blastfeed and FeedRinse are both great services, I question whether I'd want to do my filtering in a separate application to my reading.

Having said that, I like where Blastfeed is heading and I can certainly see a future for advanced stand-alone RSS filtering services. According to Patrick, here is part of what they have planned for the future:

"One other direction we want to take Blastfeed to would be to use it as a platform to manage all kinds of media generating/powering RSS. For instance, TV / VOD providers could send their programs in RSS, and users could actually set a channel for such specific topic of interest. Whenever a TV / VOD content described in the RSS feeds matches the users requirements it could trigger a notification to an application that would record the film for later viewing, etc. Combination with other media assets (Podcasts, videocast, news and blogs feeds, company feeds, etc.) can then be envisaged as well."

So this is an excellent way forward. In summary, while I'd like to have my filtering/reading wrapped up in one service - I also see the benefits of focusing on advanced filtering functionality.

PubSub once held so much promise... It was the first real 'future search' product to gain traction and it was an innovator in the area of custom RSS feeds. The somewhat clumsy term I've used for that is Topic/Tag/Remix Feeds and here's what I said about it back in January 2005:

"Tools will evolve to let people easily set-up personalized searches for information relevant to them and subscribe to the results - using, you guessed it, RSS! Google will probably be the front-runner (see this video for a hint to the future - thanks twdanny for the reminder), PubSub will be another, current players like Bloglines and Technorati will be in amongst it, and who knows who else."

As well, I thought PubSub was one of the leading companies in filtering feeds - the next layer up from aggregation. But unfortunately for all their promise, PubSub never quite lived up to it and services like Google News and Technorati have taken the limelight instead. Technorati has had its technical issues (which remain to this day), but one thing they've always done very well is the UI. Which is where I feel PubSub has let itself down, because their UI has always been a bit awkward and geeky. Technorati has also proven itself to be outstanding at PR and drumming up publicity about itself. PubSub never really got that knack, despite Bob and others best efforts to talk up its technology. If anything PubSub has been too tech-centric, whereas Technorati reaches out to 'the people' (with its tags PR blitz and the 'A-List Person's Favorite Blogs' feature, for example). 

However it's precisely because PubSub has always been very forward-thinking about web technology that made me like it and thus makes me sad about its demise. PubSub was into topic mapping circa 2004, when few other mainstream RSS companies were. PubSub was also one of the instigators of the Structured Blogging open source project. Indeed I worked with them - via Broadband Mechanics - on the recent website re-design (SB is a separate entity now, so isn't in danger of sinking with PubSub).

Their technology nous has always impressed me and at the end of 2004 they came very close to being named my Most Promising Web 2.0 Company of 2004. I gave that honor to Feedburner in the end, but here's what I said about PubSub (Dec 30, 2004):

"...PubSub has been building a technically very sound service mostly quietly in the background (but that's about to change in '05, by the looks of it)..."

That 'by the looks of it' pointed indirectly to PubSub's 2005 resolutions, which currently is a File Not Found page. That about sums up the situation, I'm afraid. Whereas Feedburner and other promising startups I mentioned at the end of 2004 (e.g. Bloglines, SixApart) have gone on to greater things, PubSub lost its way and the end looks imminent. I really hope they can somehow turn things around and get rid of the 'political' monkey on its back, because I know they have the smarts and the people to achieve success.

FeedRinse is an interesting attempt, enabling you to "rinse" your feeds by keyword, author, tag. It also has a profanity filter. To get started, you sign up and import your OPML - or filter individual feeds. Where FeedRinse's model comes slightly unstuck is that you're required to export your filtered feeds, so you can add them into your RSS Reader. This seems like a step too much for me, as I'd prefer to do my filtering and reading from the same service.

I asked Aaron Mentele from FeedRinse about this. He replied:

"The current readers on the market are pretty early in development, and there are definitely some specific ways to improve them (we're addressing that). But, we're focusing on the big item first: allowing the end-user to determine what they want to receive. We see the core product as a pass-through service that will help regardless of device, reader, etc.

What we're focusing on right now is improving the filter options as well as allowing the user to create relevant content channels. These features will integrate directly with the filtering options (that's what the mysterious "Smart aRSS" feature is all about)."

OK, so I see the value in focusing on getting the filtering technology right first - before doing an integrated filter/reader product. I'll be keeping an eye on FeedRinse's progress. Also check out TDavid's review.

NB: there's a similar product just released called ZapTXT - which seems more like a search subscription service a la PubSub. ZapTXT "allows you to define search criteria for your RSS feeds and then notifies you via email, SMS or IM when a new entry contains your search keywords."

"...it’s the little things in life that make you smarter. The little things don’t show up on Memeorandum. They do show up on RSS. Which is why I’m still subscribed to 847 smart people’s feeds."

I don’t think the problem is Memeorandum or news trackers per se, but then maybe Gabe could tweak the algorithm to weed out the flame wars and other stuff that isn't "tech". It’s not news, after all…

"I agree with Nik that there’s a huge technical chasm to cross before a “personalized meme tracker” gets really useful. I think progress I make on the memeorandum engine is approaching that, but it’s still far off enough that I’ll pass on hyping it for now."

Gabe then proceeded to give me an earful in a Skype conversation about the issue ;-) I was also interested to read Greg Linden's thoughts on the matter, as he is another very smart developer with experience in this domain. Actually Greg seemed to like my suggestion of introducing clustering to Findory, which would definitely get me using it more. 

Because let's face it, Personalization + Clustering is the next big step in RSS. If 2005 was about Aggregation, then 2006 is all about Filtering.

Nik wrote up his thoughts today, in a post entitled Memetracking Attempts at Old Issues. While he mentions lack of link data as being an issue, it seems to me the crux of the problem is this:

"generating a personal view of the web for each and every person is computationally expensive and thus does not scale, at all."

He goes on to say that "this is why you don’t have personalized Google results – we just don’t have the CPU cycles to care about you."

So it's mainly a computational and scaling problem. Damn hardware.

Nevertheless, there is a big demand for personalized clustering - among the edge cases, it must be said. And Megite and TailRank are both trying to capture that demand, which to be frank I'm very pleased about. I understand why Gabe and Nik don't want a bar of it, but there are lots of squeezed bloggers out there who are desperate for a good RSS filtering solution. The first web app that solves this, or at least gives me decent filtered RSS feeds, is going to get my business for sure.

In many ways, this is the next evolutionary step for RSS Aggregators. We all know by now that tracking hundreds of RSS feeds in an Aggregator like Bloglines or Rojo is very time-consuming and tiring. Having an automated filter for your RSS Aggregator is the holy grail for such services - because none have done it successfully so far.

btw I know Alex Barnett doesn't like the term 'Meme Tracker' - he prefers News Tracker. But I like Pete Cashmore's defence of the term Meme Tracker - it's not quite 'news' either and the term actually honors Memeorandum, the first and still the best such service.

Now I've heard that a number of the Meme Tracker services I reviewed last week are planning personalized versions, but Megite is the first I know to actually have developed one. If others have built one, then please let me know! Megite's is still pre-beta and only myself and a few others are currently testing it, but so far I have to say it's looking promising. Here's a screenshot of my personalized Megite.

The developer Matthew Chen says the clustering of Personalized Megite is determined by a combination of relevance, post popularity, blog authority and freshness. I don't think I'm giving anything away there - the secret sauce is of course how those elements are combined and mixed! He says they've been experimenting with different factors and formulas to try and get the best results. Indeed in my testing, I've noticed a gradual improvement as Matthew continues to tweak the product.

Also I should point out that it's not restricted to feeds in my OPML file - some external feeds get pulled in too, provided they are 'relevant'. So there's a lot of potential for discovery of new feeds, outside your initial OPML file.

Personalized Megite has got a long way to go, but there are signs it's coming together and I'm looking forward to see how this personalized news clustering develops. Plus I'd love to test out any other service that is doing this. I really do think this will have the traditional RSS Aggregators sitting up and taking notice. I don't know about others, but I'm struggling to keep up with my 150-odd RSS feeds on a daily basis. I desparately need a good way to filter them.

Update: Over on Robert Scoble's blog, Greg Linden reports that Findory does something similar: "Go to Findory.com, click the Favorites link in the upper right corner, then import your OPML. What you get is a personalized selection of top stories from your favorite feeds." The one difference is that Findory doesn't do any clustering of stories. That would be a great feature (hint hint Greg!).

Only it seems someone forgot to tell the Blogrunner webmaster, because according to the homepage and About page, The Annotated New York Times is "NOT affiliated with The New York Times". Huh?

In any case, it turns out NY Times plans to add Blogrunner/Annotated NYT to their own website, which is a fantastic idea. It will bring blogs well and truly into The Gray Lady's orbit, not to mention remix culture.