Hashtags for Twitter are a great start, but how are the startups moving and shaking the real-time web planning on giving users filters to control their streams in ways that make the ever-increasing volumes of information more usable? From Thing Labs and Twingly to PostRank and SocialText, read on for the problems these companies and their users have encountered and how they plan to solve information overload through clever curation and cooperation.

Thing Labs' CEO, Jason Shellen, interrupted to insist that we broaden the discussion to include the entire real-time web and all possible examples of filtration systems, not just Twitter and not just hashtags. From there, the conversation exploded into an executive-level goulash of how to make the real-time web useful.

The overall poverty of the user experience was generally deplored. "We hear from our users about what they want," said Shellen. "People say, 'Just show me the important stuff.'" The current state of real-time UXes allows for a lot of opportunity - the opportunity to make this iteration of the Internet simple for new users as well as appropriately complex for powerusers, unlike what we've all seen with RSS, which remains an underused geekcore feature.

The spectrum of data and metadata was brought up several times, as well. Keywords (e.g., hashtags) are a good start, but richer metadata would allow for filtration by sentiment or location. For example, a user might want to see blog posts about Obama's winning the Nobel Peace prize from right-leaning sources only. Or I might want to see pictures posted by people within 100 feet of me while I'm at the Real-Time Web Summit.

Overall, having the author, location, time, sentiment, and keywords automatically applied to user-generated data could lead to much richer streams with built-in filtering opportunities, both filtering content out as well as discovering new content and sources.

Another major point of emphasis for this session was the fact that a critical mass of users generally leads to the best filtering: Large datasets create very specifically defined problems and finely tuned filtration. Unfortunately, the startups involved in the real-time web often have smaller user bases than would be desired; there is simply not enough data generated by the users of the individual services. But what if all that user data was combined somehow?

"Right now," said Kallstrom, "people doing startups trying to combat information overload are mostly focused on finding high-quality signals. It's a very hard problem. The highest quality for the end user is achieved by moving from competing on gathering the signals to creating a great user experience through more open data."

One participant suggested publishing user activity to open-source the problem of how to filter real-time data. Many other participants agreed that the problem requires collaboration, data portability, and open standards between all the companies in the room and beyond. Such collaboration would make all real-time products better and lead to better experiences for users.

Then again, better filtration could be a real-time holy grail, a solution worth selling. And when the question of money comes up, will these startups be willing to sacrifice a theoretical goldmine to collaborate on a user-friendly solution?

Ironically, if anyone's to blame for spamming our Twitter timelines, it's the marketers themselves. They've managed to trick our friends into spamming us with their messages instead.

We're not sure where anyone, marketer or not, gets off telling Twitter that it's their responsibility to filter the content that flows through their service mainly because Twitter is already doing so. The company itself currently addresses the spam issue by providing an @spam account where you can report spammers and other abusers in the Twittersphere. If the account in question is indeed a spammer, Twitter boots them from the service. That sounds good to us. Simple and effective...at least for the end user. (It's probably a nightmare to deal with at Twitter HQ).

Of course, Twitter doesn't want their service overrun by spammers - no one would. However, they're probably more concerned with wasting their resources to support these fake accounts than they are with the annoyance it causes for their users. But do they have it under control? Perhaps not - fighting spam is sort of like fighting computer viruses. You block one and someone makes a new one. The same goes for spammers - kill one spammer and another appears to take his place. It's an ongoing fight, not a plague that can be wiped out overnight through some magic filter.

Besides, what you consider spam, I may consider "valuable information about a product." Probably not, but there is a gray area there that has to be taken into consideration. Some spam is out-and-out spam, but other stuff may just be "hot deals" from a legitimate company. However, if you didn't want to see said hot deals, you might consider them spam. Still, how would you see them unless you actually followed that account to begin with? Or maybe you turned on auto-follow using a service like SocialToo? If that's the case, it's a little ridiculous for you to get annoyed when half your timeline turns into a slew of "buy this" messages - you only have yourself to blame for that.

Where Actual Spam Hurts Us

The only place that honest-to-goodness spam can really affect you on an everyday basis is not in your own personal timeline of friends' tweets, but when viewing a trending topic's stream or when doing a keyword search. In these cases, spammers hijacking a currently popular hashtag may show up in the timeline, potentially diluting the results with irrelevant information. For this reason alone, we support Twitter's spam-fighting efforts.

Even More Dangerous? "Tweet to Win"

What's actually more concerning than spam, however, is the new trend we'll call "tweet to win." Legitimate companies have begun using Twitter to promote a message - essentially an advertisement about their business' offerings. To cajole twitizens into "spamming" their followers in this way, they're offering prizes or the chance to win prizes in return. (Full disclosure: this author did this once and still regrets it).

This situation hasn't gotten out of hand just yet, but it seems like it's only a matter of time before it does. Because really, how many of you could resist yourselves if all of a sudden a company started giving away free Macbook Pros? Oh, apparently not too many of you because you've already spammed up trending topics today with #moonfruit. What's Moonfruit? Why, it's a company that's giving away a free Macbook Pro every day for 10 days. Is this a brilliant social media promotion (as Adam Ostrow of Mashable claims) or just a new, inventive way to junk up the twitterstream with advertisements? We think it's closer to the latter.

The only consolation in this particular case is that Moonfruit doesn't care what your tweet says, so it can just be appended to any ordinary tweet. That's not usually the case - most companies provide a message for you to re-tweet.

What's frightening about this "it's not spam, it's a message from your friend" is that it's really not. My friend isn't actually telling me that Moonfruit is this great new company they have just heard about and that I really have to check out. This isn't a word-of-mouth recommendation - my friend just wants to win a new laptop. They know this, I know this, and the company knows this. And that makes the message just as spammy to me as any other in-stream tweet from an actual spammer.

So, what can be done? Well sure, I could unfollow that so-called friend, but why would I? It's not like they do this regularly and 99% of the time, I like what they have to say. But while one day that friend is tweeting to win a Macbook, another may be tweeting to win something else. Even if only a small percentage of an ever-shifting group of my friends tweeted a promotional message every day, it would be enough to junk up my timeline.

Sadly, that's one kind of spam that Twitter can't really block. And neither can I.

Announced yesterday (with early coverage from Louis Gray), the initial alpha has a public filter that lets anyone test the technology on a group of a few dozen feeds, and a registration-only Publisher area that allows users to add up to 5 of their own feeds to use with Shyfter Filter.

The core product is the filter itself. It consists of three types of search criteria (title, author, and article/body) that can be used independently or together to produce a customized feed of just the content you want. The public version has 44 feeds as source material to work from, of which all or just a certain subset can be chosen for the filter. Each criteria can be narrowed down to a dozen or so levels of strictness, from any of the terms to exact phrase match. Once the terms are entered and the source feeds chosen, you can grab the resulting RSS feed. I took a moment to search all the sources for the terms iphone and blackberry, you can see my results here.

You can also exclude terms that perhaps you don't want to see coverage on. Do you just hate seeing any mention of the terms iphone or twitter in a tech story? In this example we chose to exclude those terms from all sources in the technology category. And remember, you can one type of criteria with another, say searching for a particular author but excluding anything article with particular terms in it.

The Shyftr Publisher

This technology has a lot of potential, but right now it is more of a tech demo as long as you can only apply it to the 44 feeds that are listed on the public page. In recognition of that, Shyftr is building a service for muti-author blogs (like ReadWriteWeb) or blogs with a lot of diverse content to be able to build custom-filtered feeds with certain criteria. Once these filtered feeds are created, there's even a widget for the blog to display. Unfortunately, there was some trouble getting output from the Publisher feeds so all I can show you is a screenshot.

Summary

This service brings some powerful tools to the growing field of RSS feed curation, which got its start with do-it-yourself tools like Yahoo! Pipes and Tarpipe, and a more refined application in PostRank (which we cover here and here) and Grazr. How does Shyftr Filter stand up to these other tools? We can definitely say that the approach Shyftr is taking is more like the DIY tools, but makes creating a curated feed easier and with some sacrifice in flexibility. We don't think being less flexible is a problem - the DIY tools can be awful hard to get working correctly, so we are all for an easier-to-use solution.

There's really not much to the disstill web application, but that's okay with us. This is definitely an example of how the simplest web apps can be the most useful in the end.

The only thing on the disstill web page is a little slider bar that lets you filter Digg.com stories based on a minimum number of diggs. You just drag the slider to adjust the number of diggs that stories need to have in order to be included in the RSS feed. The low end of the slider is set to 100 diggs and the high end is 5000. Obviously, the higher you go, the more filtered the feed becomes and the more likely you're only going to see the really, really hot stories.

Once you have the slider set, you can either view the page or click "get the RSS feed" to add the customized feed to your preferred feed reader. It's a lot easier than using Yahoo Pipes, that's for sure!

A Couple of Suggestions

Our only complaint about this nifty little web app is that it doesn't let you choose which section the stories come from (Politics, Technology, Science, Gaming, etc.). Instead, it looks at the entire Digg website. We would also love to filter for images and videos, too. Perhaps in some future version, we hope?

At any rate, this is one of those little tools that can end up making your life a little less info-overloaded. And for that, we thank you, Mr. Alex Rabarts. (P.S. Can you build a generic version of this that lets you enter in any URL and then filter by PostRank? That would be amazing!)

Alex also created a nice visualization of Digg, Reddit, Delicious, Hacker News, and Yahoo Buzz that's worth a look. Check it out at oursignal.com.

OtherInbox attempts to have as light a touch as possible when it comes to your Gmail account. Mainly, all you will see after it has done its initial pass through your mail is a new otherinbox label that you can use to archive or delete that mail. If you happen to have more than one incoming email address pointing to Gmail, OtherInbox will also automatically create labels for them as well.

Once in your OIB mailbox, the story is different. Here, all the mail that you agreed that OIB could import is listed by category (or what OIB calls mailboxes), which you can quickly step through and perform mass actions on, such as marking as read or deleting. The mailboxes can be created manually (there is a new mailbox button at the bottom of the page) or automatically, simply by sending email directly to your custom OtherInbox email domain directly. For example, if your OIB account name was johndoe, you could fill out an online form for some free stuff with the email address freestuff@johndoe.otherinbox.com. This would create the new mailbox freestuff in your OIB inbox containing any mail that is sent to you from that site. If a spammer gets ahold of that address, simply click on the block mailbox button and you will never see any email in that mailbox again.

We have been using OIB for a few days now, just trying to get a feel for the product as a whole. Some folks may only be interested in using the service primarily for its disposable email address ability, but we think that OIB is looking further and is trying to become the primary repository for all your other mail. You know -- the stuff you don't want but can't quite get rid of. To that end, OIB is also planning to support other online mail services such as Yahoo! Mail.

Finally we should mention that the free service, while offering everything that OtherInbox features without limitation, is restricted to only showing the last 30 days of email that has been introduced into your OIB account. If you stay on top of your OtherInbox mail, this should be no problem. However, if you do want to see everything, you can sign up for the premium service for $19.99 a year.

Ambient News is a new Firefox add-on written by Mozilla developer Atul Varma and is currently available as an alpha release. The add-on tracks your browsing habits, learning which sites you visit most frequently. It then pulls the headlines in from those sites and displays them for you in a beautifully fading list every time you open a new tab in Firefox. If you see something that interests you, just click the link and you'll be taken to the web site where the headline originated. Privacy advocates, rest assured - no data is shared outside your browser.

Intelligent Agents to the Rescue!

As Michael Calore of Wired notes, the add-on is a great workaround for the biggest usability problem facing RSS. "Many people don't know what it is or how to take advantage of it," he writes. "The first hint that a feed exists is a funky orange or blue icon. Click on it and, in most cases, you get prompted to load another application. Sometimes, you just see ugly, raw XML output."

But since we're mostly web geeks here at ReadWriteWeb, we're more enthralled with another aspect to this tool: its intelligence. As we mentioned not too long ago, cloud agents are on the rise. The term, coined by blogger Chris Arkenberg, refers to automated agents that help us better deal with the volumes of data we have to sort through every day. Although Ambient News isn't necessary a full-on cloud agent - it doesn't actually work in the cloud - it can still certainly be considered an agent, especially since it helps us sort through a barrage of information in a new way.

Other Alternatives

Ambient News is not the only alternative to the traditional RSS Reader. Over the past year at ReadWriteWeb, we've also made mention of other alternative news readers like Feedly, which puts a magazine-style interface on top of Google Reader. Another popular RSS reader is Snackr, an Adobe AIR app that scrolls headlines across your screen like a news ticker. Then there is, of course, FriendFeed, a lifestreaming application that's quickly becoming an alternative way to share information among the early adopter set.

Alternative RSS readers aren't for everyone, though - journalists, bloggers, researchers, and the like may still need to use a jam-packed feed reader in order to seek out the elusive info they seek on a regular basis. But for those of you who are more casual web surfers and blog readers, alternative RSS readers are a less stressful way to get your news without the news getting to you.