TweetMeme is refocusing its efforts away from simply monitoring retweets and clicks toward deeper, richer live analysis and filtering of the Twitter Firehose. TweetMeme CEO Nick Halstead mentioned in a blog post this morning that this new focus for the company will "manifest itself in the launch of a number of new products," and today DataSift is the first of these products to be introduced.

DataSift claims it "gives developers the ability to leverage cloud computing to build very precise streams of data from the millions and millions of tweets sent everyday." Users will be able to create, share, comment and rank these tuned data streams on the DataSift site - a process the company hopes will lead to massively powerful "super streams."

As the introductory video above explains, users could curate a list of tweets from coffee shops in San Francisco, or photos posted near the Eiffel Tower. DataSift is currently taking applications for alpha testing, which Halstead says they hope to launch within the month.

"We hope that the product will allow developers to focus on the building of amazing new user experiences and let us worry about delivering the right content to the right person," says Halstead.

While TweetMeme is continuing to offer its retweet button, DataSift is the first in a series of products in a new direction for the company. Highly anticipated official analytics and analysis tools from Twitter have yet to drop, and today's Tweet Button announcement could be an early first step. It seems, however, that TweetMeme has their own plans with DataSift and other products, and perhaps that is what kept today's announcement from being that of an acquisition.

According to Twitter, nearly a quarter of all tweets include a link and, until now, they have been shared using third-party services or by "copying and pasting, link shortening, and bouncing between browser tabs" which the company calls "just too much work".

For the user, the Tweet button will bring up a pop-up window with a suggested tweet and a shortened link, identifying both the full link to be shared and the URL shortener used. After tweeting, users may also see a list of suggested Twitter accounts to follow, as designated by the site they're visiting.

On the developer end, the new button takes only a few lines of code and allows the publisher to decide the suggested tweet text, who the tweet should be attributed to and the list of suggested Twitter accounts to follow. The Tweet button API does not, however, appear to include a way for publishers to choose a preferred URL shortener. It seems likely that the move is a precursor to Twitter's long-awaited analytics, as it may be difficult to lure publishers away from third-party URL shorteners that already offer such statistics. When Twitter launched native URL shortener t.co in June, it signaled that eventually all links would be wrapped in t.co as a way to protect users and keep track of links to determine if they are "relevant and interesting".

Tweetmeme founder Nick Halstead said in a blog post today that the TweetMeme team "will be assisting Twitter with the technical challenges involved with the button", for which it has likely faced many. TweetMeme had recently grown to share more than 750 million daily retweet button impressions. Tweetmeme is recommending publishers use the official Twitter Tweet button, but is continuing to offer its button as well.

Additionally, using Google Translate, Tweetmeme will now automatically translate tweets users wish to share in their native language. "This automatically detects if translation is required and prompts you when it has made the change, it is a simple one-click to revert the translation if that is not what you desire," said Tweetmeme in a blog post today.

It's encouraging to see language support reaching more third-party services as the international Twitter community grows. Personally I am still waiting for the popular desktop and mobile applications to begin automatically translating tweets from users I follow who tweet in multiple languages. This, of course, is challenging due to the shorthand slang that Twitter's character limit produces, but it would be an invaluable feature to users like myself who happen to follow people who often tweet in a language other than my own.

Simply by automating the collection of content, Toyota minimizes the number of people, time and level of effort required to run a highly engaging website. No one is needed to create content, no complicated workflow and approval process - no manual push to live. The content is published for the most part by fans and nay-sayers of Toyota alike. This means Toyota staff can concentrate on curation, conversation and engagement.

Reuse

The entire site, from front to back, is reusing existing resources. The hardware, software and interface design is developed and delivered by Tweetmeme. The platform existed - Toyota is just tweaked existing data to deliver relevant, real-time content to the press, its customers, its employees - and let's not forget the competition.

Recycle

Instead of building from scratch, Toyota is recycling an existing idea and repurposing it for its own needs. As such, the solution was most likely delivered in a quarter of the time (or less) that it would have taken Toyota to build a similar solution. And even if Toyota tried to create the experience from scratch, it most likely couldn't replicate the Tweetmeme model as it also gained the brains, years of research and development and data management skills of Tweetmeme.

Management is RELATIVELY happy as the solution is inexpensive, quick to market, and from a development perspective, generally risk free as it is a proven solution that Tweetmeme will continue to evolve. The data geeks are happy because they have a new toy to play with, while the marketing team is thrilled because they have a platform to view real-time what customers are saying, and respond accordingly.

We got a look at the back-end infrastructure of the Tweetmeme portal system and have screenshots displayed below. These aren't for the Toyota project in particular, but they are the same tools being put to use in a different campaign. We know you feed and data geeks fantasize about building the ultimate feed moderation system. Check out the one that Toyota put down no small sum to get to use. It's a nice combination of heavy duty and easy to use, just like you'd expect for a big corporate customer like this. The best news? This system will be opened up to the public soon.

Below, an item page for a popular link shared about Toyota. Below is what appears to be the company's direct response. Thus the name of the site, Conversations.

Easy to Use Logic Chains

Tweetmeme portal customers set up complex combinations of rules for which tweets to display using what company founder Nick Halstead calls "a mini-programming language - with a drag + drop interface for setting them up. Rules can be based on tweet, text from the story, title, meta data from the story, geo location data, twitter users who are tweeting...almost any data that is associated with twitter and the linked story that we spider as well. Each channel can have a number of chains - each chain can work separately - but be valued differently - i.e. have a confidence factor associated with it."

The Big Dashboard in the Sky

This is what Tweetmeme HQ looks at, standing on top of all the channels. The ten person team calls its big set of rules "the pickle matrix". Every time someone Tweets a tweet with a new link in it, or a Retweet, that data is thrown against the pickle matrix. That's the field "access count." Then an optimized process of rules are matched. "The data isn't the problem," Halstead says, "it's the number of rules we put against it. This is 1,000 times more powerful than Twitter's Track or search because we can apply tens of thousands of rules to every Tweet we see."

Halstead's company got a big boost from this deal, but Tweetmeme has been cash-flow positive for at least the last 3 months. "I think the more interesting fact," he says, "is that I started this company for the sole purpose of doing this and companies are now only just starting to recognize the value in this kind of proposition. I think that shows how far social media has grown up. And that you have to stick at what you know is right - even if people ignore it to start with."

No word yet on when this system will be opened up to the public, but used in conjunction with other media types like Toyota has it sure seems like there's a lot of potential here.

Disclosure: FM Publishing, a partner in the Toyota project, is also RWW's advertising network.

Tweetmeme had similar beginnings and is now quite an impressive little company. Google Buzz has a lot of potential, and Buzzr is worth watching too.

Buzzr has taken a page out of Tweetmeme's book by prominently featuring a website button to share a story on Buzz, something that put the interests of publishers, users and Tweetmeme all in sync for that company.

On the other hand, there is a whole lot of backlash already concerning Buzz's intrusiveness. That's why we decided not to push content there automatically but to engage in conversations there instead. If that's how the medium shapes up then services like Buzzr may not be as widely embraced as the number-pumping Tweetmeme has been.

Time will tell; since Google Buzz's most disruptive quality is its open data standards, there should be a whole lot of services built on top of it soon.

A new class of tools intended to surface influencers and the people they are influenced by are focusing on a hub of rapid, connected conversation that's wide open for analysis - Twitter. Could analysis of individual behavior on Twitter become a valuable tool for business development and marketing? A growing number of startup companies are making a case that it could.

On the margins of the developing Twitter-as-business tool ecosystem are startups building light-weight influencer discovery and analysis tools. Two of the most interesting yet have launched in the last 24 hours, in fact.

Shining a Light on Webs of Connection

Still one of the most useful tools in this group, if still largely a proof of concept, is Pete Warden's Mailana. Give Mailana a Twitter username and it will tell you who the top 20 people that user has had reciprocal public conversations with on Twitter - who they are engaging with the most. Want to get into the world of Lewis Shepherd, CTO of Microsoft Institute for Advanced Technology in Governments? (It's his birthday today, by the way.) Then you might consider joining CTOvision.com author Bob Gourley, Purdue University Cyber War Researcher Samuel Liles and Dr. Mark Drapeau in conversation on Twitter. That's who Shepherd communicates with the most there, according to Mailana.

Friendship Discovery As a Service

It may be a sign of market maturation that some companies are now offering APIs for this kind of functionality and others are integrating it into their core products. Yesterday social media aggregator Seesmic integrated into its web application a feature from Twitter recommendation service Mr.Tweet that surfaces what other users or anyone "pays the most attention to" on Twitter. This is really useful - you can discover key people for any influencer you follow with just a few clicks. Did you know that local-blogging guru Lisa Williams is a very close contact of media consultant Amy Gahran? I did because I've been following them both for years - but now you can discover that connection programmatically, with a click.

Performance of the new feature integration is spotty and the user experience leaves a lot to be desired - but it's only been out for 24 hours.

The platform is wide open for analysis and exposing a person's "social graph" is wildly valuable for discovery, context, relationship building and more. There's going to be a whole lot more development in this direction.

Influencer Discovery For Sale

Will people pay for this kind of information? If they are smart they will. Popular Twitter shared link tracking service Tweetmeme released its first paid product this morning and is betting it can monetize influencer discovery.

The new Tweetmeme Analytics service does a number of things, but the most useful feature may be what it calls "retweet trees." Publishers can surface the chain of retweets that passed their links around and see which Twitter users had the biggest impact on a chain of retweets.

Tweetmeme is surfacing other types of data as well and there's potential for that data to be organized, cross referenced and displayed in a wide variety of ways. Tweetmeme is already performing text analysis of the pages being shared on Twitter, so content type could become another axis point to look at the data around.

All of these services are in early days. If people will in fact pay for them, then they will develop all the more. It's a greenfield of possible added value. Finding out who is of interest to people of interest to you is a classic business activity, open social networking like Twitter makes that possible in newly interesting and efficient ways.

Twitter has acquired two other companies so far, that we know of. Search engine and sentiment analysis service Summize became Twitter's own search engine and Values of N sold its assets so engineer Rael Dornfest could be brought into the company. Here are ten other startups we think that Twitter should consider acquiring next. Which kind of company would you most like to see become part of Twitter itself? We've got a poll below.

Some of these we think are likely acquisitions, some less so. In making this list we considered both functionality that would be helpful to have added to Twitter's own site and technology that would be worth buying instead of just building in-house. Whenever a platform company builds technology that a number of other startups offer, there is a risk of scaring other people away from investing in development that the platform could just reproduce. Acquisitions of startups on a platform probably increase the appeal of development though, as it's a chance to get in on the game.

Quite Likely, if It Hasn't Happened Already

Bit.ly is the most full-featured and popular URL shortener on the market right now and was recently selected as Twitter's own shortener of choice, dethroning TinyURL. Bit.ly offers all kinds of smart analytics, from real-time click tracking to semantic analysis of topic keywords from the links that people tweet.

One trusted industry source speaking on the condition of anonymity told us that Bit.ly servers "were moved into Twitter's racks months ago in preparation for this change" [of becoming the default shortener]. Bit.ly is becoming too important to Twitter to keep that functionality outside the company's own shop and the two companies share some investors. We will not be surprised at all if a Bit.ly acquisition by Twitter is announced sometime in the near future.

Could Happen...

Tweetmeme is another fast growing Twitter analytics service that tracks sharing on the service. With another chunk of new features just added today, the service is looking a whole lot like "Feedburner for Twitter" but with even more viral distribution possibilities. The Tweetmeme API is quite interesting and could complement Bit.ly quite well.

Twitpic is a popular way to share images on Twitter. The site faces a strong challenge from ImageShack's YFrog, but independent Twitpic would be a cheaper acquisition and is already well known among Twitter users. (Twitter should probably look at Enjoysthin.gs; it's got the best user experience.) An increase in imagery on Twitter would probably offer the company a lot more advertising real-estate.

Tweepz is a fascinating Twitter search engine that acts like a directory that lets you parse your results using various metrics gleaned from Twitter. Check out this search, for example. Twitter could benefit from making this kind of search available to users, advertisers and researchers - and Tweepz has already built it. See also Twazzup, another company doing interesting things with Twitter data.

Longer Shots

An iPhone app company could be a good buy for Twitter; there's certainly plenty of options. M.Twitter.com is a good mobile service already but someone specializing in super high-quality Twitter apps for the iPhone, Android and Pre could be good to bring in house. It could be AteBits, makers of Tweetie. There may not be enough reason for Twitter to buy one of these companies, though.

A desktop Twitter app company could help Twitter increase user engagement. Many of the most serious Twitter users (though not all) swear by desktop access. Twitter could acquire the most popular and arguably most innovative desktop app, Tweetdeck, or it could bring Seesmic in house. Tweetdeck would be cheap and shares investors with Twitter. Desktop apps may be too limited in appeal to be a compelling acquisition target.

Geo-location could be a good feature to add to Twitter. Search by user location could be made much more meaningful and the list of things that could be done with it is very long. Brightkite is popular and well developed, Shizzow is pretty and wouldn't be expensive. On the other hand, browsers themselves will likely all become more location aware in the near future and Twitter may be satisfied with its current location data.

A semantics company could bring structure to the Tweets, making them more useful and easier to advertise against. Right now links Tweeted are semantically analyzed by Reuters' Calais and sent to Bit.ly, but we wouldn't be surprised if Twitter was interested in scooping up a small semantics shop and helping it scale so that analysis was being done in house. Twitter may feel like semantics don't need to get that close to consumer users, though. (Disclosure, Calais is a ReadWriteWeb sponsor.)

Topify is a widely loved service that intercepts your new Twitter follower notification emails and sends you much more useful ones. It's great but probably too easy for Twitter to just reproduce itself.

FriendFeed plus Twitter would be a match made in heaven. It would be an engineering powerhouse. It would be a step towards mainstream user adoption of FriendFeed, a service that can't make up its mind which end of the sophistication spectrum it's targeting. It's also quite unlikely to happen. If there's one related startup we can imagine turning down a Twitter acquisition offer, it's probably FriendFeed. (Though the investment-laden and highly ambitious OneRiot is a close second.) Nonetheless, it would be awesome if FriendFeed's cross-network aggregation, threaded conversations, groups, media support, search and more joined forces with Twitter.

Ultimately, it may be most likely that Twitter's next acquisition will be something vapid. A service that aggregates shopping Tweets, or celebrity Tweets, or something else that will fall short of taking advantage of the Twitter platform's huge potential to change the world. Twitter staff makes relatively simple use of its own service, so hoping that it will acquire companies that make it all the more powerfully sophisticated may be an early adopter's pipe dream. [Update: After some discussion this afternoon, I am thinking it's time to reconsider this position I've held for some time. Twitter staff is not full of dummies, I'm sure, and it has probably been inappropriate of me to write as if that's the case.]

Maybe not, though. We wouldn't be shocked to see Twitter pick up at least a few of the companies above. What do you think? Are there other services you'd like to see become part of the Twitter team even more than the above? It's a wild and woolly micro-content ecosystem out there - anything could happen.

You can find ReadWriteWeb on Twitter, as well as the entire RWW Team: Marshall Kirkpatrick, Bernard Lunn, Alex Iskold, Sarah Perez, Frederic Lardinois, Doug Coleman, Jolie O'Dell, Dana Oshiro , Lidija Davis and Steven Walling.

Channels

In addition to these real-time streams, Tweetmeme is also focusing on providing channels about specific topics including this one for Earth Day, for example. Thanks to this, it might soon be a lot easier to filter out the noise during a big conference, for example, where it is usually impossible to keep distinguishing between high-value tweets and random invitations for lunch.

Sadly, these channels don't yet feature the new live streams, which is really a shame. It would also be great if we could create custom channels based on keyword searches that are then filtered by Tweetmeme, and presented in a real-time stream. We can't imagine, however, that the Tweetmeme team isn't working on something like this already (and maybe even implementing real-time updates in the mobile version as well).

Tweetmeme's backend is sponsored by Sun through the company's Startup Essentials program. The company also has an interesting business model, and it features some interesting leaderboards and stats about every item.

Twitter and Real-Time: A Marriage Made in Heaven

Twitter and  real-time clearly go hand-in-hand. Just in the last few weeks, we saw the launch of two new real-time Twitter search engines, Twazzup (our review) and Tweetmi, and, of course, there is also Monitter, the grandfather of Twitter real-time services. Tweetmeme's new real-time stream looks like a great compliment to those, and from what we've heard, the team still has a lot of great ideas about how to make the service even better.

Thanks to Marjolein Hoekstra (@cleverclogs) for alerting us to this new feature.

TweetMeme takes the TechMeme model and adapts it nicely, using Twitter as its source of news. Even the new revenue model echoes TechMeme's advertising system. We do like the democratic process it uses to find fresh new content without giving too much weight up front to authority. However, if you like that sort of thing, TweetMeme also has a leaderboard page where it tracks a spin of this concept, weighting the freshest sources first. Louis Gray posted about those features last month.

TweetMeme has not been sitting still with other refinements and features either. They recently rolled out a mobile version of their site, as well as a promotional site button and WordPress plugin.

Patrick Ruffini over at TechPresident thinks this could be the year of Twitter. He cites downtime that Twitter recently experience when two major news events overloaded the service, Macworld and the US State of the Union address, as evidence of the service's potential mainstream appeal. "While these spikes reveal some troubling capacity issues that Twitter will need to deal with, this is the surest sign that the service has gone mainstream in a way not anticipated by its founders," writes Ruffini.

Twitter is fast becoming a serious platform for discourse and discussion. More than a status app, it is being used as a first alert mechanism for the dissemination of news and for immediate discussion surrounding that news. It is the coverage of news events and the continued emergence of citizen journalism that will push Twitter toward the mainstream this year.

Why Twitter Works for News

It's fast. Increasingly mainstream news reporters and bloggers are utilizing Twitter to put up news tid bits as they happen, and commentary as it pops into their heads. For example, Ana Marie Cox, the Washington Editor of Time.com, maintains a Twitter account that is both informative and hilarious. As we recently reported John Dickerson, a political correspondent for Slate, uses Twitter to report from the US presidential campaign trail in near real-time. "It is much more authentic, because it really is from inside the room," says Dickerson of Twitter, which has a visceral nature that reporters are beginning to embrace.

It's open. By embracing an open API architecture from the start, Twitter has smartly nourished a large set of tools that help people use the service. This makes it easier for people to get content on Twitter in the manner most convenient and most comfortable to them, which in the long run should help drive adoption of the service. "The API has been arguably the most important, or maybe even inarguably, the most important thing we've done with Twitter," Twitter co-founder Biz Stone told us in September.

The API has also allowed for mashups that filter Twitter content making it easier to find. One relevant Twitter aggregator is Politweets (our coverage), which brings together all the messages sent over Twitter about the US election.

It's two-way. Unlike TV or newspaper, Twitter allows for a conversation. Like its new media brethren, blogs, Twitter encourages discourse and feedback. For reporters that aren't afraid to get down and dirty, Twitter is a golden opportunity to build a rapport with readers and gauge public opinion. It also makes readers feel more connected to the news when they can participate in a discussion about it as it happens, often times with the people reporting it first hand.

It fills a void. As Ruffini points out, Twitter is built for the new news cycle. "Traditional news operated on a 24-hour cycle. Blogs shortened this to minutes and hours. Twitter shortens it further to seconds," he writes. "It's not right for every piece of information. It's certainly not well suited for longer analysis. But when it comes to instantly assembling raw data from several sources that then go into fully baked news stories, nothing beats it."

Some Hurdles to Twitter Discourse

Sometimes, it's too fast. Twitter happens in moments. If you think keeping up with the blogging cycle at big blogs like Engadget is tough, then keeping up with a thousand voices on Twitter is damn near impossible. For the tech-obsessed -- the people like you and I who are on their computers all day already -- keeping tabs on Twitter could easily become part of the routine. But for the mainstream audience, Twitter might need better filtering tools before people can really wrap their heads around it.

Third-party clients like Twitterific can help filter to a certain extent, but they're not perfect.

It can be muddled. One of the strengths of Twitter is that it is a two-way street -- you an talk back to the people who are talking to you. But it's not threaded, so replies get shuffled around and often times, out of context, just become confusing. Further, when everyone is having a conversation at once, things get noisy. Twitter desperately needs a filter.

One recent attempt is Tweetmeme, a Twitter memetracker based on the concept of Techmeme. It works relatively well at figuring out what people are linking to on Twitter, but isn't well suited to figuring out what people are talking about, and separating out those individual discussions (plus, it doesn't filter for language, which can make it a bit confusing if you're not multi-lingual).

It's hard to navigate. There is a learning curve to Twitter. Finding people isn't as easy as it should be (certainly not as easy as on mainstream social networks like MySpace or Facebook, which people are used to), and figuring out who to follow to get involved in the conversations you want to take part in requires some work as well.

Conclusion

Despite some potential hurdles, Twitter is being used more and more for mainstream news coverage. KPBS News San Diego uses Twitter to put out updates about stories, for example, and during the California wildfires last fall it was a must read. The potential for Twitter to be used for news dissemination is something the site's founders realized early on during an earthquake.

With citizen journalism on the rise, it seems likely that Twitter will become an increasingly more important point for the distribution of breaking news during 2008, to the extent that traditional journalists will begin to pay more and more attention to it the way they have to blogs. Twitter won't replace blogging or newspapers, but as Ruffini says, it "open sources the process of developing ideas and gathering news tips, giving us a complete window onto the news cycle."