Today Collecta.com gave up the ghost and is now a parked domain. The company made a strong go of it and but apparently despite having a unique and smart product, talent, money and attention - it just wasn't enough.

It quickly became apparent that what the company had to offer wasn't something that enough people wanted to buy. Six months ago Collecta announced it was shutting down its API and moving in a different direction.

It doesn't look like that new direction ever emerged. Two months after that announcement, CEO Campbell left the company to resume his focus on tech investing, according to LinkedIn. And today the site went dark.

It's really a shame. Jared Smith, ReadWriteWeb's main man for all things technical and design, says there's nothing quite like Collecta. "Collecta's emphasis on a search experience that went beyond Twitter into photos and videos made it a great tool to truly watch a story unfold in real time," he told me. "Their embeddable widget, which I used regularly on ReadWriteWeb's event sites, was far more powerful than what Twitter provided and is still unmatched in my mind."

Personally, I suspect that there is too small a market for real time search in the consumer world. Tell a company that you'll search in real time for actionable information about itself, for example, and you could find some interest - but that's a feature not a product and is something that other companies already offer as B2B services. I don't think consumers are interested in real time search, though they are clearly interested in real time messaging and content delivery on sites they already use.

Neither Collecta, nor competitor OneRiot were able to build growing companies around filling this need. The next most likely may be Topsy, which raised another $15m of its now $30m in funding just this Spring. Real-time social stream service Echo appears to be thriving in a related market - that company says it serves up 40,000 real time media items per minute at peak to customers around the web.

Rest in Peace, Collecta and thanks for all the searches.

Will developers go for it? Though Digg's currency appears to be dropping fast, real-time streaming data from millions (?) of social media users, concerning links to content from all around the Web, has got to hold some interest for programatic analysis, UI innovation and publishing industry analytics. The flow of data coming through the API seems a little anemic, though. We spoke with some of the Web's leading data developers who see today's announcement in very different ways.

(Disclosure: Musser's blog is now owned by Alcatel-Lucent, who is also a ReadWriteWeb sponsor.)

The Upside

Pete Warden, independent social graph analysis consultant, is optimistic for what this means to the ecosystem of mashup-friendly startups.

"Despite its recent troubles, Digg is still a massively popular tool for uncovering new stories. Offering a streaming API opens up all sorts of possibilities for mining that data, everything from spotting up-and-coming stories to offering detailed real-time analytics to content publishers. I know that startups like OneTrueFan are already salivating over what they'll be able offer."

Likewise, Tweetmeme founder Nick Halstead said today that he hopes to incorporate the new Digg streaming API into his much-anticipated new startup DataSift. "Our curation abilities on top of Digg submissions would be very powerful," he told us.

The Downside

Is Digg's community exciting enough for it to still matter, though? Jeremie Miller, inventor of the Jabber/XMPP real-time/instant messaging protocol, is not so sure.

"Honestly, my gut instinct was 'digg is struggling to be relevant.' They should have published approximate flow rates with this too; some sparkly graph eyecandy woulda been nice diggs-per-second, etc. The stream is showing a flow rate of a few per second at best - Twitter's firehose is 500-1000 per second.

"Technically, their implementation definitely trumps Twitter's... whose firehose is about as un-sexy as it gets. Digg did a better job at that for sure - but it was such a low bar already.

"The only firehose I'd be surprised by and care about at this point would be Facebook's - but I expect we'll have first contact before that happens."

What Does it Mean for Digg?

If a wide range of third party startups and innovators can make meaningful use of Digg's streaming API, that could help form a virtuous circle that makes Digg more interesting to people again. If nothing substantial comes of the API, that doesn't bode well.

Witness, for example, the case of OneRiot. OneRiot is a well-funded real-time search service that put a whole lot of eggs in the basket of its ambitious API earlier this year. It's unclear that was effective in making OneRiot really stand out, though - and now the technically admirable, world-changing startup has put its tail between its legs this month and turned entirely into an advertising network.

Something like that may very well happen to Digg. Or, a streaming API could help make Digg hip and fun again.

What do you think, readers?

According to the data the research team collected, most real-time queries come from third-party applications and don't happen on the search engine's own page. On Collecta, API queries account for about 60% of all queries and we've heard similar - and often even higher - numbers from other real-time search engines. As these API-based queries are often repeated multiple times throughout the day, real-time search engines also tend to see fewer unique queries than regular search engines.

Top Queries

The researchers also collected a list of the most popular queries on Collecta during a 190-day period at the end of last year. This data shows that the typical queries on real-time search engines are quite different from what we would expect to see on a regular search engine. Even the most popular search term ("naomi watts"), only accounted for 0.003% of all queries. According to Hitwise, "facebook" is currently the most popular search term on all the major search engines (Google, Yahoo, Bing) and accounts for between 1.17% of all queries on Google to 2.6% of all queries on Bing.

Where's the Porn?

As the researchers note, Collecta registered almost no searches that were pornographic in nature. "Sex" was the ninth most popular search term and only accounted for 0.002% of all queries.

Early Adopters

As the researchers rightly note, the list of most popular search queries on Collecta reflects the early-adopter nature of the service. The fact that the second-most popular query was "jQuery CSS" is a good example for the nature of real-time searches. To some degree, of course, these numbers are also biased towards Collecta's users and the kind of applications that have been developed on top of Collecta's API.

It will be interesting to see what these numbers look like by the end of this year. If "jQuery CSS" is still in the top 10 of most often used queries, we can safely assume that Collecta - and maybe real-time search as a whole - hasn't reached a mainstream audience yet.

Twitter-Style Ads and Vertical Searches

In addition to this update, OneRiot now also offers developers new tools to filter search results for vertical searches such as news, videos and images, as well as for niche searches that only return results about topics like gadgets, politics and music.

Given that virtually all of OneRiot's growth and traffic dependents on keeping its partners happy, these updates are an important step for OneRiot. In this same context, it's also an important move for OneRiot to offer Twitter-optimized results. The company already offers a wide variety of advertising solutions, but for Twitter developers, being able to just plug these new ads into their products will surely prove to be very helpful.

Some of the developers that have already implemented ads from OneRiot include the popular multi-service IM client Digsby and the BlackBerry Twitter client ÜberTwitter. Ads are delivered through OneRiot's API and developers have considerable freedom in how they use these ads.

For additional information about OneRiot's advertising network, also have a look at our earlier coverage here.

There are two ways to access real-time search results in the app. The homepage now displays the hottest trending topics, and a single click on one of these opens up OneRiot search results.

In addition, you can also do a normal search and then select "latest buzz" from the options menu right next to the search box. This same menu also allows you to restrict search to images, videos, blog or Wikipedia articles, as well as to sites and blogs about sports, news or apps.

For a closer look at Taptu's feature set, also have a look at our review of the service's web app.]]> Discuss]]> http://www.readwriteweb.com/archives/taptu_brings_real-time_search_to_its_mobile_apps.php http://www.readwriteweb.com/archives/taptu_brings_real-time_search_to_its_mobile_apps.php News Tue, 22 Dec 2009 12:30:51 -0800 Frederic Lardinois OneRiot, one of the leading real-time search engines, just announced the launch of a new advertising product for real-time apps. RiotWise Trending Ads will give OneRiot's partners a feed of ads related to currently trending topics on the Web. These ads can, for example, be integrated in a user's stream of updates in Twitter apps or displayed as regular mobile ad units. Digsby, for example, plans to place these ads directly in its users' streams, but because the units are delivered as a feed through OneRiot's API, developers are free to use them in whatever way they see fit.

2010: It's All About Monetizing the Real-Time Web

As OneRiot's general manager Tobias Peggs told us earlier this month, "2010 will be all about monetizing the real-time Web." Real-time advertising, however, comes with its own set of challenges. While Google AdSense, for example, can take its time to learn about what works best for a certain keyword, advertising systems for the real-time web have to work with a different set of signals and react to an environment that is always in flux. In this context, it makes sense for OneRiot to launch a product that focuses on trending topics and not on trying to match an individual user's stream to the right advertiser.

Initially, OneRiot will work closely with developers to ensure that the initial implementations of the RiotWise Trending Ads work well for users. For now, developers will have to get approval to use the new ads by applying through the OneRiot Developer Network. The company plans to roll out the program more aggressively in the next year.

However, nearly everything else about the new Android application is the same as its mobile counterpart, including its search verticals of "web," "images," and "buzz" which sit above Taptu's search box on the app's main screen.

The "buzz" section contains real-time results pulled from sites like Twitter and Digg as well as from other social sharing websites, blogs, and data pulled from select panel of internet users who have downloaded the OneRiot toolbar and are anonymously sharing their web-browsing data with the company. This aggregate information is actually the most prominent source of real-time data for OneRiot's search service. As of September of this year, OneRiot claimed to have 3 million active toolbar users out of the 20 million or so who have downloaded the toolbar to date. Meanwhile, they're indexing around 20,000 links from Digg and 5 million from Twitter.

It's this data which powers Taptu's "buzz" section where you can find breaking news and other currently "hot topics" being discussed on the web. And unlike Twitter's trending topics, for instance, OneRiot doesn't link to raw tweets but to the actual news stories and blog posts that are sharing the information.

The main Taptu homepage also links to these trending items by way of a tag cloud whose colorful blue bubbles take you directly to the buzz section when tapped.

If you're interested in trying the new Android application, you can download a free copy from the Android Market on your mobile phone. If you don't own an Android device (or iPhone), you can still use Taptu via its mobile website available at www.taptu.com.

You give Factery a list of links and a keyword and it will build an index of all the facts asserted in those links about your topic of interest, delivered in XML or JSON format. The service can run on top of a search engine but could also be used in any number of other ways. I've been feeling unsatisfied with other search engines all day since seeing a Factery demo Monday morning.

Compare that with the Factery results page - I don't even need to click through if I don't want to, I feel like I got a great overview of the story just from my search page. Perhaps that's a problem - for a publishing industry that already says it's scared of search engines - but as a reader it sure isn't my problem, it's great. Why would I want Google News to tell me where I can go to find information if someone else will just give me the information?

The company's test demo searches Twitter and Yahoo Boss - neither search is as exciting as I'd hoped 100% of the time, but it's often remarkably good. Factery is also testing an interesting integration with Silverlight stream reader Sobees, in which linked pages from Twitter or Facebook are annotated with automatically extracted highlights via Factery.

I expect a whole lot of companies are going to at least try this API out and I'm excited to see the results.

How This is Unlike Other Real-Time Search Services

Factery is talking a lot about its ability to analyze links shared over Twitter, but that's probably just because Twitter is easy for people to understand. The fact is, the service can perform on-demand analysis of text behind any set of links. That's what differentiates it from other real-time search engines like OneRiot, which also analyzes the text of pages linked to on networks like Twitter and offers an API to display real-time search results on other sites. Competitor Collecta analyzes Twitter streams in real time and offers an XMPP API to push new search results live to any page.

Factery is a different kind of animal, though. It's more like a smart search inside any other search. It doesn't even have to be search, though. The company talks a lot about how they make mobile reading more efficient by pulling the salient information up to the surface of a page, instead of requiring mobile readers to load multiple pages.

I thought of five or six different ways I'd like to use it just while talking to the company on the phone. (I'm not going to share those here, either. I think some could offer an important competitive advantage.)

I'd Love to See This Work Everywhere

Yesterday I was testing a new Android app from the Sunlight Foundation that lets you track members of congress. One tab in the app is a search for your congressperson in the news. Unfortunately, the page excerpts give no indication why the politician you searched for appeared in that news story - just that their name did, somewhere. That search is powered by a Yahoo API, probably BOSS, but it's not any fun to use at all. How unsatisfying, I thought, when I could have a list of key facts concerning my search query in the list of links that the search brought back. But that was yesterday, and Factery is just launching today.

The possibilities are truly endless. That's probably why Ron Conway, one of the leading investors in the real-time economy, joined others in investing in the company. With $1.2 million in the bank, Factery is a modest developer play with a whole lot of potential.

Give Factery's API a try and let us know what you think. It's free to use; the company says it may start inserting "sponsored facts" (isn't that an interesting phrase) into results later but things like business model and to a lesser degree de-duplication are still works in progress. I sure do love this idea.

Using the OneRiot API, Taptu's new homepage presents a mobile-friendly search engine interface complete with search box and verticals for searching just the web, images, and now, "buzz." Previously, the site included verticals for music and video searches too, but those have seemingly been done away with in an effort to simplify the interface.

The new "buzz" section is where you can find the real-time results. Here you'll find content pulled from sites like Twitter, Digg, other social sharing sites and the company's own panel of users who have downloaded the OneRiot toolbar and are sharing their web-browsing data in anonymous aggregate.

While Twitter is clearly a source of breaking news, we've always found it a bit odd that OneRiot includes Digg in its "real-time" search engine. We've never thought of Digg as anything near real-time - in fact, it pales in comparison to Twitter when it comes to the speed with which information spreads. With the immediacy possible on today's web, sites like Digg seem much slower - painfully slow at times - often taking hours on end to feature the news that had already been buzzing on Twitter for half a day.

That said, Digg and other social news sharing sites can sometimes unearth news that had been overlooked by major media outlets, especially when focused on a particular niche like technology. For example, just think of how many stories you read on someone's personal blog or Reddit and never saw anywhere else on the web. By tracking niche websites like these as well as Facebook, MySpace, Bebo, social bookmarking sites like Delicious and StumbleUpon, microblogs and URL-shortening services, Oneriot can discover links that may have otherwise gone unnoticed.

Taptu's new mobile search engine interface currently works on major touch-enabled devices including the iPhone, iPod touch, G1, Nokia N97 and 5800, and the BlackBerry Storm 1. The Taptu iPhone application will also be updated soon to include the additional functionality. You can test the new service yourself starting at 9 AM EST by pointing your mobile browser to www.taptu.com.

Twitter has acquired two other companies so far, that we know of. Search engine and sentiment analysis service Summize became Twitter's own search engine and Values of N sold its assets so engineer Rael Dornfest could be brought into the company. Here are ten other startups we think that Twitter should consider acquiring next. Which kind of company would you most like to see become part of Twitter itself? We've got a poll below.

Some of these we think are likely acquisitions, some less so. In making this list we considered both functionality that would be helpful to have added to Twitter's own site and technology that would be worth buying instead of just building in-house. Whenever a platform company builds technology that a number of other startups offer, there is a risk of scaring other people away from investing in development that the platform could just reproduce. Acquisitions of startups on a platform probably increase the appeal of development though, as it's a chance to get in on the game.

Quite Likely, if It Hasn't Happened Already

Bit.ly is the most full-featured and popular URL shortener on the market right now and was recently selected as Twitter's own shortener of choice, dethroning TinyURL. Bit.ly offers all kinds of smart analytics, from real-time click tracking to semantic analysis of topic keywords from the links that people tweet.

One trusted industry source speaking on the condition of anonymity told us that Bit.ly servers "were moved into Twitter's racks months ago in preparation for this change" [of becoming the default shortener]. Bit.ly is becoming too important to Twitter to keep that functionality outside the company's own shop and the two companies share some investors. We will not be surprised at all if a Bit.ly acquisition by Twitter is announced sometime in the near future.

Could Happen...

Tweetmeme is another fast growing Twitter analytics service that tracks sharing on the service. With another chunk of new features just added today, the service is looking a whole lot like "Feedburner for Twitter" but with even more viral distribution possibilities. The Tweetmeme API is quite interesting and could complement Bit.ly quite well.

Twitpic is a popular way to share images on Twitter. The site faces a strong challenge from ImageShack's YFrog, but independent Twitpic would be a cheaper acquisition and is already well known among Twitter users. (Twitter should probably look at Enjoysthin.gs; it's got the best user experience.) An increase in imagery on Twitter would probably offer the company a lot more advertising real-estate.

Tweepz is a fascinating Twitter search engine that acts like a directory that lets you parse your results using various metrics gleaned from Twitter. Check out this search, for example. Twitter could benefit from making this kind of search available to users, advertisers and researchers - and Tweepz has already built it. See also Twazzup, another company doing interesting things with Twitter data.

Longer Shots

An iPhone app company could be a good buy for Twitter; there's certainly plenty of options. M.Twitter.com is a good mobile service already but someone specializing in super high-quality Twitter apps for the iPhone, Android and Pre could be good to bring in house. It could be AteBits, makers of Tweetie. There may not be enough reason for Twitter to buy one of these companies, though.

A desktop Twitter app company could help Twitter increase user engagement. Many of the most serious Twitter users (though not all) swear by desktop access. Twitter could acquire the most popular and arguably most innovative desktop app, Tweetdeck, or it could bring Seesmic in house. Tweetdeck would be cheap and shares investors with Twitter. Desktop apps may be too limited in appeal to be a compelling acquisition target.

Geo-location could be a good feature to add to Twitter. Search by user location could be made much more meaningful and the list of things that could be done with it is very long. Brightkite is popular and well developed, Shizzow is pretty and wouldn't be expensive. On the other hand, browsers themselves will likely all become more location aware in the near future and Twitter may be satisfied with its current location data.

A semantics company could bring structure to the Tweets, making them more useful and easier to advertise against. Right now links Tweeted are semantically analyzed by Reuters' Calais and sent to Bit.ly, but we wouldn't be surprised if Twitter was interested in scooping up a small semantics shop and helping it scale so that analysis was being done in house. Twitter may feel like semantics don't need to get that close to consumer users, though. (Disclosure, Calais is a ReadWriteWeb sponsor.)

Topify is a widely loved service that intercepts your new Twitter follower notification emails and sends you much more useful ones. It's great but probably too easy for Twitter to just reproduce itself.

FriendFeed plus Twitter would be a match made in heaven. It would be an engineering powerhouse. It would be a step towards mainstream user adoption of FriendFeed, a service that can't make up its mind which end of the sophistication spectrum it's targeting. It's also quite unlikely to happen. If there's one related startup we can imagine turning down a Twitter acquisition offer, it's probably FriendFeed. (Though the investment-laden and highly ambitious OneRiot is a close second.) Nonetheless, it would be awesome if FriendFeed's cross-network aggregation, threaded conversations, groups, media support, search and more joined forces with Twitter.

Ultimately, it may be most likely that Twitter's next acquisition will be something vapid. A service that aggregates shopping Tweets, or celebrity Tweets, or something else that will fall short of taking advantage of the Twitter platform's huge potential to change the world. Twitter staff makes relatively simple use of its own service, so hoping that it will acquire companies that make it all the more powerfully sophisticated may be an early adopter's pipe dream. [Update: After some discussion this afternoon, I am thinking it's time to reconsider this position I've held for some time. Twitter staff is not full of dummies, I'm sure, and it has probably been inappropriate of me to write as if that's the case.]

Maybe not, though. We wouldn't be shocked to see Twitter pick up at least a few of the companies above. What do you think? Are there other services you'd like to see become part of the Twitter team even more than the above? It's a wild and woolly micro-content ecosystem out there - anything could happen.

You can find ReadWriteWeb on Twitter, as well as the entire RWW Team: Marshall Kirkpatrick, Bernard Lunn, Alex Iskold, Sarah Perez, Frederic Lardinois, Doug Coleman, Jolie O'Dell, Dana Oshiro , Lidija Davis and Steven Walling.

Although results based on freshness alone are available through the search engine's real-time firehose setting, the results returned through the Pulse Rank setting are weighted based on several factors that riff off similar considerations for the static web and Google's PageRank system.

Pulse results will now be a product of the following considerations: Timeliness, domain authority, individual reputation, and acceleration. The new algo attempts to weed out spam links (or links from known spammers) and give preference instead to "more thoughtful sharers whose links tend to get retweeted and dugg."

According to the blog, the algo tweaks will also improve detection of whether a page is trending or whether it's simply a large, popular destination and always has been.

"We're also getting a lot of leverage from our Artificial Intelligence systems that constantly 'learn' how to improve the way we rank results," the post continues.

Collecta purports to pull in blog posts, comments, Twitter and Identi.ca updates and photos concerning your search query, as fast as technically possible. There are some rough edges for sure at launch, but Collecta has a lot of promise. Pagerank or other systems of authority are in many cases not what you're looking for in search - timeliness is.

Three of us at ReadWriteWeb tested Collecta this morning and only one of us got a consistent flow of new results coming in. It was slow and choppy for two of us, on three different internet connections. The ability to filter out certain kinds of results, to run multiple persistent queries at once and to preview items before clicking off site to read them were all great features.

Twitter search results, unfortunately, link out to a user's front profile page, not the particular Tweet that is shown on the results page. We imagine that's a small oversight that will be fixed promptly.

Blog posts are limited to WordPress.com blogs at launch, as the companies have common investors and an agreement. Sometimes filtering for comments on blogs brings up results when filtering for blog posts does not.

Despite warts and bumps, there's really nothing else quite like Collecta available on the market right now. It's a good way to get a handle on the real-time flow of information about a topic and it has an incredibly strong team.

We expect real time search to be an important part of the search world as Twitter search has already changed the way we research things online dramatically. Expanding that out into more media types is something that just has to happen.

We'll be keeping a close eye on Collecta as it expands and improves.

OneRiot's Tobias Peggs said the new capability is "a key feature. They are the first of these type of desktop applications to offer real time web search results in addition to the usual Twitter conversation search. This means Nambu users don't just keep track of conversations, but they also get to find related fresh content (news, blogs, videos, etc) as it emerges on the real-time web."

"OneRiot's real-time web search engine is the best way to find fresh, socially-relevant content from across the web," said Eric Woodward, CEO of Nambu. "Now our users can track realtime conversations on any topic, and discover new related content at the same time. It's a great combination."

In addition to OneRiot real-time web search, Nambu now offers Twitter and FriendFeed search to index conversations and links. The search offering is rounded out with Yahoo! News, which provides feeds from AP, Reuters, and other outlets.

"With the increase of noise and spam in real-time conversation search," said Peggs, "it's good for the users to also have real-time content search. Now you can track a term ... see what people are saying, and also discover emerging content. It's a nicely rounded experience for the user."

Here's an example of what those results would look like; OneRiot results are highlighted:

According to the IE Addons Gallery page, "This special version of Internet Explorer 8 comes loaded with fresh OneRiot goodies, putting the real-time web directly into your browser."

For users who have already installed IE8, Microsoft also offers three real-time add-on packages, which add the same OneRiot components contained in the fully optimized browser. Add-ons include OneRiot's real-time search results, top videos, and top shared items of the day.

Here's a composite screenshot of the webslices, featuring top shared results and top videos:

Users will notice the OneRiot search bar in the top right of the browser; the add-ons also give users the option of selecting OneRiot as the default search engine. OneRiot search results are indexed based on an algorithm that weighs the number of social media shares each link receives as well as the authority of the sharer, among other factors. OneRiot results are intended to be highly recent as well as highly relevant and include the exact number of shares along with the context of the share.