With this partnership, DuckDuckGo has added a new setting option that allows you replace the favicons that appear next to site results with WOT's traffic-light ratings. In other words, green is go; red, no go.

Earlier this week, we reported that Google had tweaked its search algorithm following a story in The New York Times about unscrupulous merchants gaming Google's search rankings via negative merchant reviews.

The solution implemented by DuckDuckGo goes beyond just "algorithmic search" to generate results, something that works particularly well when you're shopping or downloading.

While other services, like StatCounter have the two engines neck in neck, Chitika's latest stats put Bing ahead of Yahoo by 4.5%.

Search Engine Land's Danny Sullivan took a look at Chitika's numbers, noting that most every other U.S. rating service still shows Yahoo as ahead of Bing and that something must be going on. According to Chitika's numbers, Bing has been ahead of Yahoo for several months now, while every other service shows that Yahoo leads Bing, if not narrowly, then by several percentage points.

While other rating services rate traffic according to the number of searches performed on a given search engine, Chitika measures traffic according to hits that reach members of its advertising network. Sullivan concludes that the disparity must come in how traffic reaches Chitika and likely not in how much the individual engines are actually used, noting that "less of that traffic is 'escaping' out of Yahoo and to third-party web sites such as those in Chitika's network."

If that's true, then it might really mean something in the search engine battle. What does it mean to have more searches but not have that traffic escape your own network? Even if all these other ratings services find Bing as taking a smaller piece of the search engine pie, is their piece more valuable because it gets around more?

This partnership is sure to dramatically increase the number of people reached by Twitter's current user-base and could mean some big things for the microblog, not only in terms of exposure but in terms of its much rumored ad platform.

The Firehose, as compared to standard API access, does not have the same limits on how much a program accesses the data, which means much more real-time interaction is possible.

Gerry Campbell, CEO of Collecta, said that these partnerships show that "real-time is real," calling this "the natural progression of the validation of real-time as a mode of information gathering."

Twitter's blog post, titled "Enabling A Rush of Innovation", reads at times like a proclamation from above, but we have to admit we're excited to see what can come of this. "Full investment," reads the post "in this ecosystem of innovation, means all our partners should have access to the same volume of data, regardless of company size. [...] With access to the full Firehose of data, it is possible to move far beyond the Twitter experiences we know today. In fact, we're pretty sure that some amazing innovation is possible. "

Twitter seems to be making a point that even the little guy can get access to the full wealth of the Firehose and if this is true, then real-time interaction is just moments away from the smallest of startups.

But for now, the fact that this wealth of data is suddenly available to a number of real-time search engines can mean something big for the ad platform. As we wrote last Friday, Peter Kafka uncovered some details on the proposed advertising platform, one being that "the ads will only show up in search results."

These partnerships could mean that these advertisements just gained a new real-time audience, outside of the big three. We asked Twitter about this but have yet to receive comment.

Widgets can be created around any search term imaginable and customized in a number of ways. Results are automatically refreshed, just as they are on the Collecta site, and include results from blogs, microblogs, news feeds and photo sharing services.

Still, Facebook statuses encompass only a limited amount and type of information available on the real-time web. Collecta, on the other hand, boasts more than 10 million unique content sources, including Twitter, WordPress, Flickr, The Associated Press, CNN, and Reuters.

Customization options include the ability to choose one's search terms as well as the ability to customize the header, control the scroll rate and link to an external style sheet to integrate the widget with a website's existing look and feel.

You can see the widget in action as it powers the streaming results for MySpace's breaking news page that's focused on Haiti - or just check out the Real-Time Tech widget we've embedded below.

For more on Collecta and the evolving market, check out the ReadWriteWeb Collecta and Real-Time Search archives.

Yesterday the UK government released its new data site, data.gov.uk, to rave reviews (including ours). The new Guardian search engine searches across the UK, US, New Zealand and Australian governments' data sites. The company also offered up a gallery of the 10 best visualizations and mash-ups built on top of government data like this.

Following an era when the quantity of data available online increased in orders of magnitude, thanks largely to easy publishing tools for end-users like blogging and social networks, many people expect the next era of development online to focus on strategic moves to make the most valuable data available in standardized formats that facilitate innovation by 3rd parties independent of the original sources of the data.

If large, standardized data sets are a new language, then it's time for a new period of literature to be written.

What 80Legs does is no easy feat. It provides its users a service which offers up 50,000 computers which can crawl up to 2 billion web pages per day. Yes, it's like having your own little search engine that you can rent for a small fee. How small? 80Legs is about 50% less expensive than any other competitive service out there.

While consumers may not have a use for a service such as this, there is an extensive market that does. 80Legs aims to attract customers from a wide variety of disciplines including alternate search engine developers, market researchers, IP protection services (those who go after copyright infringements, article theft, etc.), competitive intelligent services, and ad networks looking to audit their own ads and see where other ad networks are placing theirs.

80Legs App Store

In addition to the big news about the service's public launch, 80Legs is also revealing their development program. With this new feature, developers can write their own applications that run on top of the 80Legs service and then make them available to others through a soon-to-launch "app store." Here, other customers can browse and purchase apps that suit their needs whether it's something for media analysis, market research, sentiment analysis, or whatever else the developer comes up with. The developers get to set their own CPR price for the apps and get to keep 100% of the revenue earned, too.

The API for app building was actually made available to beta users a couple of months prior to DEMO, but the company plans to soon support multiple languages which will include Java, .NET, Perl, and Python, so developers can work in whichever they feel most comfortable with.

While there are other ways to crawl the web, 80Legs wants to make sure that there's nowhere else you can do it for such a small fee. If you're interested in trying 80Legs for yourself, you can do so as of today by signing up at 80Legs.com. Use the code "RWW" to receive an additional 50% credit on top of the amount you put in. (First 50 users only).

Become a Fan

One feature that makes Tweetmi unique is that it gives users the option to become 'fans' of a certain topic. While this is definitely an interesting concept, users actually have to send out a tweet about the fact that they are now fans of 'RWW' or 'Follow Friday,' which somehwat limits the usefulness of this feature.

Another feature we liked is that the application can show you a list of all the Twitter users who tweeted a popular link. Like most Twitter search engines, Tweemi displays a list of the most popular links about a topic in a sidebar.

Given that there are already numerous Twitter search engines and more comprehensive real-time search engines like OneRiot on the market, Tweetmi will probably have a bit of a struggle to attract a dedicated user base. It is however, a perfectly capable Twitter search engine that offers all the typical features you would expect and definitely worth a try.

"It's the first step in a process that will let us push the envelope on size, indexing speed, accuracy, comprehensiveness and other dimensions," says a joint post from engineers Matt Cutts and Sitaram Iyer. "Web developers and power searchers might notice a few differences, so we're opening up a web developer preview to collect feedback."

In other words, the company aims to crawl the web, index it well, and deliver fast, relevant results - something it's always aimed to do. After a number of searches, it's clear Caffeine offers increased results in a shorter timeframe; however, how results appear remains a mystery.

From a consumer standpoint, Caffeine is identical to regular Google search save for shaving off a few precious half seconds. But for search engine marketers, the new indexing system adds another hurdle to their jobs. While it's likely that Google will have at least a month head start in outrunning them, in the past savvy keyword specialists have always managed to make their mark.

To help improve Caffeine, search in the sandbox and click the "Dissatisfied? Help us improve" link to offer feedback.

Much in the vein of the failed Wikia Search (the abandoned brain child of Wikipedia founder Jimmy Wales), these engines purport to "crowdsource" intelligence about URLs and search terms by allowing users to create profiles and submit, submit, submit content. Stumpedia and Gurutoy are two products in this category. Each offers the excitement of multimedia, semantic, "neue search" capabilities; and each delivers astonishingly dysfunctional results.

Because god knows those algorithms have done nothing for search in the past. As for the "company insiders" part, we're drawing a blank on precisely what that means (Megan McCarthy, was this aimed at you?) and defer to the wisdom of the all-knowing RWW commenters to fill us in.

Stumpedia currently boasts around 28,000 URLs and 75,000 search terms in its digital lexicon - hardly enough to allow for a good or interesting browsing experience. By way of comparison, Wikia Search had indexed about 30 million websites before Jimmy Wales could say with a straight face that the product didn't suck. Just because we know he likes the attention, we ran a search on Robert Scoble:

As you can see, the single returned result was entirely irrelevant to the search term; Scoble's name was nowhere to be found on the linked-to page.

And sadly, for all the talk about insiders not gaming the system, the most relevant results in many searches we tried came from the Stumpedia founder/CEO. Here's a look at his profile and submissions:

We wanted to run a search for irony, but apparently the CEO hasn't submitted anything ironic lately.

Exhibit B: Gurutoy

Assuming that the 99 percent of Internet users who are not tech bloggers use search engines because they need to find accurate, relevant results, the bar of expectations rests rather high.

For example, if a user searches for "orange juice," he might not expect to see this:

As can be seen by mousing over the thumbnails, the two results returned for that search term were both uploaded by a Los Angeles haberdasher. The results were tagged with relevant ("plaid," "headware") as well as damn perplexing ("brad suzuki," Gurutoy's CEO) terms, and we're still not sure how this cap was returned as a result for "orange juice."

Distressingly, a recommended search for "action figures" returned dismally irrelevant results:

Two of the 13 featured results had information on action figures, and none of the images contained action figures.

The Problem with Reliance on UGC

Expecting that users will do the kind of data entry necessary to create a competitive product in this arena is ludicrous. The Internet already has a Wikipedia, so the kind of people with the knowledge and skill sets and the sheer time to invest have likely already picked their hobby and are eyeball-deep in barnstars.

However, Suzuki sees it differently: "The goal of Gurutoy is to become a visual directory of websites (any subject) on the net. But in a cool way, with the pictures." He compares the site to YouTube and has every faith in the power of user-submitted content.

"Gurutoy does not use any spiders to search the web for content. What we're counting on is for the masses to catch on with Gurutoy and to grow the content to make it relevant."

I asked SproutBox cofounder and venture tech/capital expert Mike Trotzke what he thought of algorithm-free social search engines.

"Oh, you mean a purely spam search engine with no users? Yeah, they suck.

"If you are going to try to introduce UGC into search engines, you've got to have some indexing first. It has to have some value out of the gate or no one will care. Not even Jimmy Wales could pull that off."

Trotzke continued to say that if any company would be able to incorporate valuable user-generated information into search, it would be Google. And he doesn't imagine that the search giant would be interested in buying a smaller company for their data or technology.

"[Google has] the vote-up technology already ready in waiting. They just need to tweak and start giving weight to all the data they have been collecting in SearchWiki notes for months already."

The Spam Question

Right now, most of the "users" interested in submitting content to these sites are retailers, enterprise sites, or others with a vested fiscal interest in driving traffic to their URLs. As you can see in this screenshot, MyJewelersPlace.com is spamming the heck out of Stumpedia:

Any site that permits user-submitted links is going to suffer the predictable, lamentable onslaught of black-hat, link-stuffed atrocities, especially for competitive verticals (I personally dare you to search any of these sites for iPods or Viagra.) Especially when adoption rates are low to begin with, UGC search engines are at high risk for being overrun by this kind of spam. This begins a circular process wherein potential users are scared or bored away from the site when search results are irrelevant, desperate pleas for clickthrus and credit card information.

For generic, noncommercial queries, few or no results will be returned. For more consumer-minded searches, results will be skewed and often uninformative. Allowing the community to police itself by flagging suspicious content is a necessary feature for any UCG site. However, when the amount of spam already outnumbers the amount of useful content on a relatively new search platform, what users are going to stick around long enough to register an account, let alone slog through the spam, planting flags left and right.

So, with more apologies to the startups named above, social search still needs to amass and index content using traditional search algorithms if results are to be useful to the end user. Then again, you could just let Google have this one and wait for your next big idea.

Wolfram and his team will chronicle the launch in a live webcast on justin.tv, which will start at 5pm Pacific/8pm Eastern tonight. We are not quite sure how Wolfram will manage the gradual launch over the weekend, though we assume that if you are on the preview waiting list, you will get first dibs.

Some Things to Keep in Mind

Here are a few things to keep in mind as you start experimenting with Alpha tonight or over the weekend:

• Wolfram Alpha is not a general purpose search engine - it does not directly compete with Google and if you treat it like Google, you will inevitably be disappointed.
• Check out the copious examples from the home page - they will give you a good idea of the type of queries that Alpha can handle best.
• Here is one thing we can almost guarantee: you will be disappointed at first (especially if you were expecting a Google killer).

  Alpha is a great tool, but it takes some time to learn its limits and strengths. Unlike Google, some searches simply don't return any results at all.

Using Alpha

Once you get access to Alpha, here are some tips for how to structure your searches and searches you should try:

• If Alpha doesn't give you the results you are looking for, try a different way of phrasing the query - sometimes even capitalization can make a difference!
• Try to search for anything that can be packed into data snippets (height of a mountain, chemical formulas, population stats, stars, planets, etc.) .
• Try combining two searches. Alpha usually does a great job with these kind of queries.
• Feed it some math problems. The fact that Alpha is based on Mathematica really shines through here.
• Do some test searches for food items or drugs.
• Let it solve some word puzzles for you. Just head to the "Words & Linguistics" section for some good examples.
• If you're a sports fan, look up some baseball or football stats: "passing touchdowns Dallas Cowboys, Denver Broncos."

Our Wolfram Alpha resources

Screenshots: See Wolfram Alpha in Action

Our Preview: Wolfram|Alpha: Our First Impressions

Wolfram|Alpha will be an amazing product, but it's quite different from Google and other search engines. Indeed, maybe it is actually wrong to call it a search engine at all (and Wolfram prefers to call itself a "computational knowledge engine"). If you wanted to know what sights to see on your next trip to New York City, for example, Alpha, from what we've seen so far, will not be able to help you.

Our Review: Mixed Emotions: Our First Hands-On Test Of Wolfram|Alpha

At the end of the day, Wolfram Alpha is a tool; and once you take some time to learn its ways, it can become a very powerful tool. While a lot of media outlets have compared Alpha to Google, we think that this is a moot question. Alpha simply doesn't want to be a Google killer and, in its current form, won't take market share away from Google. As we reported in our first look at Alpha a few weeks ago, Alpha will take away some users from Wikipedia (but it's no Wikipedia killer either), as it can give those users quick and easy access to a wide range of data.

For now, we expect Alpha to remain a niche player. It will be a highly valuable tool for a small subset of potential users. Though, hopefully, over time the team will add more and better databases to draw information from so that Alpha will become more useful to a mainstream audience as well.

Videos

Stephen Wolfram's screencast demo of Alpha.

First Public Demo of Wolfram Alpha at the Berkman Center:

Stephen Wolfram and colleagues discuss the launch preparations:

Setting up the Wolfram Alpha data center (one of five W|A data centers):

Other Wolfram Alpha Reviews

Technology Review (compares Alpha to Google):

Generally, I did not use search terms that clearly had no computable answer (and therefore would have stumped Wolfram). But I also didn't throw any softballs in areas close to the heart of its makers: physics, chemistry, engineering, and genomics. On hard-core scientific questions, it gives you tons of symbols and graphics and other information that would be useful to a researcher but obscure to most people. But on many common questions for which there is no obvious data element, you will not get much help. In any event, if its plans hold, you should be able to test it out yourself in two or three weeks.

Search Engine Land (very in-depth look):

Wolfram Alpha's edge may be that it's a unique repository of general knowledge that imitates a search engine (unlike Wikipedia, which has no search engine feel). Of course, the killer combination would be for Wolfram Alpha to be partnered with a major search engine. It's something Wolfram said is being considered, though there are no formal discussions at the moment. The focus is really getting the service opened to the public and seeing how the initial reaction goes.

Telegraph:

How many times have you used to the internet to calculate the answer to a simple mathematical problem, for help with calculus, or for information on the GDP of Gibraltar? If the answer's, "not often", then it's going to be quite some time before Wolfram Alpha crops up as your search engine of choice.

What Will You Ask?

If you are looking forward to the launch of Wolfram Alpha, let us know what questions you want to ask in the comments. We'll try to answer the most interesting questions (try to give us specific queries!) with links to screenshots from the Wolfram Alpha preview in the comments until about 3pm PST today.

Let's get this out of the way quickly: Wolfram Alpha is not (yet?) geared towards mainstream Internet users, who, for the most part, are still better served by Google. Of course, comparing Alpha to Google isn't even fair, but most users will treat it like Google, and will most likely come away sorely disappointed. Instead, Alpha, for now, is going to be a great tool for students, engineers, and academics - and anybody who needs data quickly and knows how to interpret it. It takes some time to learn how to best use Alpha, and it still has its rough patches, but, overall, we have come away quite impressed, though, at times we were also frustrated.

One thing to keep in mind about Alpha is that it will give you data - but it will not supply meaning. Users have to interpret the data themselves.

Results: Great in Some Areas - Very Limited in Others

Sometimes, Alpha's data set can also be uneven. You can, for example, get unemployment data for states, but if you want to drill down to specific cities, Alpha has to pass. Alpha can also answer odd trivia questions like "wingspan B-29 Superfortress" (141' 2.882"), how many pharmacists there are in the U.S. (and their median wage), or how much money "The Wrath of Khan" made at the box office (including, oddly enough, the conversion of those $78.91 million into Japanese yen, British pounds, and euros).

It can also do impressive calculations (though some of our more complex queries timed out), draw a Sierpinski gasket for you, or tell you what a safe heart rate for exercising is when you are 25yrs old. But while it knows who the German president was in 1984, it refused to tell us who the German chancellor was in that year. And the only info about World War I or II we got were basic dates, but at the same time, Alpha can tell us how many people die per minute in Germany today (1.698) and compare that to current birth rates. Alpha can also give you nutritional information, but we weren't able to figure out how to scale this data to different weights.

Some Humor

Sometimes, some humor also shines through in the search results. When you look for "5 kilo," for example the results will give you basic conversions, but Alpha will also tell you that 5 kilo is roughly equivalent to the weight of 2 copies of Stephen Wolfram's A New Kind of Science.

Limits

Clearly, there are holes in Alpha's data set. And most of these holes are in non-technical areas which, in many ways, is understandable as it would be harder to make that kind of information parsable for a system like Wolfram Alpha (though Alpha is great at solving word puzzles and anagrams). Thankfully, Alpha ads link relevant Wikipedia articles to every results page.

Lack of Interactivity

At times, there is also a lack of interactivity that can quickly become frustrating. All the images on Alpha are just static images, for example, which means that you can't zoom in or out of a map. Or, when you search for biographical data, none of the information is linked, so that you can't just click on a person's birthplace to get more information. This means that really drilling down into a subject can be hard as you constantly have to type in new queries.

Capitalization Matters

Alpha can also be extremely finicky. When we typed in 'pdx,' it didn't know what to do with it, but when the query was capitalized it returned both information for Pursuit Dynamics, which uses PDX as its trade symbol, and the option to get info about Portland International Airport, which was the information we were actually looking for. While Google is completely agnostic when it comes to capitalization, Alpha obviously cares (maybe that is also the legacy of a complex tool like Mathematica that is the foundation of Alpha).

Alpha provides new users with an extensive set of sample queries (all of which, of course, work great). To get the most out of Alpha, it really helps to look at those to see how to best formulate your queries.

Alpha For Developers

Alpha is going to have an extensive API for third-party developers. We only had a quick look at the documents that are aimed at developers, but from what we can see, developers will pretty much get full access to Wolfram Alpha's datasets. It should be interesting to see how the developer community manages to mash this data up with other sources to even out some of the areas where Alpha doesn't quite shine yet.

A Great Tool - But Not for Everybody

At the end of the day, Wolfram Alpha is a tool - and once you take some time to learn its ways - it can become a very powerful tool. While a lot of media outlets have compared Alpha to Google, we think that this is a moot question. Alpha simply doesn't want to be a Google killer and, in its current form, won't take market-share away from Google. As we reported in our first look at Alpha a few weeks ago, Alpha will take away some users from Wikipedia (but it's no Wikipedia killer either), as it can give its users quick and easy access to a wide range of data.

Alpha's biggest problem, right now, is interpreting search queries. Too often, a minor change in a query can mean the difference between no result, and finding exactly what you are looking for.

We also hope that Wolfram will find a way to link more of the data and search results together. It is rather frustrating when you find something interesting in your search results, only to have to type yet another query, even though a simple click should suffice.

Great for Engineers - Not For the Mainstream

For now, we expect Alpha to remain a niche player. It will be a highly valuable tool for a small subset of potential users. Though, hopefully, over time the team will add more and better databases to draw information from so that Alpha will become more useful for a mainstream audience as well.

Note: If you would like to see more screenshots of Wolfram Alpha in action, you can find them here.

When we got the demo, Wolfram asked us to refrain from publishing any screenshots. Today, however, the Berkman Center posted a video of the public demo Wolfram gave earlier this week, so we think it's only fair that we share our own screenshots with our readers at this point.

Query #1: internet users in Europe

Query #2: weather oakland

Query #3: oakland

Query #4: uncle's uncle's brother's son

Query #5: water 550C 3 atm

Query #6: integrate x^3 sin^2 x dx

Query #7: bob

Example of a copy and paste dialog:

Embedding Search Results:

Here is the video of the public demo at the Berkman Center. It is a bit blurry, but it does show Wolfram Alpha in action:

And if you really want a look behind the scenes, here is a look behind the scenes of the Wolfram Alpha datacenter:

Whenever you do a search on Duck Duck Go, the service will try to bring up the most 'official' page first, and if the search terms has a Wikipedia page, it will also include a short blurb from Wikipedia, as well as related search terms in a box at the top of the page.

For some topics, Duck Duck Go features special category pages, and it can also recognize calculations, phone numbers, zip codes, ISBN numbers, and product codes, as well as street and IP addresses.

Judging from the results we have seen, it seems like Duck Duck Go actually gets a lot of its information from Wikipedia, though it also uses Yahoo's BOSS service to provide users with standard search results when the service can't find better information on Wikipedia.

Duck Duck Go also does a great job at providing users with options for disambiguation, which also look like they are based on Wikipedia's disambiguation pages. If you search for "Berlin," for example, Duck Duck Go will ask you if you are looking for the German capital, an album from Lou Reed, or a town in Connecticut.

Firefox Toolbar and iPhone App

Duck Duck Go also has a Firefox toolbar, which just came out of beta today, and the company boasts that this toolbar can prevent users from going to over 44 million spam or parked domains (based on a list maintained by the Parked Domains Project).

The company also provides an iPhone app, as well as a number of blog widgets that are not directly related to its core business.

We like the simplicity of the service, and the company's focus on getting users to results quickly by mashing up data from Yahoo and Wikipedia works well for most search terms. In many ways, it actually feels a bit like an automated version of Mahalo. Of course, Duck Duck Go's name might not exactly help it gain mainstream traction, but other search engines before it also had seemingly silly names and they did quite well in the marketplace.

Google, of course, also shows thumbnails for YouTube clips in its search results, but clicking on these will take you to YouTube and won't open the video player right on the page.

Yahoo says that it wants to make it easier for developers to make use of SearchMonkey. SearchMonkey is an extremely powerful tool, but it can also be very hard to use for somebody who doesn't have the technical knowledge required to create a SearchMonkey app. Thanks to this new feature, even novice webmasters will now be able to embed some of the most popular forms of content on Yahoo's search results page.

As we have said before, Yahoo continues to develop new and innovative ways to enhance its search, but so far, this hasn't made too much of a dent in Google's market share. Breaking Google's momentum will be very hard for any player in the search engine market, but if anything, Yahoo is clearly showing that it is not willing to throw in the towel just yet.

Hitwise's latest data also confirms that Google's market share in the search business is continuing to grow at a steady clip (9% year-over-year). Year-over-year, all of Google's larger competitors lost ground, though at least between December and January, both Yahoo and Ask.com saw a very minor increase in their market share.

Longer Search Queries

Year-over-year, using one and two-word search engine queries became slightly less popular, while the number of three-word queries remained flat. Instead, a growing number of users are now opting to use longer queries. Overall, longer search queries have increased ten percent over the last year.

This is an interesting trend, and it could be interpreted in a variety of way. This could mean that a growing number of users is finding less value in the search results they get from relatively unspecific, short queries. It could also indicate that users are becoming more sophisticated in how they structure their queries when they are looking for very specific answers.

Do you have a theory why more users are turning to longer search queries? Feel free to let us know in the comments.