The wisdom of crowds is all around us these days. Wikipedia is one of the best known examples of the concept at work. Thousands of Wikipedia users have created an encyclopedia that studies have shown is as accurate as traditional volumes like Britannica. Another well-known project is the Yahoo! Buzz Game, which is a prediction market for "high-tech products, concepts, and trends." Their memetracker market, for example, has predicted the state of the market in line with Alexa data. Note: Read/WriteWeb covered a Yahoo! event about prediction markets here.
Perhaps the best exponent so far of Web 'wisdom of crowds' is Google, which organizes websites based on how they link to each other. Google sees links as votes for the relevance of a page. It is of course more complicated than that, but one can make the argument that Google works by utilizing the wisdom of crowds to determine which websites are the most relevant.
]]> Crowdsourcing can be looked at as an application of the wisdom of crowds concept, in which the knowledge and talents of a group of people is leveraged to create content and solve problems. The official definition from the term’s originator, Jeff Howe, is "the act of a company or institution taking a function once performed by employees and outsourcing it to an undefined (and generally large) network of people in the form of an open call."Crowdsourcing can be broken down in to three categories:
1. creation (like Wikipedia);
2. prediction (like Yahoo! Buzz); and
3. organization (like Google).
Let’s take a look at those...
Creation
Cambrian House
At Cambrian House, people submit ideas for software
products and then vote on which ideas are the best, commenting on changes or improvements they would like to see made. Development of the most popular ideas is then sourced to members of the community, who earn "royalty points" that determines how much each contributor makes. Cambrian House can be looked at as a commercial spin on the classic
site, halfbakery.
Out of over 4,400 ideas submitted, the site has so far launched three products:
1) Prezzle, which wraps online gift certificates to popular retailers in flash
e-cards;
2) Robinhood Fund, at which people post their sob stories and readers vote on who needs the money the most.
That person is then cut a check from an ad-supported fund;
3) Gwabs, a desktop-to-desktop online arcade game where animated characters destroy your actual Windows desktop.
It appears possible that people are actually making money from Cambrian House. Their website has a calculator that estimates that with "good" growth, a person with 100 royalty points (the amount you receive for coming up with an original idea) would make $153,600 over a three year period. Not sure exactly what constitutes "good growth" though...
Although Cambrian House crowdsources the conception and creation of its web products, ideas are subject to editorial review by a core team and actual production is subject to a set of quality guidelines. In the case of conflicting code or design contributions, the community decides which is the best.
CrowdSpirit
CrowdSpirit is a very ambitious project that aims to utilize crowdsourcing to develop and bring to market tangible, sub-$200 electronic devices (think MP3 players, digital cameras, or game controllers). Community members will decide what
the product is, from concept to design to technical specification, by submitting and voting on product and design ideas. Winning ideas will then be funded by members of the
community - and after prototyping and beta testing, the completed products will be delivered to market.
CrowdSpirit has a core team who, along with a select group of community members and distributors, have the final say on any community decision. The idea though is that because the ideas are coming directly from the people who will be using the products - and because the core team will be acting on polling data about those ideas - the resulting products will be very customer-centric. Further, because the research and development of the products will be funded by the community itself, the community will have a significant interest in getting things right.
One potential flaw in the model is that responsibility for support for the resulting products will fall to the community. While online forums have often given me the answer to computer problems, many consumers might balk at the lack of a reliable, trained tech support department (though that actually might be an oxymoron).
Other projects in this category: A Swarm of Angels (creating a movie), AMillionPenguins (creating a novel), We Are Smarter Than Me (creating a business book)
Prediction
PicksPal
PicksPal is a sporting event prediction site that allows users to vote on amateur and professional sporting events. The site is set up like a game, allowing you to spend points to try and beat the odds makers
- you win or lose points depending on the accuracy of your picks and how ambitious you are. PicksPal awards weekly prizes, like sports tickets and flat screen televisions, to the top performing players. PicksPal says they have over 100,000 users.
Because the site mimics a professional sports book, you can bet on a lot more than just the outcome of the game. For example, you can place bets on things like who will be winning after the first quarter of a basketball game, which player will score the most points, or the final score of the game.
PicksPal sells "genius" picks, based on the picks of their top performing members, to people betting actual money. So far the site is doing pretty well. As I write this, they have a 52% win rate against the spread for their last 25 picks, and they report a 63% win rate overall.
Marketocracy
Marketocracy was launched in 2000 with the goal of finding the best investors and tapping their collective knowledge to create a superior mutual fund. Anyone can sign up for free and run a virtual fund, starting with $1 million. The site has attracted over 60,000 users to date. In November 2001 the company launched the Masters 100 Index, a
real mutual fund based on the virtual investments of its 100 most successful members (as determined by a computer ranking).
The fund, which now has $44 million in assets, has outperformed the S&P 500 Index (generally considered a good gauge of the U.S. equities market) and has an average annual return of 11.4% since its inception. However it has managed just a 2-star rating from Morningstar. That equates to an above average risk and a below average return, according to the Morningstar rating system.
So does that mean Marketocracy is a success or a failure? In this case the wisdom of crowds has not turned out better than the wisdom of a single top investor. The m100 Index isn’t doing anything miraculous, like outperforming Warren Buffet, and it doesn’t stack up well against other, traditional mutual funds in the same category. But, neither have its investors lost their shirts - so it's certainly not a failure. By utilizing the knowledge of thousands of investors, and culling the best data from the crowd, the fund has performed relatively well for 5 years.
Other projects in this category: Hollywood Stock Exchange (predicting the ups and downs of celebrity careers or entertainment properties), Foresight Exchange (predicting the likelihood of future results), Threadless (predicting which designs will sell well on a t-shirt), Yahoo! Suggestion Board (determining which issues are the most important), Dell IdeaStorm (predicting which ideas for new Dell products will sell), The Sim Exchange (predicting how well video games will sell).
Organization
digg
digg is not only a poster-child for Web 2.0 success, it is also an excellent example of using the wisdom of crowds to organize and highlight information. digg's concept is simple: users submit links to websites, articles, news stories, photos, or videos, and other users vote on whether they are worth checking out. Depending on how many people, who, and how quickly links are "dugg" they may be promoted to the main page as a worthy link of the day.
digg has proved capable of consistently highlighting links that people find interesting, and drawing attention to news stories in a timely fashion. The large number of digg users and growing traffic (digg overtook Slashdot sometime last year according to Alexa) can attest to that. digg has, however, received criticism over its susceptibility to being "gamed". Wired reported earlier this month that it is relatively easy to purchase votes on digg and get a story promoted to the main page (although it should be noted that Wired owns a competing product, reddit).
Another concern that people have expressed with digg, and one that is more relevant to this article, is the susceptibility to groupthink. As a story starts to become popular, users will often vote for it simply because it is popular, without really thinking whether this is something worth voting for in their own opinion. This is something that happened in the Wired article, in which at least half of the votes for the author's sub par submission were unpaid - and likely made by users who were simply digging something that appeared popular because it appeared popular.
StumbleUpon
StumbleUpon is a browser plugin for Firefox and Internet Explorer that allows users to rate and share web pages. The plugin is very straightforward: users vote thumbs up or thumbs down on pages and are served random pages based on their previous picks, specified topic areas of interest, and the picks of other like-minded users. The idea is that the sites with the most thumbs up votes,
will be the most relevant. Further, users are fed sites that were voted up by others who share a similar vote
history - i.e. the more you use StumbleUpon, the more it learns about what you
like and the better recommendations it gives to everyone.
With over 2 million users, Stumble Upon has amassed an exceptional amount of data on websites, photos, and videos. While the data has so far only been used to enhance their ability to deliver random sites that match users’ interests, there is no reason why it couldn’t be used for other reasons - such as enhancing or augmenting traditional search engine rankings.
Other projects in this category: del.icio.us (organizing the web based on user tags), Techmeme (organizing the day’s buzz based on what people are linking to), Netscape (organizing information by popularity).
Conclusion
I would be remiss not to mention Kathy Sierra’s widely read blog post "The Dumbness of Crowds." In it, Sierra argues that there is more to harnessing collective intelligence than just letting a group of people run wild with an idea. It is true that crowds are susceptible to "mania" - in which a less-than-stellar idea catches on and becomes popular. This is a well-documented and very old concept. One of my favorite books, "Extraordinary Popular Delusions and the Madness of Crowds" by Charles Mackay, was first published in 1841 and documents various manias of the 17th and 18th century in which decidedly bad ideas became all the rage.
But can crowdsourcing be successful at creating products, predicting markets, or organizing data? In a post about crowdsourcing on my personal blog (which inspired this post), I came up with the following rules to help assure success in tapping the wisdom of crowds:
- Crowds should operate within constraints. To harness the collective intelligence of crowds, there need to be rules in place to maintain order.
- Not everything can be democratic. Sometimes a decision needs to be made, and having a core team (or single person) make the ultimate decision can provide the guidance necessary to get things done and prevent crazy ideas and groupthink from wreaking havoc on your product.
- Crowds must retain their individuality. Encourage your group to disagree, and try not to let any members of the group disproportionately influence the rest.
- Crowds are better at vetting content than creating it. It is important to note that in most of the above projects, the group merely votes on the final product; they do not actually create it (even at Cambrian House, where the group collaborates to create the product, individuals are still creating each piece on their own and the group votes on whose implementation of an idea is best).
What do you think? We invite you to add your thoughts on crowdsourcing in the comments. If you know of any other websites that utilize the wisdom of crowds, please share them!
]]>