The "wisdom of crowds" is a popular web 2.0 buzzword, popularized by James Surowiecki’s book of the same name. At its most basic, the term means that two heads are better than one, and that still more heads will yield even better results.
The wisdom of crowds is all around us these days. Wikipedia is one of the best known examples of the concept at work. Thousands of Wikipedia users have created an encyclopedia that studies have shown is as accurate as traditional volumes like Britannica. Another well-known project is the Yahoo! Buzz Game, which is a prediction market for "high-tech products, concepts, and trends." Their memetracker market, for example, has predicted the state of the market in line with Alexa data. Note: Read/WriteWeb covered a Yahoo! event about prediction markets here.
Perhaps the best exponent so far of Web 'wisdom of crowds' is Google, which organizes websites based on how they link to each other. Google sees links as votes for the relevance of a page. It is of course more complicated than that, but one can make the argument that Google works by utilizing the wisdom of crowds to determine which websites are the most relevant.
Crowdsourcing can be looked at as an application of the wisdom of crowds concept, in which the knowledge and talents of a group of people is leveraged to create content and solve problems. The official definition from the term’s originator, Jeff Howe, is "the act of a company or institution taking a function once performed by employees and outsourcing it to an undefined (and generally large) network of people in the form of an open call."
Crowdsourcing can be broken down in to three categories:
1. creation (like Wikipedia);
2. prediction (like Yahoo! Buzz); and
3. organization (like Google).
Let’s take a look at those...
Cambrian House
At Cambrian House, people submit ideas for software
products and then vote on which ideas are the best, commenting on changes or improvements they would like to see made. Development of the most popular ideas is then sourced to members of the community, who earn "royalty points" that determines how much each contributor makes. Cambrian House can be looked at as a commercial spin on the classic
site, halfbakery.
Out of over 4,400 ideas submitted, the site has so far launched three products:
1) Prezzle, which wraps online gift certificates to popular retailers in flash
e-cards;
2) Robinhood Fund, at which people post their sob stories and readers vote on who needs the money the most.
That person is then cut a check from an ad-supported fund;
3) Gwabs, a desktop-to-desktop online arcade game where animated characters destroy your actual Windows desktop.
It appears possible that people are actually making money from Cambrian House. Their website has a calculator that estimates that with "good" growth, a person with 100 royalty points (the amount you receive for coming up with an original idea) would make $153,600 over a three year period. Not sure exactly what constitutes "good growth" though...
Although Cambrian House crowdsources the conception and creation of its web products, ideas are subject to editorial review by a core team and actual production is subject to a set of quality guidelines. In the case of conflicting code or design contributions, the community decides which is the best.
CrowdSpirit
CrowdSpirit is a very ambitious project that aims to utilize crowdsourcing to develop and bring to market tangible, sub-$200 electronic devices (think MP3 players, digital cameras, or game controllers). Community members will decide what
the product is, from concept to design to technical specification, by submitting and voting on product and design ideas. Winning ideas will then be funded by members of the
community - and after prototyping and beta testing, the completed products will be delivered to market.
CrowdSpirit has a core team who, along with a select group of community members and distributors, have the final say on any community decision. The idea though is that because the ideas are coming directly from the people who will be using the products - and because the core team will be acting on polling data about those ideas - the resulting products will be very customer-centric. Further, because the research and development of the products will be funded by the community itself, the community will have a significant interest in getting things right.
One potential flaw in the model is that responsibility for support for the resulting products will fall to the community. While online forums have often given me the answer to computer problems, many consumers might balk at the lack of a reliable, trained tech support department (though that actually might be an oxymoron).
Other projects in this category: A Swarm of Angels (creating a movie), AMillionPenguins (creating a novel), We Are Smarter Than Me (creating a business book)
PicksPal
PicksPal is a sporting event prediction site that allows users to vote on amateur and professional sporting events. The site is set up like a game, allowing you to spend points to try and beat the odds makers
- you win or lose points depending on the accuracy of your picks and how ambitious you are. PicksPal awards weekly prizes, like sports tickets and flat screen televisions, to the top performing players. PicksPal says they have over 100,000 users.
Because the site mimics a professional sports book, you can bet on a lot more than just the outcome of the game. For example, you can place bets on things like who will be winning after the first quarter of a basketball game, which player will score the most points, or the final score of the game.
PicksPal sells "genius" picks, based on the picks of their top performing members, to people betting actual money. So far the site is doing pretty well. As I write this, they have a 52% win rate against the spread for their last 25 picks, and they report a 63% win rate overall.
Marketocracy
Marketocracy was launched in 2000 with the goal of finding the best investors and tapping their collective knowledge to create a superior mutual fund. Anyone can sign up for free and run a virtual fund, starting with $1 million. The site has attracted over 60,000 users to date. In November 2001 the company launched the Masters 100 Index, a
real mutual fund based on the virtual investments of its 100 most successful members (as determined by a computer ranking).
The fund, which now has $44 million in assets, has outperformed the S&P 500 Index (generally considered a good gauge of the U.S. equities market) and has an average annual return of 11.4% since its inception. However it has managed just a 2-star rating from Morningstar. That equates to an above average risk and a below average return, according to the Morningstar rating system.
So does that mean Marketocracy is a success or a failure? In this case the wisdom of crowds has not turned out better than the wisdom of a single top investor. The m100 Index isn’t doing anything miraculous, like outperforming Warren Buffet, and it doesn’t stack up well against other, traditional mutual funds in the same category. But, neither have its investors lost their shirts - so it's certainly not a failure. By utilizing the knowledge of thousands of investors, and culling the best data from the crowd, the fund has performed relatively well for 5 years.
Other projects in this category: Hollywood Stock Exchange (predicting the ups and downs of celebrity careers or entertainment properties), Foresight Exchange (predicting the likelihood of future results), Threadless (predicting which designs will sell well on a t-shirt), Yahoo! Suggestion Board (determining which issues are the most important), Dell IdeaStorm (predicting which ideas for new Dell products will sell), The Sim Exchange (predicting how well video games will sell).
digg
digg is not only a poster-child for Web 2.0 success, it is also an excellent example of using the wisdom of crowds to organize and highlight information. digg's concept is simple: users submit links to websites, articles, news stories, photos, or videos, and other users vote on whether they are worth checking out. Depending on how many people, who, and how quickly links are "dugg" they may be promoted to the main page as a worthy link of the day.
digg has proved capable of consistently highlighting links that people find interesting, and drawing attention to news stories in a timely fashion. The large number of digg users and growing traffic (digg overtook Slashdot sometime last year according to Alexa) can attest to that. digg has, however, received criticism over its susceptibility to being "gamed". Wired reported earlier this month that it is relatively easy to purchase votes on digg and get a story promoted to the main page (although it should be noted that Wired owns a competing product, reddit).
Another concern that people have expressed with digg, and one that is more relevant to this article, is the susceptibility to groupthink. As a story starts to become popular, users will often vote for it simply because it is popular, without really thinking whether this is something worth voting for in their own opinion. This is something that happened in the Wired article, in which at least half of the votes for the author's sub par submission were unpaid - and likely made by users who were simply digging something that appeared popular because it appeared popular.
StumbleUpon
StumbleUpon is a browser plugin for Firefox and Internet Explorer that allows users to rate and share web pages. The plugin is very straightforward: users vote thumbs up or thumbs down on pages and are served random pages based on their previous picks, specified topic areas of interest, and the picks of other like-minded users. The idea is that the sites with the most thumbs up votes,
will be the most relevant. Further, users are fed sites that were voted up by others who share a similar vote
history - i.e. the more you use StumbleUpon, the more it learns about what you
like and the better recommendations it gives to everyone.
With over 2 million users, Stumble Upon has amassed an exceptional amount of data on websites, photos, and videos. While the data has so far only been used to enhance their ability to deliver random sites that match users’ interests, there is no reason why it couldn’t be used for other reasons - such as enhancing or augmenting traditional search engine rankings.
Other projects in this category: del.icio.us (organizing the web based on user tags), Techmeme (organizing the day’s buzz based on what people are linking to), Netscape (organizing information by popularity).
I would be remiss not to mention Kathy Sierra’s widely read blog post "The Dumbness of Crowds." In it, Sierra argues that there is more to harnessing collective intelligence than just letting a group of people run wild with an idea. It is true that crowds are susceptible to "mania" - in which a less-than-stellar idea catches on and becomes popular. This is a well-documented and very old concept. One of my favorite books, "Extraordinary Popular Delusions and the Madness of Crowds" by Charles Mackay, was first published in 1841 and documents various manias of the 17th and 18th century in which decidedly bad ideas became all the rage.
But can crowdsourcing be successful at creating products, predicting markets, or organizing data? In a post about crowdsourcing on my personal blog (which inspired this post), I came up with the following rules to help assure success in tapping the wisdom of crowds:
What do you think? We invite you to add your thoughts on crowdsourcing in the comments. If you know of any other websites that utilize the wisdom of crowds, please share them!
Listed below are links to blogs that reference this entry: Crowdsourcing: A Million Heads is Better than One.
TrackBack URL for this entry: http://www.readwriteweb.com/cgi-bin/mt/mt-tb.cgi/2072
Below is what happened in search today, as reported on Search Engine Land and from other places across the web:... Read More
Below is what happened in search today, as reported on Search Engine Land and from other places across Read More
Just came across this very interesting article on “Crowdsourcing: A Million Heads are Better Than One.”¬† The article talks about examples of harnessing human behaviors and interactions into three distinct areas: Creation (Ex. Open Source ... Read More
According to Read/WriteWeb {Crowdsourcing: A Million Heads is Better than One} Marketocracy was launched in 2000 with the goal of finding the best investors and tapping their collective knowledge to create a superior mutual fund. Anyone can sign up for... Read More
We are working on a few co-creation projects with clients. These are my favorite as they let us get in touch with customer or subject-matter enthusiasts. Getting customers involved in creating a product, service or experience leads to greater loyalty Read More
This is one of those days when the blog is about 15 on my to do list, but I'd be unforgivably remiss if I didn't point my readers a post Josh Catone published on crowdsourcing at the Read/Write Web blog Read More
Here is a summary of the week's Web Tech action on Read/WriteWeb, with the results of our poll at the end. Top Web News Much of the discussion in the blogosphere this week revolved around two pieces of big news.... Read More
Tags: 300, DRM, Firefox, Microsoft, RIAA, Web2.0, addiction, alist, amazon, analytics, appletv, apps, beastiality, blog, blogging, blogs, books, bubble, bubble2.0, business, buzz, calcanis, calendar, cambrianhouse, cellphone, code, community, complaini... Read More
I'm on holiday all this week and during this time Josh Catone will be Guest Editor of Read/WriteWeb. You may remember Josh from such posts as Internet Killed The Television Star and Crowdsourcing: A Million Heads is Better than One.... Read More
Comments
Subscribe to comments for this post OR Subscribe to comments for all Read/WriteWeb posts
An excellent companion to The Wisdom of Crowds is Howard Rheingold's 'Smart Mobs' -- focusing more on the mobile revolution. A must-read -- Japan (ok Korea as well) and Finland are so ahead of the rest of the world that his revolutionary stuff from 2002 is still yet to crash on the shores of other countries.
He has a follow-up site: http://www.smartmobs.com.
Posted by: Julian | March 22, 2007 1:11 PMA comment about" Stumble Upon ... , there is no reason why it couldn’t be used for other reasons - such as enhancing or augmenting traditional search engine rankings."
Actually they already do that. In my search results from Google I already get a thumbs up or down and one to 3 stars from StumbleUpon, plus the category and a link to people's reviews, all integrated in the results.
Is not ranking the results, but for sure influences me if the top result from Google has one single star: I would never click on it.
Excellent service, btw.
Posted by: hombrelobo | March 22, 2007 1:19 PMWe're leveraging this idea towards a new distributed model of equity research.
We've developed http://www.whiteboxresearch.com which provides a "netvibes" like platform with modules that gives users a quantitative model to analyze stock prices relative to each other. It also enables users to subscribe to RSS feeds, and post directly to their blog through the site.
The objective of the site is to provide a platform for users to objectively discuss valuation while enabling the flow of information through RSS and blog posts. We'd appreciate your feedback. Thanks.
Posted by: Mark | March 22, 2007 1:24 PMGreat article.
For more on A Swarm of Angels, see my post:
Posted by: steve | March 22, 2007 1:48 PM‘A Swarm of Angels’ crowdsourcing film production
This fine article makes me think of - surprise! - some alt search engines. I would call some of these Prediction \ Recommendation Engines; they predict what books, songs, or movies you might like and recommend them to you: Amazon, Gnod {Music-Map and Live Plasma}, and What to Rent.
Posted by: Charles Knight | March 22, 2007 2:10 PMAs far as Organization or Social Search Engines, there is Sproose, Yoople!, url.com, Ujiko, and VMGO. Finally, I would make my standard pitch for ChaCha: Yahoo! Answers and MSN's Live QnA "let the world answer" with, how shall I put it, "varied results." Here is a recent question on Yahoo! Answers: "My 4 week old puppy just bit me. Should it be put to sleep?" The result of the "answers" is currently a tie. Poor puppy! Google Answers, which charged money for "Expert" Answers, is no more. But when you ask a ChaCha guide a question, you get help finding the answer from a trained guide.
Hello Josh,
thanks a lot for that very interesting article. Let me come back on your comment on CrowdSpirit regarding the support of the products. Hopefully as a former Support employee of HP consumer product, I can tell you how support is important for a brand ;-). So no worry, the real technical 2nd level support will be handled by qualified technician and not by the Crowd. Thanks again for this article.
Posted by: Lionel David | March 22, 2007 2:20 PMBest regards
Lionel DAVID, CrowdSpirit Founder
For crowdsourcing to really work, applications need to prevent information cascades - things that bias the outcome of the vote. A study at Columbia University last year found that, when given a choice of watching music videos for one of two bands, test subjects always chose the video shown to have the most views, even though view numbers for the videos were arbitrarily assigned - sometimes high, sometimes low. The results were a perfect example of an information cascade.
So we really thought this through for crowdrules.com, as we found that decision markets (James Surowiecki's suggested approach to harnessing the wisdom of crowds) and digg-like sites had huge information cascade issues, and couldn't be used for business purposes.
CrowdRules.com lets members ask questions in video form. Like StumbleUpon, CrowdRules selects videos for crowd members to rate. But this is from a pool of videos that are in response to a question posed by a Member. Information cascades - things that bias voting - are blocked. That means members can’t find out how others have voted. They don’t know the number of views, or how others have rated a video. They can’t send a link to game the vote. And all videos get an equal number of votes. When voting ends, results are aggregated into a ranked list.
That's where I think crowdsourcing is going: real-world uses.
Posted by: David | March 22, 2007 2:36 PMYou forgot about Prosper.com, which allows you to crowdsource a loan.
Posted by: Hashim | March 22, 2007 3:19 PMShopify.com successfully had their services internationalized by crowdsourcing the translations and localizations. Not sure to what extent but they got what they wanted. I suppose it depends on the amount of loyalty you have to really make it work on the quality level.
Posted by: baron | March 22, 2007 4:57 PMPower of crowds will definitely rule the web 2.0 arena.
Posted by: nay min thu | March 22, 2007 7:43 PMSorry to inform, but stumbleupon is not really crowdsourcing. It's more like an active listing of sites and you can choose to bookmark it or not, you don't really work together to achieve some ultimate goal like digg - which could be getting it listed on the frontpage.
Posted by: watch free tv online | March 22, 2007 7:43 PMFour more "crowdsourcing" projects:
Sodahead (www.sodahead.com) - startup out of MySpace. The founder says in a recent interview: "Here, when someone votes, someone leaves a comment, or says something--should I pay attention to this person?...Should I go look at some experts, or just the wisdom of the crowds? James Surowiecki always talks about how the masses are right more often that the experts are."
Lijit Networks (www.lijit.com) - browser extensions that use "wisdom of crowds" to determine how safe web sites and software is.
Megabuzz (www.megabuzz.com) - uses "wisdom of crowds" in combination with contests to answer questions and polls. Founders out of Expedia.
Storage Markets (www.storagemarkets.com) - uses "wisdom of crowds" in a prediction marketplace for the storage industry. Based on the Hollywood Stock Exchange.
References:
Interview with Jason Feffer, Sodahead
http://www.socaltech.com/interview_with_jason_feffer,_founder,_sodahead/s-0008088.html
MegaBuzz Launches Social Media Site
http://www.nwinnovation.com/story/0008122.html
Interview with John Ives, Storage Markets
Posted by: Ben | March 22, 2007 8:28 PMhttp://www.techrockies.com/story/0006024.html
Thanks a lot for the interesting post. It appears that someone the whole approach of web2.0 will be fully documented ;)
Posted by: Acronyms | March 22, 2007 8:44 PM@#11: I would argue that StumbleUpon is very similar to digg -- users vote on whether web content is worthwhile, and SU uses the info gleaned from the crowd to push some content over the rest. The main difference being that with StumbleUpon you have less control over what you view next (there is an element of randomness to it). The "ultimate goal" with StumbleUpon is to deliver relevant, interesting links to users via the tool bar (just like the ultimate goal with digg is to deliver relevant, interesting links to the main page).
Posted by: Josh | March 22, 2007 9:32 PMCambrian House actually got rid of the editorial oversight of idea submission.
Good article overall. Crowdsourcing is going to be a big thing over the next number of years.
Posted by: techguy | March 22, 2007 11:03 PMAnyone for a post-modern, feminist or post-colonial reading on this?
Posted by: Rebecka | March 22, 2007 11:55 PMWe've recently launched www.ideawicket.com as an 'Open Innovation Portal' where our goal is to sign up corporations who are interested in sourcing their innovation from the crowds. As the site evolves, users will be able to pitch their innovations to specific corporations.
Like the other sites described in the article we believe that research and development is not something that can be left only to employees. Consumers are likely to chart the course for the development of new and improved goods and services. Ideawicket is based out of New Delhi, India.
Thanks
Amar
Posted by: Amar | March 23, 2007 1:38 AMwww.ideawicket.com
Interesting summary-- very much appreciate the work that went into this. I think the marketocracy example shows best that the wisdom of the crowds doesn't handle problems with high inherent entropy.
Posted by: friend | March 23, 2007 2:43 AMThe wisdom of crowds can be applied to pretty good effect in certain areas but falters on things where there is wide variability in individual tastes, especially where there is a high emotional element. Rating a camera probably lends itself well to the wisdom of crowds; rating a movie does not.
Given enough people, the average camera rating probably makes the cream rise to the top (or at least close enough that people don't notice or don't care). The average movie rating, however, just gives the watered down opinion of the masses and has generally low utility to the individual.
A number of popular movie recommendation sites fall into this trap. They recommend movies based on the wisdom of crowds and the result is obvious - you get a few decent recommendations but overall they aren't very good. A better system is one where you are matched up with individuals with very similar movie tastes to yours and then given recommendations based on that relationship (e.g. MovieFreak).
Posted by: moviefreak | March 23, 2007 6:29 AMI agree with the post about the wisdom of crowds being applied in the wrong places, movie recommendation sites being one of them. I use MovieFreak (I assume you meant moviefreak.org and not moviefreak.com, right?) and it works pretty well. There are a number of other movie recommendation sites but I haven't found their recommendations to be that great, mainly because I think the matching is too generic.
Posted by: jadams | March 23, 2007 6:56 AManother site that I have developed is a Social search engine, where users can vote for a particular result and also add comments for others to view.Can be viewed here: SansJ.
Posted by: trk7 | March 23, 2007 7:06 AMAn interesting article that touches several points of interest to me. Congratulations!
""If you know of any other websites that utilize the wisdom of crowds, please share them!""
I am the founder and main editor of
http://trendirama.com
which somehow is a good example of the wisdom of crowds you mention.
Just some more comments for debate:
1-""studies have shown is as accurate as traditional volumes like Britannica.""
This is, in IMHO, a myth.
If people really believe this, they should have a look at this:
http://niquel757.blogspot.com/2007/03/quality-of-wikipedia-entries.html#links
What's the use of information that can't be verified? Are (some) wikipedia leaning too much on the entertainment side? Where is the important information in the article cited in that post? How much of it is there?
2- The "democracy" in some web 2.0 projects, is a myth. This was exposed with examples here
http://trendirama.com/articles/internet/future-and-collaboration-changing-the-world-through-peer-power.html
a "democratic" project will pull in so many different directions that it'll be very hard to take it to a short-term profitable completion. When you do it for fun or to rub each other's ego (e.g. discussing silly things as the title of a common project/wikipedia article/whatever) that's fine, but when you do it to stay in business and increase market share quickly, you can't follow that model 100%. You need a strong lead. You need editing of articles (e.g. Trendirama.com) You need editing of people's posts (e.g. Digg vs old Netscape), you need what some will call "censorship" (it has happened to me at Trendirama, when some author felt offended by our editing!)
I think that yes, we are in the middle of a very interesting revolution where a lot of untapped common knowledge is unleashed for eveyrone's good.
However, I am also of the opinion that a lot of these companies are lying to their users by pretending they are 100% democratic, when they are not.
Trendirama.com has many similarities with these projects, but as stated in the FAQs, we are not an exercise in democracy.
In our example (we write articles about The Future), I can't see today a way to provide great quality articles to people without proper editing and polishing. I can't see a way to deliver on your promises of building a profitable business where everyone benefits if you let everyone have an equal voice (by the time we all agree, the opportunity vanished a long time ago) And if you say you do give an equal voice, you are lying, or you are destined to business failure sooner rather than later...
I think a lot of people should understand this if we all want to find the best compromise between unleashing all the common talent dormant in people, but building strong and profitable business at the same time to share their profits with them.
In any case, thank you for your post!
Please accept my invitation to write an article about The Future in Trendirama!
Regards
Posted by: Javier Marti | March 23, 2007 7:07 AMJavier
Thanks for an excellent summary, Josh. I've added a link to this from my site.
Posted by: Alan | March 23, 2007 10:43 AMBut what does the crowd think of readwriteweb? Check the dex: http://alexadex.com/ad/url/readwriteweb.com
Posted by: Colin Saunders | March 23, 2007 3:05 PMCheck out Dell's IdeaStorm. It's an excellent example of crowdsourcing and Dell is assessing and implementing submitted ideas. (full disclosure: Dell is one of my clients and I assisted them with the development of this program)
Posted by: Paul Walker | March 23, 2007 7:28 PMI've seen so many superfluous reviews of crowdsourcing companies. What I'd like to see is a more in depth analysis of these companies. What kind of infrastructure exists and/or is required to manage products from concept thru to production and delivery? Are the products being generated truly using crowdsourcing or is the majority of the product being developed in-house by employees?
Posted by: Will | March 23, 2007 8:08 PMMy issue with the concept is innovation. The wisdom of crowds works with the lowest common denominator among participants. Any deviant ideas is diluted to fit the law of average. While it may be effective to correct entries in Wikipedia (according to popular knowledge of course), it is useless at thinking outside the crowd box. All the examples used by such sites (we had this problem for years, couldn't solve it, went to "insert website name here" and it was cracked) have nothing to do with crowd sourcing. The website value was to connect a problem with an individual who could solve it. Where is the crowd here? As the adage goes: no statues was ever erected to honor a committee.
Posted by: Joel Cere | March 24, 2007 8:52 AMI love Web 2.0 features and I love the Wikipedia. But please stop misreporting that the Nature study found it "as accurate as" Britannica. It did not.
The Nature review found 162 errors in Wikipedia and 123 in Britannica. That is not "as accurate as." In fact, Wikipedia is only 75% as accurate as Britannica.
This is a quote from Nature: "the average science entry in Wikipedia contained around four inaccuracies; Britannica, about three." (http://www.nature.com/nature/journal/v438/n7070/full/438900a.html)
So if Wikipedia made 1000 errors, Britannica would make only 750. That's a huge difference.
Posted by: Howard | March 24, 2007 3:43 PMYou forgot to mention SitePoint's Design Contests ;-)
Posted by: Matt Mickiewicz | March 27, 2007 12:31 PMThere is also an emergent process called crowdcasting (full disclosure: we coined the term).
Crowdcasting-Business 2.0
Posted by: Anil Rathi | March 29, 2007 8:58 PMHavent seen anything about people. So here we are www.hypesphere.com sign up for the alpha and be one of the first to try out social people ranking with a wiki twist!
So I would like to invite you all to Hypesphere. Where you can find whos the hype in different social spheres.
Power to the People!!!
Posted by: Rizal | April 8, 2007 10:39 AM/Rizal
I would say that wikipedia and digg are horrible examples of wisdom of crowds. If the author of this post had actually read the book he would understand that for a crowd to be wise they can't know what everyone else in the crowd is doing. Sites like digg and wikipedia promote group-think and consensus.
Posted by: Motorcycle Guy | April 21, 2007 8:16 PM