Researchers at the University of California at Santa Cruz have built a test extension called WikiTrust that evaluates every word on Wikipedia and highlights text that was added by authors with a less than stellar reputation.
The research is only live on a cached copy of Wikipedia right now, but it targets what's possibly the biggest need Wikipedia has as it tries to grow into greater mainstream use - trust in an open system.
Reputation in this case is determined by an author's history of past contributions - getting your additions to the site quickly reverted away hurts your reputation in the system and casts doubt on what you add to other articles in the future. Suspect text is color coded in different shades of orange and clicking on orange text will load the version of a page where that text was added. We think this is a really exciting idea.
First blogged about on the fantastic group blog SmartMobs, WikiTrust has the potential to be applied to any wiki anywhere. The researchers building it say they are now working on a version that will process the live content of Wikipedia in real time.
For now you can navigate around the WikiTrust dump of Wikipedia, including using the search box. It's very fun to use. The entry for file format RSS, for example, is written by solid authors with relatively little history of contested text. The entry for "God", on the other hand, is almost entirely written by authors whose reputations are highly suspect! That's the case with the version of the God article saved in WikiTrust at least.
We think that this system for determining author reputation seems like a good one - do readers agree?
Comments
Subscribe to comments for this post OR Subscribe to comments for all Read/WriteWeb posts
Based on the WikiTrust logic, if a person contributes to a controversial subject, they will get a lower reputation level. (Indeed, it might create an incentive that, in order to preserve your reputation, you stick to non-controversial topics - which is a counter-productive outcome, IMHO.) Or, if you happen to have someone with a strong bias on the topic to which you contribute, you'll also suffer. OTOH, if you contribute to topics that few people care about, your reputation will grow.
I don't see how any of those outcomes will be positive.
Posted by: Terry Steichen | June 27, 2008 11:45 AM
Terry, good points. Not much way for reputations to grow here, however. It's all negative points. If you are someone who regularly contributes to controversial topics, then your contributions elsewhere may well be marked as such. That might not be such a bad thing, I'm not sure. It would be nice, however, if getting your additions reverted by the same person counted against you less than getting them reverted by multiple different people. I'd almost assume this is the case. Thanks for sharing your thoughts!
Posted by: Marshall Kirkpatrick
|
June 27, 2008 11:55 AM
I think this is a great idea.
The rise of the internet as a legitimate source of data has always been binded to the "What if this site/info isn't credible? In a medium in which anyone can say anything and pronounce it as fact, who is writing this stuff?" issue.
I've reached the ripe old age of 20, and I grew up *never* being allowed to use the internet for the primary sources of my papers.
To this day, I use Wikipedia to find out info on new subjects for my own personal use, but I tend to credit it as about one rung higher on the credibility scale than Urban Dictionary. It's something I do for a "quick fix" on a topic I'm unfamiliar with, then I move forward to a more reliable source if I want to learn more.
Apparently, this mentality is increasingly being viewed as the internet's answer to the wooly mammoth: weird, hairy, and needing to be put on ice.
And, simultaneously, if not consequently, the decline in quality of student reports is becoming a serious issue
(http://www.readwriteweb.com/archives/our_kids_are_failing_-_and_its_wikipedias_fault.php#58579)
I've had multiple peers at my uberranked U.S. News & World Report college admit to writing 5-page English papers using ONLY Wikipedia, allegedly without even touching the book.
So, Wikipedia is viewed as a legitimate resource by young students and by the slackers among us. But, it's also viewed as legitimate by many high-achievers and adults.
Any way you slice it, Wikipedia is used by almost everyone at one time or another. Whether you're researching for school, work, or just for fun, it never hurts to know who you can trust.
My only issue would be simply that: the way in which "who you can trust" is determined. Some subjects (ie. G-d, abortion, guns) will always be controversial, and those entries are in a perpetual state of evolution. If I post my feelings about G-d in a respectful manner and someone who disagrees edits them, does that make me unreliable or extreme? To deem someone untrustworthy simply because someone disagrees their opinion and deletes their comments implicitly assumes that the deleters *are*.
Posted by: Lauren | June 27, 2008 12:03 PM
Sorry for the repeat. Terry hadn't yet commented when I embarked upon my evident Epic Quest for the longest post EVER.
Posted by: Lauren | June 27, 2008 12:06 PM
While I don't think the WikiTrust logic works well (for reasons given in my earlier post), I've been playing with some related ideas in the design of my own (still beta) system, which supports challenges (moderated deletes). One idea is to award 'points' based on how many successful changes a user makes, and deduct points on how many times his/her work is removed after a challenge. If a person unsucessfully challenges posts, they also lose points.
Another is that if you have a topic with lots of activity and a persons post remains relatively intact for a decent period, award them with extra points (compared to a posting on a less active topic).
Also, my system is based on a hierarchical arrangement of content, so a contribution 'higher up the chain' would justify more points than another lower-level posting.
Just a few ideas, for what they're worth.
Posted by: Terry Steichen | June 27, 2008 12:26 PM
Correction in my 12:26pm post above: it should read, in part, "based on how many successful challenges a user makes" instead of "based on how many successful changes a user makes"
Posted by: Terry Steichen | June 27, 2008 12:29 PM
Really interesting, both the post and the comments. Maybe there's some way to tag certain topics as "controversial" - perhaps if things are getting changed more than a certain threshold the article becomes marked as controversial within the system, meaning that having your words changed hurts you less than it would in a non-controversial article. That's probably a really inelegant way to do it, just what came to the top of my head, but something that meant that you got dinged fewer points in those articles. And Marshall, I definitely agree that if the same person keeps changing your words, that should hurt you less - no personal Wiki-vendettas!
Posted by: Emily Williams | June 27, 2008 1:28 PM
Q.
http://wiki-trust.cse.ucsc.edu/index.php/Windows_Vista
vs
http://wiki-trust.cse.ucsc.edu/index.php/Linux
A. ?
Posted by: Stephen | July 8, 2008 9:56 AM