As part of their ongoing quest to create an open, people-powered search platform, Wikia has acquired Grub, a ditributed web crawling technology, from Looksmart, Wikia co-founder Jimmy Wales announced this morning during his keynote address at the at the O’Reilly Open Source Convention in Portland, Oregon. The technology will be added to their Search Wikia Labs project in an effort to create an "open and transparent" search engine that combines a "distributed user-contributed processing network, with the power of a wiki to form social consensus," according to a press release
Grub, which works something like the SETI@Home project by using spare processor cycles donated by users to crawl the web, will become an open source technology located at Grub.org. The source code will be released "as soon as possible." Emre Sokullu wrote about building an open source distributed Google clone back in May, and envisioned a project that seems similar to Wales' goal.
"We’ve had a tremendous response from very interesting commercial players in the search space," said Wales. "The desire to collaborate and support a transparent and open platform for search is clearly deeply exciting to both open source and businesses. Look for other exciting announcements in the coming months as we collectively work to free the judgment of information from invisible rules inside an algorithmic black box."
Michael Grubb, Senior Vice President, Technology, and Chief Technology Officer at LookSmart -- name just a coincidence as far as I know, said in a release that Looksmart was "pleased to collaborate with Wikia and believe that Grub will thrive under an open source license."
The financial terms of the acquisition were not disclosed.
Comments
Subscribe to comments for this post OR Subscribe to comments for all Read/WriteWeb posts
Reminds http://www.readwriteweb.com/archives/how_to_build_an_open_source_google.php
Posted by: Emre Sokullu | July 27, 2007 10:41 AMwww.faroo.com goes even a step further: distributed crawling and distributed search.
Posted by: wolf | July 27, 2007 10:44 AM@Emre: Right you are! I added a link to the post.
@Wolf: We'll have to take a closer look at Faroo. Sounds interesting.
Posted by: Josh Catone | July 27, 2007 11:13 AM"with the power of a wiki to form social consensus"
It will be interesting to see if how social consensus helps with the long tail.
Posted by: James | July 27, 2007 1:09 PMIf anyone knows how much they pay for Grub ?
Posted by: Hostimal | July 27, 2007 1:57 PMIs grub.org working for anyone?
Posted by: Eric | July 28, 2007 12:04 PM@Eric - yes, it's working.
Posted by: Matt | July 28, 2007 4:27 PM