craigslists gets heavy with Oodle
Uber classifieds site craiglist has requested that Oodle, a classifieds 'meta' search engine, refrain from scraping its content. This has the potential to be the first high-profile case of a mash-up site being slapped for taking another site's content.
In a recent ZDNet post, I wrote that the business models for Web 2.0 mash-ups are beginning to ramp up. Some of the revenue possibilities for sites like Oodle are advertising, lead generation and/or affiliates, transactional, subscription.
Oodle wrote on their blog that they send craigslist "free traffic" and they "don't compete with them by taking listings." John Battelle said that craigslist's response to Oodle "feels counter to the vibe craigslist has always had".
However Jason Calacanis summed up the counter argument with this response in Battelle's comments:
"Fair use is one thing, wholesale scraping/syndication is another. Oodle, Indeed, etc. should a) get permission and b) consider paying Craig a licensing fee for his content."
For more context, this is how the Oodle FAQ describes what Oodle does:
"How do I place a classified listing on Oodle?
Oodle doesn't directly accept classified listings. As a search engine for local listings, we regularly scan hundreds of online sources for classifieds like local newspapers and eBay. To have your listing show up in Oodle, just place your listing with a local classified provider and we'll find it."
It's a difficult issue and I don't have an easy answer with which to finish my post. On the one hand, craigslist is the source of some of Oodle's data and so craigslist has a right to protect that from mis-use. On the other hand, Oodle is clearly providing value to users on the UI side - which is one of the things Web 2.0 and mash-ups is about. It would be best for all concerned if craigslist and Oodle made an agreement with each other, for the benefit of all users. And it would be fair for Oodle to share some of their revenue with craigslist. This is all easier said than done, of course. It's an interesting test case.
Share3 TrackBacks
TrackBack URL for this entry: http://www.readwriteweb.com/cgi-bin/mt/mt-tb.cgi/2530
Comments
Subscribe to comments for this post OR Subscribe to comments for all ReadWriteWeb posts
I don't see much difference between Oodle, Indeed and Google. They all spider sites and show snippets to the user who can then click the link to the original source.
If you have a memory that goes back to Web 0.1, you might recall that the original Gopher-based "Online Career Center" (later acquired by Monster) got a lot of its original content by reading newsfeeds of the Usenet newsgroup misc.jobs.offered and by posting job listings to that group. So resyndication of jobs is not by any means new.
Wow, edward, thanks for the memory trip... them's were simple times.
Perhaps the CraigsList gang could invest some time and energy into developing an API and possibly license its content via the Creative Commons?
Just spit balling here.
I side with CraigsList on this issue, as wholesale screenscraping can be very expensive. Oodle should at least ask permission. It's just good etiquette.
>This has the potential to be the first high-profile case of a mash-up site being slapped for taking another site's content
You must be new here. I remember lawsuits against "mash-up" sites from when I was in college and we used quaint names for them like "screen scrapers". Here's a link from the history books, http://www.wired.com/news/politics/0,1283,37643,00.html .
Hate to be defensive but just wanted to through in a couple of quick comments...
1. Oodle respects anyone's right not to participate. As such, we obey the search engine etiquette robots.txt. If search engines needed to pro-actively ask each site for permission, there would be no billion page indexes.
2. I don't know why the term screen scraper bothers me... Is Google considered a "screen scraper"? Seems that term is typically reserved for services that take the content from another site and present it in its entirety (usually without attribution). In this case, Oodle follows search engine etiquette: presenting a short summary (similar to what Google produces) and linking that summary directly to the listing page.
3. With respect to that wired article, it's a bit ironic that eBay now provides an API so that sites like Oodle can easily index their content and point users to their site. And are willing to pay sites to do so.
Maybe Craig just doesn't like Craig. Or Oodle Craig has some dirty coded scrapers that use brute force on CL servers.
He should have just asked CL to include listings. Vertical Content search is by no means anything like algorithmic search. Anyone can whip up a per site scraper.
I don't think this makes an Oodle of a differnce. CL is popular but many don't even know of the site.