Earlier this morning the Internet was buzzing with news first reported by The New York Times Bits blog: Doug Cutting was leaving Yahoo. In specific, he was leaving Yahoo for Cloudera, a small startup.
Cutting was one of Yahoo's best and brightest, especially in the area of search and software infrastructure. He got to work on the largest installation of his wildly successful software Hadoop, an open source solution for dealing with huge data sets. Both Yahoo and Cloudera use Hadoop, so the work is the same on the most basic level. Why would he leave to join a 20-person team at a young company?
]]> Blame Bing?The gut reaction of many, including Ashlee Vance in her his Bits post, was to speculate about the effect that the Microsoft-Yahoo search deal had. With Bing now in charge and Yahoo search gutted, was it the reason behind Cutting's departure?
In a phone conversation, Cloudera CEO Mike Olson staunchly denied any connection with the deal. He stated that the discussions with Yahoo about Cuttings move predated the search deal announcement.
While Yahoo representatives were unavailable to confirm the claim, Yahoo will definitely continue to do much of the work with Hadoop that was Cutting's job, even if Bing is now front-and-center. In other words, the deal doesn't affect him that drastically.
But if it wasn't the Microsoft-Yahoo search deal that caused Cutting to depart, then what was it?
The Question of Scale
Both Yahoo and Cloudera use Hadoop for mission critical work. The latter built an entire business around it, and for Yahoo it's become key to their operations. The difference is in scale.
In his blog post explaining the move, Cutting specifically states that he joined Yahoo in order to get the resources needed to scale the system so it could process the full Web.
Today, Yahoo uses what it claims is the largest implementation of Hadoop in the world, and a lot of work that goes into it revolves around the scaling problems that so many clusters bring. Cloudera, on the other hand, is a relatively young startup working with a greater variety of clients. "Going forward, Cloudera presents an opportunity to work with a wider range of Hadoop users," Cutting said.
A Proving Ground for Hadoop
Working with Cloudera might mean working on a smaller scale, but it presents the opportunity just about every passionate developer wants: an exciting (and potentially profitable) proving ground for their work. And while Cutting lauded his peers at Yahoo, he's moving on to work with ex-Google and Facebook engineers, some of the best minds in the business.
That kind of proving ground is the appeal that a startup like Cloudera has for a developer like Cutting, and it's what serial entrepreneurs get addicted to.
Photo by Tim Bray, via Wikipedia
]]>