Comments for CERN Officially Unveils Its Grid: 100,000 Processors, 15 Petabytes a Year

CERN Officially Unveils Its Grid: 100,000 Processors, 15 Petabytes a Year

2008-10-03T18:28:56Z

CERN today officially unveiled the massive computer network that will crunch the enormous amount of data coming from CERN's Large Hadron Collider (LHC). CERN expects that the LHC will produce around 15 petabytes of data every year. While the LHC was in its planning stages, CERN's IT department decided that the only realistic way to handle this amount of data would be by relying on the then still novel idea of grid computing. CERN's grid consists of 100,000 processors at 140 scientific institutions in 33 countries.

]]> How to Crunch 15 Petabytes of Data?

As Science reported last month (subscription required), CERN's IT department quickly realized that no known data center could handle the amount of information the LHC would create. It was not even clear that Geneva's power grid could supply the energy necessary to run this massive data center. In addition, most of the money for the LHC project was going toward the collider itself, so that very little funding was left for the actual computing resources.

In order to distribute this data, CERN relies on dedicated 10Gbit/s fiber-optic lines that connect CERN with the 11 Tier-1 data centers on the grid. The Tier-1 data centers (pdf) will do some processing, but will also function as the main archives for the LHC data. These Tier-1 centers then farm out a large part of the actual data crunching to the Tier-2 data centers spread around the world. The Tier-2 centers are connected to the grid via regular, public Internet connections.

Large Hadron Collider @ Home

While grid computing has been around for quite a while now and has been implemented successfully on the public Internet by projects like SETI@home or Folding@home, CERN's grid is most likely the largest and most powerful grid established for scientific research so far.

CERN has also set up a project similar to Folding@home called (somewhat unimaginatively) LHC@home, which, thanks to the current shut-down of the LHC does not have much to do right now, but will allow individuals to contribute to CERN's efforts by donating computing time on their own computers.

Image of CERN Computer Center used courtesy of CERN.

Comment from Jason Kiesel on 2008-10-03

2008-10-03T22:16:49Z

When can I get one of these for my home office? Dang...

Jason Kiesel
Founder & CEO
http://www.freedomspeaks.com

Comment from Jeff Barr on 2008-10-03

2008-10-03T22:30:20Z

Jason -- why buy when you can rent? Click on my name for more info :-)

Comment from Christian Decker on 2008-10-05

2008-10-05T16:51:54Z

I've actually seen the two computer rooms while they were being filled up with equipment. They are all standard PCs, which are booted using a LiveCD that runs a Linux Kernel and then connects to the rest of the grid to do the processing :-)
Also the huge tape silos are quite impressive

Comment from michael.chelen.myopenid.com on 2008-10-06

2008-10-07T04:52:11Z

@Christian - That seems well considered, as often consumer equipment provides excellent performance value. The choice to use CDs seems a bit odd, since network boots could offer more flexibility, but it is difficult to speculate without knowing more details.
Is there documentation for the methods that were used? It would be great to see their procedures available to researchers around the world who face similar challenges, albeit usually on a smaller scale ;)