CERN BOFH needs a bigger storage array

Networking the secrets of the universe

By John Leyden • Get more from this author

Posted in IT Director, 1st October 2007 13:45 GMT

Webcast: Building Applications for the 21st Century

Preparations are well underway at CERN to commission the world's largest particle accelerator. Advances in networking technology have allowed the particle physics lab to bring in scientists from around the world to analyse the data is will generate.

When activated in May 2008 it's hoped that the Large Hadron Collider (LHC) will uncover evidence of the elusive Higgs boson particle, theorised as a fundamental building block for matter.

Observing signs of the particle would mark a significant milestone in formulating a Grand Unified Theory that explains the four fundamental forces in nature: electromagnetism, the strong nuclear force, the weak force, and gravity. The previous collider wasn't powerful enough to peer into the energy space where Higgs boson particles are theorised to exist.

The LHC is being built 100 metres underground in a circular chamber that runs from Geneva airport out under the Jura Mountains and back, giving the collider a diameter of 27km. During commissioning parts of the accelerator were cooled to -271°C, less than two degrees from the absolute zero of temperature or colder than outer space.

Supercooled superconducting magnets in the LHC are used to accelerate beams of protons to close to the speed of light, before two beams travelling in opposite directions collide with each other around 40 million times a second.

The detectors have millions of output channels, generating data at around one million Gigabytes a second. That's way beyond the capability of existing technology to capture and store, so systems have been put in place to filter this data down into more manageable chunks. In practice, data acquisition occurs at around 100MB per second.

Enter the Matrix

Even after information is filtered to concentrate on interesting events the collider generates a phenomenal volume of data, around 15 million Gigabytes of data a year. This data is distributed from CERN to partner laboratories, from Academia Sinica in Taipei to Fermilab in Chicago, via fibre optic links.

This network (with CERN at its hub) is the backbone of the Worldwide LHC Computing Grid, the largest scientific grid of its kind. At its core is a 10Gbps network that uses kit supplied by HP and Force10. The network feeds data to rack of blade servers at CERN and out through edge switches to data centres at its partners.