Server platforms Toolkit

Grid computing to solve Big Bang queries?

Steve Ranger silicon.com

Published: 24 Nov 2005 13:55 GMT

...run particular tasks. Individual scientists can then access data from Tier-3 sites which could be local clusters or individual PCs.

RAL hosts the UK's Tier-1 site, with the universities of Lancaster and Edinburgh and Imperial College operating Tier-2 sites.

While real data won't start flowing until 2007, scientists are already using lots of processing power on simulations. "They need to know what they are looking for so they do lots of simulations," Gordon said.

Commodity hardware and open source software are being used to keep costs down. "Because it's worldwide we are all looking at open source", said Gordon. "All the grid stuff is done in open source, that's taken for granted. Grid should use standard protocols, it's across administrative domains."

Network bandwidth will also be key — at the moment it has a 2Gbps dedicated link to CERN — the same amount of bandwidth RAL uses for all the rest of its Internet traffic — and the plan is to build a dedicated fibre-optic network between the sites.

"What we are looking at is setting up a network of private light-paths to Tier-1 sites," said Gordon.

Managing the huge number of files the experiments will generate is another problem the team is working on, according to Gordon. "You end up with millions of files and the problem comes in handling them and that's where the data management comes in. Data management is key."

But beyond all the exciting technology, much of the work will be in persuading different organisations to share, Gordon said. "A lot of it is sociological — you are persuading people that they gain by connecting all their computers together. It's about collaboration; it's not about people sitting in London using computers all over the world, it's about groups of people working on the same problem."