Stock exchange for ‘grid’ computing?

As you read this, millions of computers all over the world are sleeping peacefully after a busy day at work. Computing power is going to waste simply because these machines do not have enough work to occupy them round the clock.

The idea of using the internet to help make better use of computing resources is not new, but according to Professor Torsten Eymann of the University of Bayreuth, Germany, we need a better way to match resources with the people who want to use them.

Current approaches to “grid computing” tend to be organised like an online auction, Eymann says, with everything routed through a central server. This works well for small or slow-moving markets, but soon bogs down when many thousands of traders need to negotiate deals on a timescale of minutes or seconds.

Eymann says that a free-market approach, without central oversight, may be the best way to trade computing resources on a large scale. He is the coordinator of the EU-funded CATNETS project, which has developed a promising technical foundation for a free market in grid computing.

Spreading the load

Grid computing is well known through public projects like SETI@home, which since 1999 has used spare capacity on thousands of small computers to search for evidence of extra-terrestrial life. Peer-to-peer (P2P) file sharing is another example of how PCs can co-operate to share the work of distributing large files, such as music and videos.

Grid computing has great potential in industry, too. Computing-intensive tasks include production scheduling and engineering design tools, such as stress analysis and fluid flow modelling. On office computers, these programs typically run overnight. Using the internet to create a “virtual supercomputer” could increase productivity or solve more complex models.

Existing grids generally use a centralised system, rather like an eBay auction, to match “sellers” to “buyers”. According to Torsten Eymann, this is fine for small grids, but does not scale up well: “Centralised systems struggle with a network of just 3,000 buyers and sellers, so a grid to cover the whole of Europe, say, would be very complex.” And because each transaction is for just a few seconds or minutes of computer time, he adds, administration needs to be quick and cheap.

A better solution might be a self-organising free market that does not depend on a central authority. Eymann, who at this point was at Freiburg University, set up a project called CATNET to investigate the idea.

Leave it to the market

CATNET took its name from catallaxy, a term used by the 20th-century Austrian economist Friedrich Hayek. Catallaxy refers to the order that stems from many independent transactions in a free market. It attempts to show how, not why, market prices reach the values they do, and rejects the idea of central control.

CATNET, a one-year feasibility study with just two partners, was promising enough to lead on to CATNETS, a three-year project involving researchers in Spain, the UK, Italy and Germany.

Free markets work well for trading energy or stocks and shares, but these applications involve only a few thousand people at a time, Eymann points out. The main challenge in CATNETS was, therefore, to create models that can handle thousands of transactions at once without becoming unreasonably slow.

CATNETS tackled this in two ways, of which the first was a simulator capable of modelling complex grids. Physics researchers at CERN in Geneva had already created a powerful simulator for their European DataGrid, and with some difficulty the CATNETS team were able to adapt this for their own use. The second part of the work was to build a real prototype network to validate the results from the simulator.

The results showed that the free-market system is actually more complex than a central auction, in the sense that more messages need to be transmitted to arrange the same number of transactions. Crucially, however, these overheads are distributed among all the participants, rather than being loaded preferentially onto a central administrator.

As a result, CATNETS showed that a free-market network can be much bigger than a centrally-administered one without becoming bogged down by administrative overheads.

CATNETS brought together computer scientists and economists. The team is now planning a follow-up EU-funded project. Whereas CATNETS looked at deals made as a result of price alone, the new project will investigate other factors likely to be significant in real-world grid computing: the effect of trust and reputation, service level agreements, and different pricing levels for different services.

The team members have no plans to commercialise their work directly, but would happily offer consultancy services to a commercial developer, Eymann says: “Creating a commercial product would not be too hard: perhaps a one-year project for ten software engineers.” Your computer might soon be earning money while you sleep.