As Reinefeld points out the investment is in fact even larger. Another 4 million euro will be spent on the archive system, bringing the total investment cost to around 30 million euro.

The system called the HLRN-II will be installed in three phases. In the first phase, end of March 2008, an MPP system with 30 Tflop/s and an SMP system with 4,2 Tflop/s will be in production. The MPP system will have

2.560 processor cores and 5.120 Gbyte of main memory. The SMP system will have 352 processor cores and 2.816 Gbyte of main memory.

In comparison, the current HLRN-I system in Berlin/Hannover has only a top speed of 5,2 Tflop/s.

In the second phase, beginning of 2009, the MPP system will be expanded by 7.328 processor cores, 29.312 Gbyte of main memory and a peak of 100 Tflop/s.

In the third phase in September 2009, the SMP system will grow by another 2.176 processor cores, 8.704 Gbyte of memory and 20 Tflop/s peak - at each site (Berlin and Hannover). In total, the system comprises 25.000

processor cores and 312 Tflop/s of peak performance.

One thing Reinefeld likes about the SGI solution is that HLRN already will have a pretty large system in April next year. The system will then already have 364 nodes of which 44 nodes are SMP systems with extra large memory. The next extension phase by the end of 2008 will be the major performance improvement. It will use the Intel Xeon "Gainestown" processor with Intel CSI and a Fat-Tree InfiniBand 4X-QDR interconnect network. By then, each of the two subsystems - called "complexes" by Reinefeld - will have a 134 Tflop/s performance. With

the other vendors they would have to wait much longer for that type of computing power.

As Reinefeld pointed out, this system is a general purpose supercomputer. They have a much broader user community (currently about 70 different research groups) than sites running Blue Genes for instance.

According to Reinefeld, the biggest challenge ahead is to get the applications running efficiently on 25.000 cores. It is already clear that some applications really can use this type of power and parallelism. For instance the BQCD, the Berlin QCD kernel, which has been developed at ZIB, so they know how it works. The kernel has

already shown to be able to run with 1 TFlop/s sustained performance on the supercomputers in Juelich and Munich. A couple of users, especially chemistry users, are using packages, like CPMD, MOLPRO, or Gaussian. For

those it is, of course, difficult to reach the scalability needed. But there are other codes like PALM (parallel large eddy simulation) or the ocean modelling codes OPA or MOM, or the TAU CFD codes which all will be adapted to the new machine to utilize its full performance.

The reason that HLRN chose for SGI was based on the benchmarks and some additional requirements. On the benchmarks, SGI offered the best sustained performance based on the actual measurements and on projections for larger systems and new processors. The Intel Gainestown and Beckton processors are for instance not available yet and it is therefore difficult to estimate their sustained speed.

Another important criterion to decide for SGI was the support that will be provided by the vendor to improve the user codes, also in relation to multicore. In addition there were several other criteria such as sustained performance on which SGI also did score very well.

Important was also that the whole system, with 25.000 cores presents itself to the user as a single image. The users will have a single home-directory. There is a single queue, with subqueues. The user sees the whole system as just one virtual supercomputer.

The new system will be 60 times faster than the current computer. And although the current system is oversubscribed by a factor of three, this still leaves a big gap, and a challenge to get the machine efficiently

used even though, of course, the system has been bought based on the demands of the users. Hence Reinefeld sees that even more than before, it is important to get a good co-operation between the users, the HLRN consultants and the vendor.