While the scientific results of the project are important, the success in building reliable high-bandwidth connections linking key research facilities and institutions addresses a problem facing many science communities.

"A lot of researchers stand to benefit from this successful demonstration", stated Eli Dart, an ESnet engineer who helped the team achieve the necessary network performance. "While the science itself is very important in its own right, the ability to link multiple institutions in this way really paves the way for other scientists to use these tools more easily in the future."

"This couldn't have been done without ESnet", Rick Wagner stated. Two aspects of the network came into play. First, ESnet operates the circuit-oriented Science Data Network, which provides dedicated bandwidth for moving large datasets. However, with numerous projects filling the network much of the time for other demos and competitions at SC09, Mike Norman and Rick Wagner took advantage of OSCARS, ESnet's On-Demand Secure Circuit and Advance Reservation System.

"We gave them the bandwidth they needed, when they needed it", stated ESnet engineer Evangelos Chaniotakis. The San Diego team was given two two-hour bandwidth reservations on both Tuesday, November 17, and Thursday, November 19. Evangelos Chaniotakis set up the reservations, then the network automatically reconfigured itself once the window closed.

At the SDSC booth, the live streaming of the data drew a standing-room-only crowd as the data was first shown as a 4,096^3 cube containing 64 billion particles and cells. But Mike Norman pointed out that the milky white cube was far too complex to absorb, then added that it was only one of numerous time-steps. In all, the data required for rendering came to about 150 terabytes of data.

In real time, the data was rendered on the Eureka Linux cluster at the Argonne Leadership Computing Facility and reduced to one-sixty-fourth of the original size for a 1,024^3 representation, making it more manageable and able to be explored interactively. The milky mesh was shown to contain galaxies and clusters linked by sheets and filaments of cosmic gases.

The project, Mike Norman explained, is aimed at determining whether the signal of faint ripples in the universe known as baryon acoustic oscillations, or BAO, can actually be observed in the absorption of light by the intergalactic gas. It can, according to research led by Mike Norman, who said they were the first to determine this. Such a finding is critical to the success of a dark energy survey known as BOSS, the Baryon Oscillation Spectroscopic Survey. The results of his proof-of-concept project, Mike Norman said, "ensure that BOSS is not a waste of time".

Creating a simulation of this size, even using the petaflops Cray XT5 Kraken system at the University of Tennessee can take three months to complete as it is run in batches as time is allocated, Mike Norman said. The data could then be moved in three nights to Argonne for rendering. The images were then streamed to the SDSC OptiPortal for display. Mike Norman said the next step is to close the loop between the client side and the server side to allow interactive use. But the hard work - connecting the resources with adequate bandwidth - has been done, as evidenced by the demo, he noted.

But it wasn't just an issue of bandwidth, according to ESnet's Eli Dart. "We did a lot of testing and tuning", stated Eli Dart. ESnet is managed by Lawrence Berkeley National Laboratory (LBNL).