Researchers at the Department of Energy’s SLAC National Accelerator Laboratory (SLAC) and Zettar Inc. recently transferred 1 petabyte of data—including encryption and checksumming—in 29 hours, beating last year's record by five hours, an almost 15% improvement. The test was part of a project aimed at increasing data transfer rates to accommodate the data that will be generated by the Linac Coherent Light Source II (LCLS II).
The LCLS II, which is expected to come online in 2020, will be the first hard X-ray free-electron laser (XFEL) with strobe-like pulses just a few millionths of a billionth of a second long and a billion times brighter than previous X-ray sources. The new LCLS will give researchers a major jump in capability, going from 120 pulses per second to 1 million pulses per second. Scientists and engineers will use LCLS to take crisp pictures of atomic motions, watch chemical reactions unfold, probe the properties of materials, and explore fundamental processes in living things.
The increased capability is expected to generate data transfers of multiple terabits per second and refine hardware and software that could eventually find its way onto the next generation of the internet. First, though, it will send LCLS experimental results via ESnet from SLAC to the DoE’s supercomputing facilities at Lawrence Berkeley and Oak Ridge National Laboratories for analysis.
This screenshot from MyESnet shows the southern links of the network (dark red), indicating data transfers exceeding 50 Gbps during the 5,000-mile transfer conducted by SLAC and Zettar Inc.
ESnet is the DoE’s dedicated network user facility for scientific research and carries data between universities and DoE’s national laboratories and user facilities along a multi-100 gigabits-per-second (Gbps) backbone network. The demonstration of high speed data transport is also part of a program to develop and evaluate tools to address the upcoming decade’s critical data transfer requirements for the next-generation XFELs at SLAC.
The firm helping in this project, Zettar, is a National Science Foundation-funded software firm in Palo Alto. It develops hyperscale data distribution software capable of multi-100+Gbps transfer rates and works with ESnet and DOE national labs. For the trial, the team used ESnet’s On-Demand Secure Circuits and Advance Reservation System (OSCARS) to reserve bandwidth for the run.
“The innovative and versatile nature of ESnet’s OSCARS let us design a 5,000-mile 100 Gbps loop, which ESnet set up in 2015,” says Zettar Founder and CEO Chin Fang. “The design of ESnet lets us use a separate dedicated channel for the test without degrading or interfering with normal traffic. We were also able to have data storage clusters and data transfer nodes in the same rack, although they were at either ends of the loop. Physically they were separated by inches, but digitally they were 5,000 miles apart.”
Fang said the five-hour improvement in transferring the petabyte of data was due to improved software and hardware. This year, Zettar significantly improved the parallelism of its software to speed up data transfers for both encrypted and non-encrypted data. Intel also provided faster processors and solid-state memory products for moving data in and out of main storage systems faster.
