NASHVILLE, Tenn. – When a natural disaster hits Texas, P.R. Blackwell has a problem. As director of the TexasView consortium, he needs to quickly get satellite photos of the affected areas to a number of places in support of recovery efforts. But the images are very large and take about 30 minutes each to send over the highest speed network available.
Because dozens or more images are needed, just transmitting them can take a day or more.
In hopes of reducing distribution times of large amounts of satellite data from hours to seconds, Blackwell, assistant director for operations for the Columbia Regional Geospatial Service Center at the Stephen F. Austin State University, is participating in a new project. The Research and Education Data-Depot Network, or REDDnet (pronounced “ready-net”), is a collaboration among a diverse group of universities and other groups that handle and distribute large data sets. The project, administered by Vanderbilt University, has received $594,000 from the National Science Foundation to set up this new kind of network in the United States. Vanderbilt’s Center for the Americas has contributed funding to extend it to Brazil.
“Sharing large amounts of data among geographically distributed groups creates a big logistical problem. You need more than high-speed pipes and massive, centralized data centers to deal with it,” said Paul Sheldon, the Vanderbilt professor of physics who heads the project. “You also need distributed storage depots that can handle large amounts of data before sending them along to their destinations.”
By large amounts of data, Sheldon and his colleagues mean a trillion bytes (or a terabyte) per day or greater. By comparison the U.S. Library of Congress holds an estimated 20 terabytes of information.
Currently, the nation’s high-speed networks have enough capacity to transmit terabytes of data. What they lack is adequate working storage capacity to manage all this data while it is in transit. That is where REDDnet comes in. It will install data depots at six host universities – California Institute of Technology, University of Florida, University of Michigan, North Carolina State University, Stephen F. Austin State University and Vanderbilt – and two national labs – Fermi National Laboratory and Oak Ridge National Laboratory.
The support of the Center for the Americas will allow REDDnet to extend to Brazil. Brazil’s research network, the Rede Nacional de Pesuisa, is connected to the U.S. by fiber-optic cable and is installing a major storage system using REDDnet technology. This will allow Brazilian physicists and medical imaging researchers at the University of Sao Paolo, the Federal University of Campina and the Brazilian start-up Nevoa Networks to work closely with U.S. colleagues.
“The project exemplifies the center’s goal to encourage collaborative research that is both likely to advance knowledge in its field and to make a real difference in the world outside of academia,” said Vera Kutzinski, director of the Center for the Americas.
Several major collaborations that handle terabytes of data will test the new network. These include: two particle-physics groups with about 5,000 collaborators worldwide who are associated with the Large Hadron Collider currently under construction in Europe; AmericaView, a national consortium of state-based consortia, including TexasView, dedicated to improving satellite remote sensing for applications, education and training; a consortium of eight universities and ORNL doing large-scale simulations of supernovas; and a collaborative effort between Phoebe Stewart, an associate professor at the Vanderbilt University Medical Center, and colleagues at Lawrence Berkeley National Laboratory who are constructing detailed images of large biological molecules.
The REDDnet data depots will combine state-of-the-art hardware with system software called L-store developed by researchers at Vanderbilt’s Advanced Computing Center for Research & Education to aid in the transfer and storage of large amounts of data being generated by cutting-edge scientific instruments, sensors and simulations. L-store was used by the university team that set a network speed record in 2005.
“We got into this from the storage side,” Sheldon said. He is involved in one of the physics experiments that is designed to produce petabytes (thousands of terabytes) of data per year and is interested in finding efficient ways to handle these large amounts of data.
REDDnet will also use an innovative technology called the Internet Backplane Protocol developed by group member Micah Beck, director of the Logistical Computing and Internetworking Lab at the University of Tennessee, Knoxville. IBP applies some of the key features that have made the Internet so successful for sharing network storage. The technology has been successfully tested at lower data rates on Internet 2, a high-speed network that links a number of U.S. universities and is dedicated to academic research.
Additional information about the project is available at www.vanderbilt.edu/americas and http://mimir.accre.vanderbilt.edu/cgi-bin/public/DocDB/ShowDocument?docid=73.