Texas State researchers to explore state-of-the-art lossy compression

The Department of Computer Science at Texas State University, in partnership with the Argonne National Laboratory, has received a $1 million grant from the U.S. Department of Energy (DOE) Office of Science for research to advance the state of the art in computer science and applied mathematics.

Martin Burtscher, a professor in the Department of Computer Science, will serve as principal investigator (PI) on the project, "Automatic Generation of Algorithms for High-Speed Reliable Lossy Compression," with Sheng Di of the Argonne National Lab serving as Co-PI. Approximately $600,000 of the grant will go to Texas State.

The research project is one of nine funded by the DOE across the country to address the challenges of moving, storing and processing the massive data sets produced by leading-edge scientific instruments and computer simulations, accelerating the pace of scientific discovery. Lossy compression is a method of data compression in which the size of the file is reduced by eliminating data in the file.

Fast reliable data compression is urgently needed for many high-performance computing applications and scientific instruments because they produce vast amounts of data at extremely high rates. The goal of the research project is to develop a high-speed reliable lossy-compression framework named LC that meets three critical needs:

• Improving the amount of data reduction and the trustworthiness of lossy compression methods
• Increasing the compression/decompression speed to match the high data generation/acquisition rates
• Supporting progressive compression and decompression with multiple levels of resolution
   

The LC framework will allow users to synthesize customized compression pipelines, thus optimizing the speed and compression ratio. To accomplish this, LC will provide numerous algorithms to choose from and automatically emit the source code of the optimal configuration with reliable execution time bounds. Increasing speed will be accomplished with the development of new decorrelation strategies, high-speed data predictors, efficient quantization methods, parallel execution, GPU acceleration, and a new class of encoders called 'essentially lossless' that will compress faster and better than the current state of the art.

 The resulting fast reliable lossy compression framework will greatly benefit the many scientific applications that need not only high trustworthiness but also high performance.

The projects are managed by the Advanced Scientific Computing Research (ASCR) program within the DOE Office of Science. For more information, visit www.energy.gov/science/articles/doe-invests-137-million-research-data-reduction-science.