Presentation
Significantly Improving Lossy Compression Quality Based on an Optimized Hybrid Prediction Model
SessionCompression
Event Type
Paper
TP
Algorithms
Data Compression
Data Management
Graph Algorithms
I/O
Memory
Parallel Application Frameworks
Performance
TimeWednesday, 20 November 201910:30am - 11am
Location405-406-407
DescriptionWith ever-increasing volumes of data produced by large-scale scientific simulations, error-bounded lossy compression has come into a critical place. In this paper, we design a strategy to improve the compression quality significantly based on an optimized, hybrid prediction framework. The contribution is four-fold. (1) We propose a novel, transform-based predictor and optimize its compression quality. (2) We improve the coefficient-encoding efficiency for the data-fitting predictor. (3) We propose an adaptive framework that can select the bestfit predictor accurately for different datasets. (4) We perform the evaluation by running real-world applications on a supercomputer with 8192 cores. Experiments show that our adaptive compressor can improve the compression ratio by 112~165% over the second-best state-of-the-art lossy compressor. The parallel I/O performance is improved by about 100% because of significantly reduced data size. The total I/O time is reduced by up to 60x with our compressor compared with the original I/O time.
Download PDF
Archive
