Logo image
Optimizing the Query Performance of Block Index Through Data Analysis and I/O Modeling
Conference paper   Open access

Optimizing the Query Performance of Block Index Through Data Analysis and I/O Modeling

Tzuhsien Wu, Jerry Chou, Shyng Hao, Bin Dong, Scott Klasky and Kesheng Wu
International Conference for High Performance Computing, Networking, Storage and Analysis, SC, Vol.2017-November
2017

Abstract

I/O system Indexing Modeling Performance analysis Scientific data Computer Networks and Communications Computer Science Applications Hardware and Architecture Software
Indexing technique has become an efficient tool to enable scientists to directly access the most relevant data records. But, the time and space requirements of building and storing indexes are expensive in the traditional approaches, such as R-tree and bitmaps. Recently, we started to address this issue by using the idea of 'block index', and our previous work has shown promising results from comparing it against other well-known solutions, including ADIOS, SciDB, and FastBit. In this work, we further improve the technique from both theoretical and implementation perspectives. Driven by an extensive effort in characterizing scientific datasets and modeling I/O systems, we presented a theoretical model to analyze its query performance with respect to a given block size configuration. We also introduced three optimization techniques to achieve a 2. 3x query time reduction comparing to the original implementation.
url
https://doi.org/10.1145/3126908.3126934View
Published (Version of record) Open

Related links

Metrics

1 Record Views

Details

Logo image