xCache: An Adaptive Compression Strategy for Metadata Cache

Abstract

Advancements in technologies such as Artificial Intelligence (AI), big data, and Internet of Things (IoT), have driven an exponential growth in the amount of data managed by storage systems. In systems laden with significant data and metadata, rapid and immediate file access becomes crucial. These attributes are essential for maintaining operational efficiency and enhancing user experience. Partially caching of metadata in memory is acknowledged as an effective method to improve file access efficiency. However, the limited capacity of memory can compromise caching effectiveness and cause adverse effects when exceeded. To address this issue, this paper introduces xCache, an effective metadata caching strategy that reduces the memory footprint and improves cache hit rates. The proposed architecture consists of three levels of metadata caching, complemented by an adaptive compression algorithm to maximize its efficiency. Evaluation results demonstrate that xCache can reduce the memory footprint of cached metadata to 13% of its original size, leading to a significant improvement in memory utilization

Publication
2024 IEEE International Conference on High Performance Computing and Communications