Publicação
Comprehensive study of the energy impact of LSM-based key-value stores
| Resumo: | The exponential growth in digital data—from mobile devices, social media and IoT devices—has created an escalation of information travelling from the edge to the core layer. The core layer plays a crucial and central role in storage, analytics, and service delivery. Key Value Store (KVS) as RocksDB, Cassandra, among others, are widely used in the core layer due to their high throughput and low-latency solutions for managing and storing real-time data of large volumes of small-sized data. Recent efforts have been made to reduce the carbon footprint of data centres while improving their energy efficiency. However, due to the end of the Dennard scaling, reducing hardware energy consumption comes at the cost of worse performance. A shift to improving the energy efficiency of data centres through a software-centric approach is expected. As KVS support a significant share of services hosted in data centres, it is crucial to evaluate and improve their energy efficiency. Nonetheless, improvements in KVS have focused on performance and have neglected the energy impacts of their decisions. To the best of our knowledge, no existing KVS is solely designed with energy efficiency in mind, and there are no studies evaluating the energy profile of existing KVS. Evaluating the energy of these systems requires access to fine-grained energy monitoring tools, capable of capturing the energy of internal events. Notwithstanding, existing tools cannot capture energy at such fine granularity. To facilitate the development of energy-aware KVS, we present an in-depth experimental study that characterises the energy consumption patterns of RocksDB, a widely used LSM-Tree KVS. We also present PowMon, a fine-grain energy monitoring tool capable of monitoring energy at the application, thread, and event levels to support this study’s energy readings. The study shows that energy efficiency is highly dependent on workload, data distribution, and hardware properties, underscoring the need for profiling when choosing the most energy-efficient configuration setup. Furthermore, the insights in this work can be used by KVS designers to create a more energy-aware KVS and also enable them to compare energy efficiency results against a RocksDB baseline efficiency benchmark. |
|---|---|
| Autores principais: | Fernandes, José Pedro Dias Nascimento |
| Assunto: | Key-Value Store Fine-Grained Energy Attribution Energy Efficiency Sistema chave valor Atribuição de energia grão fino Eficiência energética Engenharia e Tecnologia::Engenharia Eletrotécnica, Eletrónica e Informática |
| Ano: | 2025 |
| País: | Portugal |
| Tipo de documento: | dissertação de mestrado |
| Tipo de acesso: | acesso aberto |
| Instituição associada: | Universidade do Minho |
| Idioma: | inglês |
| Origem: | RepositóriUM - Universidade do Minho |
| Resumo: | The exponential growth in digital data—from mobile devices, social media and IoT devices—has created an escalation of information travelling from the edge to the core layer. The core layer plays a crucial and central role in storage, analytics, and service delivery. Key Value Store (KVS) as RocksDB, Cassandra, among others, are widely used in the core layer due to their high throughput and low-latency solutions for managing and storing real-time data of large volumes of small-sized data. Recent efforts have been made to reduce the carbon footprint of data centres while improving their energy efficiency. However, due to the end of the Dennard scaling, reducing hardware energy consumption comes at the cost of worse performance. A shift to improving the energy efficiency of data centres through a software-centric approach is expected. As KVS support a significant share of services hosted in data centres, it is crucial to evaluate and improve their energy efficiency. Nonetheless, improvements in KVS have focused on performance and have neglected the energy impacts of their decisions. To the best of our knowledge, no existing KVS is solely designed with energy efficiency in mind, and there are no studies evaluating the energy profile of existing KVS. Evaluating the energy of these systems requires access to fine-grained energy monitoring tools, capable of capturing the energy of internal events. Notwithstanding, existing tools cannot capture energy at such fine granularity. To facilitate the development of energy-aware KVS, we present an in-depth experimental study that characterises the energy consumption patterns of RocksDB, a widely used LSM-Tree KVS. We also present PowMon, a fine-grain energy monitoring tool capable of monitoring energy at the application, thread, and event levels to support this study’s energy readings. The study shows that energy efficiency is highly dependent on workload, data distribution, and hardware properties, underscoring the need for profiling when choosing the most energy-efficient configuration setup. Furthermore, the insights in this work can be used by KVS designers to create a more energy-aware KVS and also enable them to compare energy efficiency results against a RocksDB baseline efficiency benchmark. |
|---|