Publicação
Performance and carbon footprint modelling of irregular tasks
| Resumo: | High-Performance Computing (HPC) systems face increasing challenges in energy efficiency and resource management as computational demands grow. A central concern is the carbon footprint of irregular tasks, which exhibit unpredictable runtime and power consumption patterns, complicating job scheduling. Efficient scheduling requires differentiating between regular, predictable jobs and irregular ones, allowing for customized scheduling that reduces energy consumption and enhances performance. Furthermore, there is a notable lack of real-world HPC data, which necessitates the creation of synthetic datasets. These datasets are essential for enabling predictive modeling of job characteristics, as machine learning models require substantial data volumes for optimal accuracy. To address these challenges, this study explores machine learning-based techniques for predicting job characteristics in HPC systems. By generating and analyzing a synthetic dataset derived from existing job metrics, predictive models capable of classifying regular versus irregular jobs and estimating key performance metrics was developed. Specifically, an ExtraTreesClassifier model, achieving 96.7% accuracy in job classification. Additionally, subsequent regression models predict runtime and power consumption, attaining a Mean Absolute Error (MAE) of 495 seconds and 280 watts, respectively. These predictions support more efficient scheduling strategies, optimizing both resource utilization and energy demands in HPC systems. |
|---|---|
| Autores principais: | Carvalho, Nuno Filipe Quintela Teixeira Carvalho |
| Assunto: | High-Performance Computing Power Consumption Prediction Runtime Prediction Machine Learning Supervised Learning Synthetic Data Computação de Alto Desempenho Previsão de Consumo de Energia Previsão de Tempo de Execução Aprendizagem Automática Aprendizagem Supervisionada Dados Sintéticos Engenharia e Tecnologia::Outras Engenharias e Tecnologias |
| Ano: | 2025 |
| País: | Portugal |
| Tipo de documento: | dissertação de mestrado |
| Tipo de acesso: | acesso aberto |
| Instituição associada: | Universidade do Minho |
| Idioma: | português |
| Origem: | RepositóriUM - Universidade do Minho |
| Resumo: | High-Performance Computing (HPC) systems face increasing challenges in energy efficiency and resource management as computational demands grow. A central concern is the carbon footprint of irregular tasks, which exhibit unpredictable runtime and power consumption patterns, complicating job scheduling. Efficient scheduling requires differentiating between regular, predictable jobs and irregular ones, allowing for customized scheduling that reduces energy consumption and enhances performance. Furthermore, there is a notable lack of real-world HPC data, which necessitates the creation of synthetic datasets. These datasets are essential for enabling predictive modeling of job characteristics, as machine learning models require substantial data volumes for optimal accuracy. To address these challenges, this study explores machine learning-based techniques for predicting job characteristics in HPC systems. By generating and analyzing a synthetic dataset derived from existing job metrics, predictive models capable of classifying regular versus irregular jobs and estimating key performance metrics was developed. Specifically, an ExtraTreesClassifier model, achieving 96.7% accuracy in job classification. Additionally, subsequent regression models predict runtime and power consumption, attaining a Mean Absolute Error (MAE) of 495 seconds and 280 watts, respectively. These predictions support more efficient scheduling strategies, optimizing both resource utilization and energy demands in HPC systems. |
|---|
Atividades financiadas
Não foram encontrados projetos financiados.