Publicação

Consistency matters: neural ode parameters are dependent on the training numerical method

Ver documento

Detalhes bibliográficos
Resumo:Neural Ordinary Differential Equations (Neural ODEs) are continuous-depth models that use an ordinary differential equation (ODE) to capture the dynamics of data. Due to their modelling capabilities several works on applications and novel architectures using Neural ODEs can be found in the literature. In this work, we call for the attention to the need of using the same numerical method for both training and making predictions with Neural ODEs since the numerical method employed influences the prediction process, thereby impacting the loss function and introducing variance into parameter optimisation. We provide theoretical insights into how numerical methods of varying orders or with different step sizes influence the loss function of the network. To validate our theoretical analysis, we conduct a series of simple preliminary numerical experiments employing a regression task, demonstrating how the training numerical method influences model performance for testing. Our findings underscore the need for consistency in numerical methods for training and prediction, a consideration not previously emphasised or documented in the literature.
Autores principais:Coelho, C.
Outros Autores:Costa, M. Fernanda P.; Ferrás, Luís Jorge Lima
Assunto:Ciências Naturais::Matemáticas
Ano:2024
País:Portugal
Tipo de documento:comunicação em conferência
Tipo de acesso:acesso aberto
Instituição associada:Universidade do Minho
Idioma:inglês
Origem:RepositóriUM - Universidade do Minho

Registos relacionados

Atividades financiadas

Carregando projetos financiados...

Ver mais

Registos relacionados