Neural Ordinary Differential Equations (Neural ODEs) have emerged as a promising approach for learning the continuous-time behaviour of dynamical systems from data. However, Neural ODEs are black-box models, posing challenges in interpreting and understanding their decision-making processes. This raises concerns about their application in critical domains such as healthcare and autonomous systems. To address this challenge and provide insight into the decision-making process of Neural ODEs, we introduce the eXplainable Neural ODE (XNODE) framework, a suite of eXplainable Artificial Intelligence (XAI) techniques specifically designed for Neural ODEs. Drawing inspiration from classical visualisation methods for differential equations, including time series, state space, and vector field plots, XNODE aims to offer intuitive insights into model behaviour. Although relatively simple, these techniques are intended to furnish researchers with a deeper understanding of the underlying mathematical tools, thereby serving as a practical guide for interpreting results obtained with Neural ODEs. The effectiveness of XNODE is verified through case studies involving a Resistor–Capacitor (RC) circuit, the Lotka–Volterra predator-prey dynamics, and a chemical reaction. The proposed XNODE suite offers a more nuanced perspective for cases where low Mean Squared Error values are obtained, which initially suggests successful learning of the data dynamics. This reveals that a low training error does not necessarily equate to comprehensive understanding or accurate modelling of the underlying data dynamics.

Neural Ordinary Differential Equations (Neural ODEs) are continuous-depth models that use an ordinary differential equation (ODE) to capture the dynamics of data. Due to their modelling capabilities several works on applications and novel architectures using Neural ODEs can be found in the literature. In this work, we call for the attention to the need of using the same numerical method for both training and making predictions with Neural ODEs since the numerical method employed influences the prediction process, thereby impacting the loss function and introducing variance into parameter optimisation. We provide theoretical insights into how numerical methods of varying orders or with different step sizes influence the loss function of the network. To validate our theoretical analysis, we conduct a series of simple preliminary numerical experiments employing a regression task, demonstrating how the training numerical method influences model performance for testing. Our findings underscore the need for consistency in numerical methods for training and prediction, a consideration not previously emphasised or documented in the literature.

The continuous dynamics of natural systems has been effectively modelled using Neural Ordi nary Differential Equations (Neural ODEs). However, for accurate and meaningful predictions, it is crucial that the models follow the underlying rules or laws that govern these systems. In this work, we propose a self-adaptive penalty algorithm for Neural ODEs to enable modelling of constrained natural systems. The proposed self-adaptive penalty function dynamically adjusts the penalty parameters during training, allowing for a more flexible and efficient optimization process. Furthermore, by explicitly incorporating prior knowledge through constraints, the method enhances the interpretability of Neural ODE-based models, making them better suited for applications involving physical, biological, or other natural processes. The proposed approach is validated through the modelling of three natural systems: population growth, chemical reaction dynamics, and the motion of a damped harmonic oscillator. The numerical experiments and a comparison with other penalty Neural ODE approaches and vanilla Neural ODE, demonstrate the effectiveness of the proposed self-adaptive penalty algorithm for Neural ODEs in modelling constrained natural systems. Moreover, the self-adaptive penalty approach provides more accurate and robust models with reliable and meaningful predictions.

Optimal control problems (OCPs) are essentials in various domains such as science, engineering, and industry, requiring the optimisation of control variables for dynamic systems, along with the corresponding state variables, that minimise a given performance index. Traditional methods for solving OCPs often rely on numerical techniques and can be computationally expensive when the discretisation grid or time horizon changes. In this work, we introduce a novel approach that leverages Neural Ordinary Differential Equations (Neural ODEs) to model the dynamics of control variables in OCPs. By embedding Neural ODEs within the optimisation problem, we effectively address the limitations of traditional methods, eliminating the need to re-solve the OCP under different discretisation schemes. We apply this method to a coastal ecosystem OCP, demonstrating its efficacy in solving the problem over a 50-year horizon and extending predictions up to 70 years without re-solve the optimisation problem.

Neural Ordinary Differential Equations (Neural ODEs) have gained popularity for modelling real-world systems, thanks to their ability to fit ODEs to data. However, numerous systems in science and engineering often exhibit intricate memory behaviours, being classical ODEs inadequate for such tasks due to their inability to handle strong and complex memory effects. In this work, we introduce the Neural Fractional Differential Equation (Neural FDE), a Neural Network (NN) architecture to fit a FDE to data. With this we leverage the capabilities of FDEs allowing the architecture to take into account all past states and their influence on a system’s current and future behaviours. Neural FDE inherently exhibits memory, providing a more accurate representation of complex phenomena in systems with long-term dependencies. Numerical experiments show Neural FDE generalises better and has faster convergence than Neural ODEs.

Neural Ordinary Differential Equations (Neural ODEs) have been used to model natural systems because of their continuous dynamics. However, for meaningful predictions, it is critical to accurately extract from the data the underlying rules or laws governing these systems. To overcome the limitations of commonly used penalty methods, we propose a filter-based approach for Neural ODEs that uses a filter methodology to ensure convergence to a feasible and optimal solution for constrained natural systems. The proposed filter methodology prioritises feasible solutions followed by infeasible and non-dominated solutions. The proposed algorithm provides explicit control over the parameter optimisation process, increases the interpretability of the model, and improves the overall performance. We validate the proposed filter-based Neural ODE approach by modelling two different constrained natural systems. Experimental results show that the models trained by our method provide robust generalisation and satisfy the imposed constraints. Our findings indicate that the proposed method represents a promising approach for incorporating prior knowledge constraints into Neural ODE models.

This work presents and compares different methodologies for the joint optimisation of the fractional derivative order and the parameters of the right-hand-side neural network in Neural Fractional Differential Equation models. The proposed strategies aim to tackle the training difficulties typically encountered when learning the fractional order (Formula presented.) together with the network weights. One approach is based on regulating the gradient magnitude of the loss function with respect to (Formula presented.), encouraging more stable and effective updates. Another strategy introduces an online pre-training scheme, where the network parameters are initially optimised over progressively longer time intervals, while (Formula presented.) is updated more conservatively using the full time trajectory. The study focuses only on a foundational setting with one-dimensional problems, and numerical experiments demonstrate that the proposed techniques improve both training stability and accuracy. Nonetheless, the issue of non-uniqueness in the optimal derivative order remains, particularly in less well-posed scenarios, suggesting the need for further research in data-driven modelling of fractional-order systems.

Hydropower plants are crucial for stable renewable energy and serve as vital water sources for sustainable agriculture. However, it is essential to assess the current water management practices associated with hydropower plant management software. A key concern is the potential conflict between electricity generation and agricultural water needs. Prioritising water for electricity generation can reduce irrigation availability in agriculture during crucial periods like droughts, impacting crop yields and regional food security. Coordination between electricity and agricultural water allocation is necessary to ensure optimal and environmentally sound practices. Neural networks have become valuable tools for hydropower plant management, but their black-box nature raises concerns about transparency in decision making. Additionally, current approaches often do not take advantage of their potential to create a system that effectively balances water allocation. This work is a call for attention and highlights the potential risks of deploying neural network-based hydropower plant management software without proper scrutiny and control. To address these concerns, we propose the adoption of the Agriculture Conscious Hydropower Plant Management framework, aiming to maximise electricity production while prioritising stable irrigation for agriculture. We also advocate reevaluating government-imposed minimum water guidelines for irrigation to ensure flexibility and effective water allocation. Additionally, we suggest a set of regulatory measures to promote model transparency and robustness, certifying software that makes conscious and intelligent water allocation decisions, ultimately safeguarding agriculture from undue strain during droughts.

Electrokinetic flows driven by electro-osmotic forces are especially relevant in micro and nano-devices, presenting specific applications in medicine, biochemistry, and miniaturized industrial processes. In this work, we integrate analytical solutions with numerical methodologies to explore the fluid dynamics of viscoelastic electro-osmotic/pressure-driven fluid flows (described by the generalized Phan–Thien–Tanner (gPTT) constitutive equation) in a microchannel under asymmetric zeta potential conditions. The constitutive equation incorporates the Mittag–Leffler function with two parameters (α and β), which regulate the rate of destruction of junctions in a network model. We analyze the impact of the various model parameters on the velocity profile and observe that our newly proposed model provides a more comprehensive depiction of flow behavior compared to traditional models, rendering it suitable for modeling complex viscoelastic flows.

Neural Fractional Differential Equations (Neural FDEs) represent a neural network architecture specifically designed to fit the solution of a fractional differential equation to given data. This architecture combines an analytical component, represented by a fractional derivative, with a neural network component, forming an initial value problem. During the learning process, both the order of the derivative and the parameters of the neural network must be optimised. In this work, we investigate the non-uniqueness of the optimal order of the derivative and its interaction with the neural network component. Based on our findings, we perform a numerical analysis to examine how different initialisations and values of the order of the derivative (in the optimisation process) impact its final optimal value. Results show that the neural network on the right-hand side of the Neural FDE struggles to adjust its parameters to fit the FDE to the data dynamics for any given order of the fractional derivative. Consequently, Neural FDEs do not require a unique (Formula presented.) value; instead, they can use a wide range of (Formula presented.) values to fit data. This flexibility is beneficial when fitting to given data is required and the underlying physics is not known.