Learning flow functions : architectures, universal approximation and applications to spiking systems

Abstract: Learning flow functions of continuous-time control systems is considered in this thesis. The flow function is the operator mapping initial states and control inputs to the state trajectories, and the problem is to find a suitable neural network architecture to learn this infinite-dimensional operator from measurements of state trajectories. The main motivation is the construction of continuous-time simulation models for such systems. The contribution is threefold.We first study the design of neural network architectures for this problem, when the control inputs have a certain discrete-time structure, inspired by the classes of control inputs commonly used in applications. We provide a mathematical formulation of the problem and show that, under the considered input class, the flow function can be represented exactly in discrete time. Based on this representation, we propose a discrete-time recurrent neural network architecture. We evaluate the architecture experimentally on data from models of two nonlinear oscillators, namely the Van der Pol oscillator and the FitzHugh-Nagumo oscillator. In both cases, we show that we can train models which closely reproduce the trajectories of the two systems.Secondly, we consider an application to spiking systems. Conductance-based models of biological neurons are the prototypical examples of this type of system. Because of their multi-timescale dynamics and high-frequency response, continuous-time representations which are efficient to simulate are desirable. We formulate a framework for surrogate modelling of spiking systems from trajectory data, based on learning the flow function of the system. The framework is demonstrated on data from models of a single biological neuron and of the interconnection of two neurons. The results show that we are able to accurately replicate the spiking behaviour.Finally, we prove an universal approximation theorem for the proposed recurrent neural network architecture. First, general conditions are given on the flow function and the control inputs which guarantee that the architecture is able to approximate the flow function of any control system with arbitrary accuracy. Then, we specialise to systems with dynamics given by a controlled ordinary differential equation, showing that the conditions are satisfied whenever the equation has a continuously differentiable right-hand side, for the control input classes of interest.