Dynamic Programming and Time-Varying Delay Systems

Abstract: This thesis is divided into two separate parts. The first part is about Dynamic Programming for non-trivial optimal control problems. The second part introduces some useful tools for analysis of stability and performance of systems with time-varying delays. The two papers presented in the first part attacks optimal control problems with finite but rapidly increasing search space. In the first paper we try it reduce the complexity of the optimization by exploiting the structure of a certain problem. The result, if found, is an optimal solution. The second paper introduces a new general approach of relaxing the optimality constraint. The main contribution of the paper is an extension of the Bellman equality to a double inequality. This inequality is a sufficient condition for a suboptimal solution to be within a certain distance to the optimal solution. The main approach of solving the inequality in the paper is value iteration, which is shown to work well in many different applications. In the second part of the thesis, two analysis methods for systems with time-varying delays are presented in two papers. The first paper presents a set of simple graphical stability (and performance) criteria when the delays are bounded but otherwise unknown. All that is needed to verify stability is a Bode diagram of the closed loop system. For more exact computations, the last paper presents a toolbox for Matlab called Jitterbug. It calculates quadratic costs and power spectral densities of interconnected continuous-time and discrete-time linear systems. The main contribution of the toolbox is to make well known theory easily applicable for analysis of real-time systems.