Projection Techniques for Classification and Identification

Abstract: It is very well understood how to evaluate and find, in different senses, optimal linear projections of measurements on linear systems. The solution to the linear least squares problem, the principal component analysis and partial least squares are all examples of well known techniques that work very well as long as the dependencies in data are fairly linear. When the measurements are due to nonlinear systems or due to measurements on a finite number of objects (classification), it is much more difficult not only to find an optimal projection, but also to asses a general quality measure to a particular projection. The dimension of the measurement is often high, so the search for projections is motivated by the desire to visualize in 2-dimensional diagrams, the need to reduce data with minimal loss (data compression) and to efficiently parameterize nonlinear models.For classication of measurements on a finite number of objects, the projection quality is naturally connected to how accurate the classification can be conducted despite the data reduction due to the projection. In this work, methods to estimate this accuracy as well as numerical methods to find the optimal projection with respect to this estimated accuracy are investigated. It is found that the nonlinear conjugate gradient method on the Grassmann manifold is one of the most time efficient numerical optimization methods to use with this type of problems.For nonlinear dynamical systems modeled by nonlinear ARX model structures, the challenge is to find projections of the regression vector that isolate a nonlinear curve or surface. It is not obvious that such low-dimensional projections that well fit the system always exist. When they do exist, for instance when the nonlinearities enter the system additively, we show that the projection can rather efficiently be estimated by using multi-index models fitted by least squares criteria.There is also an interesting hybrid case where a continuous system is analyzed by measurements on a nite number of calibration classes. We show how the resulting measurement classes can be used by techniques borrowed from discriminant analysis in order to find good projections where a continuous regression can be performed.