Topics in Sparse Least Squares Problems

Abstract: This thesis addresses topics in sparse least squares computation. A stable method for solving the least squares problem, min ||Ax-b||2 is based on the QR factorization.Here we have addressed the difficulty for storing the orthogonal matrix Q. Using traditional methods, the number of nonzero elements in Q makes it in many cases not feasible to store. Using the multifrontal technique when computing the QR factorization,Q may be stored and used more efficiently. A new user friendly Matlab implementation is developed.When a row in A is dense the factor R from the QR factorization may be completely dense. Therefore problems with dense rows must be treated by special techniques. The usual way to handle dense rows is to partition the problem into one sparse and one dense subproblem. The drawback with this approach is that the sparse subproblem may be more ill-conditioned than the original problem or even not have a unique solution. Another method, useful for problems with few dense rows, is based on matrix stretching, where the dense rows are split into several less dense rows linked then together with new artificial variables. We give and analyze the conditioning of the matrix obtained by this method and show that no ill-conditioned subproblem arise.In many least squares problems upper and lower bounds on the variables have to be satisfied at the solution. This type of problem arises, for example, in reconstruction problems in geodesy and tomography. Here methods based on direct factorization methods for sparse matrix computation are explored. Two completely different approaches for solving the problem are discussed and compared, i.e. active set methods and primal-dual interior-point methods based on Mehrotra's predictor-corrector path following method. An active set block method suitable for sparse problems is developed and a convergence proof is presented. The choice of barrier parameter, multiple corrections and finite termination for the interior-point method are discussed. Numerical comparison is given of the active set method, the interior-point method, together with an trust region method based on the interior-reflective Newton implemented in the optimization toolbox for MATLAB. The numerical tests show that the block active set method is faster and gives better accuracy for both nondegenerate and degenerate problems.