Detection for multiple input multiple output channels : analysis of sphere decoding and semidefinite relaxation

Abstract: The problem of detecting a vector of symbols, drawn from a finite alphabet and transmitted over a multiple-input multiple-output (MIMO) channel with Gaussian noise, is of central importance in digital communications and is encountered in several different applications. Examples include, but are not limited to; detection of symbols spatially multiplexed over a multiple-antenna channel and the multiuser detection problem in a code division multiple access (CDMA) system. Two algorithms previously proposed in the literature are considered and analyzed. Both algorithms have their origin in other fields of science but have gained mainstream recognition as efficient algorithms for the detection problem considered herein. Specifically, we consider the sphere decoder and semidefinite relaxation detector. By incorporating assumptions applicable in the communications context the performance of the two algorithms is addressed. The first algorithm, the sphere decoder, offers optimal performance in terms of its error probability. Further, the algorithm has proved extremely efficient in terms of computational complexity for moderately sized problems at high signal to noise ratio (SNR). Although it is recognized that the algorithm has an exponential worst case complexity, there has been a widespread belief that the algorithm has a polynomial average complexity at high SNR. A contribution made herein is to show that this is incorrect and that the average complexity, as the worst case complexity, is exponential in the number of symbols detected. Instead, another explanation of the observed efficiency of the algorithm is offered by deriving the exponential rate of growth and showing that this rate, although strictly positive for finite SNR, is small in the high SNR regime. The second algorithm, the semidefinite relaxation (SDR) detector, offers polynomial complexity at the expense of suboptimal performance in terms of error probability. Nevertheless, previous numerical observations suggest that error probability of the SDR algorithm is close to that of the optimal detector. Herein, the near optimality is of the SDR algorithm is given a precise meaning by studying the diversity of the SDR algorithm when applied to the (real valued) i.i.d.~Rayleigh fading channel and it is shown that the SDR algorithm achieves the same diversity order as the optimal detector. Further, criteria under which the SDR estimates coincide with the optimal estimates are derived and discussed.