Perceptual Surface Reconstruction

Abstract: How does the brain transform the 2-D light arrays in our eyes into a meaningful 3-D description of surfaces around us? What assumptions does the visual system make about the world when information is incomplete? And how are these assumptions computationally expressed in this perceptual reconstruction process? These questions, and other aspects of binocular depth perception are analysed from a theoretical and computational perspective, as well as through empirical investigations. In paper one, the fundamentals of stereopsis are briefly reviewed, and the difficulties related with resolving the (stereo) correspondence problem are particularly discussed. A computational model of stereopsis is further proposed that seek (binocularly) matching left-right image regions, by finding the highest area-correlation, taken on a derivative of the original images, at three different scales. A number of simulations are presented and discussed. In the second paper, the computational difficulties that are related with the identification of object boundaries are addressed. A computational model is proposed that given contextual information selects image primitives that are (statistically) common along occluding edges, and connects such primitives into smooth contours. The selection is guided by a set of simple heuristics, which are based on findings that the response of cortical cells, which are tuned to a certain image primitive, can be modulated by information from outside their ?classical? receptive field. In paper three, the justification for using the uniqueness constraint, as an absolute constraint in stereo models, is questioned. A stereo algorithm is proposed that uses a relaxed form of this constraint, and allows multiple matches when a one-to-one correspondence does not exist between the left and right image primitives. The central mechanism in the model produce binocular matches that preserve the relative ordering of image primitives. Paper four describes an empirical study where sparse random-dot stereograms were used to investigate how depth is perceived in ambiguous image regions that lack explicit disparity information. The results of the study suggest that the binocular disparity content, as well as interocularly unpaired image elements, both affect interpolation of depth in stereoscopic displays.