The effect of haptic feedback in visual-manual human-machine interaction

Abstract: Humans use all their senses when they explore and interact with the environment. In human-machine interaction (HMI) vision is the dominant sense, followed by audition. Haptic information - information concerning the sense of touch - is not commonly available. The overall aim of this thesis was to investigate how haptic feedback affects visual-manual HMI. In the experiment presented in paper I the spatial haptic properties shape and location were compared. Shape encoding often relies on users sequentially exploring differently shaped knobs, levers or buttons. The experiment revealed that physical shapes available through a shape-changing device can be as efficient as adjacently located push-buttons to encode functions in an interface. The experiment presented in paper II investigated the extent to which interface information can be transferred between the haptic and visual modalities. The feedback - rendered textures - was displayed haptically through a rotary device and visually through a computer monitor. There was a cross-modal transfer between the modalities, although not effortless, and the transfer from haptics to vision seemed to be easier than the transfer from vision to haptics. The asymmetry of the cross-modal transfer and the enhanced visual performance might be a result of the visual information being more useful for the task at hand. Paper III presents an experiment carried out in a car simulator. The experiment was conducted to investigate how haptic feedback in an in-car interface affects driver behaviour. Visual feedback was provided on a screen at the centre panel of the simulator. Haptic feedback was provided through the interaction device - a rotary device. The results revealed that, although driving performance degradation did not differ between the different haptic and visual feedback conditions, all conditions caused a degradation in driving performance. Visual behaviour did not differ between conditions including visual feedback. It is therefore apparent that the haptic feedback was not actively used when visual interface information was provided. Using haptic feedback only was shown to be more time-consuming. In addition it was revealed that tasks with only haptic feedback induce a cognitive load on the driver. It was apparent in studies II and III that the haptic information is not actively used if the visual information is more easily achieved.