Talk the walk : Empirical studies and data-driven methods for geographical natural language applications

Abstract: Finding the way in known and unknown city environments is a task that all pedestrians carry out regularly. Current technology allows the use of smart devices as aids that can give automatic verbal route directions on the basis of the pedestrian's current position. Many such systems only give route directions, but are unable to interact with the user to answer clarifications or understand other verbal input. Furthermore, they rely mainly on conveying the quantitative information that can be derived directly from geographic map representations: 'In 300 meters, turn into High Street'. However, humans are reasoning about space predominantly in a qualitative manner, and it is less cognitively demanding for them to understand route directions that express such qualitative information, such as 'At the church, turn left' or 'You will see a café'. This thesis addresses three challenges that an interactive wayfinding system faces in the context of natural language generation and understanding: in a given situation, it must decide on whether it is appropriate to give an instruction based on a relative direction, it must be able to select salient landmarks, and it must be able to resolve the user's references to objects. In order to address these challenges, this thesis takes a data-driven approach: data was collected in a large-scale city environment to derive decision-making models from pedestrians' behavior. As a representation for the geographical environment, all studies use the crowd-sourced Openstreetmap database. The thesis presents methodologies on how the geographical and language data can be utilized to derive models that can be incorporated into an automatic route direction system.