Learning and Evaluating the Geometric Structure of Representation Spaces

Abstract: Efficient representations of observed input data have been shown to significantly accelerate the performance of subsequent learning tasks in numerousdomains. To obtain such representations automatically, we need to designboth i) models that identify useful patterns in the input data and encode theminto structured low dimensional representations, and ii) evaluation measuresthat accurately assess the quality of the resulting representations. In thisthesis, we present work that addresses both these requirements, where weextensively focus on requirement ii) since the evaluation of representations hasbeen largely unexplored in the machine learning research. We begin with anoverview of representation learning techniques and different structures that canbe imposed on representation spaces, thus first addressing i). In this regard,we present a representation learning model that identifies useful patterns frommultimodal data, and describe an approach that promotes a structure on therepresentation space that is favorable for performing a robotics task. We thenthoroughly study the problem of assessing the quality of learned representations and overview the pitfalls of current practices. With this, we motivatethe evaluation based on analyzing geometric properties of representations andpresent two novel evaluation algorithms constituting the core of this thesis.Finally, we present an application of the proposed evaluation algorithms tocompare large input graphs.