Simple principles of cognitive computation with distributed representations

Abstract: Brains and computers represent and process sensory information in different ways. Bridging that gap is essential for managing and exploiting the deluge of unprocessed and complex data in modern information systems. The development of brain-like computers that learn from experience and process information in a non-numeric cognitive way will open up new possibilities in the design and operation of both sensor and information communication systems. This thesis presents a set of simple computational principles with cognitive qualities, which can enable computers to learn interesting relationships in large amounts of data streaming from complex and changing real-world environments. More specifically, this work focuses on the construction of a computational model for analogical mapping and the development of a method for semantic analysis with high-dimensional arrays. A key function of cognitive systems is the ability to make analogies. A computational model of analogical mapping that learns to generalize from experience is presented in this thesis. This model is based on high-dimensional random distributed representations and a sparse distributed associative memory. The model has a one-shot learning process and an ability to recall distinct mappings. After learning a few similar mapping examples the model generalizes and performs analogical mapping of novel inputs. As a major improvement over related models, the proposed model uses associative memory to learn multiple analogical mappings in a coherent way. Random Indexing (RI) is a brain-inspired dimension reduction method that was developed for natural language processing to identify semantic relationships in text. A generalized mathematical formulation of RI is presented, which enables N-way Random Indexing (NRI) of multidimensional arrays. NRI is an approximate, incremental, scalable, and lightweight dimension reduction method for large non-sparse arrays. In addition, it provides low and predictable storage requirements, and also enables therange of array indices to be further extended without modification of the data representation. Numerical simulations of two-way and ordinary one-way RI are presented that illustrate when the approach is feasible. In conclusion, it is suggested that NRI can be used as a tool to manage and exploit Big Data, for instance in data mining, information retrieval, social network analysis, and other machine learning applications.