Knowledge based perceptual anchoring : grounding percepts to concepts in cognitive robots

Abstract: A successful articial cognitive agent needs to integrate its perception of the environment with reasoning and actuation. A key aspect of this integration is the perceptual-symbolic correspondence, which intends to give meaning to the concepts the agent refers to { known as Anchoring. However, perceptual representations alone (e.g., feature lists) cannot entirely provide sucient abstraction and enough richness to deal with the complex nature of the concepts' meanings. On the other hand, neither plain symbol manipulation appears capable of attributing the desired intrinsic meaning.We approach this integration in the context of cognitive robots which operate in the physical world. Specically we investigate the challenge of establishing the connection between percepts and concepts referring to objects, their relations and properties.We examine how knowledge representation can be used together with an anchoring framework, so as to complement the meaning of percepts while supporting linguistic interaction. This implies that robots need to represent both their perceptual and semantic knowledge, which is often expressed in dierent abstraction levels and may originate from dierent modalities.The solution proposed in this thesis concerns the specication, design and implementation ofa hybrid cognitive computational model, which extends a classical anchoring framework, in order to address the creation and maintenance of the perceptual-symbolic correspondences. The model is based on four main aspects: (a) robust perception, by relying on state-of-the art techniques from computer vision and mobile robot localisation; (b) symbol grounding, using topdown and bottom-up information acquisition processes as well as multi-modal representations; (c) knowledge representation and reasoning techniques in order to establish a common language and semantics regarding physical objects, their properties and relations, that are to be used between heterogeneous robotic agents and humans; and (d) commonsense information in order to enable high-level reasoning as well as to enhance the semanticdescriptions of objects.The resulting system and the proposed integration has the potential to strengthen and expand the knowledge of a cognitive robot. Specically, by providing more robust percepts it is possible to cope better with the ambiguity and uncertainty of the perceptual data. In addition, the framework is able to exploit mutual interaction between dierent levels of representation while integrating dierent sources of information. By modelling and using semantic & perceptual knowledge, the robot can: acquire, exchange and reason formally about concepts, while prior knowledge can become a cognitive bias in the acquisition of novel concepts.