Remember me! Supporting Reminiscence through Digital Capture of Lifestories and Activity Recognition

Abstract: Lifelogging is the act of digitally capturing a person’s life experiences in the form of digital lifestories. A digital lifestory is a view of the person’s life based on a single activity or a set of activities, where activities are defined by content data, such as images, and context data, such as related places and persons. An example of a digital lifestory is a wedding where a set of selected images visually represents a person’s activities at the wedding. The person can use this representation as a digital means for later review and reminiscence of the captured activities in the lifestory about the wedding. This thesis discusses how new lifelogging technologies and methods for activity recognition can be used to create such digital lifestories and how these lifestories can be utilized for digital reminiscence. Digital capture of lifestories requires a lifelogging system capable of capturing daily activities. The digital representation of the person’s life should be interpreted and organized as activities that provide an insight into: What activities did the person do, When did the activities take place, Where did the activities take place, and Who was involved in the activities. Presenting the person’s life as activities provides an overview for reminiscing and helps the person selecting the significant activities to keep in digital lifestories.This thesis proposes a system that automatically filters captured lifelog data and then organizes the filtered data in the form of activities identified by time, location, movement data, and knowledge of context. The detection of significant places is implemented based on location clustering techniques that utilizes the density of the collected location data. The time periods when the user lingers at the same place of significance are then used to identify activities. The thesis shows that the required activity recognition can be improved by using knowledge of prior context and by automatic detection of significant places. The thesis also shows that everyday activities can be recognized using a single accelerometer, for which the wrist is the best placement of the accelerometer to analyze body movements and to detect daily activities.Two important aspects of lifelogging system design are to avoid encumbering or stigmatizing a person with too many devices, and to minimize required user interaction. The proposed lifelogging system is therefore highly automated, where the pervasively captured data is filtered from noisy data, segmented into representative activities, annotated with captured images, and organized as digital lifestories. A key finding is that one accelerometer plus one device for capturing location and images constitute a sufficient set of devices required to capture digital lifestories, hence supporting a person in reminiscing past activities by using the proposed system.The work has been evaluated through proof-of-concept prototype systems, which demonstrate the potential of reminiscence tools based on capture and review of digital lifestories. This work has the potential to make digital reminiscence systems more affordable, acceptable and easy to use, which also would lead to a positive impact on utilization. This can in particular be important for persons with special needs, such as persons with mild dementia, who generally cannot cope with too complex interaction. They can thus use such digital reminiscence systems for recollecting and reflecting on past life experiences.