Towards Reliable, Stable and Fast Learning for Smart Home Activity Recognition

Abstract: The current population age grows increasingly in industrialized societies and calls for more intelligent tools to monitor human activities.  The aims of these intelligent tools are often to support senior people in their homes, to keep track of their daily activities, and to early detect potential health problems to facilitate a long and independent life.  The recent advancements of smart environments using miniaturized sensors and wireless communications have facilitated unobtrusively human activity recognition.  Human activity recognition has been an active field of research due to its broad applications in different areas such as healthcare and smart home monitoring. This thesis project develops work on machine learning systems to improve the understanding of human activity patterns in smart home environments. One of the contributions of this research is to process and share information across multiple smart homes to reduce the learning time, reduce the need and effort to recollect the training data, as well as increase the accuracy for applications such as activity recognition. To achieve that, several contributions are presented to pave the way to transfer knowledge among smart homes that includes the following studies. Firstly, a method to align manifolds is proposed to facilitate transfer learning. Secondly, we propose a method to further improve the performance of activity recognition over the existing methods. Moreover, we explore imbalanced class problems in human activity recognition and propose a method to handle imbalanced human activities. The summary of these studies are provided below. In our work, it is hypothesized that aligning learned low-dimensional  manifolds from disparate datasets could be used to transfer knowledge between different but related datasets. The t-distributed Stochastic Neighbor Embedding(t-SNE) is used to project the high-dimensional input dataset into low-dimensional manifolds. However, since t-SNE is a stochastic algorithm and  there is a large variance of t-SNE maps, a thorough analysis of the stability is required before applying  Transfer learning.  In response to this, an extension to Local Procrustes Analysis called Normalized Local Procrustes Analysis (NLPA) is proposed to non-linearly align manifolds by using locally linear mappings to test the stability of t-SNE low-dimensional manifolds. Experiments show that the disparity from using NLPA to align low-dimensional manifolds decreases by order of magnitude compared to the disparity obtained by Procrustes Analysis (PA). NLPA outperforms PA and provides much better alignments for the low-dimensional manifolds. This indicates that t-SNE low-dimensional manifolds are locally stable, which is the part of the contribution in this thesis.Human activity recognition in smart homes shows satisfying recognition results using existing methods. Often these methods process sensor readings that precede the evaluation time (where the decision is made) to evaluate and deliver real-time human activity recognition. However, there are several critical situations, such as diagnosing people with dementia where "preceding sensor activations" are not always sufficient to accurately recognize the resident's daily activities in each evaluated time. To improve performance, we propose a method that delays the recognition process to include some sensor activations that occur after the point in time where the decision needs to be made. For this, the proposed method uses multiple incremental fuzzy temporal windows to extract features from both preceding and some oncoming sensor activations. The proposed method is evaluated with two temporal deep learning models: one-dimensional convolutional neural network (1D CNN) and long short-term memory (LSTM) on a binary sensor dataset of real daily living activities.  The experimental evaluation shows that the proposed method achieves significantly better results than the previous state-of-the-art. Further, one of the main problems of activity recognition in a smart home setting is that the frequency and duration of human activities are intrinsically imbalanced. The huge difference in the number of observations for the categories means that many machine learning algorithms focus on the classification of the majority examples due to their increased prior probability while ignoring or misclassifying minority examples. This thesis explores well-known class imbalance approaches (synthetic minority over-sampling technique, cost-sensitive learning and ensemble learning) applied to activity recognition data with two temporal data pre-processing for the deep learning models LSTM and 1D CNN. This thesis proposes a data level perspective combined with a temporal window technique to handle imbalanced human activities from smart homes in order to make the learning algorithms more sensitive to the minority class. The experimental results indicate that handling imbalanced human activities from the data-level outperforms algorithm level and improved the classification performance.