Device-free human localization and activity recognition for supporting the independent living of the elderly

Abstract

Given the continuous growth of the aging population, the cost of health care, and the preference that the elderly want to live independently and safely at their own homes, the demand on developing an innovative living-assistive system to facilitate the independent living for the elderly is becoming increasingly urgent. This novel system is envisioned to be device-free, intelligent, and maintenance-free as well as deployable in a residential environment. The key to realizing such envisioned system is to study low cost sensor technologies that are practical for device-free human indoor localization and activity recognition, particularly under a clustered residential home. By exploring the latest, low-cost and unobtrusive RFID sensor technology, this thesis intends to design a new device-free system for better supporting the independent living of the elderly. Arising from this live-assistive system, this thesis specifically targets the following six research problems. Firstly, to deal with severe missing readings of passive RFID tags, this thesis proposes a novel tensor-based low-rank sensor reading recovery method, in which we formulate RFID sensor data as a high-dimensional tensor that can naturally preserve sensors’ spatial and temporal information. Secondly, by purely using passive RFID hardware, we build a novel data-driven device-free localization and tracking system. We formulate human localization problem as finding a location with the maximum posterior probability given the observed RSSIs (Received Signal Strength Indicator) from passive RFID tags. For tracking a moving target, we mathematically model the task as searching a location sequence with the most likelihood under a Hidden Markov Model (HMM) framework. Thirdly, to tackle the challenge that the tracking accuracy decreases in a cluttered residential environment, we propose to leverage the Human-Object Interaction (HOI) events to enhance the performance of the proposed RFID-based system. This idea is motivated by an intuition that HOI events, detected by pervasive sensors, can potentially reveal people’s interleaved locations during daily living activities such as watching TV or opening the fridge door. Furthermore, to recognize the resident’s daily activities, we propose a device-free human activity recognition (HAR) system by deploying the passive RFID tags as an array attached on the wall. This HAR system operates by learning how RSSIs are distributed when a resident performs different activities. Moreover, considering that falls are among the leading causes of hospitalization for the elderly, we develop a fine-grained fall detection system that is capable of not only recognizing regular actions and fall events simultaneously, but also sensing the fine-grained fall orientations. Lastly, to remotely control the smart electronic appliances equipped in an intelligent environment, we design a device-free multi-modal hand gesture recognition (HGR) system that can accurately sense the hand’s in-air speed, waving direction, moving range and duration around a mobile device. Our system transforms an electronic device into an active sonar system that transmits an inaudible audio signal via the speaker and decodes the echoes of the hand at its microphone. To test the proposed systems and approaches, we conduct an intensive series of experiments in several real-world scenarios by multiple users. The experiments demonstrate that our RFID-based system can localize a resident with average 95% accuracy and recognize 12 activities with nearly 99% accuracy. The proposed fall detection approach can detect 90.8% falling events. The designed HGR system can recognize six hand gestures with an accuracy up to 96% and provide more fine-grained control commands by incorporating hand motion attributes.