Device-Free Localization for Human Activity Monitoring

Abstract

Over the past few decades, human activity monitoring has grabbed considerable research attentions due to greater demand for human-centric applications in healthcare and assisted living. For instance, human activity monitoring can be adopted in smart building system to improve the building management as well as the quality of life, especially for the elderly people who are facing health deterioration due to aging factor, without neglecting the important aspects such as safety and energy consumption. The existing human monitoring technology requires additional sensors, such as GPS, PIR sensors, video camera, etc., which incur cost and have several drawbacks. There exist various solutions of using other technologies for human activity monitoring in a smartly controlled environment, either device-assisted or device-free. A radio frequency (RF)-based device-free indoor localization, known as device-free localization (DFL), has attracted a lot of research effort in recent years due its simplicity, low cost, and compatibility with the existing hardware equipped with RF interface. This chapter introduces the potential of RF signals, commonly adopted for wireless communications, as sensing tools for DFL system in human activity monitoring. DFL is based on the concept of radio irregularity where human existence in wireless communication field may interfere and change the wireless characteristics.

1. Introduction

Human movement and behavior while performing their daily activities have inherent hierarchical structure. As enabling technology, real-time human activity monitoring plays an important role in many human-centric applications in different areas such as healthcare, security, surveillance, smart building, etc., particularly to protect elder people and children from some bad incidents. Due to the advanced development in medical, science and technology, the human average lifespan has increased rapidly where people are getting healthier and having longer lives, thus increased the aging population worldwide. The United States’ medical research agency, known as National Institute of Health (NIH), reported that in 2012, 8.0% (or 562 million) of the 7 billion global human population are aged 65 and over, and the percentage has increased by 0.5% (or 55 million) in 2015 [1]. Based on the aging trends, NIH has projected that by 2050, the older population will grow substantially up to 17% (or 1.6 billion) throughout the world [1]. However, most of elderly people spend their extra lifespan in unhealthy manner, with often debilitating illness and disability due to the deterioration of physical or mental functions caused by age-related diseases. In fact, the increase in the older population has a slight impact on the increase in disability rates of world’s population [2].

With the increasing of older and disabled population in most countries and regions across the world, human and activity monitoring has gained substantial attention from the research community for ambient assisted living or elderly care application. As reported in [3], majority of the elderly people are more comfortable to live independently at their own home and community. Nevertheless, in the modern society, the conventional ways of taking care of elders in the family are no longer effective. As a result, there is higher demand in the society for the assistive technologies such as an intelligent monitoring system that can record the elders’ daily activities in which family can respectfully monitor their loved ones who live alone at home. Due to the lower income earner after retirement and higher standards of living, many of the elders cannot afford to pay their healthcare cost as well as the expensive healthcare system or private nursing home care services. Nonetheless, various human monitoring technologies have been developed that help elderly people to age in place.

Traditionally, the human activity monitoring technology is a vision-based [4], which requires the use of video camera to monitor the human activity. Although this vision-based is an effective security measure approach as it can retains the records with high resolution, it also has several drawbacks that involves cost-inefficiency for large-scale deployments, energy consumption, and serious user privacy concerns if it is used in inappropriate places such as lavatory or bathroom, bedroom, and even nursing room. However, in several applications like elderly care and assisted living, monitoring human activities in these privacy areas is very crucial and necessary. For instance, lavatory or bathroom is one of the potential places for falling due to its slippery condition, thus activity monitoring in this place is very important for elderly fall detection system in detecting the falling event [5]. Meanwhile, the activity monitoring in the bedroom is very important for patient sleep monitoring system in detecting unusual sleeping behavior. In fact, the video camera requires a good lighting area ineffective in the dark and has limited view angles.

In recent years, thousands of research study on human activity monitoring has been conducted involving the replacement of traditional vision-based approach with various technologies such as acoustic-based [6, 7], motion-based [8, 9], body-worn sensors [10, 11], gyroscope [12], as well as smartphone [13, 14]. While such approaches address the privacy concern issue, they are sensor-based or in other words impose the requirement that special sensors, that is to be attached to, carried or worn by the subject for an effective activity monitoring. This is inconvenient and inappropriate for human usage especially the elders or people with brain-related diseases (Alzheimer, amnesia, dementia, etc.) to remember each day to wear or to activate those sensors. Furthermore, the whole monitoring process is ineffective and futile if the subject forgets to carry the sensor. Besides, the acoustic-based approach is range limited and prone to false detections since it can only be used in a short range and can easily be influenced by other audio signals [15]. The motion-based sensor such as a single accelerometer is not able to provide sufficient information to the system if used alone, hence need to combine with other sensors for more efficient activity monitoring [16]. Nevertheless, both vision- and sensor-based approaches bear with huge costs due to expensive equipment, installation, and maintenance. All the advantages and disadvantages of above approaches are summarized in Table 1.

Category	Sensor technology	(+) Advantages; (−) Disadvantages
Vision-based	Video camera	+ Effective security measure; + Maintain records; – Interfere with privacy; – Ineffective in the dark; – High computational cost
Motion-based	Accelerometer; Gyroscopes; PIR	+ No privacy issue; + Lower cost (PIR); + High detection accuracy; – Raise physical discomfort issue (accelerometer and gyroscopes); – No direct linear or angular position information; – Low range and line-of-sight restriction (PIR); – Prone to false detection; – Insensitive to very slow motions
Sound-based	Ultrasonic; Acoustic; Audio (microphone)	+ Very sensitive to motion; + Objects and distances are typically determined precisely; + Inexpensive (audio); – Work only directionally (ultrasonic); – Sensitive to temperature and angle of the target (ultrasonic); – Easily be influenced by other audio signals/noise; – Prone to false detections; – Range limited
Sensor-based	Body-worn sensors; (Body sensor networks)	+ High detection accuracy; + No privacy issue; – Expensive devices (sensors); – Disturb or limit the activities of the users; – Required sensors installation and calibration

Table 1.

Advantages and disadvantages of existing sensor technologies.

Recently, Radio Frequency (RF)-based approaches have received significant research attentions to be employed in the human presence detection and activity monitoring based on different wireless radio technologies such as RFID [17, 18], Wi-Fi [19, 20], ZigBee [21, 22], FM radio [23, 24], microwave [25], etc. According to studies on the impact of human presence and activity on the RF signal strength [26, 27, 28], it has been proven that the existence and movement of human body in wireless radio network environment will interfere the wireless signal profiles, either in constructive or destructive manners, in which will change the RF communication pattern between the wireless transceivers. This phenomenon is called radio irregularity, which often consider as a drawback in RF communication. In RF-based human detection and activity monitoring, researchers have seen the radio irregularity phenomenon as a benefit in which it can be exploited as sensing tools to locate the human presence in the indoor environment and discriminate human activities or gestures. Since RF-based human activity monitoring approaches only exploit the wireless communication features, there is no need for expensive physical sensing equipment and modules, which accordingly reduce the cost, ease the deployment, reduce the energy consumption, and protect user privacy [29].

The RF-based approaches can be classified into device-bound and device-free. Like the sensor-based, the device-bound RF-based approach requires the on-body wireless sensors or devices (such as RFID tags or cards, Bluetooth wristbands, smart watches, etc.) to be attached to the subject, which has been known as one of the drawbacks. Hence, the subject is required to actively participate in the activity recognition and monitoring process by always remembering to activate and carry the wearable wireless devices. This device-bound system is also known as active monitoring system and the subjects are usually willing to be monitored by the system. Therefore, we refer the subject in this active monitoring system as an active target. As an example, daily activities, such as walking, sitting, lying, falling, etc., of an active target wearing a simple RFID tag can be tracked using RFID readers [30, 31]. Another example is that an active target carrying mobile phone or other Wi-Fi-embedded devices can be easily tracked by Wi-Fi detectors or monitor [32, 33].

Although the on-body wireless sensors such as RFID tags and RFID cards are commercially available and relatively low cost compared to other wireless technologies, their placement on the target’s body may cause physical discomfort [34], especially the elders under long-term monitoring. Recent research works introduce the placement of RFID in the environments and objects instead of target’s body for activity monitoring [35, 36]. However, reading multiple RFID tags at once may cause malfunction due to signal collision, thus anticollision algorithms are required in which incur an extra cost [37]. On the contrary, device-free RF-based approach, known as device-free localization (DFL), is a passive monitoring system that can locate and monitor human position and activity without the subject’s participation, where the subjects do not need to carry or wear any radio devices. They are usually unaware with the system’s existence, and possibly want to avoid being monitored [21]. The subject in this passive monitoring case is referred as passive target.

In this chapter, we review the recent progress of DFL for indoor environment prioritizing on human activity monitoring with a particular focus on the monitoring systems targeting personal health and assisted living applications. Our aims are to provide a comprehensive review on the topic and to quickly update the researchers beyond this field the state of art, potential, opportunities, challenges, opens issues, and future directions of activity recognition using DFL technology. To the best of our knowledge, although there exist surveys on human activity monitoring and recognition using vision-based [4, 38], wearable sensors [10, 39, 40], mobile phones [41, 42], there are only few surveys published in this research field on human activity monitoring using device-free RF-based [29, 43, 44, 45], including a general architecture of existing work especially in the context of healthcare and assisted living applications. Surveys as in Refs. [46, 47] are specifically on the Wi-Fi-based approaches. However, we do not focus on the classification approaches of human activity, as there exist several in-depth literatures on human activity classification methods [48, 49, 50, 51].

The organization of the chapter is as follows: Section 2 “RF-based DFL Technology” briefly discusses the concept of DFL in the perspective of human activity monitoring as understood within this study and provide an extensive review on the existing works. We decompose the taxonomy of the existing RF-based DFL technologies for human activity monitoring into measurement-based categories, regardless the type of wireless radio technologies used. Section 3 “Opportunities and Potential” presents the potential applications based on the state-of-art of RF-based DFL technology. Based hereon, in the last Section 4 “Challenges, Open Issues and Future Directions,” we outline the challenges and the possible solutions, discuss the open issues, and comment on the possible future research direction of activity recognition using DFL technology.

2. RF-based DFL technology

Historically, the DFL analogy was firstly introduced by Youssef et al. in 2009 as device-free passive (DfP) for location determination, in which the subject is not equipped with a radio device, or not required to actively participate in the localization system [52]. The concept of DFL relies on the fact that any changes on the radio network environment will fluctuate the received signal profiles, i.e., due to reflection, diffraction, absorption, or scattering phenomena. DFL exploits the potential of ubiquitous deployed Internet of Things (IoT) [53] devices for indoor localization by leveraging the RF fluctuations as an indicator of presence of obstruction, i.e., object or human body. In [54], we have briefly defined the concept of DFL technology in the context of human detection and counting, together with the comprehensive review on the publications related to DFL research.

In correspondence to the tremendous progresses on DFL research, Scholz et al. have expanded the area of DFL technology in the context of activity recognition by introducing the concept of device-free RF-based for human activity monitoring as device-free radio-based activity recognition (DFAR) [55]. Instead of utilizing radio signal analysis for object detection and tracking, it can also be utilized in the DFL technology to recognize specific human movement, and even their activities and gesture. For instance, the fluctuations of ambient and local continuous signals have been exploited in detecting human daily activities such as walking, lying, crawling, or standing [56]. To ease the reader’s understanding, we defined the DFL and DFAM systems as:

DFL: device-free localization system: a system which detects the presence of a passive target and locates the target’s position using radio signal information while the target is not equipped with a wireless device, nor required to actively participate in the localization system.

DFAM: device-free activity monitoring system: a system which monitors and recognizes the activity performed by a passive target using radio signal information while the target is not equipped with a wireless device, nor required to actively participate in the localization system.

We illustrate the overall conceptual framework of RF-based DFL technology for human activity monitoring as in Figure 1, including the three important modules: wireless radio sensor network (WRSN), human detection (HD), and human monitoring (HM). The WRSN is a self-configured wireless network consisting of radio devices connected wirelessly, acting as the sensors, for monitoring and recording of the physical or environmental conditions, and organizing the collected information to a predefined central location for processing. The WRSN module works by detecting the availability of radio-embedded devices (sensors) for human presence detection, localization, and activity monitoring, as well as the deployment of the radio sensor networks. WRSN can be deployed in the real-world environments using any radio devices that utilize the similar technology or IEEE standard. For instance, Wi-Fi-based sensor network can be deployed using any devices that utilized the IEEE802.11 wireless local area network (WLAN), while ZigBee-based sensor network can be deployed using devices that utilized the IEEE802.15.4 wireless personal area network (WPAN).

Figure 1.

The overall conceptual framework of the RF-based DFL system.

The sensors in WRSN collect the information of current environment and forward the information to be processed by the HD module. HD module consists of detection and localization algorithms which analyze the information and automatically discover the presence of target, the number of target, the location of the target, the body temperature of the target, the activities performed by the target, the humidity of the environment, etc. HM module consists of activity classification algorithms connected with the designated context aware-based activity reasoning engines depending on the applications. Once an activity is detected, the HM module will observe, retrieve, and recognize the activities and alert the designated context aware-based activity reasoning engine to interpret ongoing events successfully or initiate actions as needed. For example, a ZigBee-based sensor network is deployed in a single bedroom apartment for elderly care application as depicted in Figure 2.

Figure 2.

ZigBee-based sensor network deployed for elderly care application with the integration of mobile apps visualization.

Following on the DFAM research in [55], many research works on human motion detection and activity monitoring have been presented utilizing different radio technologies such as RFID [35, 36, 37], WiFi [5, 19, 20], ZigBee [21, 22], FM radio [23, 24], microwave [25], etc., adopting different signal descriptors such as Receive Signal Strength (RSS) [57, 58, 59, 60, 61], Channel State Information (CSI) [5, 20, 62, 63, 64], Doppler effect [25, 65], and Packet Received Rate (PRR) [66], without neglecting the easy-of-use problem and physical discomfort issue. In the following subsection, we decompose the taxonomy of the existing RF-based DFL technologies for human activity monitoring into signal descriptor categories such as RSS-based, CSI-based, amplitude-based, Doppler-based, and PRR-based, regardless the type of wireless radio technologies used.

2.1. RSS-based

Similar to human presence detection, activity monitoring using RSSI-based DFL technology exploits the RF-signal fluctuation features, in which the components of the received signal are blocked, absorbed, and reflected by the human while performing an activity, inducing the RF signal in the vicinity of receivers into a specific characteristic pattern. Such pattern can be identified and classified for the corresponding activity by exploiting the changes on the RSS of the affected wireless links.

In [57], Sigg et al. introduced three types of RF-based DFAR systems: active continuous signal-based, active RSSI-based, and passive continuous signal-based DFAR; which exploit the fluctuation of RSS due to human movement and activities. Both active and passive continuous signal-based proposed are USRP Software Defined Radio (SDR)-based system, which are deployed using specialized SDR devices. Meanwhile the RSSI-based DFAR system utilized the 2.4 GHz INGA sensor nodes [57]. The performance accuracy of the proposed DFAR systems is then compared with the performance accuracy of the motion-based recognition system. In the motion-based recognition system, accelerometers are attached to the subjects while performing the activities. By implementing three well-known classifier algorithms that are Naive Bayes, Classification Tree, and k-nearest neighbor (k-NN), their proposed RF-based DFARs are able to achieve comparable results with the motion-based system. Furthermore, they evaluated the performance of the proposed RF-based DFAR system in the presence of multiple subjects performing different activities and the impact of increasing the number of receiving devices. However, the proposed systems required specialized SDR devices, where the hardware availability remains as an open issue [60].

Sigg et al. expanded their work by designing an RSS-based activity recognition system for the mobile phones [58, 59] based on the advantages of mobile phones as personal devices that often carried everywhere. The proposed system utilized the Wi-Fi-RSSI values of incoming packets at a mobile phone for the activities classification. Unlike other body-worn devices, the function of mobile phone in an RSS-based activity monitoring system remains feasible even when it is not carried by the user. By default, the firmware and operating system (OS) of a standard mobile phone do not provide privilege for user to access its hardware as well as desired RSSI information. Thus, work in [58] utilized a modified firmware, which allows mobile phone to run Wi-Fi interface in monitor mode and developed tools to process RSSI sample captured on mobile phone in monitoring simple human activities such as walking and phone handling. Meanwhile, work in [59] focused on recognizing 10 different single-handed gestures utilizing the same modified firmware and tools developed in [58] with average accuracy of 0.51 when distinguishing all gestures and is able to achieve average accuracy of 0.60 and 0.72 when reducing to 7 and 4 gestures, respectively. Unfortunately, the OS root access incompatibility, complicated firmware modifications, and low accuracy are the major issues in the real-world applications.

The proposed RF-based DFAR systems in [57, 58, 59] utilized the RSS features as per listed in Table 2 and several combinations of those features for the activities classification. Assume that a wireless network environment consists of a static transmitter node or access point (AP), and a static receiver node or monitoring point (MP). Let $r_i\left(t\right)$ denote the RSS of sample $i$ at time $t$. Assume that $\left|R_t\right|$ samples of $r_i\left(t\right)$ are captured on a received signal for a sample window, $R_t=r_1\left(t\right),\dots ,r_{\left|R_t\right|}\left(t\right)$, the RSS features of the samples are defined in Table 2.

Feature	Description	Definition
Mean	Represents the static changes in RSS; Provides means to distinguish a presence of static person as well as the exact location	$\mathit{\text{Mean}}\left(R_t\right)=\frac{{\sum }_{r_i\in R_t}{r}_i}{\left|R_t\right|}$
Variance	Represents the volatility of RSS; Provides the estimation on changes in nearby receivers such as movement of person	$\mathit{Var}\left(R_t\right)=\sqrt{\frac{{\sum }_{r_i\in R_t}{\left(r_i-\mathit{\text{Mean}}\left(R_t\right)\right)}^2}{\left|R_t\right|}}$
Standard deviation (SD)	Can be used instead of the variance; The interpretation of SD and variance is identical	$\mathit{Std}\left(R_t\right)=\sqrt{\mathit{Var}\left(R_t\right)}$
Median	Represents static changes in RSS.; More robust to noise than the mean; Let the ordered set of samples; $R_{t,\mathit{ord}}={\overline{r}}_1,\dots ,{\overline{r}}_{\left|R_t\right|};i<j⟹r_i\le r_j$	$\mathit{Med}\left(R_t\right)=r_{⌈|R_{t,\mathit{ord}}|/2⌉}$
Normalized spectral energy	Represents a measure in the frequency domain of the RSS; Can be used to capture periodic patterns such as walking, running, or cycling	$E_i=\sum _{k=1}^n{P}_i{\left(k\right)}^2$
Minimum and maximum	Both represent extremal signal peaks; Can be used to estimate movement and any changes in environment	$\mathit{Min}\left(R_t\right)=r_i\in R_t\mathit{\text{with}} \forall r_j\in R_t:r_i\le r_j$; $\mathit{Max}\left(R_t\right)=r_i\in R_t\mathit{\text{with}} \forall r_j\in R_t:r_i\ge r_j$
Signal peaks within 10% of a maximum	Reflections of the obstructed signal strength at a receive antenna; Peaks of similar magnitude indicate that movement is farther away; Can be used to indicate near-far relations and activity of individuals	$h\left(r_i\right)=\left\{\begin{array}{l}1\mathit{\text{if}}{r}_i\ge \text{max}\left(r_1,\dots ,r_{\left|R_t\right|}\right)·0.9\\ 0\mathit{\text{else}}\end{array}\right.$; ${\mathit{max}}_{0.9}\left(r_i\right)={\displaystyle \sum _{r_i\in R_t}}h\left(r_i\right)$
Mean difference between subsequent maxima	Similar magnitude of maximum peaks within a sample window indicates low activity in an environment or static activities; The opposite will indicate dynamic activities	$R_{\mathit{max}}\left(R_t\right)=\left\{r_i|r_i\in R_t,r_{i-1}<r_i\bigwedge r_i>r_{i+1}\right\}$; $a\left(R_t\right)={\displaystyle \sum _{\begin{array}{c}\forall r_i,r_j\in R_{\mathit{max}}\left(R_t\right);\\ i<j;\\ \nexists r_k \mathit{\text{with}}i<k<j\end{array}}}\frac{\left|r_i-r_j\right|}{R_{\mathit{max}}\left(R_t\right)}$

Table 2.

Several features considered for RF-based DFAR [57, 58, 59].

Since works in [58, 59] focused more on hand gestures, Gu et al. [60] proposed an online Wi-Fi RSSI fingerprint-based DFAM concentrated on human activity, which has a flexible architecture and can be integrated in any existing indoor WLANs, regardless the environment conditions. Based on the preliminary results of the human activities impact on the Wi-Fi characteristic study [60], the Wi-Fi RSSI fingerprint can be extracted and exploited to distinguish different activities since each activity has their own RSSI fluctuation patterns. To reduce the difficulties in distinguishing activities having similar RSSI footprints, such as sitting and standing, the proposed system adopted a novel fusion classification tree-based algorithm. The system has been evaluated through extensive real-world experiments based on six main activities (that are sleeping, sitting, standing, walking, falling, and running) and achieved average accuracy of 72.47% for all activities, thus outperforms Naive Bayes, Bagging, and k-NN classifiers.

Monitoring human activity using RFID technology is often associated with the physical discomfort issues as user needs to wear or carry the RFID devices. However, there exist several studies that implemented the RFID technology in the different way for the device-free activity monitoring [61, 67]. Instead, the RFID devices are attached to the walls, furniture, and daily objects. This approach is known as passive RFID-based DFL. Thanks to the rapid advancement and sophistication in cheap sensing and wireless technology for introducing various RF-embedded devices with an open-source platform such as TelosB [68], IRIS [69], Waspmote [70], etc., that can operate in real-time environment based on the “plug and sense” concept where information like RSS can easily be captured. However, RSS measurements suffer from high uncertainties since the signal profiles tend to fluctuate depending on the environment, thus unpredictably experience interference, complex multipath propagation, and being noise-sensitive. In addition, RSS-based system experiences accuracy and coverage limitation due to the lack of the frequency diversity. Thus, RSS-based approach is only suitable for coarse-grained human activity monitoring.

2.2. CSI-based

Most of the research on Wi-Fi-based DFL utilized the CSI, one of the Wi-Fi features extracted from the physical layer of radio wireless system, for indoor location estimation and human motion and activities monitoring due to its stability and robustness in complex environment compared to RSSI. CSI information are available in commercial wireless devices such as network interface controller (NIC), which is also known as network interface card, network adapter, LAN adapter, or physical network interface. Unlike RSSI value which is usually measured from one packet, CSI value is measured per orthogonal frequency-division multiplexing (OFDM) from each packet and uses the frequency diversity technique to reflect the multipath propagation signals caused by human motion and activity, thus making it suitable for monitoring the fine-grained signals of human activities and motions.

Based on [19, 63], consider a Wi-Fi-based DFL system with NICs continuously measure the CSI variations in every received Wi-Fi frame of multiple wireless channels. Let $\mathit{NTx}$ and $\mathit{NRx}$ represent the number of transmitting and receiving antennas, respectively. Assume that at time $t$, the frequency domain representation of transmitted and received signals with carrier frequency $f$ is denoted as $X\left(f,t\right)$ and $Y\left(f,t\right)$, respectively. The relationship of the transmitted and received signals can be expressed as:

$$Y\left(f,t\right)=H\left(f,t\right)\times X\left(f,t\right)$$E1

where $H\left(f,t\right)$ represents the complex-valued channel frequency response (CFR) of the same carrier frequency $f$ and time $t$. Based on Eq. (1), the CFR depends on the received signal $Y\left(f,t\right)$ with noise channel, where the noise channel captured in the measured $H\left(f,t\right)$ can be expressed as:

$$H\left(f,t\right)=\frac{Y\left(f,t\right)}{X\left(f,t\right)}$$E2

The CFR values consist of S metrices of CSI measurement with dimension of $\mathit{NTx}\times \mathit{NRx}$, where S represents the number of OFDM subcarriers. Each CSI matrix represents the CFR value of one received Wi-Fi frame between a pair of $\mathit{Tx}-\mathit{Rx}$ antennas at a particular OFDM subcarrier frequency and time. CSI are usually measured at S = 30 subcarriers and starting from here onward, the time series of CFR values of a particular OFDM subcarrier for a given antenna pair is denoted as CSI stream. For instant, at S = 30, there are 30 × NTx × NRx CSI streams in a time series of CSI values.

Since the radio signals travel from a transmitter to a receiver through multiple paths depending on the surrounding, the measured $H\left(f,t\right)$ of a received signal through K different paths can be expressed as:

$$H\left(f,t\right)=e^{-j2\pi ∆\mathit{\text{ft}}}\prod _{k=1}^K{a}_k\left(f,t\right)e^{-j2\mathit{\pi f}{\tau }_k\left(t\right)}$$E3

where $a_k\left(f,t\right)$ is the complex-valued representation for both attenuation and initial phase offsets of the $k$-th path, $e^{-j2\pi ∆f{\tau }_k\left(t\right)}$ is the phase shift on the $k$-th path with the propagation delay of ${\tau }_k\left(t\right)$, and $e^{-j2\pi ∆\mathit{\text{ft}}}$ is phase shift caused by the carrier frequency offset (CFO) with frequency difference ∆f between the sender and the receiver. Any changes in the length of particular path will affect the phase of the Wi-Fi signal travel on that corresponding path.

Figure 3 shows the scenario where Wi-Fi signals transmitted from an AP (Tx) to an MP (Rx) are traveled through different paths, which are the line-of-sight (LoS) path and paths reflected by wall and human body. Let the path of reflected signal due to human body is the $k$-th path. When the human body moves by a small distance $d$ between time interval 0 and $t$, the $k$-th path length changes from $d_k\left(0\right)$ to $d_k\left(t\right)$. Based on the fact that the radio signal travels at the speed of light, the propagation delay ${\tau }_k\left(t\right)$ experienced by the $k$-th path can be written as:

$${\tau }_k\left(t\right)=\frac{d_k\left(t\right)}{c}$$E4

Figure 3.

Multipaths scenario experienced by the Wi-Fi signals caused by human movement.

where $c$ is the speed of light, which is related to the carrier frequency $f$ and wavelength $\lambda$ based on the function $\lambda =c/f$. Thus, the phase shift $e^{-j2\pi {\mathit{f\tau }}_k\left(t\right)}$ of the $k$-th path can be written as $e^{-j2\pi d_k\left(t\right)/\lambda }$, which describes the relationship of the changes in path length by one wavelength with the changes of phase shift of $2\pi$ at the receiver subcarrier signal of the given path.

The phase of each path can be precisely measured only if the transmitter is in synchronization with the receiver. Unfortunately, due to hardware limitation and environment variations, the CFO of the commercial Wi-Fi devices, denoted as ∆f in Eq. (3), cannot be ignored. The impact of CFOs of devices running on IEEE 802.11n standard causes random variation in the phase of CSI, which allows devices to continuously transmit Wi-Fi frames based on frame aggregation mechanism, thus creating a phase interference scenario. It is difficult to precisely measure even the small phase shift in $e^{-j2\pi {\mathit{f\tau }}_k\left(t\right)}$ under this interference scenario.

To ignore the phase interference introduced by CFO, Wang et al. [63] introduced a CSI-speed model into their activity recognition and monitoring system (CARM), which considers the relationship of CFR power variations instead of CFR phase variation to the human movement speeds. Since the CSI streams of human movements are correlated, it is hard to extract the real trend of CSI caused by the human movement for feature classification purpose. Therefore, works in [5, 19, 63] applied the principal component analysis (PCA) to discover the principal component of the CSI fluctuation pattern caused by human activity motion to be used as features for activity classification. In [5], Li et al. analyzed five features from CSI principal component which are normalized standard deviation (STD), median absolute deviation (MAD), interquartile range (IR), signal entropy, and duration of human motion to recognize seven different human daily activities. By applying random forest-based classification algorithm, work in [5] verified the validity of their proposed human monitoring system in both the LoS and Non-LOS (NLoS) scenarios as 95.43% and 91.4%, respectively. Meanwhile, activity monitoring system based on hidden Markov model (HMM) classifier algorithm proposed by Wang et al. [19, 63] achieved an average recognition accuracy of 96%.

Although undesired noise from the environment may disturb some of the streams, since CSI is measured using OFDM method, other streams which are not affected by the noise still can provide the real trend of CSI information. Since CSI contains more information than RSSI, it is suitable for fine-grained activity monitoring. However, unlike RSSI which is available in almost all wireless devices, CSI only can be obtained from devices with specific NIC cards such as Intel 5300 and Atheros 9390 [19].

2.3. Doppler-based

When wave such as ultrasonic and radio wave is transmitted to moving target, the wavelength of the reflected wave shifts depending on the direction and velocity of the movement. This is known as Doppler effect or Doppler shift. Recently, the principle of the Doppler effect has been proposed by researches in device-free radio sensor network for human activity monitoring and data gathering of real-world environment [25, 65] since the Doppler-based technology has the ability to accurately detect movement and eliminate the stationary noise of the environment [66]. The same principle of Doppler effect is applied to a Doppler sensor, having a beat signal as an output, in which frequency is defined as the difference between transmitted and received waves. Due to its high detection accuracy, work in [25] has deployed a 24-GHz microwave-Doppler sensor for a device-free activity monitoring system to recognize the daily activity of three passive targets with an average recognition rate of 90.6% based on eight different activities.

Based on the Doppler possibility study in [25], assume that a radio wave source at a fixed position transmits a radio wave with frequency $f_t$ and velocity $c$. An object is moving relatively to the source with velocity $\pm v$. The received wave with frequency $f_r$ can be defined as:

$$f_r=f_t\pm f_D$$E5

where $f_D$ is the Doppler frequency, which is defined as the difference between frequency of transmitted and received waves. The value of $f_D$ is higher when the object moves toward the source, and lower when the object moves away from the source. Thus, the calculation of $f_D$ can be simplified as:

$$f_D=\mid f_r-f_t\mid =f_t\left(\frac{c+v}{c-v}-1\right)=\frac{2v}{c-v}{f}_t\simeq \frac{2v}{c}{f}_t.$$E6

Let the signal of the transmitted wave $S_t$ with amplitude $A_t$ at time $t$ is defined as:

$$S_t\left(\mathrm{t}\right)=A_t\text{sin}\left(2\pi f_t\right)$$E7

and $∆t$ is the time delay between the transmitted and received signal. The received signal $S_r$ with amplitude $A_r$ at time $t$ is defined as:

$$S_r\left(\mathrm{t}\right)=A_r\text{sin}\left(2\pi (f_t\pm f_D\right)t-2\pi f_t∆t)$$E8

From Eq. (8), the received signal depends on the object size and its distance from the source. The beat signal $S_D$ with amplitude $A_D$ at time $\mathrm{t}$ is then observed to be the output signal of the Doppler source as:

$$S_D\left(\mathrm{t}\right)=A_D\text{sin}\left(2\pi f_{D}t-2\pi f_t∆t\right).$$E9

From Eq. (9), the amplitude and frequency of the Doppler shift are highly correlated with the range of the object and its motion speed. Thus, any human movement and activities with different speeds will have different Doppler shifts. Those human activities can be estimated and analyzed by extracting the features of Doppler signature in the frequency and time domains.

Work in [65] proposed an in-home Wi-Fi signal-based activity recognition framework for e-healthcare applications utilizing the passive micro-Doppler (m-D) signature classification. A fast Fourier transform (FFT) was used on the cross-correlation product of the baseline and monitored signals to find the exact delay $∆t$ and frequency shift $f_D$ of the strongest signal. This was defined as cross ambiguity function (CAF) and the equation was represented as follows:

$$\mathit{CAF}\left(∆t,f_D\right)={\int }_{-\infty }^{+\infty }{e}^{-j2\pi f_{D}t}{S_B}^H\left(t-∆t\right)\times S_M\left(t\right)\mathit{dt}.$$E10

where $S_B\left(t\right)$ and $S_M\left(t\right)$ are the baseline and monitoring signals, respectively. The m-D signature of an activity at a specific time $t$ is defined as the frequency vector $\widehat{f_D}$ induced by the passive target movement at a specific delay $∆t$. All the recorded Doppler signatures are then concatenated together in a time line history of Doppler signature for the database construction.

Although the constant false alarm rate (CFAR) detection is not suitable for the indoor environment due to the ambiguity peaks and direct signal interference (DSI) problems [65], DSI is an important feature in Doppler-based as it can be used to distinguish different signatures. Instead, a weighted standard deviation is proposed as the indicator to detect the m-D signature without eliminating the ambiguity peaks and DSI. PCA can be applied to reduce the dimension of dataset and eliminate the undesired noise. Finally, the Doppler signature is classified using a sparse representation classifier (SRC) with subspace pursuit (SP) technique, which outperforms the well-known support vector machine (SVM) in terms of classification accuracy and coverage. The sparsity level in SRC can easily be controlled and adjusted, thus making the proposed activity recognition framework a feasible tool, which is very suitable for the real-time healthcare applications, especially for the new users since it is not required to re-training the system.

2.4. PRR-based

It has been proved that RF signal features extracted from RSS and CSI information discussed in Sections 2.1 and 2.2 can be used to distinguish the type of movement as well as recognize the activities performed. However, RSS is sensitive to the shadowing effect and experiences the complex multipath propagation behavior, which makes it only suitable for monitoring coarse-grained activity. Meanwhile, CSI, which provides powerful information suitable for fine-grained activity monitoring, faces hardware issues since the information is only available from NIC embedded devices.

In [66], Huang and Dai presented a novel PRR-based DFL system for human movement recognition under the NLoS scenario based on packet state characteristic from link state information (LSI). The LSI, which contains more physical information such as RSSI, packet delivery rate, packet state, packet delay, packet loss, time arrival, and time interval of the received packet, etc., can be accessed from the network layer. Human movement in the radio network environment will block or reflect the signal and cause significant changes on the signal propagation path. This results in the fluctuation of channel link quality as well as slow fading effect.

By exploring the LSI features such as packet state and packet arrival time, different activities performed by a person in the monitoring area can be identified. Work in [66] exploited the PRR measurement to identify the link state. Assume the $i$-th window of size $w_i$ at fixed interval $L$. The packet state is denoted as $s\left(\mathrm{i}\right),$ and labeled as “1” if the packet is successfully arrived with no error and “0” if the packet is lost or contains an error. The PRR $P\left(W_i\right)$ of the link state is defined as the proportion of successfully arrived packets among all transmitted packets and can be expressed as:

$$P\left(W_i\right)=\frac{1}{w_i}\sum _{j=L\times i}^{L\times i+w_i}s\left(j\right).$$E11

Consider a wireless network environment in a hallway consists of a transmitter Tx and a receiver Rx as shown in Figure 4. When a person is moving into the hallway area, there will be four possible trajectories: walking from Tx to Rx, walking from Rx to Tx, walking from Tx to Tx, and from Rx to Rx. When the person moves into the hallway area, the link state quality tends to fluctuate in terms of the PRR. Different moving trajectories in the hallway will generate different fluctuation patterns of PRR with respect to the person position in hallway, thus the direction of walking can be identified. The distance of moving traces with different trajectories can be calculated using the Euclidean distance equation as in (12) and the walking direction of the traces can be identified using the K nearest neighbors (KNN) algorithm. The Euclidean distance between the PRR of the testing trace $P_i$ and training trace ${\overline{P}}_i$ can be calculated as:

$$E_d=\sqrt{{\sum }_{i=1}^n{\left(P_i-{\overline{P}}_i\right)}^2}.$$E12

Figure 4.

Node deployment in the hallway area.

Since PRR cannot be used to distinguish the speed of the movement, other link state information known as the received packet arrival time is used to measure the speed. However, the time interval of received packets is highly correlated with the moving speed. Therefore, several parameters, such as autocorrelation function $\mathit{acf}$, Budget Rate $R_B$, and expected total latency (ETL), can be applied to the link state-related information in order to classify the different speed [66]. The proposed PRR-based approach introduced in [66] is able to achieve a high accuracy of 95% in recognizing four different movement directions and 44% improvement on the average accuracy in classifying four different speeds compared to the RSSI-based approach.

3. Opportunities and potential applications

DFL for human and activity monitoring is the promising technology for collecting data about the human presence and activity patterns. The technology is much cheaper than the existing traditional monitoring system using video camera. It consists of radio nodes comprising the appropriate sensor array along with computational devices that transmit and receive data wirelessly, and capable of providing information on an unprecedented temporal and spatial scale. The DFL system is an easy-to-install motion tracking system developed based on the IoT to improve the quality of life as well as provide intelligence and comforts to the user especially the disabled. Users, especially family, can respectfully monitor their loved ones who live alone at home, without requiring them to wear devices or change their habits. The system can be integrated with mobile and web apps which allow user to easily monitor their home/office from anywhere, in real time. The system can be made to replace the existing RFID monitoring system which always raises physical discomfort and is less reliable since more than one tag can respond at the same time.

3.1. Remote home healthcare services

In recent years, there has been an increase in the number of patient admission in hospitals worldwide, whether federal or nonfederal, especially in the developed countries due to the increase of older and disabled population [71, 72]. In England, for instance, the older population (aged 65–69) has grown by 34% in 2016 after a decade from 2.2 million in 2006, together with the series of increasing hospital admissions by 57% from 0.8 million, over the same period of time [71]. This causes most hospitals to experience inadequate bed problem to admit patients, thus slowing down the work of medical staff, especially at the casualty department or emergency department (ED). Patients started to complain about the slow services, which lead to bad reputation of the hospital. By implementing DFL system, federal and nonfederal bodies can introduce remote home healthcare services where patients can be monitored and advised from anywhere. These services help patients to improve their function and live with greater independence. Using this system, existing patients are taught to manage their wellness level, and safely manage their medication regimens; meanwhile, medical staff can remotely monitor and estimate the health condition of patients by interpreting the patients’ daily routines. In this situation, patients will remain at home, avoiding hospitalization or admission to long-term care institutions. If the daily routine of a patient is abnormal as expected such as too long sleeping or resting in bed, the patient might be sick and should be visited soon for closer examination.

3.2. Ambient assisted living tool

Recent advances in medicine allow people to live longer and healthier compared to the previous generations, which lead to an increase in the number of elder people. Aging brings many challenges to them due to cognitive decline, chronic age-related diseases, as well as limitations in physical activity, vision, and hearing. With an increase in age-related diseases, there will also be a rise in individuals unable to live independently. However, due to the higher standards of living, children nowadays are too busy working to earn money for living and have no time to care for their parents. This leads to an increase in the number of elderly people in the federal- and nonfederal-owned welfare or nursing home; meanwhile, there will be a shortage of professionals trained or care-giver to work with the aging population. Given the fact that most of the elderly people prefer to stay in the comfort of their own homes, and given the costs of private nursing home care, it is imperative to develop technologies that help elderly people to age in place. By implementing the DFL technology as an ambient assisted living tool, family can respectfully monitor their loved ones who live alone at home, without requiring them to wear devices or change their habits. The DFL system can be integrated with mobile and web apps which allow user to easily monitor their home or office from anywhere, in real time. This advantage makes DFL technology very suitable for monitoring persons’ activities (especially the elderly, disable people, and patient suffering from Alzheimer’s disease) without causing them physical discomfort with the wearable devices or sensors. In addition, it is a challenge for them to remember each day to wear or to activate those devices.

3.3. Smart buildings for home and office

Automatic and monitoring control in “smart” building, i.e., for home or office, was developed based on the IoT and WSN technologies to improve the quality of life as well as provide intelligence and comforts to the user especially the disabled. The DFL technology can be expanded not only for monitoring purposes, but also as an application server that can control and initiate actions as needed. For example, in an office building where few people are working together, the proposed DFL technology can enhance the existing lighting, heating, and air conditioning system by providing information of current environment such as the presence of people, the number of people as well as their location, the body temperature of the occupants, the activity performed by the occupants, the humidity of the environment, etc. If there are too many electronic devices in use or too many occupants in the office which leads to an increased temperature of the room, the building heating system can be adjusted or automatically lowered based on the information provided by the DFL system. If there is no people presence in several areas inside the office especially during lunch break, the lights and air-conditioning in those areas can be automatically switched off. Government as well as private bodies can implement this technology in all their buildings which definitely will reduce the utility cost.

4. Challenges, open issues, and future direction

In the recent years, the RF-based DFL approach has made tremendous achievements and becomes a popular research topic in localization and activity monitoring area. In the previous section, we have provided a review of existing human activity monitoring system based on different approaches. However, there are still significant challenges and open issues worth exploring and require further in-depth research. Moreover, the performance of existing systems can be further optimized, improved, and extended. In this section, we present a list of challenges and open issues together with the possible future research directions to be addressed by the researchers.

Unstructured approaches: Although several theories and models have been presented in the existing human activity monitoring researches, still, there is no general technique, methodological approach, and framework to DFAM. Most of the existing research focused on a particular application and specific technology based on empirical observation result. This topic requires more complete theoretical model as well as its general technique or framework for RF-based activity recognition. Additional in-depth research on the characteristic of human body which relates to the radio signal based on human body model is required. One of our research directions is to deploy the representation learning method such as deep learning method with high recognition rate. Unlike the traditional machine learning algorithm, which required manual feature selection as well as rules definition, deep learning approach is able to learn the correct auto-generated features and accurately predict the correct feature. In addition, although it is a challenging task to implement unsupervised learning technique into the DFL human activity monitoring system, it is worth to be explored.

Inconsistency and unreliability of the sensors: Most of the presented studies used common sensor such as RFID, MEMSIC motes, and other RF-devices used the continuous sensing technique, where the sensor will continuously collect sample for activity recognition. However, this continuous sensing technique is challenging since it depends on the battery lifetime in order to be consistently operated. In addition, various continuous sequences of human performing activity with several periodic variations may result in wrong activity prediction. In order to continuously monitor the human activity without disrupting the system, energy-efficient mechanisms can be implemented to the sensor, where under certain conditions, the sensors can be turned off or put under sleep mode. This technique is known as duty-cycling technique. Only selected sensors are active for sampling at specific state, whereas other inactive sensors are under sleep mode, waiting for any possible state transitions. It has been proven that by using this duty-cycling technique, the battery lifetime can be improved by 75%; however, the recognition latency will increase. Additional research work is needed to compare the performance of activity recognition system in terms of hardware or devices diversity. Some of sensors or devices can be used together for a complete human activity recognition.

Hardware, maintenance, and labor costs: For larger-scale environment, especially for system with finger-printing approaches, the deployment of hundreds of sensors in a building of multiple rooms will obtain good detection accuracy. However, deploying such high density of sensors will affect the overall energy consumption of the building and demand for additional installation, maintenance, and labor costs. If these sensors are powered up using battery, the maintenance and energy requirements should be taken into consideration for the long-term deployment. Thus, it is required to select the best hardware, feature extraction approach, and classification technique to be deployed in the system without the requirement of additional cost.

Noise and interference from other appliances: Recognizing activities in noisy environment, that is, communication channel consisting noisy channel and interferences from other devices running on the same channel, is quite challenging. With the presence of noise and interference due to the inherent volatility of wireless signal, the activity recognition becomes less accurate. Most of the studies are evaluated under controlled condition or laboratory environment, which allowed selected devices to be present in the monitoring environment. Some of the research implementation of frequency diversity technique may help to increase the system accuracy. However, frequency diversity technique is not suitable for indoor environment due to interference from nearby wireless devices. In order to develop the DFL system in the practical or real-world environment, the activity recognition algorithm should be performed in the large-scale area which consists not only the required devices, but other devices running on the same communication channel. Thus, further in-depth research on the noise elimination technique is required to effectively remove the noise of different sources in the radio channel.

Offline classification and training: Most of the presented studies proposed the offline activity classification methods where the data collected by the receiver are being trained and classified offline by the application server. The system performances presented in those studies are based on the offline recognition. In order to build reliable real-world activity monitoring applications, the activity classification and system performance should be performed and evaluated online on the application server. Offline classification process is suitable for application which does not require online recognition such as monitoring daily routine of a person. In this scenario, the data of the daily routine can be collected and stored into the application server and can be processed offline. However, online classification and recognition are required for those applications that interested in specific human activity, duration and sequence of activity, such as fitness coaching, fall detection and remote healthcare. In an ideal and reliable system, the system performance and classifier accuracy can always be improved and optimized as long as the system continuously collects enough data. This will make the system benefit to human centric applications.

Recognizing complex activity: Most of the presented studies focus on the coarse-grained or human basic activities such as walking, running, standing, falling, etc. However, the patterns of these activities are not strong enough to be directly linked to the more complex or fine-grained activities. Human behavior is spontaneous, and they tend to perform multiple tasks at the same time which introduce confusion in the activity recognition process, and sometimes may result in incorrect classification. For instance, it is rather straightforward to detect if the user is lying down on couch but inferring if the user is sleeping or watching television, or fainted is different. Although there exist several attempts in addressing this issue, further research is needed in exploring the information collection of the complex activities recognition and mapping for human-centric application domains, especially in persuasive applications for a behavior or lifestyle change.

Recognizing multiuser activities: It is noticeable that most of the presented studies focus on recognizing activity of a person. In fact, the real-world applications usually involve multiple user presence in environment such as people walking together, queuing in a line, watching television together, family dinner, etc.; however, none of the presented methods are applicable for the situation. This open issue should be further investigated for different application domains.

5. Conclusion

In this chapter, we provided an extensive review on human activity recognition using RF-based DFL technology, targeting human-centric applications such as healthcare, well-being, and assisted living applications. We provided the details information on concept of DFL and DFAM, together with the feature selection approaches based on different signal descriptors and the potential applications. We presented an extensive review on the existing and on-going works qualitatively and discussed on the challenges, limitations, and future research directions relevant to this field. We believe that this DFL technology has great potential in the future, which can benefit humans and will be one of the key areas of research that worth to be explored.