Piezoelectric Sensors Used for Daily Life Monitoring

Abstract

This chapter presents an unrestrained and predictive sensor system to analyze human behavior patterns, especially in a case that occurs when a patient leaves a bed. Our developed prototype system comprises three sensors: a pad sensor, a pillow sensor, and a bolt sensor. A triaxial accelerometer is used for the pillow sensor, and piezoelectric elements are used for the pad sensors and the bolt sensor that were installed under a bed mat and a bed handrail, respectively. The noteworthy features of these sensors are their easy installation, low cost, high reliability, and toughness. We developed a machine-learning-based method to recognize bed-leaving behavior patterns obtained from sensor signals. Our prototype system was evaluated by the examination with 10 subjects in an environment representing a clinical site. The experimentally obtained result revealed that the mean recognition accuracy for seven behavior patterns was 75.5%. Particularly, the recognition accuracies for longitudinal sitting, terminal sitting, and left the bed were 83.3, 98.3, and 95.0%, respectively. However, falsely recognized patterns remained inside of respective behavior categories of sleeping and sitting. Our prototype system is applicable and used for an actual environment as a novel sensor system without restraint for patients.

1. Introduction

Along with the longevity in our society, labor shortages will be severe, especially at hospitals, nursing homes, and nursing care facilities [1]. Currently, few caretakers care for numerous elderly patients. For this situation, nurses and caretakers should monitor them inadequately, especially during sleep at night [2]. As related to this situation, Mitadera and Akazawa addressed that approximately half of falling or tumbling accidents are among elderly patients [3]. Numerous accidents occurred when elderly patients leave their bed and these accidents occurred in a place where nurses and caretakers are hard to keep their eye on. Statistically, the tumbling and falling accident rates were 83.3% and 85.4%, respectively, when they had no support at an accidental moment. For protecting it, suitable provision is necessary after conducting assessments for respective elderly patients. Actually, one approach is to use bed-leaving sensors that signal when they leave from their beds. The number of hospitals and nursing care facilities using these sensors, which can be used to prevent falling from their beds, has increased recently [4, 5].

As a simple sensor system attached to a patient’s clothing [6], clip sensors are the lowest-cost sensors that can be introduced easily. However, the usage of clip sensors has been prevented because of the necessity of wearer constraint, especially for the protection of human rights. Moreover, malfunctions and anomaly detections occur frequently for clip sensors because of the binary output used to detect bed-leaving behaviors. Regarding the reliability and perspective of management, clip sensors are insufficient to prevent falling from a bed completely. Furthermore, accidents caused by cable binding to the neck have been reported as clinical serious accidents [6]. Therefore, clip sensors are regarded as inadequate for using clinical sites, although it is easy to introduce them as a cost-effective solution.

Mat sensors are also widely used as a cost-effective and convenient sensor that can be installed easily [7]. Using mat sensors, Haruyama et al. developed an alarm system to detect patients leaving from their beds [8]. Medical and welfare suppliers have already released numerous models and various types of consumer used mat sensors installed on a floor, a bed, or on rolling handrails. For authentication, mat sensors used on a floor are unnecessary used for medical devices under the pharmaceutical law. Although mat sensors are easy to produce and to sell, a delayed problem remains as detection accuracy because of the response after sitting at the end of the bed. Moreover, sensor responses are apparent when medical staff members such as nurses or physicians walk on. Therefore, it is a challenging task to distinguish the responses of patients and medical personnel for signal pattern recognition.

The sensors that are rolled over handrails not only obstruct a view of a bed, but also present a removal risk when a patient finds it and feels negatively about being restrained. Moreover, false detection occurs when patients leave their bed without gripping a sensor installed handrail. Mat sensors installed on a bed under a sheet can detect bed-leaving behavior patterns with superior reliability to mat sensors of other types. Nevertheless, existing mat sensors are actuated by a binary response similar to that for clip sensors. Early detection is not feasible, especially in the initial bed-leaving behavior.

Using a camera as a bed monitoring sensor system provides a cost-effective solution combined with state-of-the-art computer vision technologies and algorithms that can use as open source software. Although camera can obtain much information for a monitoring subject, it is a challenging task to predict behavior patterns obtained from images as pixel signal information. It is impossible to monitor numerous subjects simultaneously and is unrealistic for a few operators or medical staff members if they observe images directly. Moreover, they must consider the aspect of human rights and quality of life (QoL). Because behavior patterns differ among people [9], it is impossible to recognize bed-leaving behavior patterns using only sensor responses with detailed analyses. Moreover, vision-based monitoring using a camera imposes a mental load on patients because they feel as though they are under surveillance all daytime and nighttime.

For solving abovementioned problems of various types, this chapter presents an unrestrained and cost-effective sensor system regarding QoL. The proposed system can analyze predictive behavior patterns that occur when a patient leaves from their bed. A machine-learning-based method was developed to obviate the setting of thresholds in advance. Moreover, an integrated system was developed for sending output signals obtained from sensors to a monitoring computer using a close-range wireless module. For this chapter, a novel multiple-sensor system is proposed to improve accuracy according to a usage of a subject. The amount of data was reduced to use predicting and minimizing incorrect recognition which is given to a minimum number of sensors. Our sensor system was evaluated in an environment that represents a clinical site. Experimentally obtained results demonstrated that our method predicted bed-leaving behavior patterns of seven types, especially for longitudinal sitting and terminal sitting.

2. Related studies

Hatsukari et al. developed a sensor system to detect bed-leaving three major patterns using strain gauges installed inside of actuators to obtain weight changes of a person on a bed [15]. For their system, three actuators are used in a bed. Each actuator has four biaxial strain gauges installed on diagonal lines. This is the most popular bed-leaving sensor system in practical use currently. As a new product of their company Paramount Bed Co. Ltd., they embedded sensors and a controller to their beds of the highest models. The detection accuracies for longitudinal sitting and terminal sitting were 87.0 and 98.1%, respectively [15]. This performance measure addressed that this system has both high performance and reliability. However, an important limitation of this system is that it is used only for the lifting beds of that company’s products with installed actuators. Moreover, it is necessary to install sensors into actuators at the bed manufacturing process. Therefore, users must buy this product if they use this bed-leaving detecting function. Moreover, this sensor system is not casually used, especially for a use case that can be installed later. In addition, strain gauges have an uncertain response to thermal changes. For a characteristic gap caused by thermal changes, they described that they accommodated it with software. Furthermore, this system requires the initial weight setting of a subject to be selected from three divisions in advance.

Using a watch-type triaxial acceleration sensor, Iomoto et al. [12] measured body motion patterns during sleeping as quality of sleep (QOS). They detected body motion patterns at 1 min intervals in band-pass signals of 2–3 Hz acceleration changes using the threshold of $\pm$ 0.01 G. They recognized sleeping or arousal for time-series active mass datasets. Their proposed method was evaluated for 1 week using datasets obtained from six subjects in their 50s who were diagnosed as needing daily life guidance. The experimentally obtained results revealed that each subject assessed sleeping parameters quantitatively. However, the results contained a shortcoming to demonstrate a global tendency in light of relations to temporary arousals during sleep. Moreover, this watch-type sensor decreases QOS when worn on an arm during sleeping because of the necessity for a subject to be restrained.

As an unrestrained monitoring sensor system, Okada et al. proposed a method to install an acceleration sensor in a comforter cover [10]. This system was constructed to measure the heart rate of a subject during sleeping. Near the position of the heart in a subject, this sensor was stitched to the surface of a cover on the upper-left chest region. The piezoelectric ceramics-based acceleration sensors have advanced sensitivity. They obtained measurement results resembling electrocardiogram (ECG) waves during face-up sleeping under resting status. The acceleration signal peak was obtained in the corresponding position on the ECG. Moreover, the signal peaks, which were equivalent to peaks of the R wave, were counted to assess the side sleeping behavior. The system with a compact sensor of 4 g weight requires amplifier circuits and convertors from analog signals to digital signals with capturing and processing in a computer. However, numerous details are remained in their system in terms of sensor wiring with a comforter cover as a remaining obstacle to practical use of this device.

Bed-leaving sensors and other systems of several types were proposed by Madokoro et al. [19, 20, 21, 22, 23] as simple and unrestrained sensors that can be installed in existing beds. Their first prototype was pad sensors [19] installed between a bed and a cover to detect behaviors of a subject globally. For this system, piezoelectric films were used for the bound with acrylic resin boards to detect pressure for measurement of the upper body movements of a subject on a bed. Their second prototype was pillow sensors [20] for measurement of the upper body movements of a subject on a bed. For this system, a three-axis gyroscope protected with a small capsule was used for installation into a pillow. Their third prototype was bolt vibration sensors [18] installed to a safety railing beside a bed. The size of bolt vibration sensors [17] was downsized because of the use for measuring bridge vibration. Using these three prototypes, we proposed multiple sensor systems [21] concomitantly with use cases and the balance of cost and accuracy. Our latest prototype was caster sensors [22] used for measuring the weight distribution of a bed.

However, several problems persisted in terms of a slip or drift of the pad sensors and the pillow sensors. No response is apparent for bolt sensors without gripping of the safety railing by a patient. Moreover, a medical doctor at a hospital made the comment that caster sensors present difficulties for moving a bed in a ward. Therefore, it is necessary to develop a new sensor that can resolve these problems, not merely with a slip and drift, but also for a frequently moving bed.

3. Sensors and their integration system

Various functional and high-performance sensors were proposed for existing bedside monitoring systems with targeting expensive care or medical treatments [3]. Alternatively, attaching them to the body is necessary to realize steady and long-term monitoring [12, 10]. For this chapter, a sensor system is designed while being practical, convenient, inexpensive, and simple.

Particularly, the following features are examined as (1) a bed mat to distribute the body weight patterns, (2) a pillow to measure head weight patterns, and (3) a handrail which is gripped by a subject while standing immediately before leaving. For multiple sensing, our system comprises following three sensors: (1) a pad sensor under the bed mat, (2) a pillow sensor to detect acceleration according to head movements, and (3) a bolt sensor used for a handrail joint. The features in each sensor and the overall system architecture are described as the following.

3.1. Pad sensor

Our originally developed pad sensors are easy to install under a bed sheet. Figure 1(b) depicts the design structure and an overview picture of our prototype sensors. The shape of pad sensors is round because they can detect weight changes from all directions. Piezoelectric films by Measurement Specialties Inc. [11] were used for this sensor prototype. The piezoelectric films are fixed between two polyethylene terephthalate (PET) plates of laminated polyester. The polyester and PET plate sizes were, 125 $\mu$ m and $\varphi$ 70 $\times$ 0.5 mm, respectively.

Figure 1.

Developed sensors: (a) pad sensor, (b) pillow sensor, and (c) bolt sensor.

Output signals are generated from the bent piezoelectric films when a subject transfers body weight on the bed. This sensor is possible to measure recursively because the reference potential is offset when the bending stops. Moreover, the weight strength according to body changes is obtainable linearly because the bending angle of the piezoelectric film and output voltage has a relation of proportionality. Herein, piezoelectric films are less troublesome and provide no false operations because of simple wiring without electric power supply cables for measurements. Therefore, a cost-effective system can be provided as requiring no maintenance related to replacement of a battery.

3.2. Pillow sensor

For the pillow sensor, a triaxial acceleration sensor module MMA7361LC produced by Freescale Semiconductor Inc. is used. Figure 1(b) presents a photograph of the device installed in the case. As shown on the left side of the photograph, after storing the device, a cap for sealing was inserted. Herein, basic characteristics of this sensor module were evaluated by our former study of attitude control for a flying robot [16].

A lithium-ion polymer battery with 0.85 Ah capacity is used as the power source of this sensor. Additionally, an original case was designed for storing the board and the battery to a pillow. Using a three-dimensional printer, the cylindrical case was made from acrylonitrile butadiene styrene (ABS) with the size of 55 mm diameter and 105 mm long. The board and battery were stored to the upper and lower part in the case using sliding rails without a margin that is set for the slides because the sensor boards were fixed inside the case.

3.3. Bolt sensor

A bolt sensor is used for the joint between a bed and a handrail. This sensor performs overall bed sensing, whereas pad sensors are used for partial subject sensing. Weight changes are detected with this sensor that occurs when a subject tries to stand up and grip a handrail. Figure 1(c) depicts our bolt sensor prototype that was developed originally by Shimoi et al. [17] as a simple and smart sensor to measure bridge strain. His bolt sensor has been commercially available for some time. Based on his bolt sensor, a novel sensor was originally developed with $\varphi$ =10 mm to fit the size of the bolt to the joint of the bed used for our evaluation experiment.

A piezoelectric cable was inserted into hollow polyurethane resin. Subsequently, a bolt-shaped adapter was attached as a cover for the exterior. Using this adapter, the bolt sensor can be fixed to a nut. Although no electric power source is necessary for piezoelectric cables, electric wires were extracted to obtain sensor signals. The data sheet denotes that piezoelectric cables generate voltage linearly according to bending angles. This linear feature was reported by Shimoi et al. [17] through a preliminary experiment and earlier experiments.

3.4. System structure

Figure 2(a) portrays the entire structure of our sensor system that comprises six pad sensors installed over the bed, one bolt sensor mounted on the joint part between a handrail and a frame of the bed, and one pillow sensor using a triaxial acceleration module installed inside of a pillow.

Figure 2.

Block diagram of our proposed system and assignment of our developed sensors on the bed. The symbols of 1–6 correspond to the sensor assignment of S1–S6.

The sensor installation positions are annotated from S1 to S6. The assignment of six pad sensors was S1 and S2 for the shoulder part, S3 and S4 for hip part, and S5 and S6 for the terminal part. These six sensors were installed with referring to the literature related to the monitoring system development used for welfare care beds [4, 5]. A bolt sensor was mounted to the joint of a handrail on one side of the bed. The size of the bed used for this experiment is 2100 mm long, 1000 mm width, and 700 mm height. Although the bed equips an actuator used for back-panel reclining, this function was not used for the experiment.

A commercially available pillow was used for embedding the sensor. This low-resilience urethane pillow purchased from Nitori Co. Ltd. was 500 mm wide, 310 mm deep, with 90 mm height. The pillow backside was cut out as shown in Figure 2(c) to produce a void space to insert the sensor case. The sensor case was fixed with three points using a hook and loop fastener to prevent rolling inside the pillow. One advantage of our sensor system is its easy installation as an aftermarket add-on part. Herein, it is necessary to open a pillow and partially empty it during repairs.

The sensor system requires no calibration. Regarding the place to store the sensor, it would feel uncomfortable or decrease the cushioning feature when putting the head to the pillow. Participated subjects in the experiment reported that they felt no negative impression from using this pillow.

The primary purpose of our system is to realize real-time monitoring using unrestrained and smart sensors with keeping QoL, which is important for a patient to live life normally. Our proposed system requires no supervision using infrared cameras or constraining sensors such as collar clip sensors. Moreover, a cost-effective system is concerned using piezoelectric films and cables as sensors that can function with minimum trouble or missed operations, and with remarkable duration and characteristics for pressure resistance.

Output voltage signals from each sensor were obtained using a board. The board was developed using Open Source Hardware Arduino FIO with a wireless communication module. With consideration of power consumption, a short-range wireless module of ZigBee was used for the communication with a monitoring terminal computer. Measured signals are displayed to a monitoring computer in real time. The method based of machine learning was embedded as real-time execution software for bed-leaving prediction.

4. Prediction method

4.1. Target behavior patterns

The target behavior patterns for bed-leaving prediction comprise three groups: sleeping, sitting, and leaving. Detailed behavior patterns were classified from output signals of six sensors. The sleeping includes three patterns: face-up sleeping, left sleeping, and right sleeping. The sitting includes three patterns: longitudinal sitting, lateral sitting, and terminal sitting [14]. The total prediction target is to produce seven patterns including leaving. The following descriptions are global features and estimated sensor responses in respective patterns.

Face-up sleeping: sleeping on the bed normally to the upper side of the body.

Right sleeping: rolling over on the bed to the right side.

Left sleeping: rolling over on the bed to the left side.

These three patterns occur as sleeping behaviors. The following patterns are the target of bed-leaving prediction.

Longitudinal sitting: sitting longitudinally on the bed after rising.

Lateral sitting: sitting laterally on the bed after turning the body from longitudinal sitting.

Terminal sitting: sitting in the terminal position on the bed trying to leave the bed. Rapid and correct detection are necessary because of the terminal situation for leaving the bed.

Left the bed: left from the bed.

Therein, sensor output signals disappear in the status of losing consciousness or a life crisis in terms of cardiopulmonary arrest. Monitoring devices in terms of electrocardiographs are used for these circumstances. Therefore, such circumstances are beyond our detection and prediction targets.

Longitudinal sitting is a behavior pattern by which a subject sits up on the bed. In numerous cases, subjects will return to sleeping again. In lateral sitting, a subject will move to leave from their bed because of turning their body to the terminal. For this position, our system should determine lateral sitting immediately to predict a behavior pattern leaving from the bed. Moreover, it is necessary to detect rapidly and correctly because of the terminal situation for leaving the bed if a subject moves to longitudinal sitting. Patients are protected from injury or accidents caused by falling from the bed if our sensor system can detect such phenomena before leaving completely.

4.2. Preprocessing

Frequently, original sensor signals include much noise, which decreases the recognition accuracy and increases calculation costs. As a noise removal method, threshold-based filters have been used widely for preprocessing. In contrast, calibration in advance is necessary according to a monitoring target because thresholds are set empirically or subjectively. Moreover, it is a challenging task to absorb characteristic variation of sensors used only for fixed thresholds. For targeting humans as a measurement objective, individual variations have strong effects.

Various methods have been proposed for machine-learning algorithms. For this chapter, self-organizing maps (SOMs) [27] are used for similarities of input data in the training process. Based on competitive and neighborhood learning, SOMs create clusters using unsupervised learning-based self-mapping characteristics. For practical applications, SOMs are validated as effective and superior performance through existing studies [33]. In the advanced computing performance, SOMs have facilitated various applications such as facial image processing, medical image processing, character recognition, text mining, and remote sensing data analysis.

As a classical approach, $k$ -means [24] is widely used for unsupervised learning-based clustering. Vesanto et al. evaluated that SOMs have superior performance to that of $k$ -means in their numerical experiments [31]. Moreover, Terashima et al. [29] demonstrated that false recognition accuracy of SOMs decreased to the minimum compared with that of $k$ -means used for clustering. Therefore, SOMs are used in our method because of these benefits.

The SOM network comprises two layers: an input layer and a mapping layer. Input layer units are assigned as the number of features of input data. Two units were set for a triaxial acceleration sensor. The mapping layer consists of units in a low-dimensional space. For our method, mapping units were assigned as one dimension because of using it for vector quantization on clustering. The learning process is executed for bursting a unit on the mapping layer when a set of input signals is given.

SOM training algorithms are the following. $w_{i,p}\left(t\right)$ are weights from an input layer unit $i$ $\left(i=\left[1,I\right]\in \mathrm{\mathbb{Z}}\right)$ to a mapping layer unit $j$ $\left(j=\left[1,J\right]\in \mathrm{\mathbb{Z}}\right)$ at time $t$ . These weights are initialized randomly before training. The Euclidean $L2$ distance $d_j$ between $x_i\left(t\right)$ and $w_{i,j}\left(t\right)$ is calculated as

$$d_j=\sqrt{\sum _{i=1}^I{\left(x_i\left(t\right)-w_{i,j}\left(t\right)\right)}^2}.$$ E1

The unit for which $d_j$ is the smallest is sought as the winner unit $c$ .

$$c=\text{argmin}\left(d_j\right).$$ E2

The neighborhood region $N_j\left(t\right)$ around $c$ is defined as

$$N_j\left(t\right)=S\left(1-\frac{t}{O}\right).$$ E3

Therein, $S$ $\left(0<S\le J\right)$ is the initial size of $N_j\left(t\right)$ ; $O$ is the maximum iteration for training. Subsequently, $w_{i,j}\left(t\right)$ of $N_j\left(t\right)$ are updated to close input feature patterns.

$$w_{i,j}\left(t+1\right)=w_{i,j}\left(t\right)+\alpha \left(t\right)\left(x_i\left(t\right)-w_{i,j}\left(t\right)\right).$$ E4

Therein, $\alpha \left(t\right)$ is a training coefficient that has decreasing value with the progress of training as

$$\alpha \left(t\right)=\alpha \left(0\right)\left(1-\frac{t}{O}\right),$$ E5

where $\alpha \left(0\right)$ $\left(0<\alpha \left(0\right)\le 1.0\right)$ is the initial setting value. In the first stage, the training speed is rapid when the rate is high. In the final stage, the training converges while decreasing the rate.

4.3. Prediction and recognition

After noise removing from originally obtained sensor signals, behavior patterns are recognized using supervised learning-based methods. Supervised learning intends to acquire information representation [30]. In contrast, unsupervised learning intends to create mapping relations among data [26]. Herein, support vector machines (SVMs) are popularly used methods for supervised learning. SVMs provide advanced performance with mapping to a high dimensional space using a kernel function. Moreover, Boosting is a method combining numerous weak learning machines. As a simple algorithm approach, counter propagation networks (CPNs) [25] are used as a supervised learning algorithm to be expanded based on unsupervised learning on SOMs.

CPNs were used as our method, not SVMs or Boosting, because of the following two features: (1) the learning algorithm is easy to implement for inserting a Grossberg layer to SOMs that are used for preprocessing; (2) relations among signals are visualized through the creation process of mapping structures. Formulas are described as learning algorithms on CPNs in different parts of SOMs. Visualization of relations among signals is actualized on a category map [33]. Particularly, it presents visualization of mapping results obtained as figures in the evaluation experiment.

CPNs, which were proposed by Nielsen [25], contain the network structure to append a Grossberg layer as the third layer to be supplied teaching signals. The input and mapping layers of CPNs are similar to those of SOMs. The Grossberg layer is assigned to the counter position of the input layer. For this chapter, a two-dimensional mapping layer was selected for visualization of similarity among features of input data.

The CPN training algorithm from presenting input data through updating weights after searching the winner unit $c$ consists of the similar procedure of SOMs. However, weights between the input layer and the mapping layer and neighborhood regions are changed respectively to $w_{i,j,k}\left(t\right)$ and $N_{j,k}\left(t\right)$ because of the use of a two-dimensional mapping layer. The $v_{j,k,l}\left(t\right)$ are weights from a Grossberg layer unit $l$ $\left(l=\left[1,L\right]\in \mathbb{L}\right))$ to a mapping layer unit $\left(j,k\right)$ at time $t$ . $v_{i,j,k}\left(t\right)$ and its neighborhood units inside $N_{j,k}\left(t\right)$ are updated based on the following Grossberg-learning algorithm.

$$v_{j,k,l}\left(t+1\right)=v_{j,k,l}\left(t\right)+\beta \left(t\right)\left(T_j\left(t\right)-v_{j,k,l}\left(t\right)\right),$$ E6

where $T_j$ represents teaching signals. Similarly to $\alpha \left(t\right)$ , $\beta \left(t\right)$ is a training coefficient that decreases its value with the progress of training as

$$\beta \left(t\right)=\beta \left(0\right)\left(1-\frac{t}{O}\right),$$ E7

where $\beta \left(0\right)$ $\left(0<\beta \left(0\right)\le 1.0\right)$ is the initial value. Finally, as the maximum value of $v_{j,k,l}\left(t\right)$ for the Grossberg layer unit $l$ , label $S_l\left(t\right)$ is searched for the following.

$$S_j\left(l\right)=\underset{1\le l\le L}{\text{argmax}}\left(v_{j,k,l}\left(t\right)\right).$$ E8

After labeling all units on a mapping layer, a category map is automatically created as a training result. Subsequently, test datasets are given to CPNs. A mapping layer unit is bursted as the minimum value of the Euclidean $L2$ distance in the formula (1). Recognition results are presented as corresponding labels.

5. Sensor characteristics evaluation

5.1. Setup

For evaluating characteristics of our developed film-load sensors, preliminary experiments were conducted using the load test machine (Multi Force Analyzer FWT-1000; DigiTech Co. Ltd.). The major specifications of the machine are as follows: 1 kN rated weight; 100 mN resolution; 600 mm/min maximum test speed; and $\pm$ 0.2% weight precision. The majority of loads are attained from the vertical side as a surface load because of the installation of the sensors on the bed frames. For this load test, a fixture was made of A2017 duralumin, as depicted in Figure 3(a). The major specifications of the fixture are 100 $\times$ 100 mm with 15 mm basement thickness and 70 $\times$ 50 mm with 5 mm top thickness.

Figure 3.

Load test: (c) fixture, (d) schematic diagram, and (e) load test.

The output characteristics were evaluated using our developed sensors of five sets with the default test speed of 5 mm/min. Figure 3(b) depicts a schematic diagram of the sensor output that occurs from the range except that of the rivet parts. For attaining a load, the sensor is fixed to the removal part of 10 mm from the boundaries. The output voltages are measured from respective sensors using a data logger (LR8431; Hioki Corp.) concomitantly with the test load.

5.2. Basic sensor characteristics

Figure 4 depicts the output characteristics of five sensors. The vertical and horizontal axes, respectively, depict output voltage and test loads. The output voltage increases concomitantly with the load until the peak for the maximum load. Subsequently, reverse voltage appears during a slight time as a steady state for removing the load cell from the device. The output characteristics were calculated using our prototype sensors obtained from this peak voltage obtained for this test. The results address the linear relation between the sensors and the test load patterns, although the gradients differ among sensors. We consider that the output voltage increases concomitantly with the weight of a person.

Figure 4.

Relation between output voltage and load of sensors.

Figure 5 presents results of changing test speeds from 1 to 8 mm/min step by 1 mm/min. The output voltage increases concomitantly with speed changes that are similar characteristics that resemble the load-test results presented earlier.

Figure 5.

Relation between output voltage and load speed.

Figure 6 depicts characteristics of the side and orientation of the sensors. Herein, four patterns were evaluated: a top/longitudinal side, a bottom/longitudinal side, a top/lateral side, and a bottom/lateral side. The top and bottom sides are defined by rivets of the piezoelectric film. The results are depicted in Figure 6. The output voltage of the longitudinal side is 3.12 times higher than that of the lateral side. This directivity characteristic is reflected in the sensor installation.

Figure 6.

Relation between output voltage and load of side and orientation.

6. Experimental results

6.1. Parameters

For evaluation of our developed sensor system, a similar experimental environment was set as a clinical site. Herein, the number of subjects is 10 persons: Subjects A–J. As attribute information, the body weights of the subjects are from 50 to 80 kg. They repeated the behavior sequences of seven patterns for five times. Therefore, 35 pattern datasets were obtained in each subject. Each behavior is switched in 20 s intervals. The sampling rate is set for capturing signals to 50 Hz. Herein, the number of subjects of our former study [20] was merely three persons.

Table 1 denotes optimized parameters used for SOMs and CPNs. These values were determined based on our former study [13] and the literature by Hosokawa et al. [32]. As an evaluation method, leave-one-out cross validation (LOOCV) [28] was used. For this experiment, four datasets and the remaining one dataset were used, respectively, for training and testing. Therefore, our method was evaluated for five combinational patterns in each subject.

Methods	I	J	P	Q	S	$\alpha \left(0\right)$	$\beta \left(0\right)$	O [epoch]
SOMs	10	—	10	—	16	0.5	—	1,000,000
CPNs	10	7	20	20	16	0.5	0.8	100,000

Table 1.

Setting values of parameters on SOM and CPN for this experiment.

6.2. Measured signals

Figure 7 depicts time-series output signals obtained from the nine sensors. The vertical and horizontal axes depict the voltage output and transition time, respectively. The dashed lines depict boundaries of each behavior pattern. Output signal patterns are different in each subject.

Figure 7.

Time-series changes of sensor output in each behavior pattern (Subject A, first trial).

For pad sensors, the voltage output range on S1 through S4 was expanded, especially in face-up, right, and left sleeping behavior patterns. Although salient voltage output included some noise, they were generated from S5 and S6. The voltage output on S3 became high for longitudinal sitting. The body weight was concentrated to the hip area according to the upper body rising behavior. For terminal sitting, the voltage output was generated from S5 and S6 and no voltage output was generated from S1, S2, and S3. Moreover, S4, which is located near S5, gave voltage output slightly. After leaving from the bed, no voltage output was generated from any sensor.

For the pillow sensor, the Z axis signal output of the upper and lower directions was greater than that of either the X or Y axis. The output became small during longitudinal sitting after body raising, which was similar tendency to that of lateral sitting. For terminal sitting, the sensor output expanded because the subject attempted to stand up at the bed terminal. This widening of output results from the behavior by which the subject tried to grip the safety handrail by hands. The weight shift at the bed terminal produced severe shaking all over the bed compared with that occurring around the center.

Signal outputs from the bolt sensor were presented only by the terminal sitting. The bolt sensor captures slight strain with the $\pm$ 0.1 width of output signals. The signal-noise rate was high compared with other two sensors. This is the dataset of Subject A at the first trial. Output signal patterns and their intensity differ in each subject and each trial. Therefore, it is solely a challenging task for setting thresholds of various types used for noise removal or for recognition solely from original signals. In contrast, our proposed sensor system can automatically remove noise using a machine-learning-based method.

6.3. Noise removal results

Using SOMs, our method removed noise patterns not only for improving recognition accuracy, but also for reducing calculation cost. The unit numbers were sorted according to the maximum of the bursting iterations from the left to the right. Most voltage output remained near the offset value because no voltage output when no load change was given without bending.

Figure 8 depicts signal output patterns of the X axis from the pillow sensor during the first 70 s period after removing sixth-unit mapped signals. As the whole tendency, noise signals around $\pm 0.1\left[m/s^2\right]$ were holistically removed. However, the signals around 60 s with higher amplitude were also removed, while those with smaller amplitude were remained.

Figure 8.

Noise removing result using SOM (pillow sensor, X axis).

6.4. Recognition results

For our method, category maps are used for classifiers and recognizers. Figure 9 depicts category maps for Subjects A, B, and C. The numbers from 1 to 7 on the category maps correspond to respective behaviors that were labeled through learning. The label distributions differ among subjects. Respective category maps created clusters of similar labels for similar behaviors.

Figure 9.

Category maps of Subjects A–C. The grids correspond to the units on the mapping layer of CPNs. Labels 1–7 denote face-up sleeping, right sleeping left sleeping, longitudinal sitting, lateral sitting, terminal sitting, and left the bed.

The boundaries were divided with units on the map for thick lines that depict segments of sleeping, sitting, and leaving from the bed as global changes of behavior patterns. Similar behavior patterns created recognizable behavior regions. Units, which are assigned to the seventh label as left from the bed, are distributed in the cluster of units labeled sitting. Dash marks were inserted in the category maps to the labels in each subject. Our method created a classifier in each subject through learning on CPNs. Moreover, our method enables visual verification of the relations among datasets.

Subsequently, our method was evaluated for recognizing bed-leaving behaviors from matched labels that correspond to burst unit labels after presenting test datasets to the category maps. Table 2 portrays a result of recognition accuracies with five combination patterns evaluated as LOOCV. The recognition accuracies were calculated as a ratio of the number of ground truth (GT) labels and the number of category labels in each behavior pattern that reached the maximum of responses during 20 s.

Subject	Face-up	Right	Left	Longitudinal	Lateral	Terminal	Left the bed	Average
A	100	83.3	83.3	100	33.3	100	100	85.7
B	100	100	100	100	16.7	100	100	88.1
C	0	66.7	100	100	83.3	100	100	78.6
D	66.7	50.0	33.3	100	66.7	100	100	73.8
E	50.0	83.3	16.7	50.0	33.3	83.3	100	59.5
F	50.0	83.3	66.7	100	66.7	100	100	81.0
G	66.7	66.7	50.0	66.7	50.0	100	83.3	69.0
H	66.7	33.3	83.3	66.7	66.7	100	100	73.8
I	66.7	50.0	66.7	66.7	66.7	100	100	73.8
J	100	66.7	33.3	83.3	50.0	100	66.7	71.4
Ave.	66.7	68.3	63.3	83.3	53.3	98.3	95.0	75.5

Table 2.

Recognition accuracies of respective subjects and positions for seven patterns (%).

The recognition accuracies of Subjects B and E were, respectively, 88.1 and 59.5% as the highest and the lowest. The recognition accuracies of leaving and lateral sitting were 95.0 and 53.3%, respectively. The recognition accuracies of longitudinal sitting and terminal sitting, which are the most important positions for bed-leaving detection, reached 83.3 and 98.3%, respectively. The mean recognition accuracy for 10 subjects was 75.5%. The lateral sitting as a status that can accommodate diversity among individuals. The recognition accuracy of Subject B for lateral sitting was 16.7%. For sleeping, the recognition accuracy of face-up sleeping was higher than that of right or left sleeping. The body weight gathered to the two sensors on the left or right sides in sleeping, whereas the load from their upper body was distributed on S1–S4 at face-up sleeping.

The performance was compared with a single use of pad sensors, a pillow sensor, and a bolt sensor. Table 3 presents the recognition accuracies of three sensor patterns. The recognition accuracy of the pad sensors as the highest performance was 75.0%. Comparison with our proposed multiple sensor system denoted that the performance difference was 0.5%. The recognition accuracies using a pillow sensor and a bolt sensor as a single sensor system were 58.3 and 33.3%, respectively. During the use of setting priority to simplicity, to use each single sensor is possible except for the bolt sensor. Our method for a multiple sensor system is useful for situations favoring performance over simplicity.

Sub.	Face-up	Right	Left	Longitudinal	Lateral	Terminal	Left the bed	Average
Pad sensors	35.0	81.7	75.0	90.0	55.0	96.7	91.7	75.0
Accuracies	68.0	34.0	46.0	52.0	46.0	74.0	88.0	58.3
Bolt sensor	33.3	0	11.1	0	22.2	100	66.7	33.3

Table 3.

Recognition accuracies used for stand-alone sensor (%).

6.5. Discussion

Table 4 portrays the confusion matrix (CM) for analyzing recognition details. For this matrix, the number of correct images is assigned to the diagonal cells that are marked as boiled numbers. Other cells refer the number of incorrect images and its category name depicted in the column. The maximum numbers of incorrect categories are marked as underlined.

Position	Face-up	Right	Left	Longitudinal	Lateral	Terminal	Left the bed
Face-up	40	3	15	2	0	0	0
Right	5	41	6	2	0	6	0
Left	9	12	38	0	0	1	0
Longitudinal	0	0	1	50	7	2	0
Lateral	0	0	0	20	32	5	3
Terminal	0	0	0	1	0	59	0
Left the bed	0	0	0	3	0	0	57

Table 4.

Confusion matrix of respective positions for seven patterns.

Underlines show the numbers of correct data.

The recognition accuracy of lateral sitting was the second lowest. The number of correct recognition datasets was 32 sets. For this behavior pattern, 20, 5, and 3 datasets are falsely recognized as longitudinal sitting, terminal sitting, and left the bed, respectively. The behavior patterns from longitudinal sitting to lateral sitting had a large gap because they turned their body about 90° on the bed. For both sitting positions, the load provided by the pad sensors was concentrated to the hip at the bed center. The false recognition in both behavior patterns was occurred from this feature. For lateral sitting, the legs load was gathered to the bed terminal as an exit. The false recognition was occurred from unclear boundaries between lateral sitting and terminal sitting. Since no dataset of lateral sitting was falsely recognized as sleeping or bed leaving, fatal errors of bed-leaving prediction can be avoided in this status transition.

Twelve and nine datasets of left sleeping were falsely recognized as right sleeping and face-up sleeping, respectively, as shown in the fourth line of Table 4. False recognition of left sleeping remained for sleeping except for one dataset. For face-up and right sleeping, four datasets were falsely recognized as longitudinal sitting. The results from the load to be gathered S3 merely depended on a subject, although the load on right sleeping was gathered to S1 and S3 normally.

The recognition was examined from three patterns: sleeping, sitting, and leaving. For bed-leaving prediction, these are basic behavior patterns. Table 5 presents the CM with respective recognition accuracies. The recognition accuracies of sleeping, sitting, and leaving improved to 93.9, 97.8, and 95.0%, respectively. The mean recognition accuracy improved to 95.7%. Our system provides sufficient recognition accuracy, especially for recognition of standing behavior patterns.

	Sleeping	Standing	Left	Acc. (%)
Sleeping	169	11	0	93.9
Standing	1	176	3	97.8
Left	0	3	57	95.0

Table 5.

Confusion matrix with respective recognition accuracies for three patterns.

Comparison with the bed-leaving sensor system that used strain gauges in actuators proposed by Hatsukari et al. [15] revealed that the recognition accuracies of their method were 87.7% for longitudinal sitting and 98.1% for terminal sitting. Although the experimental environment and the number of subjects differ in the results, the recognition accuracies of terminal sitting and longitudinal sitting of our method are, respectively, 0.2% higher and 4.4% lower than their method. Their method remains to three target patterns: longitudinal sitting, terminal sitting, and leaving. Moreover, their method is necessary for setting a subject body weight in advance. The calibration procedure differs from our system.

7. Conclusion

This chapter presented a multiple smart sensor system for predicting human behavior patterns that occur when a person leaves a bed. The proposed system was designed for early prediction of bed-leaving behavior patterns with regarded low privacy and high QoL. The behavior pattern prediction and recognition method was developed using machine-learning algorithms of two types from signals obtained using the sensors. Our system is applicable to an actual environment as a novel sensor system that does not restrict patients.

For future work, steady detection will be achieved to expand the range of applications and thereby increase the number of subjects. Moreover, our system must be applied to care facilities or single senior’s homes for security and safety observation that simultaneously maintains QoL and privacy.