Many service robots have been developed to assist patients with sit-to-stand movement (STS). However, little research has focused on users’ negative psychological changes during the STS movement when assisted by a robot. The STS movement accompanied with a negative psychological change is defined as a fault. The main purpose of this study was to propose a method of conveying faults to a service robot through the center of mass (CoM). Experiments on the STS movement were executed five times with 10 healthy subjects under four conditions: two self-performed STSs with seat heights of 43 and 62 cm, and two robot-assisted STSs with a seat height of 43 cm and end-effector speeds of 2 and 5 s. Time series data on the CoM were measured with high-speed camera system. A classifier was designed according to the data on the CoM in the frequency domain. The results showed that the proposed classifier had a high probability of discriminating fault classes from others. Then, the vertical ground reaction force (vGRF) under the same experimental conditions was used to cross-check the experimental results. It was concluded that faults in the assistance of service robots can be detected from the CoP-related items.
- center of mass
- fault classification
- service robot
- sit-to-stand movement
The sit-to-stand (STS) movement is essential to daily life and is used to change from a sitting position to a standing position. Dall and Kerr  pointed out that healthy adults performed 60 ± 22 STS movements every day on average. Grant et al.  reported that healthy older adults living in the community performed significantly more STS movements per day (71 ± 25) than older adults attending a day hospital (57 ± 23) or frail older patients in a rehabilitation ward (36 ± 16). For the elderly, their ability to perform this basic task weakens from the deterioration of muscle strength, joint range of motion, and balance. This can increase the risk of institutionalization, impaired functioning and mobility for activities of daily living (ADL), or even death [3–5]. With the continuous decrease in the number of nursing care specialists, there is an urgent need for the development of STS-assisting service robots.
Only a few studies have focused on service robots for assisting STS movement. Gervand et al.  formulated unassisted and assisted STS transfers as a control problem and found a balance among the end-point accuracy, human balance, energy consumption, and smoothness of motion. Chuy et al.  presented two approaches to assisting with the STS movement by using a robotic walking support system and showed that this system can track the desired support force. Rather than using kinematic data, Burnfield et al.  compared muscle demands through electromyography during self-performed and device-assisted STS transfers.
However, studies on service robots have neglected the psychological impact on subjects so far. A good understanding of psychological issues may help facilitate better integration of medical devices with users. Thomson et al.  emphasized that medical devices have both positive and negative psychological impacts on users, and simply addressing safety requirements does not guarantee user satisfaction. Because the duty of a service robot is to provide satisfaction with its service, cases where robot fails to meet this need can reasonably be defined as a fault.
A robot does not have the ability to tell whether its service satisfied its users. In other words, present service robot is not aware of its fault. Inferring human psychology is an essential step toward understanding human actions and hence is critical for realizing human-robot interaction. Recent advances in sensors and algorithms have allowed researchers to improve the perception ability of robots. However, only perception may not be sufficient for the efficient interaction between a human and robot because the robot’s reaction should depend on its understanding of human actions. This observation raises the question of how to correlate human psychological changes to legible data. Answering this question can provide methods for giving feedback to a service robot so that it can offer better health care service and daily-life assistance. In contrast to a laboratory environment, the situation in hospitals, rehabilitation centers, and nursing homes always have conditions that do not allow the use of large-size experimental equipment such as motion capture system or traditional force plate board. Thus, our aim in this study is to propose a practical, economical, and reliable method of evaluating service robot and conveying faults to service robot. Psychological results were collected by asking subjects to answer a questionnaire. The results showed that the proposed method discriminates faults with a high probability and is suitable for clinical research.
This chapter is organized into five sections. Section 2 explains the subjects, experimental system, and proposed method for analyzing the STS movement under different experimental conditions. Section 3 describes the experimental results. Section 4 cross-checks the results with center of pressure (CoP) and introduces other advanced signal analysis and classification approaches. Finally, Section 5 concludes this chapter.
2. Subjects and methods
Most service robots were originally designed for the elderly or patients with particular diseases. The elderly and patients were allowed to freely participate in our experiment because no clinical evidence had demonstrated the effectiveness and availability of service robots.
Ten human subjects (age: 38.6 ± 12.2 years old, height: 1.72 ± 0.06 m, body mass index: 22.37 ± 2.60 kg/m2) volunteered for the experiment. No subjects reported a major back pain or lower limb pathology, use of medication, or history of neurological disease that may influence the standing balance. There was no large difference in BMI for all subjects (p > 0.005).
The experimental procedures of the present study were in accordance with the Declaration of Helsinki and approved by the Ethics Committee on the Division of Health Science, Graduated School of Medicine, Osaka University (No. 305, August 21th, 2014). Informed consent was obtained from all subjects.
2.2. Experiment system
Figure 1 shows the overview of experiment system. Service robot evaluates experiment system consists of three equipment: a service robot for assisting with the STS movement, a height-adjustable chair for initializing the STS movement, and a high-speed camera system for measuring the STS movement.
Figure 2(a) shows a prototype of service robot whose end-effector speed can be regulated between 1.5 and 5.0 s. This robot can help users stand up, walk, and sit down. Figure 2(b) shows each step of the STS movement with assistance from service robot. The steps followed the definitions provided by Schenkman et al.  and are marked by five events. The path of the end-effector could be controlled so that service robot could match the user’s trunk movement with self-performed STS. Individual differences in body height could be also accounted for through the control of service robot.
Figure 3 illustrates a design scheme and trajectories of service robot utilized in our study, which is a robot entailing a chain-motor system containing a geared motor and three chains, see Figure 3(a). Trajectories of robot arms and end-effector are presented in Figure 3(b), wherein the start and end points are programed to match the position of user’s chests while they are sitting and standing still. At present, this service robot can only support users’ trunks.
A chair (CS-320A) was used to adjust the seat height during STS movement. The model could adjust the height to six levels (32–62 cm). The chair had the dimensions of a 35 cm width, 48 cm depth, and 78 cm height.
In order to acquire the details of users’ STS movement, we used a high-speed camera system, where there is a high-speed camera (VW-9000©KEYENCE, Japan) with 250 Hz sampling frequency and horizontal and vertical resolution of 0.5 mm (set through pre-experiment calibration). Five LED markers were placed on the users at: acromial, greater trochanter, lateral patellar ligament, lateral malleolus, and instep. Each marker was placed according to the ISO 7520:1996 standard.
2.3. Class design
Janssen et al.  categorized determinants that may result in stress during the STS movement such as chair-related factors. The literature indicated that the chair influences the performance of STS movement. Lowering the height of the chair apparently makes the STS movement more demanding or even unsuccessful. For young subjects (25–36 years old) without impairments, lowering the seat of chair from 115 to 65% of the knee height increased the trunk flexion angular velocity to stand by almost 100%. Accordingly, seat heights of 62 cm (>115% of subject’s knee height) and 43 cm (<115% of subject’s knee height) were set as Classes 1 and 2, respectively.
Although various determinants influence the STS performance, once the STS movement is assisted by the service robot, most determinants including the age, gender, foot position, trunk position, and arm movement are restricted by the robot. The assisting speed of the service robot naturally becomes the main factor that may influence how the STS movement is performed and directly affects the psychological changes of the user. Thus, we set up two robot assistance classes: Classes 3 and 4 correspond to end-effector speed of 2 and 5 s, respectively, for a chair height of 43 cm.
In order to ensure that subjects got used to the experimental procedures and assistance from the service robot, all subjects performed several training runs before the experiment. Hesse et al.  emphasized that training can be a determinant in an experiment study. For the self-performed STS experiment, subjects were told to stand up from the chair with their self-selected speed five times. For the robot-assisted STS experiment, because the trajectory of the end-effector was fixed and subjects had different body heights, it was difficult to make the robot accommodate every subject. As a result, robot-assisted STS was only performed with subjects at a suitable height five times in the experiment and utilized for analysis.
As a medium between psychological changes and physical movement, subjects were asked to answer a questionnaire on aspect of the STS movement that they felt demanding during both self-performed and robot-assisted experiments. Based on the results of the questionnaire, we were able to define a particular class as a fault when accompanied by a demanding feeling.
2.4. Fault measurement
The center of mass (CoM) and center of pressure (CoP) of the human body are widely utilized to analyze human motion. The location of CoP is highly variable, especially when vertical forces are small near the beginning or end of the stance phase. In addition, the trajectory of the CoP only represents the location of the vertical ground reaction force vector. It is difficult to understand the mechanism and interaction between the upper and lower limbs. Consequently, a method that can vividly describe human motion is necessary.
CoM is frequently calculated and proven to be significant especially in studies on human postures. As the location of CoM is not fixed and changes continuously with alteration of postures, the CoM needs to be calculated with time series data, and details about the CoM trajectories of each body segment are required.
Figure 4 shows an example on how to calculate CoM. Center of mass on a segment is located at the point that creates the same net gravitational moment of forces about any point of a segment as did the original distributed mass. We can simplify the CoM of segment as a single mass M located at a distance XM from end of the segment as:
In our experiment, human model was simplified as a three-segment model as shown in Figure 4. First, we assume that CoM for a total link model is located at point M0, coordinate (XM0, YM0). Coordinates can be calculated as follows:
Frequently, the mass of each segment M0 is calculated as a particular percentage of whole body weight. Then we have the calculation of coordinates of CoM of total segment model in our study:
where XMi and YMi are the horizontal and vertical coordinates of CoM of each segment.
The abovementioned questions make the calculation of CoM simple because the only knowledge required is the fraction of total body weight and the coordinates of each segment CoM. In this study, f1, f2, and f3 are set as 0.433, 0.433, and 0.5 for shank, thigh, and trunk, respectively.
A time-varying signal can be represented by successively adding individual frequencies present in the signal. Note that the frequency spectrum is mandatory during signal analysis. Thus, we used fast Fourier transform (FFT) to replace the time domain data with frequency domain data. Based on the FFT results, we are able to remove undesirable noise and depict all phases during the STS movement without any omissions. Because it is difficult for human to execute the STS movement as fast as 3 Hz, frequency domain data of less than 3 Hz were used in this study. Then, the maximum amplitude and its frequency were utilized as classifying features. We chose the Bayesian classifier for fault classification.
The main indices about FFT are given as follows:
Feature 1: maximum amplitude value of horizontal velocity of CoM
Feature 2: frequency of maximum amplitude value of horizontal velocity of CoM
Feature 3: area of frequency domain data of horizontal velocity of CoM with 3 Hz
Feature 4: maximum amplitude value of vertical velocity of CoM
Feature 5: frequency of maximum amplitude value of vertical velocity of CoM
Feature 6: area of frequency domain data of vertical velocity of CoM with 3 Hz
In order to discriminate features in each class, FFT was calculated from differential data of horizontal and vertical coordinates of CoM. The area of FFT curves was also calculated in order to consider the second and third highest frequency as well as the first highest frequency.
2.5. Fault classification
Bayesian classification and decision-making are based on probability theory and the principle of choosing the most probable or the lowest risk option. The major problem with the Bayesian classifier is the class-conditional probability density function, which describes the distribution of feature vectors in the feature space of a particular class. The distribution can be estimated from a training set with a range of methods. The probability density function is defined as a weighted sum of Gaussians:
where is the weight of the component c, 0 << 1 for all components, and . The following parameter list defines a particular Gaussian mixture probability density function: .
In D-dimensional space, it is defined in matrix form as:
where is the mean vector and is the covariance matrix. In order to calculate the discriminant function, Eq. (4) is rewritten as a nature logarithm:
When = = σ2I (unitary covariance matrix), the discriminant function is determined by:
When = σ2I (common covariance matrix), the discriminant function is determined by:
When (general covariance matrix), the discriminant function is determined by:
3.1. Results of fault classes
Based on the psychological changes, for self-performed STS, all subjects felt a 43 cm chair height (Class 2) was demanding. For the robot-assisted STS movement, adjusting the assisting speed affected the psychological change. Specifically, all subjects reported that they felt supported with an assisting speed of 2 s. However, when the assisting speed was slowed to 5 s, executing the STS movement became demanding. Accordingly, we defined Classes 2 and 4 as faults.
3.2. Results of calculated CoM and faults classification
Figure 5 shows the experiment images and results of CoM at time domain and frequency domain. Figure 5(a) shows four-frame multiple exposure photography to describe subjects’ performance under four conditions. Figure 5 (b)–(d) shows time series data of CoM at time domain, including trajectories of CoM, CoM velocities at horizontal (X) and vertical (Y) directions. Figure 5(e) and (f) shows results of CoM data at frequency domain with FFT.
It is obvious to see that due to different seat heights among experimental conditions, CoM trajectories under Class 1 were lower than those under other classes. However, much shorter trajectories of CoM were found under Class 1. As to the velocities at X and Y directions, lowering the seat height increased the CoM speed at both directions almost two times at the horizontal direction and 1.5 time at the vertical direction. On the other hand, velocities under Class 4 were smaller than those under Class 3 due to slower assisting speed.
From the raw data of frequency domain, it is found that self-performed STS movements showed a dramatic difference compared with the robot-assisted STS movement. It is necessary to identify which indices hold the most probability to distinguish four classes in better way.
Figure 6 shows the results of probability density analysis (PDA) for six features, which can be calculated from Figure 5(e) and (f). The major contribution of PDA is to demonstrate in which feature factor, the proposed classification approach would distinguish different classes much better. Principle on how to choose the appropriate features is to find out the largest disparity of distributions.
Accordingly, the maximum amplitude of CoM velocity at the vertical direction (feature 4) and its frequency (feature 5) were used to further classify process.
Figure 7 shows the classification results. Different classes were divided into four areas. Classes where subjects felt supported were located in the middle of the scatter diagram, and classes where subjects felt demanded upon were located on two flanks. Class 4 was isolated from the other classes and more concentrated; the frequency and amplitude in Class 4 showed the smallest magnitude compared with other classes.
In our study, senior citizens were free to participate in our experiment, even though many service robots were originally designed for the elderly or patients with functional limitations. Schenkman et al.  argued that a more successful approach would be if we understood how healthy individuals execute the STS movement under different conditions and utilize this information to interpret the performance of those with functional limitation or impairments.
Two main aspects were discussed in this chapter. First, we cross-check our experiment results through CoP using the vertical ground reaction force (vGRF). Second, we reviewed works on the advanced signal analysis method and other promising fault classification approaches.
4.1. Cross-check of experiment results
Compared with CoM, which requires the masses of body segments to be known and is often not directly determined, the CoP position is the projection on the ground plane of the centroid of the vertical force distribution; thus it can be obtained directly from a force plate or Wii Balance Board (WBB).
The WBB (23 cm × 43 cm platform), which was designed to support people weighing up to 136 kg, fed data into the computer through a Bluetooth connection. Soangra and Lockhart  reported that the force sensors are linear with CoP noise levels of approximately ±0.5 mm. Although it was originally designed as a video game controller, the WBB has become a proven tool for assessing the CoP position and has been confirmed to be both accurate and reliable. Moreover, the WBB provides a portable and inexpensive balance assessment system that is widely available. Jeong et al.  used a WBB to analyze the difference in manual material handling between experts and rookies at a logistics workplace. Huurnik et al. , Clark et al. , and Park and Lee  demonstrated the validity and test–retest reliability of the WBB by measuring the CoP position and comparing the data with that from an identical study using a laboratory-grade force plate. They found that the WBB can provide reliable and consistently repeatable data. Yamada et al.  reported that vGRF parameters have high reliability with an intra-class correlation coefficient (ICC) of 0.70–0.95. Then, vGRF data on the knee-hip joint extension phase and hip-lift off phase during the STS movement phase can be used to evaluate the lower limb muscle function of the elderly.
Figure 8 describes the WBB and proposed a method for fault measurement. Time series data of vGRF were measured according to the sum of four pressure sensors: PV1, PV2, PV3, and PV4. Because subjects had different body weights, we needed to compare the time series data of vGRF (W’B) with the body weight WB. We define this value as fWR(t) and calculated as follows:
When subjects sat still on the chair, the vGRF was much smaller than the body weight. It increased and reached its peak value when the trunk and ankle began to extend. Finally, when subjects finished their STS movement and stood still, fWR(t) stayed at a value of unity (W’B = WB). These measurements are similar to the results of a related conventional study by Yamada and Demura .
In order to describe how fWR(t) changed during the STS movement, we defined the differential value of fWR(t) as the STS smoothness. This can be calculated as follows:
Figure 9 shows the results about frequency domain data of STS smoothness. In frequency domain, Classes 1 and 2 showed two means and covariance matrices of smoothness (maximum amplitude and its frequency):
Although there was no large difference between the frequencies of the maximum amplitude for the two classes (0.4234 Hz for Class 1 and 0.3861 Hz for Class 2), the maximum amplitude for Class 2 was 46.0% larger than that for Class 1.
For the robot-assisted STS movement, the maximum amplitude and its frequency for Classes 3 and 4 showed two means and covariance matrices:
Note that the maximum amplitude for Class 4 was only 44% of that for Class 3. However, the frequency of the maximum amplitude for Class 4 was 1.4 times larger than that for Class 3.
Figure 10 shows the classification results under four different experimental conditions using same algorithm for CoM. Classes 1 and 2 clearly had different distributions. Even though the subjects were assisted by service robot during STS movement, the psychological results showed that subjects did not feel supported in Class 4. This meant that Class 4 was unique among both the robot-assisted and self-performed STS movement. This phenomenon indicated that nondemanding classes (Classes 1 and 3) were concentrated, while demanding classes (Classes 2 and 4) were located outside the former classes.
After fault classes were classified and discriminated from the others, it was necessary to identify how altering the assisting speed of the service robot affects the STS performance and makes it feel demanding. A difference in the STS smoothness clearly caused a psychological change. Because we considered Class 1 to be natural and normal STS movement, we normalized the STS movement reproducibility through dividing the standard deviation (SD) of peak values of time domain data under four experimental conditions by those for Class 1.
Figure 11 shows the STS movement reproducibility. Regarding the SD ratio at the peak amplitude, there was no big difference between Classes 1 and 3. However, the value for Class 4 was 1.5 times larger than that for Class 1. Although the trunk flexion and extension were restricted by the end-effector of the robot, the SD for the peak amplitude of the smoothness changed according to the results for Classes 3 and 4. However, the larger SDs for both the amplitude and time were from the effect of lower limbs. Because of the slower speed, it took a longer time to move from trunk flexion (step 2) to trunk extension (step 3). Thus, it was necessary for the lower limbs to produce a larger force to support the upper limbs.
Regarding the SD ratio for the time of the peak amplitude, although there was no big difference between Classes 2 and 3, the value of Class 4 was 2.4 times larger than that for Class 1. A larger SD for the time indicates a different STS movement under the same experimental condition. With Classes 2 and 3, most subjects showed a similar STS movement, but with Class 4, every subject showed different STS movements.
4.2. Advanced signal analysis and classification approaches
In our study, we used a simple questionnaire to describe psychological changes, where negative psychological changes were labeled with different degrees. Subjective emotion measurement also includes interviews on emotional experiences and self-reporting. One of the most widely accepted measures for emotional states was proposed by Russell . The Russell theory consists of a 2-D emotion space defined by the valence and arousal. However, it is difficult for robots to recognize subjective descriptions such as excited or tired. Converting an emotional state to numerical values urgently needs to be addressed.
Swangnetr and Kaber  offered an effective approach of inspecting the heart rate (HR) and galvanic skin response (GSR) to measure psychological changes in human-robot interaction through a wavelet analysis. They proposed neural network structures for classification and reported an accuracy rate of about 80%.
In the past 40 years, academic interest in chaos has rapidly increased in several areas, including robotics . Perc expanded the scope of practical application of a deterministic chaotic system to the cardiorespiratory  and human locomotor systems . In the former work, basic methods of nonlinear time series analysis were demonstrated to be appropriate for the human electrocardiogram (ECG), and the maximal Lyapunov exponent was calculated to provide a new insight into human heart. The latter study presented an advanced signal analysis that can give overall insight into the dynamics of human motion.
Accordingly, it is reasonable to believe that there is a tight relationship between chaotic theory and machine learning, and it has already been discussed. Khelifa and Boukabou  used numerical simulations to confirm the capability of an adaptive neural network to approximate a multiscroll chaotic system. In the future, we intend to draw on the experience of the abovementioned signal analysis method and chaos-based technologies to upgrade our experimental system and fault classification algorithm in order to provide more precise and real-time feedback for service robot.
Besides human locomotor and physiological systems, muscle activation also plays an essential role in our daily life, including the STS movement. Roldan-Jimenez et al.  analyzed muscular activity and fatigue using electromyography (EMG) during STS tests and demonstrated that the vastus medialis of the quadriceps (QS) plays a major role in the STS movement. Meanwhile, the QM is also the muscle most likely to be fatigued. It is followed in importance by the tibialis anterior (TA), which was reported to be the second muscle with a high level of participation in STS involvement.
The EMG signal also provides data describing neuromuscular activities and has been investigated as a machine learning tool. Yousefi and Hamiltion-Wright  reviewed some classification methodologies using EMG characterizations and concluded that the artificial neural network (ANN) is the most popular method for classifying EMG signals and can achieve a high accuracy of about 97.6%. The second most popular method for classification is the support vector machine (SVM), which has an average accuracy rate of 93%.
However, to the best of our knowledge, there has been no similar work focusing on muscle activities and fault classification for EMG when the elderly stands up from a chair with assistance from a service robot. Thus, as future work, we also plan to explore this area and clarify the relationship between the neuromuscular system and robot-assisting system.
Regarding the limitations of our study, using frequency domain data for a time series signal may be too simple for the undergraduate level. The Lyapunov exponent has been reported to be an excellent approach for analyzing the chaotic behavior of time series data . However, the only concern with using a high-dimensional system for classification is the relatively slow calculation speed, which can directly affect the real-time feedback for robots. In clinical situations, the most critical factor is that the patient status must be monitored or determined constantly. Therefore, realizing a real-time response for service robot may provoke more academic interest not only in the development of hardware but also in human-robot interaction.
Although service robots have an infinite amount of latent potential, there is still a possibility that they may fail to meet users’ need and cause negative psychological results. The main purpose of the present study was to develop a method of informing a service robot of the users’ feelings based on measured CoM-related items. A classifier was designed according to the frequency domain data of CoM during the STS movement. The results showed that the designed classifier had a high probability of discriminating fault classes from others. We also presented a practical and economical experimental method to correlate negative psychological changes to data that a robot can understand.
This research was funded by Hirose International Scholarship Foundation (no. J770703161). We gratefully acknowledge the work of past and present members in our laboratory. Lastly, we appreciate all supports and understanding from those people who stood behind us and still are standing by our sides.