Smart Healthcare at Home in the Era of IoMT

Abstract

Smart Home improves the quality of our life in various aspects such as the convenience of managing our home, efficiency of energy consumption, and secure living environments. Taking advantage of the Internet of Medical Things (IoMT), smart homes in the context of healthcare have attracted a lot of attention to provide a more convenient, easier accessible, and personalized healthcare experience. Leveraging state-of-the-art techniques like Digital Twins (DT), machine learning (ML) algorithms, and human action recognition (HAR), Smart Healthcare at Home (SHAH) not only provides independent healthcare service options and social support but also gives seniors or other individuals who are in need a reliable way for real-time monitoring and safety preservation. This chapter will provide a comprehensive overview of the technical components of a SHAH paradigm, which is based on an architecture that integrates DT, IoMT, and artificial intelligence (AI) technology. The design rationales and key function blocks are illustrated in detail. In addition, taking seniors’ safety monitoring as a case study, a prototype of a SHAH system is experimentally investigated, and the performance and design tradeoffs are highlighted. Finally, this chapter also provides an overview of this exciting field’s existing challenges and opportunities.

1. Introduction

Smart Home, or Home Automation, has become one of the trending fields of the Internet of Things (IoT) since 2004. Smart Home refers to a residential space that utilizes advanced information and communication technologies (ICT) and automated systems to enhance comfort, safety, security, and energy efficiency [1]. With smart home devices and systems, homeowners can control and monitor various aspects of their homes, such as lighting, heating, cooling, security systems, and entertainment systems, from anywhere in the world, using their mobile devices or voice assistants. The proposal and development of the Internet of Medical Things (IoMT) bring more elements and functionalities into the landscape of smart homes [2]. Integrated with IoMT, Smart Home can leverage smart sensors to monitor and track residents’ vital health data such as blood pressure, heart rate, blood sugar levels, and more. Moreover, personalized health services are provided based on the data collected by the IoMT devices, such as setting up suitable room temperature and lighting schemes to improve the living experience.

A Digital Twin (DT) is a virtual mirror of a physical object or system, such as a building, machine, or city in the digital space, or cyberspace [3]. DT is created by combining data from various sources, such as sensors, cameras, and other IoT devices, and using advanced modeling and simulation tools to create a digital replica of the object or system. In the context of a smart home, a digital twin could be used to create a virtual replica of the home that incorporates real-time data on energy usage, temperature, humidity, and other factors. Moreover, digital twins could then be used to improve the quality of service (QoS) for healthcare in intelligent home systems.

Based on the above rationales, this chapter aims at inspiring more discussions and sparking more new ideas in the digital healthcare community by introducing a scenario of smart healthcare at home (SHAH) that leverages these novel technologies to envision the future of medical services, senior safety, and more exciting application domains.

The rest of this chapter is structured as follows. Section 2 provides necessary background knowledge for readers who are new in this area. Section 3 discusses the design rationals and technical components required to enable smart healthcare at home. Taking the seniors’ safety monitoring as a case study, Section 4 illustrates the feasibility of such a framework. Section 5 tries to highlight the major challenges yet to be tackled and the opportunities in the near future. Finally, Section 6 wraps up this chapter with some brief conclusions.

2. Background knowledge

2.1 Internet of Things

Internet of Things (IoT) refers to the network of physical objects or devices that are embedded with sensors, software, and other technologies to collect and exchange data over the Internet. These objects can be anything from simple household appliances such as smart thermostats to more complex devices such as industrial machinery, self-driving cars, and even medical implants. While the potential uses of IoT are virtually limitless, in this chapter we just highlight several most popular areas.

2.1.1 Smart home

A Smart Home is a home equipped with various Internet of Things (IoT) devices that are connected to a network and can be controlled remotely and automatically through a smartphone, tablet, or computer [4]. These devices are designed to make life more convenient, comfortable, and efficient by automating various tasks and functions around the house. Here are some examples of smart home devices:

• Smart thermostats: These devices can be programmed to adjust the temperature of your home automatically based on your preferences and schedule.

• Smart lighting: Smart bulbs and switches can be controlled remotely and programmed to turn on or off at certain times, or in response to other triggers such as motion detection.

• Smart surveillance and locks: These locks can be controlled remotely and allow you to lock or unlock your doors from anywhere, as well as monitor who is coming and going.

• Smart appliances: Many appliances, such as refrigerators, ovens, and washing machines, are now available with IoT connectivity, allowing you to monitor and control them remotely.

• Voice assistants: Devices such as Amazon Echo and Google Home can be used to control your smart home devices using voice commands.

By integrating these devices into a smart home ecosystem, homeowners can automate many tasks and function around the house, making life more convenient and efficient. Li et al. [5] summarized that smart home technology is highly associated with healthcare, energy efficiency, and home security. However, the authors also illustrate challenges such as privacy, security, technology anxiety and negative social influences.

2.1.2 Internet of medical things (IoMT)

IoMT refers to the network of medical devices, wearable sensors, and other healthcare technology that are connected to the Internet and designed to collect, transmit, and analyze patient health data. This data can be used to monitor patient health remotely, diagnose conditions, and deliver more personalized services. Here several examples of IoMT devices are highlighted:

• Wearable fitness trackers that monitor activity levels, heart rate, and sleep patterns.

• Remote monitoring devices that can be used to track vital signs, such as blood pressure, blood glucose levels, and oxygen saturation.

• Medical imaging devices, such as X-ray and MRI machines, that can be connected to the Internet to transmit images and other diagnostic data to healthcare providers.

• Smart pills that contain sensors to monitor medication adherence and provide real-time feedback to healthcare providers.

• Mobile health apps that can be used to monitor and manage chronic conditions such as diabetes, asthma, and heart disease.

IoMT has the potential to revolutionize healthcare by enabling remote monitoring, improving patient outcomes, and reducing costs. For example, IoMT devices can be used to monitor patients with chronic conditions and intervene early if a problem arises, reducing the need for hospitalization and improving quality of life. With the help of state-of-the-art artificial intelligence (AI) techniques, real-time monitoring can be realized using lightweight human action recognition [6].

Additionally, the data collected by IoMT devices can be used to develop more personalized treatments and improve medical research.

2.1.3 Smart grid

A smart grid is an advanced electricity distribution system that uses advanced sensors, communication technologies, and big data to improve the reliability, efficiency, and sustainability of the power grid. The smart grid allows for better integration of renewable energy sources, more efficient distribution of power, and greater control over power usage by both utilities and consumers. By improving the efficiency and reliability of the electric grid, the smart grid can help to decrease energy costs, improve energy security, and reduce greenhouse gas emissions. Additionally, the smart grid can enable the adoption of new energy services, such as electric vehicle charging and home energy management systems. Smart grid technology brings revolutions to energy management from smart homes to smart cities in a “bottom-up approach” [7]. Smart grid not only helps to build a sustainable energy consumption ecosystem for smart homes but also gives new strategies for energy trading on large city-level scales.

2.1.4 Smart city

A smart city is a city that integrates advanced ICT, such as the Internet of Things (IoT), sensors, and data analysis, into its infrastructures, administration, and daily operations to improve the quality of life for its residents, enhance sustainability, and streamline urban services. Smart city initiatives aim to optimize resource management, increase efficiency, and improve communication and connectivity. A smart city consists of various aspects of the daily routines of a modern city operation, including Smart Transportation, Smart Energy, Smart Waste Management, and Smart Public Safety. By using technology to optimize urban services, smart cities can reduce costs and increase efficiency, while also improving the environment and public health.

2.2 Digital twins (DT) and IoMT

The concept of DT was first adopted in the industrial manufacturing domain. As the definition of Physical Object (PO) evolved from industrial artifact into almost every real-world object, DT was soon introduced into the context of healthcare, especially in IoMT. The early examples of DT models benefiting healthcare date back to the maintenance of medical equipment. Until today, the application of this technology in IoMT can be roughly categorized into three major subdomains:

• Medical resource allocation: Leveraging DT models, medical/healthcare service providers can improve the allocation of medical resources including personnel, equipment, medicine, hospital beds, and appointments. The digital models help professionals to better understand the need of patients and predict or estimate future trends. These DT-enabled schemes play a more significant role in crises as was observed during the COVID-19 pandemic as medical institutions may face shortages in various medical resources under extraordinarily heavy pressure. Some hospitals are collaborating with large healthcare enterprises to establish their own DT platforms or systems to provide better QoS for patients [8]. With the help of artificial intelligence (AI) technologies and big data, DT models are powerful to optimize resource allocation and scheduling decisions to enable the entire healthcare service system to be operated more efficiently.

• Digital organs: The process of creating digital organs is different from traditional procedures in manufacturing because most of the data is collected during medical treatment and examinations rather than directly gathered by the sensors [9]. However, based on the massive data about the organs, medical professionals may create DT models and set certain presumptions or scenarios to investigate different hypotheses. This is an effective approach to improve the quality of medical service and reduce the risk of applying new methods to human beings directly. Moreover, combined with certain AI/ML methods, the medical database helps to train various models to obtain more reliable predictions and more accurate diagnoses.

• Digital patient: Digital patient utilizes the real-time collected data from various sensors and/or historical medical records to create a virtual replica of a patient enabling different medical services such as daily examination, remote diagnosis, emergency response [10]. The booming evolution of smart devices especially in IoMT like wearable devices and other light-weight medical sensors brings more opportunities to the development of digital patients. A smart home environment equipped with state-of-the-art IoMT devices allows patients more options to receive personalized medical services. Continuous monitoring and remote medical response systems free certain patients from mandatory hospitalization.

3. Smart healthcare at home (SHAH): An architectural overview

The smart home is one of the major scenarios in which digital patients are enabled. This section presents an architectural overview of a Smart Healthcare at Home (SHAH) system, which integrates DT, IoMT, and AI/ML technologies. By no means the SHAH system covers the entire design space of smart healthcare services in the era of DT, IoMT, and more emerging new technologies, but we hope it provides our readers with some useful insights for further exploration.

3.1 System architecture

Figure 1 illustrates the system architecture of the SHAH framework. This conceptual architecture utilizes DT technology to build an edge-layer virtual healthcare system, which is designed to provide smart healthcare services to the residents living in the smart home. The major components or function blocks in this framework include:

• Body sensors: Body sensors include different types of wearable devices that collect bio-signals such as photoplethysmography (PPG), blood pressure, temperature, heart rate, blood oxygen level. The data collected by these sensors will be transmitted wirelessly to the support unit for modeling and analysis, enabling healthcare professionals to monitor patients remotely and provide timely interventions if necessary.

• Tracking and recognition sensors: This type of sensor mainly refers to smart cameras and motion detectors. With the help of these devices, the system can easily locate the resident(s) and provide more information about the patient. For example, a smart camera can identify different human actions within its field of view and the motion detector is triggered if someone enters the room where it was mounted. And by combining this information with other data collected by body sensors, the system is capable of defining certain patterns and giving a timely response if any anomaly is detected.

• Environment sensors: These sensors are responsible for collecting environmental parameters from the facility. Various combinations can be personalized according to usage, such as thermometers, hygrometers, smoke detectors, and water leakage detectors. These environment sensors can be integrated into the centralized smart home system that collects and analyzes data to provide insights and trigger appropriate actions. The collected data can also be shared with healthcare providers to monitor and assess the well-being of individuals remotely or aid in preventive care strategies with the help of AI technologies.

• Support unit: The support unit plays a critical role as the brain of the smart home system. It may consist of a smart gateway, small single-board computers, a personal computer (PC), or even a small home server. The support unit is responsible for collecting all the data from the sensors in the physical world, unifying these parameters in different protocols and sampling rates, providing intelligent healthcare services, and maintaining the DTs in the virtual space.

• Smartphone: Most of the light-weight operations in SHAH can be deployed on a smartphone. The users would have all access to their data and can review the system through a Graphical User Interface (GUI). According to their personal preference, the system can set different alarm patterns and/or data-sharing policies to support customized healthcare services and protect data privacy.

Figure 1.

SHAH system architecture: An overview.

3.2 Technique components

Figure 2 shows the major technique components of SHAH.

• Communication units: SHAH is highly based on the data collected from various sensors, which requires an efficient, accurate, and unified data communication scheme. To address these requirements, technologies such as new-generation networks, Bluetooth Low Energy (BLE), Wi-Fi, and various network protocols may be utilized in different scenarios.

• AI: Artificial intelligence plays an important role in SHAH. The massive data collected from sensors relies on AI algorithms to generate different results such as anomaly detection, future prediction, and other insights which would promote healthcare services in this system.

• Modeling and analysis: The DT-based SHAH highly relies on modeling technologies for creating a digital replica inside the virtual space. This digital avatar not only reflects the real-time status of the resident(s) but also gives the user an intuitive vision of the whole living space. Tools like AutoCAD, BIM, Unreal Engine, and 3Dmax are widely used for creating 2D or 3D models of systems.

• Information fusion: Information fusion, also known as data fusion, is the process of integrating and combining data from multiple sources or sensors to obtain more accurate, reliable, and comprehensive information. The goal of information fusion is to leverage the complementary strengths of different data sources, compensate for their individual limitations, and extract valuable insights that would be difficult to obtain from any single source alone.

• Data storage: If we consider the scenario of SHAH as an isolated node that is off the grid, it would be reasonable and practical to store the data on-site and in the system. However, if the system joins the local healthcare network society or even the Metaverse, it involves data sharing and privacy issues. Then, we might introduce distributed data storage which refers to a method of storing data across multiple physical or virtual storage devices or nodes that are geographically distributed. Instead of centralizing data storage in a single location, distributed storage systems distribute data across a network of nodes for improved performance, scalability, fault tolerance, and availability.

• Blockchain and non-fungible tokens (NFT): Security and privacy are paramount in virtual healthcare, where the protection of sensitive medical data during transmission and storage is crucial to prevent unauthorized access and misuse. Blockchain technology holds promise in addressing these concerns by providing a robust solution [11]. Blockchain enables the creation of a secure and tamper-proof ledger for medical records, accessible only to authorized parties. Each transaction or modification to the record is securely recorded on the blockchain, allowing for easy tracking of record access and changes. Another promising technique is the use of NFTs, which possess characteristics such as immutability, traceability, and uniqueness [12]. NFTs can help ensure secure access to medical records while maintaining patient privacy. Ownership of the NFT can be easily transferred to other authorized parties as required, maintaining the integrity and privacy of the records.

Figure 2.

SHAH technique components.

4. Seniors’ safety monitoring: a case study of SHAH

The entire world is witnessing a fast-growing aging population body, and it is widely observed that there are more and more seniors living alone, either in their homes or in individual rooms of nursing houses. An effective healthcare service system is required to maintain 24/7 real-time monitoring and timely dangerous action recognition. The compelling need for both regular medical consultation and timely emergency assistance inspires us to adopt this topic for our case study.

4.1 Sensor types

There are two types of sensors that are normally used in action recognition: wearable sensors and remote sensors. Wearable sensors are compact and lightweight, making them convenient for real-world settings. They can provide detailed motion data, including acceleration, orientation, and angular velocity information. This type of data helps to recognize specific actions, such as walking, standing, and sitting. However, wearable sensors can be limited in terms of the scope of their coverage. They may not be able to capture specific actions or movements that occur outside of the sensor’s range or when the sensor is not worn.

Remote sensors, such as cameras, can capture visual data that complements the motion data provided by wearables. This type of data gives a more comprehensive view of the environment and enables the recognition of complex actions that may not be easily detectable from motion data alone. For example, cameras can capture facial expressions, gestures, and interactions with objects in the environment. However, remote sensors can be limited by lighting conditions, occlusion, and the need for a clear line of sight.

By combining the advantages of both wearable and remote sensors, it is possible to develop a more comprehensive action recognition system that can perform real-time and accurate recognition of a wide range of actions in different settings. In addition, data fusion is a technique used to integrate data from multiple sources to improve the accuracy and reliability of action recognition systems, which can enhance the effectiveness of healthcare and other applications.

4.2 Information fusion

Information fusion is an approach to integrating data from multiple sources to improve the accuracy and comprehensiveness of the information obtained. It is typically divided into three levels: data level, feature level, and decision level [13].

At the data level, raw data from sensors, such as wearable devices and cameras, are combined to represent the phenomenon being monitored thoroughly. This approach can be computationally intensive and requires careful calibration and synchronization of the sensors, but it can provide a more comprehensive understanding of the phenomenon.

At the feature level, features extracted from the raw data are combined to obtain a more informative and complementary representation of the phenomenon. This approach can be more efficient than data-level fusion but requires careful selection and processing of the features to ensure they are informative and complementary.

At the decision level, the decisions or outputs of different classifiers are combined to make a final decision about the phenomenon being monitored. This approach can be used when the various sources of information provide redundant information but are not necessarily complementary. It can also be used to weigh the different sources of data according to their reliability or importance.

In the IoT context, data-level fusion has been preferred due to its ability to integrate data from different sources and provide a more accurate representation of the physical environment. After the data from various sensors are fused at the data level, the resulting data is typically more compact and comprehensive than the raw data from each sensor, as shown in Figure 3. Consequently, the amount of data to be transmitted is less than the sample data combination, which is beneficial in conserving network bandwidth and reducing power consumption. In IoT-based action recognition, the fused data can be uploaded to the cloud or processed locally using fewer resources, such as computing power, memory, and energy. By minimizing the resources needed for uploading and processing, the overall system can operate more efficiently and cost-effectively while still achieving high accuracy and real-time performance.

Figure 3.

Data-level information fusion.

4.3 Data processing methodology

Singular Spectral Analysis (SSA) is a powerful signal-processing technique that has been applied to various domains, including action recognition. In the context of IoT-based action recognition, SSA can be used to implement the skeleton data by combining wearable data [14].

In SSA, time-series data is first embedded into a trajectory matrix, where each row represents a trajectory or a subsequent trajectory of the original data. The trajectory matrix is then decomposed using singular value decomposition (SVD) to separate the data into singular values and corresponding singular vectors. These singular vectors represent the fundamental building blocks of time series. Relevant patterns and features can be extracted by selecting specific singular vectors and their associated singular values.

The fundamental components obtained by SSA can be interpreted as different patterns of variation in the time series. They capture underlying trends and recurring patterns in the data. These components can be further analyzed and combined to reconstruct the original time series or for various applications such as noise reduction, feature extraction, or forecasting. SSA provides a flexible and practical approach to analyzing time series data and has applications in multiple fields such as finance, climate science, and signal processing.

Figure 4 shows the first ten components sorted by singular value, which is the analysis result of the example of accelerator data in the X-axis when falling. The components are clear enough to represent the trade of initial sensors, which can be used to implement the skeleton data, as it is shown in Figure 5.

Figure 4.

Accelerator data in X-axis for falling and sequence of the first ten components sorted by singular value.

Figure 5.

SSA-based skeleton data implementation.

4.4 Experimental results

Two databases are used in the experimental study. To represent the skeleton data, the NTURGB+D database is chosen. The NTURGB+D (NTU RGB + D) database is a comprehensive benchmark dataset widely used for human action recognition and pose estimation. It comprises a total of 56,880 action samples, recorded from 40 subjects performing 80 distinct actions. Each action has 20 instances, resulting in a diverse set of data for analysis. The database includes RGB videos, depth maps, and skeleton data, providing rich multi-modal information for studying human activities.

To represent the initial data, the SCUT-NAA dataset is used. It is a 3D acceleration-based activity dataset that consists of 1278 samples collected from 44 individuals (34 males and 10 females). The data was gathered in naturalistic settings using a single tri-axial accelerometer placed in three different locations: waist belt, pants pocket, and cloth pocket. Each participant was asked to perform ten activities, providing a diverse range of motion data for analysis. To enhance the falling data, the 2015 Fall dataset is added, where data was collected from 32 volunteers specifically focusing on falls. The dataset includes four fall postures: forward, backward, left, and right. Sensors were primarily placed on the chest and thighs to capture both acceleration and angular velocity data during falls. This dataset offers valuable insights into different fall scenarios, enabling researchers to study and develop effective fall detection and prevention algorithms.

After selecting common actions from the above databases and sorting them out, we obtained an experimental dataset with eight actions: Sitting, Walking, Step walking, Jumping, Upstairs, Downstairs, Cycling, and Falling. The total dataset number is 879.

The falling detection result achieved an impressive accuracy rate of 93.82% using the SSA implemented on skeleton data. This outcome demonstrates the effectiveness of the SSA approach in accurately identifying and detecting falls in the dataset. By analyzing the spatiotemporal patterns of skeletal movements, the SSA-based method successfully captured the distinct characteristics associated with falls, leading to highly accurate detection results. The high accuracy rate signifies the potential of SSA-based skeleton data analysis for real-time fall detection systems, which can play a crucial role in ensuring the safety and well-being of individuals, particularly seniors and those at risk of falling. The remarkable performance underscores the value of SSA in enhancing fall detection capabilities and highlights its significance in advancing research and applications in healthcare and eldercare domains.

5. Challenges and opportunities

While the general landscape of DT-enabled smart healthcare at home looks promising and our case study validates its feasibility, there are many open questions yet to be explored. Here, we identify the most critical challenges along with the opportunities in this raising area:

• Security: Although DT technology offers significant advantages for smart healthcare at home, security is still among the top concerns that need to be addressed, such as data integrity, authentication, network security. The integrity of medical/health data is essential and it is mandatory to prevent tampering, manipulation, or unauthorized modifications. Therefore, IoMT-affordable but strong data validation techniques, robust end-to-end data encryption, and secure protocols for data transmission are required. As mentioned in Section 3, a reliable authentication scheme can verify the identity of users accessing DT and their associated healthcare resources. The combination of Blockchain and NFT is a promising approach, but further studies are still needed to address multiple open questions. Network security involves countermeasures to various potential attacks on the smart home network as the system highly relies on wireless-connected devices.

• Privacy: The sensitive medical information asks for strict privacy-preserving protocols. The vital signs, medication history, and data of the resident(s) can be the target of various unauthorized and even malicious agents. Adequate measures such as encryption, access controls, and strict data handling policies should be implemented to ensure the privacy and confidentiality of patient information. Moreover, even authorized data sharing requires traceable, auditable, and transparent operations for users to have full control of their personal medical information. To meet these urgent demands, techniques like dynamic ID changing [15] and reliable time stamp tracking [16] are possible solutions for privacy-preserving.

• Scalability: The scalability problem mainly refers to the efficiency of the system when we expand the smart home into a large-scale network such as a local healthcare community or virtual community in Metaverse. The massive information brought by numerous sensors and devices would have high requirements for the capability of the network.

• Social acceptance: Social acceptance is another important issue as many residents may find it difficult to live with so many sensors, especially cameras. And the sensitive data can cause certain concerns such as misuse or exploitation of these information. Reliability is another problem since the result or decision of the systems is sometimes a matter of life and death.

6. Conclusions and future work

6.1 Conclusions

This chapter envisions the future of digital healthcare services in the IoMT era, a historical time witnessing the proliferation of many new enabling technologies. We discussed the application of Digital Twins, blockchain, and IoMT in the context of the smart home environment. A SHAH framework is introduced as an example of providing real-time monitoring and safety preservation utilizing a combination of different technologies. A preliminary experimental investigation shows the feasibility of using information fusion and statistical analysis to achieve intelligent decision-making for the system. Additionally, several most compelling challenges and open questions are highlighted. We hope this chapter could inspire more discussions in the smart healthcare community and spark new ideas, technical breakthroughs, and novel applications.

6.2 Future work

The preliminary experiment only tested the feasibility of our framework and is limited to a small-scale network. To evaluate the availability of DT-based healthcare services, we need more effort in a large-scale network including the investigation of accuracy and efficiency. Apart from the skeleton recognition and SSA approaches, we will investigate more onsite diagnosis mechanisms and integrate them into SHAH to improve the accuracy of identifying emergent events. Further investigation and studies are required and certain standards need to be established to satisfy local laws and social acceptance.

Abbreviations

AI	artificial intelligence
BLE	Bluetooth low energy
DT	digital twins
GUI	graphic user interface
IoMT	Internet of Medical Things
IoT	Internet of Things
LO	logical object
ML	machine learning
NFT	non-fungible tokens
PC	personal computer
PO	physical object
PPG	photoplethysmography
QoS	quality of service
SSA	singular spectral analysis
SVD	singular value decomposition