Application of Computational Intelligence in Visual Quality Optimization Watermarking and Coding Tools to Improve the Medical IoT Platforms Using ECC Cybersecurity Based CoAP Protocol

Abstract

To ensure copyright protection and authenticate ownership of media or entities, image watermarking techniques are utilized. This technique entails embedding hidden information about an owner in a specific entity to discover any potential ownership issues. In recent years, several authors have proposed various ways to watermarking. In computational intelligence contexts, however, there are not enough research and comparisons of watermarking approaches. Soft computing techniques are now being applied to help watermarking algorithms perform better. This chapter investigates soft computing-based image watermarking for a medical IoT platform that aims to combat the spread of COVID-19, by allowing a large number of people to simultaneously and securely access their private data, such as photos and QR codes in public places such as stadiums, supermarkets, and events with a large number of participants. Therefore, our platform is composed of QR Code, and RFID identification readers to ensure the validity of a health pass as well as an intelligent facial recognition system to verify the pass’s owner. The proposed system uses artificial intelligence, psychovisual coding, CoAP protocol, and security tools such as digital watermarking and ECC encryption to optimize the sending of data captured from citizens wishing to access a given space in terms of execution time, bandwidth, storage space, energy, and memory consumption.

1. Introduction

The emergence of the pandemic threatened the existence of humanity, which led scientists to look for solutions to fight against this scourge and reduce its severity. Many solutions have been developed mobile application CovidSafe and CovidScan to check whether the certificate presented by a citizen in the form of a QR code is valid [1], and platforms COVID-19 diagnosis using machine learning from radiography and CT images [2].

Convinced of the harmful influence of this disease on vulnerable people, fighting against the spread of this virus and especially during access to private and public spaces has become a challenge for researchers. Access to these spaces is reserved to people with a valid vaccination pass, people exempt from vaccination, and people with a negative PCR test of fewer than 48 hours.

Any person protected against coronavirus 19 by obtaining the vaccine has a personal code that allows him/her to access a computer service and that shows his/her immunization schedule. This code is still coveted by dishonest people who do not have the complete vaccination scheme to escape controls and access spaces with the help of criminals who offer falsified health passes containing stolen or false QR codes. Even though these acts are criminalized by the states, criminals are constantly innovating in their search for real QR codes.

These irresponsible actions make the task so hard because instead of concentrating on finding a definitive solution to get rid of this dangerous virus, we waste our time looking for solutions to fight against the theft of people’s data such as the QR Code. This theft is especially pronounced when a large number of people access public places such as sports stadiums, supermarkets, and events with a large number of participants at the same time. During these times, data management is more difficult in terms of execution time and quality of data sent for processing. IoT platforms have been developed to control access to public places [3].

Upon entering a controlled space an IoT node is required to scan the QR Code, read the tag, capture the citizen’s photo and capture the temperature of people to proceed to filter the people who have the right to access from those who do not, and this by controlling the validity of the unique identifier of each one and its belonging to its owner, hence the need for an application that can recognize the face in real-time, processes it, optimize the quality and size of the data. and all this while guaranteeing a fast and secure delivery. Our work aims to use two types of image coding namely visual coding for scanned QR codes, foveal coding for photos captured of people at the entrance of public spaces, Elliptic-curve cryptography (ECC), and watermark to ensure data security.

This work consists of the architecture of our platform based on ECC encryption, CoAP protocol, and technologies used including artificial intelligence, face detection, and recognition in part 2. For part 3, we talk about the psychovisual and foveal coding image using watermarking to evaluate the quality assessment and the execution time of these two coding types. Part 4 presents the comparative results of these coded types of coding/decoding, encryption/decryption, and insertion/extraction of the images and their performances in terms of quality and execution time.

2. Medical IoT platform architecture using AI for face detection and recognition

The evolution of IoT platforms is growing, several sectors apply this technique provided that they integrate Artificial Intelligence [4, 5]. The medical sector [6, 7] requires this type of platform to monitor the health status of these patients, so to be able to distinguish different patients we need facial recognition [8, 9].

Our medical IoT platform (Figure 1) is to prevent the system from crashing while processing the shipment or citizens from waiting in a long queue or using another person’s data to avoid any kind of fraudulent theft. To this end, we used artificial intelligence on the one hand to store the user’s personal information, detection of QR codes and images of citizens, face recognition, as well as decision making. On the other hand, we applied visual coding for the QR code and foveal coding for the citizen’s faces to study their impact on the image quality and the time to send and process these data.

Figure 1.

The medical IoT platform architecture using artificial intelligence.

To access space, a double verification is required; the system verifies the QR code’s legitimacy as well as the holder’s identification using facial recognition to prevent counterfeiting. There are two methods of personal identification that have been established. The first involves scanning the QR code on the health pass or a PCR test that takes less than 48 hours, and the second is face recognition to verify the individual’s identity. After the picture of the person is taken, foveation is used to create a higher-resolution image of the face. The image and QR code are then encoded and decoded; the choice to embed the entire QR code instead of the few bits of the associated information contained in the QR code because it allows us to extract information hidden inside a digital image without distorting the original or losing any data. Authorized recipients can extract not only the embedded message but also the original image, which is an intact and identical bit for bit to the image before the data was inserted. And the most important is that it guarantees and keeps the quality of the image according to the bit rate if the network speed is higher the quality of the image will be with better quality and vice versa. Finally, the images are encrypted with ECC which is a type of encryption that uses an elliptical ECC and a wall cryptosystem that is used in SSL/TLS licenses to encrypt data for devices with limited resources [10]. Moreover, it works with points on an elliptic curve and provides two major benefits; security and a short key length. The memory and bandwidth savings, as well as the processing speed and low power usage [11].

Then, the payload is sent via the communication protocol CoAP (Constrained Application Protocol) to the webserver. CoAP is a communication protocol with less memory consumable and time that we used to make nodes and servers communicate (send and receive data exchanged during execution). CoAP is focused on the transmission of tiny messages, which are often sent through UDP (each CoAP message occupies the data section of one UDP datagram). It has a simple binary format. The payload is made up of a 4-byte fixed-size header, a variable-length token, a CoAP option sequence, and a 4-byte fixed-size header. Moreover, CoAP is characterized by its client/server architecture, where the client transmits a method code, such as GET, PUT, POST, or DELETE, to the server [12]. After receiving a request, the server provides a payload and a response code. Finally, a communication layer and a request/response layer are included. The messaging layer is in charge of message redundancy and consistency, whereas the request/response layer is responsible of connectivity and communication. CoAP also provides multicast communication and asynchronous message exchange, as well as confirmable (CON) after it gets the data, unconfirmable (NON) with a unique identification in the event of an unreliable transmission, Acknowledgement (ACK), and resettable messages (RST) [13].

When the data is received by the server, it must be decrypted before it can be read and compared to the database’s existing data. The person’s entry to the area is granted or denied by a decision system.

Then with the help of the AI, the system begins to process the received information, comparing it to the existing database (Image of the person requesting the access and Image of the real owner of the Qr Code). After comparing the submitted data to the current database, the system decides whether or not to provide the guest access. If the person presenting the QR Code has a valid pass or test, he or she is permitted to access the facility. Access is given if the person is exempt from the health pass and has a fever of no more than 37 degrees. Access is prohibited if someone has a valid QR code but the picture does not match the one previously provided or cannot be identified in the database. Access is disallowed if the person has an expired QR code, a positive PCR test, or is older than 48 hours.

For the AI, the study consists of five stages. The essential phase is the storage of the user’s information in the system. The Faster-RCNN architecture is applied for QR code detection. The MTCNN architecture is used for face detection. Finally, the FaceNet architecture is dedicated to face recognition. Deep learning is performed on a database of a few people. For each face image, the MTCNN model produces a fixed-length embedding vector as a unique facial feature (distinguishing characteristics of people). Thus, each person has multiple similarity vectors using Euclidean distance or cosine similarity. The learning process follows the recognition process, namely: image acquisition, detection, and feature extraction. The acquisition phase consists of resizing the input images and normalizing them by removing the average pixel value. Thus, all detected regions of interest (ROI) are scaled to fit the input CNN architecture. In the extraction phase, its role is the feature vectors, to store them in the database. Finally, the detection phase is based on the MTCNN model based on the delimitation boxes of the faces in an image (Landmarks). The MTCNN model includes three processing blocks to perform face detection and tracking. For the first processing block, several candidate windows go through a shallow CNN (P-Net block) and then through a second more complex CNN that consists of refining the windows to reject a large number of windows that do not contain a face (R-Net block). In the third processing block, a robust CNN is used to polish the result and display the landmark positions of the faces.

For FaceNet, it is a facial recognition system, with a unified integration for facial recognition and clustering. It is an architecture that gives an image of a face, extracts high-quality features from the face, and predicts a 128-element vector representation of those features, called face integration. FaceNet directly learns a scene (images or video) from images of faces in a compact Euclidean space where distances correspond directly to a measure of face similarity. Face-Net takes a face embedding as input and predicts the identity of the face that is stored (for recognition). However, a technique that applies a standardized rotation to the face and relies on the facial cues to align the feature vectors. To do the alignment, we first need to find the facial landmarks as fast as possible (speed of execution), so we used two architectures (MTCNN and FaceNet). To extract from the image only the facial features (eyes, face contour, nose, mouth, etc.) as vectors.

Several works [14, 15] use the Faster R-CNN architecture, in our case, it involves extracting the QR code from the image. This architecture is composed of two branches that share convolutional layers. The first branch is a region proposition network that learns a set of window locations, and the second is a classifier that learns to label each window as one of the classes in the training set. We use all the layers in the network that work with object proposals and extract features from the convolutional layers. Build a global image descriptor from the faster activations of the Faster R-CNN layers. The activations of each filter have the same dimension as the number of filters in the convolutional layer. In general, the Faster-RCNN architecture is based on CNN, consisting of a feature extraction network to extract feature maps from the input image. Meanwhile, the convolution layers do not change the size of the image, so usually adopt a basic size, with a set of pad and stride, and each grouping layer reduces the image by half the original size. Faster R-CNN allows us to obtain better feature representations for detection or extraction of Qr-Code from images and improves the performance of spatial analysis and reanalysis.

Our AI platform makes the decision based on the three architectures mentioned earlier: MTCNN, FaceNet, and Faster-RCNN. The extracted features are compared to those stored in the face and QR code database by averaging a similarity metric, in our case we opted for the Euclidean distance. After exploring MTCNN face detectors, including a FaceNet facial recognition package, for verified access.

3. Watermarking for foveal and visual image coding to evaluate the quality assessment

Psychovisual coding algorithms consist of optimizing the image quality according to the image complexity. The characteristics of the human visual system (HVS) on the frequency and spatial domain are exploited for best coding results.

Our system is designed according to the scheme shown in Figures 2 and 3. The construction steps are the acquisition of the image QR code and face of the citizen, decomposition of the image by applying the wavelet transform (DWT), the psychovisual weighting model, watermark embedding, and the SPIHT scalable coding. The reconstruction steps are the psychovisual inverse weighting model, watermark extraction, and the SPIHT scalable decoding reconstructed image.

Figure 2.

Visual coding scenario for QR code image.

Figure 3.

Foveal coding scenario for Citizen’s face image.

It is important to compress the image in order to guarantee a low memory space and a fast image transmission, without degrading the image quality. First, we decompose the original image using discrete wavelet transform. This algorithm consists in cortically splitting the image into a biorthogonal set of wavelets using two filters of type low-pass and high-pass. The use of LPF allows extracting the edges and details of the image, while HPF extracts the most important information seen by the eye.

DWT is a transformation used for frequency domain analysis of image [16, 17]. DWT decomposes the image into four non-overlapping multi-resolution sub-bands: LL1 (Approximate or Low-Low sub-band), HL1 (Horizontal or high-Low sub-band), LH1 (Vertical or Low-high sub-band), and HH1 (Diagonal or High-high Sub band). Here, LL1 is a low-frequency component whereas HL1, LH1, and HH1 are high frequency (detail) components.

All wavelets used are based on the Daubechies 9/7 filter for an ideal reconstruction of the image. The benefit of this type of DWT compression is that it ensures fast computation and fewer resources with interesting mathematical properties. Then, a psycho-visual weighting filter is implemented to process the wavelet sub-bands. This model uses the contrast sensitivity filter to discriminate between low frequencies and remove invisible ones. The luminance setting and contrast calibration are adapted according to the perceptual thresholds based on the JND wavelets.

The weighting model combines the following steps; The application of the JND just noticeable distortion model which is established from the adaptation of the luminance and contrast of the image to improve the performance of the perceptual coding. The measurement of the visibility threshold is then based on the JND model. We apply a CSF contrast sensitivity filter that masks invisible frequencies by taking into account the properties of human frequency sensitivity. The luminance mask is then operated on the original wavelet spectrum to adapt the light of the image. The correction factor of the luminance mask is taken into account to varying the luminance of the coded image. The contrast mask is then used according to the perceptual thresholds to calculate the contrast correction, which allows to eliminate the invisible contrast information and to enhance the perceptible information.

The same weighting model is applied to both the visual coding of the QrCode and the foveal coding of the visual image, with an adaptation of the localization of the regions of interest by applying a foveal filter in the case of foveal coding.

Foveation is a lossy filter that reduces the size of the transmitted image by preserving the relevant information and removing all the background from the image that will not be processed by the system. This filter is used to extract the regions of interest and reduces the wavelet coefficients by applying a low-pass filter while focusing on the target region. It is important to determine the parameters of the foveation filter in order to keep the good quality of the image and the needed information. One of the characteristics of the foveation filter is the determination of the frequency spectrum of the area of interest in the function of the distance, when the observation distance increases, the high-frequency areas are higher and higher.

For Watermarking, it is one measure among others to have a good defense against copying [18]. It’s a method for embedding data into a multimedia element such as an image, audio, or video file [19]. Current watermarking methods often aim at a certain level of robustness against intrusions that aim at getting rid of the hidden watermark at the cost of destroying the data quality of the media [20]. This chapter presents an image watermarking method based on Discrete Wavelet Transform (DWT). The proposed method is based on a 3-level discrete wavelet transform (DWT). First, the original image of size $512\times 512$ is DWT decomposed into the third level using Haar wavelet providing the four sub-bands LL3, LH3, HL3, and HH3 [21]. In the same manner, 3 level DWT is also applied to the watermark image. For this Haar wavelet is used. Digital image Watermarking consists of two processes that are watermark embedding and watermark extracting [22] described below (Figure 4).

Figure 4.

Watermarked image process.

For watermark embedding, we need a host image and a watermark image, then we begin our process. Firstly, the First level DWT is performed on the host image and the watermark image to decompose it into four sub-bands LL1, HL1, LH1, HH1, and wLL1, wHL1, wLH1, wHH1 respectively. Then, the second level DWT is performed on the LL1 and wLL1 sub-band to get four smaller sub-bands LL2, HL2, LH2 et HH2 and wLL2, wHL2, wLH2, wHH2 respectively. While the third level DWT is performed on the LL2 and wLL2 sub-band to get four smaller sub-bands LL3, HL3, LH3, HH3, and wLL3, wHL3, wLH3, wHH3 respectively. Next, A embedding function is used to add the two sub-bands are added with an embedding formula with the value alpha as in is as follows: $\mathit{\text{newLL}}3=\mathit{LL}3+{\mathit{\text{alpha}}}^{\ast }\mathit{wLL}3$. After, the Inverse DWT is performed using the sub-bands newLL3, LH3, HL3, HH3 to get image new LL2. Then, the same Inverse is done using the sub-bands newLL2, LH2, HL2, HH2 to get image new LL1. The last one is performed using the sub-bands newLL2, LH1, HL1, HH1 to get the watermarked image.

For the watermark extracting phase, we start by performing the first level DWT on the host image and the watermarked image to decompose it into four sub-bands LL1, HL1, LH1, HH1, and nLL1, nHL1, nLH1, nHH1 respectively. Then, performing the second level DWT on the LL1 and nLL1 sub-band to get four smaller sub-bands LL2, HL2, LH2, HH2, and nLL2, nHL2, nLH2, and nHH2 respectively. And the third level DWT on the LL2 and nLL2 sub-band to get four smaller sub-bands LL3, HL3, LH3, HH3, and nLL3, nHL3, nLH3, nHH3 respectively. Next, the following extract is performed to get wLL3 with the extraction formulae with the same value of alpha as in embedding $\mathit{wLL}3=\left(\mathit{nLL}3-\mathit{LL}3\right)/\mathit{\text{alpha}}$. After that, we apply inverse DWT on wLL3 with all other subbands (LH, HL, HH) equal to zero to get wLL2. Finally, we repeat the last step 5 two times at each level to get the extracted watermarks (Figures 5 and 6).

Figure 5.

The DWT compression with face detection and recognition diagram.

Figure 6.

The DWT compression with QR code embedding diagram.

We developed a visible difference predictor (VDP) metric to evaluate the quality between the reference image and the decoded image. It highlights the set of SVH features by using the wavelet transform to analyze the content of an image. The VDP metric can automatically detect errors in an image that are not visible to the human eye. The principle is to compare the original image and the degraded image by associating each point of the visibility map with a visibility probability. An improved MDVP model is presented in our work [23]. We apply the previously explained weighting model to compare relevant information and neglect unseen information and use a psychometric function to examine the quality factor PS. A mathematical Minkowski summation of all wavelet subbands is then performed to determine the mean opinion score (MOS). The larger the factor determined, the higher the image quality, hence the important role this metric adopts. Similarly for foveal coding, we foveally evaluate the image quality using an FDVP metric improved from the VDP metric by applying a foveal weighting model that uses the foveal filter for the detection of areas of interest [24].

The SPIHT progressive encoder is the final phase which has the task of improving the quality of the image progressively and prioritizing the image. The SPIHT encoder is an improved version of the zero tree EZW encoder for lossless compression. The progressive aspect of this encoder is to detect the most relevant information and send it first, transmitting the most significant bits first and then the least significant bits.

4. Discussion and results

The transmission of Qr Code and CitizenPicture images classically avoids many problems in terms of size, complexity, loss of time and memory, as well as security, and to ensure their rapid transmission without loss of any of these and specifically without degradation of image quality, We have tried in this work to integrate the classical SPIHT coding, the psychovisual coding, and to highlight the impact of the graphic security based on watermarking and the payload security based on ECC encryption on the execution times.

A broad comparative analysis has been performed which is divided into three evaluations; a qualitative study presented in Tables 1 and 2, a subjective study illustrated in Figures 7 and 8, and finally the quantitative study displayed in Tables 3 and 4.

Table 1.

Lena EFIC foveal coding images versus the SPIHT coding images and QR code EVIC visual coding images versus its optimized version SPIHT with their quality scores PS using visual WVDP assessor given for varying bit rate and fixed observation distance. The bit rate varies as follow: A. 0.25 bpp, B. 0.15 Bpp, C. 0.0625 bpp, D.0.0313 bpp, a. 0.25 bpp, b. 0.15 bpp, c. 0.0625 bpp, d. 0.0313 bpp.

Table 2.

The watermarking Lena SPIHT coding images (column 1) versus the SPIHT Dewatermarking QR code images (column 2) and watermarking Lena EFIC foveal coding images (column 3) versus the EVIC version Dewatermarking QR coding (column 4) with their quality scores PS using visual WVDP assessor given for varying bit rate and fixed observation distance. The bit rate varies as follow: A. 0.25 bpp, B. 0.15 Bpp, C. 0.0625 bpp, D.0.0313 bpp.

Figure 7.

EFIC vs. SPIHT applied to non-secure CoAP CitzenPicture.Png payload.

Figure 8.

EVIC vs. SPIHT applied to non-secure CoAP deWatermarkedQrCode.Png payload.

Bit Rate (bpp)	128:1	64:1	32:1	16:1	8:1	4:1	2:1
EFIC vs. SPIHT non secure(%)	5.51	8.24	8.83	8.75	14.35	6.33	13.34
EFIC vs. SPIHT secure(%)	1.14	0	1.17	3.88	9.62	14.89	8.49

Table 3.

Execution time gain percent EFIC vs. SPIHT non secure for CitzenPicture.Png and secure version with ECC.

Bit Rate (bpp)	128:1	64:1	32:1	16:1	8:1	4:1	2:1
EVIC vs. SPIHT non secure(%)	14.04	6.86	8.58	9.13	23.96	22.27	17.20
EVIC vs. SPIHT secure(%)	10.13	2.78	1.95	8.85	9.83	15.55	21.27

Table 4.

Execution time gain percent EVIC vs. SPIHT for Dewatermarked QR code non secure and its secure version with ECC.

Starting with Table 1, the results obtained concern the different types of coding with different degrees of quality according to four types of binary budgets. Then, the foveal EFIC coding for the CitizenPicture which is in our case Lena test compared it with its reference SPIHT. And the visual coding for QrCode and SPIHT.

From these results, we notice that at the beginning (A. FPS and a. FPS) with a bit rate of 4:1 the images have excellent quality, at the level (B. FPS and b. FPS) with a bit rate of 8:1, the images hold good quality. The images presented in (C. FPS and c. FPS) with bit rate 16:1 have a medium quality, and finally for (D. FPS and d. FPS) with bit rate 32:1 the images retain bad quality. On the other hand, Lena coded with EFIC that focuses on the face and not the whole image provides a better quality comparing it with SPIHT Lena. The same goes for the visual coding of the QrCode EVIC, it is better than SPIHT in terms of perceptual quality.

Moving on to Table 2 which illustrates the Watermarking Lena SPIHT (column 1) versus SPIHT deWatermarking QrCode (column 2) and the Watermarking Lena EFIC (column 3) versus EVIC deWatermarking QrCode (column 4) according to different quality and bit rate budgets. From these images, we get four types of bit rate high bite rate which gives excellent quality, medium bite rate which offers good quality, reasonable bite rate which delivers acceptable quality, and inferior bite rate which supplies unsatisfactory quality. On the other hand, we notice that the watermarking Lena EFIC versus the Dewatermarking QrCode EVIC provides an excellent quality when compared with its reference SPIHT.

So, from Tables 1 and 2, we can distinguish that when the binary budget increases we obtain images with good quality whatever the coding. Thus, the psychovisual coding is the best codage compared to SPIHT in terms of quality.

Based on the subjective study illustrated in Figure 5, we have curves that present the bite rate execution time consumption for Citizen Picture (Lena) for EFIC versus SPIHT coding in both secured with ECC and unsecured versions. And Figure 6, presents the EVIC versus SPIHT coding for the deWatermarked QrCode. From these two figures, we can see that EFIC for the Lena test and EVIC for Dewatermaking consume little time compared to SPIHT.

Let us talk about the security, we can notice that the secured version in both figures is more consuming compared to the non-secured version which is normal when we add the security layer to secure our images but this does not prevent us from saying that ECC does not consume much time if we compare it with non-secured and other security algorithms.

Finally, moving to the quantitative study presented in Tables 3 and 4, we have the percentage gain in terms of execution time for CitizenPicture in the non-secure and secure version (Table 3), and the gain in execution time EVIC versus SPIHT for Dewatermarked QrCode non-secure and secure (Table 4). We notice that the execution is faster when the binary budget is lower than 16:1. So from all these results, we can say that even if the execution time increases when the binary budget is higher than 16:1 we obtain images with good quality but when the budget decreases the quality of the images becomes mediocre and weak.

5. Conclusion

In this work, we have processed the QR code images and CitizenPicture captured at the entrance of public or private spaces by decomposing them through the application of the DWT, the psychovisual weighting model, watermark embedding, and the SPIHT embedded coding/ decoding and for the construction of these images we have used the psychovisual weighting model, watermark extraction and the SPIHT embedded coding/decoding.

These visual optimization techniques, based on human visual cortex usage and quality optimization tests, have been used to develop the performance of the Medical IoT platform. Encouraging results were obtained in terms of reduced execution time, storage space, bandwidth and memory load. Adding the watermark to the ECC made it possible to send the citizen’s picture containing the QR Code securely with the CoAP protocol.

The use of this Medical IoT platform will help fight the corona virus pandemic by allowing a large number of people to simultaneously access a space with full security of their private data such as their CitizenPicture and QR Code.