Open access peer-reviewed chapter - ONLINE FIRST

Weighted Module Linear Regression Classifications for Partially-Occluded Face Recognition

By Wei-Jong Yang, Cheng-Yu Lo, Pau-Choo Chung and Jar Ferr Yang

Submitted: May 14th 2021Reviewed: September 24th 2021Published: November 5th 2021

DOI: 10.5772/intechopen.100621

Downloaded: 21

Abstract

Face images with partially-occluded areas create huge deteriorated problems for face recognition systems. Linear regression classification (LRC) is a simple and powerful approach for face recognition, of course, it cannot perform well under occlusion situations as well. By segmenting the face image into small subfaces, called modules, the LRC system could achieve some improvements by selecting the best non-occluded module for face classification. However, the recognition performance will be deteriorated due to the usage of the module, a small portion of the face image. We could further enhance the performance if we can properly identify the occluded modules and utilize all the non-occluded modules as many as possible. In this chapter, we first analyze the texture histogram (TH) of the module and then use the HT difference to measure its occlusion tendency. Thus, based on TH difference, we suggest a general concept of the weighted module face recognition to solve the occlusion problem. Thus, the weighted module linear regression classification method, called WMLRC-TH, is proposed for partially-occluded fact recognition. To evaluate the performances, the proposed WMLRC-TH method, which is tested on AR and FRGC2.0 face databases with several synthesized occlusions, is compared to the well-known face recognition methods and other robust face recognition methods. Experimental results show that the proposed method achieves the best performance for recognize occluded faces. Due to its simplicity in both training and testing phases, a face recognition system based on the WMLRC-TH method is realized on Android phones for fast recognition of occluded faces.

Keywords

  • face recognition
  • linear regression classification
  • occlusion tendency
  • weighted module face recognition
  • texture histogram difference

1. Introduction

As the progress of the computer vision and machine learning, person identification and verification for security considerations become practical and play an important role for modern smart living applications. Face recognition for security control has received lot of attentions recently. Designed with a camera, the face recognition becomes a simple and direct way to achieve reliable identification and verification. There are numerous algorithms [1, 2, 3, 4] made important contributions to face recognition. These approaches can be divided into two categories, holistic based and modular based methods [5]. The holistic based approaches include the principle component analysis (PCA) [6] and linear discriminant analysis (LDA) [6]. John et al.[7] exploiting sparsity representation to improve the recognition rate. On the other hand, the modular based approaches provide more valuable features for face recognition. Naseem et al.[8] proposed linear regression classification (LRC) and its modular design to solve occlusion problems. During training stage, the LRC algorithm is to form the regression surface, which has the most fit of the data distribution, of each identify. In the testing stage, the unlabeled face image vector is used to recognize the person by finding the shortest project distance to the regression surface of the identify.

In practical applications, especially in COVID-19 pandemic period, face recognition might encounter diversified challenges such as low resolution, luminous change, facial expression and partial occlusion affected by facial accessories and covering face masks and scarf. For various challenges, there are several improved LRC algorithms proposed in the literatures, such as PCA-based LRC [9], unitary LRC [10], class-specific kernel LRC [11], and sparse representation classifier [12] methods.

Figure 1 shows some examples of partially-occluded faces, where they are partially covered by facial accessories or covering obstacles. Partially occlusion is one of the practical and difficult problems in face recognition [13, 14]. Thus, some researches have focused on partially-occluded faces [15, 16, 17, 18, 19] recently. Most approaches tried to detect the testing images in several modules, avoided occlusion area influence the recognition results. The module LRC (MLRC) proposed in [8] is an effective way to solve occlusion issues in facial recognition. The MLRC treats each module equally important, however the occluded modules should be excluded or reduced their contribution for the final classification. Thus, by using regression parameters (RP), our early-version weighted module linear regression classification (WMLRC-PR) [20] is first proposed to enhance the MLRC method. To secure the access of ATM machines, the occluded face detection methods are proposed [21, 22]. By using support vector machine (SVM) to detect the occlusion modules, the LDA face recognition method is performed in the non-occlusion part [23]. By using structured dictionary learning (SDL) method, the SRC face recognition can separate occluded face images [24]. The robust face recognition algorithms based on various features, such as Huber loss [25], local binary feature [26] and topology preserving structure matching (TPSM) [27] are proposed to solve occlusion problems successfully. It is noted that the state-of-the-art CNN approaches [28, 29, 30, 31], which need pre-trained by all face images, will not be applicable for partially-occluded face recognition. Besides, the CNN approaches usually need to pre-train the model, any instance increase of new members becomes impossible.

Figure 1.

Examples of partially-occluded situations.

In this chapter, we propose a better occlusion tendency detection by using texture histogram (TH) to distinguish occluded and non-occluded modules. With the TH occlusion tendency, the WMLRC-TH methods are then proposed. The rest of the chapter is organized as follows. In Section 2, we first review the basic LRC concept and introduce the module LRC design for face recognition. Then, Section 3 first analyzes texture histograms and discusses how to use it to determine the occlusion tendency of the module. The proposed WMLRC-TH method in use of texture histogram is the proposed. In Section 4, the proposed WMLRC-TH method, which is tested on AR [32] and FRGC2.0 [33] face databases with synthesized occlusions, is compared to the well-known and robust face recognition methods. The realization of the partially-occluded face recognition system based on the WMLRC-TH method in mobile platforms are also addressed. Finally, conclusions are addressed in Section 6.

Advertisement

2. Module linear regression classification for face recognition

For face recognition, we assume that all training and testing facial images are preprocessed by face detection, face cropping and possible face alignment functions. Assume that we have Csubjects, which are characterized by training facial images used for identity recognition. For Cidentities, the ith identity is with Ntraining facial images all with the size of p × qpixels. Each pixel could be further represented with Kcolor channels. Thus, the kth channel data of the ith identity is represented by vi,j,kRp × q, i = 1, 2, …, C, j = 1, 2, …, Nand k = 1, 2, …, K, where K = 3 for most RGB face images.

For the kth channel, we can cascade its qcolumn vectors into a larger column vector, xi,j,kRpq × 1. For the ith class identify, its Ntraining vectors, xi,j,k, j = 1, 2, …, Nare grouped into the ith class vector space as

Xi,k=xi,1,kxi,2,kxi,N,kRpq×N,i=1,2,,C.E1

If νi,j,kis reformed by linearly combining channels to one grayscale channel, we set K = 1 or just remove the subscript kas xi,jRd × 1 and XiRpq×Nfor simplicity. For face recognition, the linear regression classification (LRC) and module LRC are briefly reviewed as follows.

2.1 Linear regression classification

Let ukRp × qbe the kth channel of an unlabeled query image, which is formed as a column vector ykRpq × 1. If the query data vector ykbelongs to the ith class, the prediction with the linear combination of the ith class vector space can be rewritten as

yk=Xi,kβi,k,i=1,2,,C,E2

where βi,kRN × 1 is the regression parameter vector of the ith class. The optimal regression parameter vector can be solved by the least square optimization and expressed in matrix operations as

βi,k=Xi,kTXi,k1Xi,kTyk,i=1,2,,C.E3

With optimized regression parameter vector, βi,k, the predicted response vector y˜i,kof the kth channel for the ith class can be formed as

y˜i,k=Xi,kβi,k,i=1,2,,C.E4

For all Cclasses, we first compute the kth-channel predicted response vectors. To construct the recognition system, by combining Eq. (3) and Eq. (4) together, we can store the pq×pqprojection matrix of the ith identity as:

Hi,k=Xi,kXi,kTXi,k1Xi,kT,k=1,2,,K.E5

The construction of the ith projection matrix can be treated as the training process of the ith identity. For the trained parameters of all Cidentities, we need to store Cp2q2 coefficients for recognition. The computation of predicted response vector becomes the projection process as

y˜i,k=Hi,kyk,k=1,2,,K,E6

which takes about Kp2q2 multiplications. For testing all Cidentities, one identity recognition process, it needs CKp2q2 multiplications. To determine the best class, the LRC method chooses the identity, which is with the minimum prediction error of all channels. Thus, the classified identity i* is determined by minimizing the L2-norm distance between the predicted response vector and the query data vector as

iLRC=argminidi,i=1,2,,C.E7

where

di=k=1Ky˜i,kyk2,i=1,2,,C,E8

for K-channel classifications.

2.2 Module linear regression classification (MLRC)

For real applications, if a query sample is partially corrupted or occluded, the LRC algorithm cannot handle these special situations well since the discernible messages are getting less. The module, which is in a small and clean subface, approach can efficiently overcome this problem.

For the module LRC (MLRC), each training image νi,j,kRp × qxKis segmented into Mnon-overlapped partitions. Figure 2 shows an example of segmentation of facial images with M = 16. The mth module, as the LRC approach, is formed as a column vector xi,j,km, i = 1, 2, …,C, j = 1, 2, …, N, m = 1, 2, …, M. As Subsection 2.1, in the training phase, the column vectors of all training image related to each class are grouped accordingly. Hence, for the ith class, we could obtain Mgroups of the kth channel data spaces as

Figure 2.

Segmentation of face images withM = 16 modules.

Xi,km=xi,1,kmxi,2,kmxi,N,kmRpqM×NE9

for i = 1, 2, …, C, j = 1, 2, …, N.

For the kth channel of the query image uk, as the training images, we first segment it to Mmodules and form them as column vectors ykmfor m = 1, 2, …, M. If the mth module of the query image, ykmis assumed to lie on the mth module of the ith class, the least square prediction can be expressed in the training images of the mth module in the ith class as

ykm=Xi,kmβi,km,i=1,2,,C,m=1,2,.,M,E10

where the optimal regression parameter vector is given as

βi,km=Xi,kmTXi,km1Xi,kmTykm=Ri,kmykm.E11

and the regression matrix is defined as

Ri,km=Xi,kmTXi,km1Xi,kmT.E12

Similar to Eq. (5), the corresponding module response vectors y˜i,kmcan be predicted as

y˜i,km=Xi,kmβi,km=Hi,kmykm,E13

where Hi,kmis the (pq/M) × (pq/M) projection matrix of the mth module of the ith identity as

Hi,km=Xi,kmXi,kmTXi,km1Xi,kmT,k=1,2,,K.E14

The project matrix, Hi,km, for m = 1, 2, .., M, k = 1, 2, …, Kcan be treated as the training parameters for the ith identity. For the trained parameters of Mmodules of all Cidentities, we need to store CKp2q2/Mcoefficients for recognition process. For testing all Cidentities, we can directly use (13) to compute the project result. Thus, the MLRC recognition process, which needs to compute all y˜i,kmfor M-module and K-channel p × qfacial images by using (13), will need CKp2q2/Mmultiplications. The computation of the MLRC is less than the LRC method, which needs CKp2q2 multiplications.

With the above module optimization, each module is processed by the LRC computation individually. Without the knowledge of occluded modules, the minimum distance could be used as the distance metric which implicates that the occluded modules will be automatically removed. The MLRC in min-min distance measure is expressed as.

iMLRC=argminiminmdim,E15

where

dim=k=1Ky˜i,kmykm2,i=1,2,,C;m=1,2,,M.E16

for K-channel classifications.

Advertisement

3. Weighted MLRC by detection of occluded modules

The module linear regression classification (MLRC) shows better recognition performance under occlusion situations. However, the MLRC with min-min distance measure only uses the best module for classification essentially. Thus, we can further improve its classification performance if we can properly gather more reliable non-occluded modules. As shown in Figure 2, we should fully utilize all top 8 clear modules (#1, #5, #9, #13, #2, #6, #10, #14) modules and moderately adopt 4 partially-occluded modules (#3, #7, #11, #15) for face recognition.

Thus, in the early age, we suggested an occlusion detection measure, which can be used to infer the occlusion tendency of the module. For linear regression classification method, the occlusion detection could be obtained by regression parameters [15] by the assumption that the regression parameters of the clean modules will have large variations for in-class and out-class faces. The module with small variations of regression parameters will have high probabilities to be detected as the occluded modules. The regression parameter (RP) concept is only good for dark and less texture occlusion objects. If the face recognition algorithm is not a linear regression approach, it will need extra computation to compute the regression parameters. In this chapter, we suggest a simple detection method to find the occlusion tendency of the module. Generally, the module has its particular location, the texture distribution of the module could easily be identified by its texture histogram.

3.1 Occlusion detection by texture histogram

To detect the occluded modules, instead of regression parameters, we can also perform the occlusion detection in the texture domain. First, the D-bin texture histogram of a grayscale image I(x, y) ∈ Rp × qis expressed by

hIxy=t1titDTE17

where the count of gray level I(x, y) for every b = 256/Din Dbins will be collected as.

ti=y=1px=1qqiIxyE18

and qipis to quantize pinto the ith bin as

qip=1,ifi1bp<ib,0,else`E19

For finding D-bin texture histogram of modules of the ith class among Ntraining samples, we first obtain the expected vmias

v¯i,km=1Nj=1Nvi,j,km.E20

where vmi,j,kis the kth channel of the mth module, νi,j,kRp × q. Let the kth channel of the query image uRp × q, ukbe segmented into Mpartitions as ukm. To evaluate the occlusion tendency of the modules, γim, which denotes the distance between the D-bin texture histograms of the mth module of query image ukmand the averaged module v¯i,kmafter summing all Kchannels, becomes

γ¯im=khukmhv¯i,km2.E21

It is obvious that the modules of the query image, which are with larger γ¯im, will have more occlusion tendency due to dissimilar intensity distribution. In AR face database, we test 200 scarf images, With respect to 100 identities, Figure 3 shows 200x100 overlapped curves of γ¯imfor all 16 modules. The results also show that γ¯imin the occluded modules with m = 4, 8, 12, 16 are larger than the reminding ones. Similarly, the texture histogram difference could be also used to determine the occlusion tendency.

Figure 3.

Plots ofγ¯imfor all 16 modules of 200x100 scarf query images in AR face database.

Conceptually, the texture histogram difference (HTD) of the module is a meaningful measure to identify the location of the face. For example, the modules #5 and #6 are forehead and left-center-eye modules, respectively. The average of texture histogram for module #5 has flat texture distribution of pure skin color while that for module #6 is unevenly distributed with textures of eyeball and nose regions. These two modules have their distinct and special texture histograms. If the modules are occluded by the mask, their texture histograms become very different from the original ones. For any face recognition algorithm, the classification criterion generally can be expressed by

iface=argminifiy,i=1,2,,C,E22

where fi(y) denotes the ith class score function of the face recognition algorithm. If we divide the face region into Mmodules, the adaptive weighted module face recognition algorithm generally becomes

imfaceHT=argminim=1Mgγ¯imfiym,i=1,2,,C,E23

by using the above averaged texture histogram difference stated in (21). In (23), gγ¯imis a function of HTD parameters should be properly designed to achieve the best face recognition performance. Without loss of generality, we apply the HTD parameters to the adaptive weighted MLRC method in the following subsection.

3.2 Weighted MLRC method by texture histogram

By using texture histogram difference (THD), we suggest a weighted MLRC method, called WMLRC-HT for robust face recognition. For minimization measure errors, it is obvious that occluded modules should give larger weights in the WMLRC such that we can improve the drawback of the MLRC and achieve better recognition performance. Similarly as the observation shown in Figure 3, the texture histogram distance, γ¯imhas highly correlated to the occlusion tendency of the module, In other words, the occluded module has larger γ¯imthan the normal one. Thus, we define the texture histogram (TH) weight for the mth module to be

wTHm=gγ¯im=γ¯im1.E24

With (24), the TH weight will be bigger for smaller texture histogram difference. Thus, wTHmfor the occluded module is smaller than the normal one.

As the module LRC, the response vector y˜imis predicted in terms of Xi,jmas

y˜im=Ximβ̂m.E25

By using THD weight defined in (24), the WMLRC-TH with the weighted minimum rule can be adjudicated as

iTH=argminimy˜imymwTHmmwTHm.E26
Advertisement

4. Experimental results

In our experiments, the recognition performance with different face recognition algorithms will be used to validate the proposed methods. In experiments, AR and FRGC 2.0 face databases are used by synthetically adding osculation. We compare recognition performances of the proposed WMLRC methods to those of PCA, LDA, LRC, MLRC, SRC, locality preserving projection (LPP) [34], neighboring preserving embedding (NPE) [35] methods. All the experiments are carried out on a personal computer, which is equipped with Intel Core2 Q9400 CPU associated a 4-GB RAM. The testing environment is on Microsoft Visual Studio 2013 with OpenCV.

4.1 Experiments on AR database

The famous face recognition algorithms and recent robust face recognition methods [23, 24, 27] all show the recognition performances on AR database, which contains more than 100 people’s color face images. For experiments, the cropped facial images are normalized to 40 × 40 pixels. The first experiment is to test all the algorithms on AR database [32] under sunglasses, scarf, colored blocks, black, white, and texture occlusions, overlapped onto normal faces. The AR database consists of more than 100 people’s color face images. The images also include several facial variations, such as illumination change, expression and facial disguises. We choose a subset of 100 subjects with 50 men and 50 women. In each subject, 3 normal images are used as the training set as shown in Figure 4 while 2 images with sunglasses and 2 images with scarf are used for the testing set as shown in Figure 5.

Figure 4.

Normal faces as the training set on AR database.

Figure 5.

Sunglasses and scarf occluded faces on AR database.

Before the experiments, we should first decide suitable settings for the proposed WMLRC-TH method. For module designs, the images are divided into 16 modules, i.e., M = 16. For the WMLRC-RP, we choose α, ε, and γto be fixed to 2, 0.3, 0.7, respectively. The number of bins in texture histogram is D = 64 for WMLRC-TH. For the MLRC, we further modify the min-min criterion stated in Eq. (12) to become min-min 4 criterion, the MLRC-4, which is the sum of distances of the best 4 modules as

iMLRC4=argminim=14minmdim,E27

By using original AR database, Table 1 shows the comparison results with those normal face algorithms while Table 2 exhibits the reported results from the recent robust face recognition methods. From Table 1, the results show that the proposed WMLRC-TH methods achieved with over 95% accuracy outperforms the other methods. The LDA with SVM method [23] achieves 91.5% recognition rate, the SRC with SDL method [24], the recognition rate for sunglasses plus scarf face images, the recognition rate achieves up to 92%. The topology preserving structure matching (TPSM) [27] achieve 91.7%. recognition accuracy. From Table 2, the proposed WMLRC-TH outperforms the WMLRC-RP [20], the LDA-SVM [23], SRC-SDL [24] and TPSM [27] methods for recognizing occlusion face images on AR database.

SituationsMethods
PCALDALPPNPELRCMLRC
Sunglasses34.528.016.549.064.088.0
Scarf5.018.012.59.023.045.0
Average19.7523.014.529.043.565.0
SituationsMethods
MLRC- 4SRC [12]WMLRC-RP [20]WMLRC-TH (K = 1)WMLRC-TH (K = 3)
Sunglasses72.093.089.095.097.5
Scarf92.052.087.596.097.0
Average82.072.588.2595.597.25

Table 1.

Recognition performances (%) achieved by different methods for sunglasses and scarf images on AR database.

Robust Recognition MethodsWMLRC-RP [20]LDA-SVM [23]SRC-SDL [24]TSPM [27]WMLRC-TH (K = 3)
Average Recognition Rates86.091.592.091.797.25

Table 2.

Recognition performances (%) achieved by the recently-proposed robust face recognition algorithms on AR database.

To further analyze the detailed performances of the algorithms with different conditions of occlusions. The synthesized occlusions with various occlusion levels overlapped to the normal faces are shown in Figures 69. We still keep all the face recognition algorithms with same settings, it is noted the synthesized occlusion images are used for the testing results.

Figure 6.

Added colored occlusion with occlusion ratios on AR database.

Figure 7.

Black partial occlusion with various occlusion ratios on AR database.

Figure 8.

White partial occlusion with various occlusion ratios on AR database.

Figure 9.

Different occlusion (a) - (e) color wall patterns with 50% occlusion ratio on AR database.

Tables 36 show the recognition performances of all the face recognition methods for all synthesized occlusion cases, where WMLRC-TH1 and WMLPRC-TH3 denote the WMLRC-TH preformed with gray (K = 1) and RGB color (K = 3) images, respectively. The experimental results show that the proposed WMLRC-TH with gray and color images outperform the other methods, which can achieve over 95% of accuracy. The MLRC and MLRC-4 performed badly in dark occlusion because low pixel value produces low prediction error, which leads to error recognition. All results show that the proposed WMLRC-TH methods achieve the best recognition performances. The WMLRC-TH had a few benefit from RGB color information since the WMLRC-TH (K = 3) exhibits better recognition performance than WMLRC-TH (K = 1).

MethodsOcclusion ratio
10%20%30%40%50%Average
PCA [6]31.016.07.05.03.012.4
LDA [7]40.027.012.011.06.019.2
LPP14.09.05.03.03.06.8
NPE30.020.012.09.07.015.6
LRC [8]50.033.023.011.06.024.6
SRC [12]84.070.070.068.040.066.4
MLRC [8]87.087.01.01.01.035.4
MLRC-498.098.01.01.01.039.8
WMLRC-RP [20]88.088.088.081.081.085.2
WMLRC-TH (k = 1)98.098.098.098.096.097.6
WMLRC-TH (k = 3)98.098.098.098.098.098.0

Table 3.

Recognition performances (%) of different methods with varying occlusion ratios of dark partial occlusion on AR database.

MethodsOcclusion ratio
10%20%30%40%50%Average
PCA53.063.061.032.052.052.2
LDA42.067.067.017.020.042.6
LPP21.039.031.011.016.023.6
NPE49.068.061.030.043.050.2
LRC86.053.030.012.06.037.4
SRC96.093.076.068.039.074.4
MLRC97.097.089.079.039.079.4
MLRC-497.096.094.070.061.083.6
WMLRC-RP91.088.084.081.043.077.4
WMLRC-TH (k = 1)98.098.093.085.061.087.0
WMLRC-TH (k = 3)98.098.095.086.070.089.4

Table 4.

Recognition performances (%) of different methods with varying occlusion ratios of colored block occlusion on AR database.

MethodsOcclusion ratio
10%20%30%40%50%Average
PCA13.09.08.07.02.07.2
LDA23.06.04.08.06.09.4
LPP10.010.05.04.04.06.6
NPE13.06.07.05.05.07.2
LRC28.015.012.012.09.015.2
SRC86.086.070.068.039.083.6
MLRC87.087.082.082.081.083.8
MLRC-495.095.095.095.095.095.0
WMLRC-RP89.089.084.084.078.084.8
WMLRC-TH (k = 1)98.098.095.095.095.096.2
WMLRC-TH (k = 3)98.098.095.096.096.097.2

Table 5.

Recognition performances (%) of different methods with varying levels of white partial occlusion on AR database.

MethodsOcclusion texture
ABCDEAverage
PCA20.010.022.012.016.016.0
LDA18.015.019.011.016.015.8
LPP3.04.07.03.04.04.2
NPE17.011.019.013.016.015.2
LRC33.019.034.025.022.026.6
SRC45.040.039.055.039.043.6
MLRC82.082.082.082.082.082.0
MLRC-495.095.095.095.095.095.0
WMLRC-RP86.086.087.084.085.085.6
WMLRC-TH (k = 1)96.094.094.095.095.095.4
WMLRC-TH (k = 3)96.094.096.097.095.095.6

Table 6.

Recognition performances (%) of different methods with different occlusion textures of 50% occlusion on AR database.

4.2 Experiments on FRGC 2.0 database

The second experiments are to test all the methods on the FRGC2.0 database [33, 36] under gray texture blocks, dark, white, and texture occlusions overlapped onto normal faces. The FRGC2.0 database consists of more than 4000 people’s front-view grayscale images including non-facial expressions and slightly facial variations. We choose a subset of 100 subjects with 50 men and 50 women. In each subject, two non-facial expression images are used for the training set as shown in Figure 10. In additions, the synthesized occlusions with various occlusion levels overlapped to normal faces as shown in Figures 1114 are also used for the testing. As to the experiment settings, the facial images are also normalized to 40 × 40 pixels and divided into 16 modules.

Figure 10.

Normal faces as the raining set on FRGC2.0 database.

Figure 11.

Added texture block with various occlusion ratios on FRGC 2.0 database.

Figure 12.

Dark partial occlusion with various occlusion ratios on FRGC 2.0 database.

Figure 13.

White partial occlusion with various occlusion ratios on FRGC 2.0 database.

Figure 14.

Different occlusion (a) - (e) gray wall patterns with 50% occlusion on FRGC 2.0 database.

Tables 710 illustrate the recognition performances of all the methods in all test cases. On FRGC 2.0 database, the proposed WMLRC-TH methods also achieve better performances than other approaches. From Table 8, the MLRC and MLRC-4 performed badly in dark occlusion also. In most cases, the WMLRC-TH performs better than WMLRC-RP. In flat and bright occlusions, as shown in Table 9, the WMLRC-RP carries out the better performance than WMLRC-TH. Since the sunglasses are the most common situation in daily life, WMLRC-RP is still useful in real applications in the cases of dark occlusions.

MethodsOcclusion ratio
10%20%30%40%50%Avg
PCA99.095.086.061.037.075.6
LDA99.097.081.062.040.075.8
LPP85.064.039.031.014.046.6
NPE98.094.091.076.061.084.0
LRC70.065.052.032.012.046.2
SRC98.092.080.069.042.076.2
MLRC96.096.097.094.082.093.0
MLRC-497.096.096.096.089.094.8
WMLRC-RP96.090.085.085.082.087.6
WMLRC-TH (k = 1)99.098.097.097.092.096.6

Table 7.

Recognition performances (%) of different methods with varying occlusion ratios of random block occlusion on FRGC 2.0 database.

MethodsOcclusion ratio
10%20%30%40%50%Avg
PCA63.018.05.04.03.018.6
LDA82.019.022.06.04.026.6
LPP35.032.024.012.06.021.8
NPE40.021.018.09.01.017.8
LRC49.025.014.02.01.018.2
SRC88.086.070.068.041.070.6
MLRC97.097.01.01.01.039.4
MLRC-498.098.01.01.01.039.8
WMLRC-RP96.096.095.095.095.095.4
WMLRC-TH (k = 1)98.098.097.097.097.097.4

Table 8.

Recognition performances (%) of different methods with varying occlusion ratios of dark occlusion on FRGC 2.0 database.

MethodsOcclusion ratio
10%20%30%40%50%Avg
PCA21.07.06.04.02.08.0
LDA36.023.09.08.05.016.2
LPP14.05.02.02.01.04.8
NPE20.015.010.07.04.09.8
LRC25.016.09.04.03.011.4
SRC88.081.046.032.029.055.2
MLRC97.097.093.093.093.094.6
MLRC-497.097.097.097.097.097.0
WMLRC-RP100100100100100100.0
WMLRC-TH (k = 1)98.098.096.097.096.097.0

Table 9.

Recognition performances (%) of different methods with varying occlusion ratios of white partial occlusion on FRGC 2.0 database.

MethodsOcclusion texture
ABCDEAvg
PCA20.036.015.011.026.021.6
LDA14.019.017.014.023.017.4
LPP16.019.06.010.06.011.4
NPE24.033.017.08.020.020.4
LRC34.042.018.07.021.024.4
SRC40.047.045.066.039.047.4
MLRC93.093.093.093.093.093.0
MLRC-495.095.095.095.095.095.0
WMLRC-RP91.093.092.088.092.091.2
WMLRC-TH (k = 1)96.094.095.097.091.094.6

Table 10.

Recognition performances (%) of different methods with different occlusion texture of 50% occlusion on FRGC 2.0 database.

From above experiments, the proposed WMLRC-HT method achieves best accurate recognition rate. For the cases of light color occlusions, the WMLRC-RP performs better than the WMLRC-HT. For the other cases, the WMLRC-HT achieves the best recognition performances. The simulations show that the texture histogram feature can be used to detect occluded modules effectively. Besides, we observe that the MLRC-TH in RGB space domain performs better than that in grayscale domain because the grayscale images would waste the useful colored information.

4.3 Android based system implementation

Recently, Android-based operating system successfully supports smartphones and tablets because of its computing ability, storage capacity and its handy functions for image capture and image processing. Thus, we realize the proposed WMLRC-TH method on Android system to a robust face recognition system on mobile devices since it has more robust performance in general situations, especially in color image.

In general, the face recognition system can be divided into training phase and the testing phases. As shown in Figure 15, the user needs to select the training phase first after capturing face images of a new identity. The user will be asked to enter the name of the identity then the system will store face images with the name in database. In the test phase, the user can capture face image and then identify it. The testing phase basically consists of three steps. Figure 16 shows the flow chart of testing phase. The first step is the face detection, which rapidly captures human faces when the human is located within a proper distance by the mobile phone camera. The second step is the pre-processing stage that is removing the noisy pixels like background and human hairs of the face images. The third step is face recognition that is recognizing the face according to the face database collected before.

Figure 15.

Proposed android face recognition system.

Figure 16.

Flow chart of testing phase in Andorid phones.

Considering the computation resources of mobile phones, we surveyed several face detection methods [37, 38, 39, 40, 41]. By trade-off between accuracy and computation, we finally selected the LBP-feature [37] for face detector, where the LBP operator is given as:

LBPxcyc=p=0P12psipic,sx=1,x00,x<0E28

where icis the intensity of (xc, yc), at the central pixel in the local image, ipis the intensity of neighbor pixel. Pis the total number of neighbor pixels. After face detection, the pre-processing stage, including cropping, gray transform, elliptical mask, and resizing. It is noted that the face detection and pre-processing should be identically performed in both training and testing phases. Finally, the face image vector is then classified by the proposed WMLRC-HT algorithm. Figure 17 shows the user interfaces of face registration and face recognition of the proposed system.

Figure 17.

User interfaces for registration and face recognition. (a) Registration and setup database (b) recognition and show name.

The proposed Android face recognition system is mainly developed on OpenCV SDK and Android NDK (Native Development Kit). Android NDK is used to overcome the limitations of Java, such as memory management and performance, by programming directly into Android native interface to support native development in C/C++.

Advertisement

5. Conclusion

In this chapter, we first review the linear regression classification algorithm as the base of face recognition. If we divide the face region into several subfaces, called modules, any face recognition algorithm will become a new module face recognition algorithm, which can avoid the serious degradation of recognition performance to solve the occlusion problem. We proposed the texture histogram difference of the module to detect the its occlusion tendency of the input face image. The concept of the texture histogram difference can be used for any face recognition algorithms if they adopt the module design. By using texture histogram (HT) concept, the weighted module linear regression classification (WMLRC-HT) method for partially-occluded face recognition is finally proposed. The proposed WMLRC-HT method with adaptive HT weights can effectively improve the shortcoming of the original LRC and MLRC algorithms. The experimental results show that the proposed WMLRC-HT method performs better than the existing linear regression classification methods and the contemporary approaches like SRC with various occluded situations. Since the WMLRC-HT method acquires small computation cost in both training and testing, after the face detector with LBP-features, we implement the proposed method on smart phones. Even the people wear masks, we can easily train and successfully detect the identify only with a smart phone.

Advertisement

Acknowledgments

This work was supported by Ministry of Science and Technology, Taiwan, MOST 110-2221-E-006-119.

Advertisement

Conflict of interest

The authors declare no conflict of interest.

DOWNLOAD FOR FREE

chapter PDF

© 2021 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution 3.0 License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite and reference

Link to this chapter Copy to clipboard

Cite this chapter Copy to clipboard

Wei-Jong Yang, Cheng-Yu Lo, Pau-Choo Chung and Jar Ferr Yang (November 5th 2021). Weighted Module Linear Regression Classifications for Partially-Occluded Face Recognition [Online First], IntechOpen, DOI: 10.5772/intechopen.100621. Available from:

chapter statistics

21total chapter downloads

More statistics for editors and authors

Login to your personal dashboard for more detailed statistics on your publications.

Access personal reporting

We are IntechOpen, the world's leading publisher of Open Access books. Built by scientists, for scientists. Our readership spans scientists, professors, researchers, librarians, and students, as well as business professionals. We share our knowledge and peer-reveiwed research papers with libraries, scientific and engineering societies, and also work with corporate R&D departments and government entities.

More About Us