Open access peer-reviewed chapter

Face Identification Using LBP-Based Improved Directional Wavelet Transform

Written By

Mohd. Abdul Muqeet and Qazi Mateenuddin Hameeduddin

Submitted: March 10th, 2020 Reviewed: July 22nd, 2020 Published: December 21st, 2020

DOI: 10.5772/intechopen.93445

Chapter metrics overview

398 Chapter Downloads

View Full Metrics


Face identification is the most active area of research in computer vision and biometric authentication. Various face identification methods are developed over the time, still, numerous facial appearances are needed to cope with such as facial expression, pose, and illumination variation. Moreover, faces captured in unrestrained situations also impose immense concern in designing effective face identification methods. It is desirable to extract robust local descriptive features to effectively characterize such facial variations both in unrestrained and restrained situations. This chapter discusses such a face identification method that incorporate a popular local descriptor such as local binary patterns (LBP) based on the improved directional wavelet transform (IDW) method to extract facial features. This designed method is applied to complex face databases such as CASIA-WebFace and LFW which consists of a large number of face images collected under an unrestrained environment with extreme facial variations in expression, pose, and illumination. Experiments and comparison with various methods which include not only the local descriptive methods but also local descriptive-based multiresolution analysis (MRA) based methods demonstrate the efficacy of the LBP-based IDW method.


  • face identification
  • improved directional wavelet (IDW)
  • local binary patterns (LBP)

1. Introduction

Researchers have devoted a substantial amount of effort in studying face identification methods in the context of computer vision, image processing, and pattern recognition due the wide acceptability of face as biometric [1]. Requirements for high recognition accuracy, high computational efficiency, and invariance to variations in facial expression, illumination, pose, and occlusions are the prominent challenges in face identification. The illumination problem comes from the fact that different illuminations can cause vast changes to the image of a subject’s face [2, 3]. Similarly, deviations in facial expressions along with head pose variation and occlusion can also lead to very unlike face images for the same subject. Moreover, face identification in the unrestrained environment is still a major challenge which greatly degrades the performance of various well-established methods. Additionally, there still exist challenges such as high dimensionality of feature data and intraclass variations occurring due to the effect of facial variations in restrained and unrestrained environments. A face identification method must be discriminatory for different subjects and invariant to numerous facial variations. Researchers have been extensively utilizing MRA methods and using various off–the-shelve designs of wavelet filters [4] for the implementation of isotropic 2-D DWT for facial feature extraction. Recently implemented 2-D DWT methods such as GHWFB [5] and TWFB [6] considers the handcrafted wavelet filters with additional features compared to off-the-shelve wavelet filters. But these methods do not achieve excellent results due to limited directions orientation and non-adaptation in facial feature selection.

Various local descriptors prominently the LBP [7] and weber local descriptors (WLD) [8] have been efficiently used for facial feature extraction. The constraint of the LBP-based feature extraction method is their noise intolerance and poor discrimination capability [8]. Recently, various non-adaptive MRA methods are applied as a pre-processing step before LBP feature extraction to improve the performance. The prominent methods are local Gabor binary patterns (LGBP) [9], Steerable Pyramid Transform (SPT)-LBP [10], Curvelet Transform-LBP (Curvelet-LBP) [11], Contourlet-LBP [12], and Wavelet Transform (WT) LBP [13]. Liu et al. [14] used hierarchical multi-scale LBP to create features of sparser coefficients and performed classification using sparse coding with the application of a greedy search approach. Wang et al. [15] combined the Gabor wavelet transform (GWT) and CLBP features and carried out the SRC to perform classification.

The aforementioned LBP-based MRA methods [9, 10, 11, 12, 13, 15] use non-adaptive directional transform which lacks the adaptive directional selectivity based on the image description. These methods also experience various issues, for instance, selection of transformed sub-bands, complex filter design, and the large dimension of the feature vector. Maleki et al. [16] proposed adaptive direction selection and applied directional lifting within the selected optimal direction and constructed a compact representation for adaptive MRA method. Due to such inherent characteristics, significant directional details for various face variations can be approximated by the detection of edges responsible for such variations [17].

For numerous facial variations, substantial directional details can be estimated by approximating the edges [18, 19] accountable for such variations which will considerably enhance the face identification performance which decides the basis of our method. The concept has been exploited in [17, 18, 19] for face recognition applications. This work extends the design of the adaptive directional scheme presented in [19] and presents an LBP-based IDW method to capture multi-resolution directional details from the face images. Subsequently in contrast to [19] where CLBP is used, LBP is applied to the generated IDW sub-bands to extract MRA-based local descriptive features.

The Implementation of the 2-D IDW using seven directions along with the quadtree partitioning (QTP) scheme [19] is explained in Section 2. A brief theory on LBP is described in Section 3. Further, the proposed facial feature extraction method is exhibited in Section 4. In Section 5, comparative results on the CASIA-WebFace and LFW face databases are demonstrated. Conclusions are highlighted in Section 6.


2. Implementation of 2-D improved directional wavelet (2-D IDW)

The fundamental concept of implementation of improved directional wavelet (IDW) is to carry out transform operations on a face image at a viable variety of possible directions while maintaining the properties of multi-resolution, localization, and isotropy intact. The authors in [19] considered a set of seven directions with a quad-tree partitioning scheme. Here we will provide a brief review of the work mentioned in [19].

The 2-D IDW being isotropic method performs a separable 1-D horizontal and vertical 1-D IDW on face image with variation in prediction and update steps where seven directions are considered in an adaptive direction scheme. While performing the 1-D IDW transform if non-integer sample arrives sub-pixel interpolation is performed.

Let xij be a 2-D face image which is first horizontally sub-sampled to get even subsamples xeij=x2i+j and odd sub-samples xoij=x2i+1j. Next in the prediction step odd samples xoij are predicted from neighboring even samples with strong correlation along an optimal direction θ. The outcome of the prediction step and the generated high-pass signal Hij are described as [19],


Where Kp and 2Np are the length and coefficients of the prediction filter. Here, samples from six even rows are selected to conduct the prediction step [19]. Now in the update step, odd samples of Hij along the same optimal direction θ as used in (1) are selected to modify the even samples. The update step and the generated low-pass signal Lij are given as [19],


Where Ku and 2Nu are the coefficients and length of the update filter. Similarly, samples from six odd rows are selected to conduct the update step. The values of scaling factors are considered as gL=1.3416 and gH=0.7071 [17, 19].

Due to linear phase characteristics and large vanishing moments, Neville filters with the order as six are considered as the coefficients Kp and Ku [19]. The usage of Neville filters increases the approximation ability of 2-D IDW.

In contrast to the nine directions [17] and five directions [18], we also used seven pre-assigned directions to implement 2-D IDW [19].


These directions are used to confirm a strong correlation among samples and to extract directional MRA features from face images. It is to point out that signn1tanθ term in (1) and (3) may not always locate integer samples and may not be present on the original image sampling grid [19]. So; sub-sample interpolation is conceded to compute intensity for such non-integer samples. To maintain perfect reconstruction lifting structure [4], the integer samples required to perform sub-sample interpolation for such non-integer samples at optimal direction θ must be even sampled. If optimal direction comes across the integer samples the value is computed by the nearby even sample otherwise the value of the non-integer sample is computed from the interpolation of the two nearby even samples.

To extract local edge details due to face variations that exist at different pixel regions, a quadtree partitioning (QTP) scheme is implemented to partition each face image into sub-blocks of distinct directional details. Each QTP sub-block will have the same direction. The improved QTP scheme provides an efficient direction assignment while implementing the prediction and update step.

Let each face image xij is applied with QTP to obtain non-overlapping sub-block xs. Also, consider the initial block size Sini, minimum block size as Smin and the Lagrangian multiplier as α. The energy summation of the prediction error (ESPE) for each block is computed as [19],


Where Fs,nij are the filtered responses obtained by applying the prediction filter Kp along with the predefined directions θ. Bn is the number of bits spent on signaling the selection of directions. When a sample is predicted from the nearest samples, each candidate direction from (5) is checked and the direction with the smallest ESPE is ultimately selected. The optimal direction which gives the least value of ESPE is selected as,


The value of the lagrangian multiplier α determines the complexity of the QTP scheme and its value needs to be selected sensibly. Moreover, to detect the local edge details and to suit it to the adaptability of the IDW method, a face image needs to be segmented into partitions of clear orientation bias. To resolve this problem an improved QP scheme is proposed to suit the face identification problem as mentioned in [19]. The 1-D IDW can be simply extended to the 2-D IDW where second dimension lifting is yet again performed in the horizontal direction on high-pass signal Hij and low-pass signals Lij to generate four sub-bands i.e. LHij, LLij, HHij, and HLij.


3. Local binary patterns

The LBP [7] is estimated with sampling points xp0P1 in the neighborhood of a center pixel xmicjc at a radial distance given by R [7],


Where tsd is a threshold function. The sampling points which does not fit within the center of a pixel are bilinearly interpolated [7]. Another extension of LBP is the uniform patterns and it is mapped from LBPP,R to LBPP,Ru2 [18], resulting in P×P1+3 feature dimension. After obtaining the LBP coded image, codes of the input image XLij pixels are formed into a histogram as a feature descriptor,

Hl=i,jFXLij=l,Fy=1,ifyis true0,ifyis false,l=0,1,2,n1E10

Where n is the number of different labels produced by the LBP operator. With the usage of LBP8,1u2, the feature dimension is 59 [18].


4. Implementation of LBP-based IDW method

We consider a resolution of 128×128 pixels for face images of the selected databases and face preprocessing is performed on all the face images. Thus each one of the IDW sub-bands LLHLLH is of size 32×32 pixels and each sub-band is divided into m=16 regions with the size of each region as x×y=8×8 pixels [18]. We applied LBP to each of the regions from each of the sub-bands LLHLLH. We used the uniform pattern LBP8,1u2 [7] and NN classifier with Chi-square distance measure. This form an enhanced feature vector or descriptor EFV with a combined dimension as 59×m×3=2832 [18]. The algorithm representing the proposed method is presented in Algorithm 1.

Algorithm 1: Face Identification using LBP-based IDW

Input: Test Image, Train image

Output: Rank-one recognition results of the feature vectors.


Step 1: (Preprocessing)

1.1. Consider the input face image X.

1.2. Resize the image to the resolution of 128×128 pixels.

Step 2: (Computation of IDW sub-bands)

for a number of decomposition levels do

2.1. Quadtree partitioned the face image X into several non-overlapping sub-blocks.

2.2. Estimate the value of ESPE using (6) for each sub-block.

2.3. Estimate the optimal direction which gives the least value of ESPE using (7).

2.4. Perform the prediction and the update steps as described in (1) and (3) in the selected directions in the selected sub-block.

2.5. Obtain IDW sub-bands {LL, HL, LH, HH} and proceed with the LL sub-band for the next decomposition level.

Step 3: (LBP Computation)

3.1. Consider the top-level {LL, LH, HL} sub-bands and divide each sub-band into non-overlapping regions Rk with each of size 8×8 pixels.

for each sub-band do

for each sub-block within the sub-band do

for each coefficient value within the sub-block do

3.2. Compute the LBP8,1u2 histogram features from each region Rk using (8), (9), and (10).

3.3. Concatenate all such LBP8,1u2 multi-region histograms from each sub-band {LL, HL, LH} to form the histogram feature vectors LLl,k, HLl,k, and LHl,k respectively.

 end for end for end for

3.4. Concatenate all the sub-band histogram features to form the final enhanced histogram feature vector EFV.

Step 4: (Dimensionality Reduction)

4.1. Perform dimensionality reduction using LDA on the EFV feature vector.

4.2. Save the reduced dimension train feature vector database to EFV_train.

Step 5: Repeat Step 1 to 3 and 4.1 on each test image to obtain the test feature vector EFV_test.

Step 6: (Identification)

6.1. Compare test feature vector EFV_test against train feature vector EFV_train using the NN classifier using Chi-Square distance measure and calculate the Rank-one results in an identification process.


5. Experimental results

All experiments are performed using Matlab 2018a on a standard i5–3320 2.60 GHz machine with 8.0 GB RAM. Here we prove the effectiveness of the LBP based IDW feature extraction method for which a comparison is established with other LBP based non-adaptive MRA methods. In the experiments, we randomly select a few face images for training and rest for testing to obtain the recognition results.

The face identification performance of the proposed method is performed on CASIA-WebFace [20] and LFW [21] face Database with extreme facial variations where all the images are considered under unrestrained environment. Since, LBP-based IDW histogram features are extracted, comparative face identification methods include descriptive methods such as LBP [7], WLD [8], and SRC-GSLBP [14]. Besides, a comparison with few non-adaptive MRA-based LBP feature extraction methods such as LGBPHS [9], SPT-LBP [10], Curvelet-LBP [11], Contourlet-LBP [12], and GTCLBPSRC [15] is also performed for competitive analysis.

5.1 Experiments on the CASIA-WebFace face database

The CASIA-WebFace database [22] is a huge and complex face database. This database includes 494,414 face images of 10,575 subjects. Considering the subjects with only a few samples deters the recognition results. Thus 10,575 subjects are allocated in the decreasing order by the count of their images contained in the particular subject set. Here we consider only 9067 subjects which consist of at least 15 images. The remaining images of the rest of the 1508 subjects are discarded. Within this, we considered a subset of 600 subjects with 15 images per subject out of 9067 subjects. These subjects are specifically considered based upon their extreme facial variations. Face images are normalized and resized to128x128 pixels.

A random subset is constructed with T (T = 4, 5, 6, 7, 8) images of each subject for training and in every case remaining images for testing. Table 1 tabulates the Rank-one recognition results of various comparative methods. Since the images are collected from around the web with extremely unrestrained conditions, the results are less for all the methods. Due to extreme face variations which include illumination, expression, pose, occlusion, and age difference, this database imposes an immense challenge. Figure 1 depicts the trend of the rank one recognition rates for different comparative methods along with the proposed method for Casia-WebFace face database.

MethodNumber of training samples per subject
LBP [7]14.0716.0618.2421.8124.14
WLD [8]15.6017.7121.4023.2025.09
LGBPHS [9]22.8026.0928.9331.4834.76
SPT-LBP [10]25.2230.1032.2034.5236.12
Curvlet-LBP [11]25.6630.2831.4935.4938.26
Contourlet-LBP [12]27.2231.3332.4936.5540.60
SRC-GSLBP [14]31.2433.3734.1938.2042.05
GTCLBPSRC [15]31.2532.5534.4238.6342.30
ADWTLBP [17]33.9039.4041.0942.0045.60
DIWTLBP [18]34.6540.3641.7842.1745.81
IDW-LBP (Proposed Method)35.8941.3942.2043.3146.12

Table 1.

Rank-One Recognition Results of different methods on the CASIA-WebFace face database (%).

Figure 1.

Rank-One Recognition Results of different methods for the CASIA-WebFace face database (%).

5.2 Experiments on the labelled faces in the wild (LFW) face database

The LFW [20] is a large database that contains face images of 5749 famous personalities captured in an unrestrained environment with an extreme variation of background, pose, illumination, expression, and accessories. This makes it a challenging database for face identification. Here, we used the LFW-a database [21] which is an aligned version of the LFW database. For our experimentation purpose, we created a subset with 15 dissimilar images of 150 subjects from the LFW-a database. Each image is resized to 128×128 pixels.

A subset with T (T = 4, 5, 6, 7, 8) images per subject is randomly selected to form a training set, and rest images per subject are selected to form the testing set. Rank-one recognition results of different comparative methods are tabulated in Table 2. Since the images are selected in the unrestrained environment the Rank-one recognition results are also low in this database.

MethodNumber of training samples per subject
LBP [7]24.0730.1638.2242.8144.14
WLD [8]25.1231.4240.1043.6046.12
LGBPHS [9]33.7635.1542.0545.3449.06
SPT-LBP [10]34.9040.2043.6245.9350.61
Curvlet-LBP [11]33.1139.4843.5544.6050.08
Contourlet-LBP [12]35.0940.2545.5246.4153.32
SRC-GSLBP [14]38.3041.4047.2255.0057.69
GTCLBPSRC [15]40.7242.8050.3058.5960.23
ADWTLBP [17]41.6044.5052.2859.4462.74
DIWTLBP [18]40.8044.8953.0360.1863.04
IDW-LBP (Proposed Method)40.7745.6454.8061.0164.42

Table 2.

Rank-One Recognition Results of different methods on the LFW face database (%).

Figure 2 depicts the trend of the rank one recognition rates for different comparative methods along with the proposed method for LFW face database.

Figure 2.

Rank-One Recognition Results of different methods for the LFW face database (%).


6. Conclusion

This chapter discusses the recently developed implementation of interpolation-based ADWT with seven directions and an improved QTP scheme to extract directional MRA features from face images. LBP is applied to the selected top-level IDW sub-bands to extract the multi-region histogram-based local descriptive features. Experiments conducted on the complex face databases such as CASIA-WebFace and LFW database exhibit the efficacy of our proposed method. The identification results of our method are compared with various methods which include local descriptors such as LBP and WLD. Few LBP-based non-adaptive MRA methods are also utilized for a fair comparison as our method also falls into the category of MRA based methods.

LBP and WLD suffer from issues such as the large size of histogram features, extraction of only very local texture details, limited discriminative ability, and intolerance to noise.

It also is evident from Tables 1 and 2 that LGBPHS, SPT-LBP, Curvelet-LBP, and Contour-LBP methods provide lesser results against our proposed method. We examine progress over SRC-GSLBP, GTCLBPSRC which illustrates some effectiveness of usage of sparse features at the cost of increased complexity in implementation.

SPT-LBP also exhibits comparable performance to the proposed method but the feature selection is threshold dependent and necessitates the selection of sub-bands for efficient feature extraction. Curvelet-LBP uses only the LBP coded image of the approximation sub-band and mid-frequency sub-bands coefficients for feature generation and does not consider the multi-region information. Thus LGBPHS, Curvelet-LBP, Contourlet-LBP, and SPT-LBP are both memory and time exhaustive to extract the multiresolution and multi-orientation features due to selection and feature extraction from different sub-bands.

Moreover, these methods despite capturing the directional information lack the adaptation in selecting the directional details based on the image description and suffer from various issues such as the selection of sub-bands, high computational rate, and complex filter design. The GTCLBPSRC delivers close results to the proposed method for all the databases but at cost of additional computational time and due to the implementation of Gabor wavelet transform (GWT) which exhibits over-complete representation. We also examined comparable progress over SRC-GSLBP and GTCLBPSRC for all the databases which illustrate the effectiveness of sparse representation methods. The IDW method consists of benefits such as directional lifting and adaptation in the direction selection as per the characteristics of the images within a block of samples. Moreover, as a result of lifting based factorization, perfect reconstruction is also assured and the resultant multiresolution image is completely compatible with that of the conventional 2-D DWT multiresolution image. These facts effectively consider various edge manifolds that represent different face variations.

We also compared the proposed method with recently developed methods which also considers the facial descriptions in an adaptive MRA-based structure such as ADWT [17] and DIWT [18]. We applied a similar procedure to extract the LBP based features. The improvement in our method is visible owing to the adaptation of more directions as compared to DIWT [18] and application of sub-pixel interpolation in IDW which is absent in the ADWT method.

Thus, as per Tables 1 and 2, it is verified that the IDW method exhibit high discrimination capability and offers excellent recognition results for a very complex database such as CASIA-WebFace and LFW databases which consists of facial variations with mild to intense pose, expression, and illumination variations. Experiments performed for an identification process verify that the proposed method excels with all the comparative methods.


  1. 1. Anil K. Jain, Arun Ross, and Sharath Pankanti, Biometrics: A Tool for Information Security, IEEE Transactions on Information Forensics and Security, Vol. 1, no. 2, June 2006
  2. 2. M. Turk and A. Pentland, Eigenfaces for recognition, J. Cognitive Neuroscience., vol.3, no.1, pp.71–86, 1991
  3. 3. P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, Eigenfaces versus Fisherfaces: Recognition Using Class Specific Linear Projection, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711–720, Jul. 1997
  4. 4. Mallat S. A theory for multiresolution signal decomposition: The wavelet representation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(7): 674–693, July 1989
  5. 5. M. A. Muqeet and R. S. Holambe, Face identification using LDA based generalized half band polynomial wavelet filter bank, 2016 International Conference on Electrical, Electronics, and Optimization Techniques (ICEEOT), Chennai, India, 2016, pp. 4649-4653. DOI: 10.1109/ICEEOT.2016.7755601
  6. 6. Mohd. Abdul Muqeet, Raghunath S. Holambe, Enhancing Face identification Performance using Triplet Half Band Wavelet Filter Bank, International Journal of Image, Graphics, and Signal Processing (IJIGSP), vol.8, no.12, pp.62-70, 2016. DOI: 10.5815/ijigsp.2016.12.08
  7. 7. T. Ojala, M. Pietikainen, T. Maenpaa, Multiresolution gray-scale, and rotation invariant texture classification with local binary patterns, IEEE Trans. Pattern Anal. Mach. Intell. 24 (7) (2002) 971–987
  8. 8. J. Chen, S. Shan, C. He, et al., WLD: a robust local image descriptor, IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 9, pp. 1705-1720, 2010
  9. 9. W. Zhang, S. Shan, W. Gao, H. Zhang, Local Gabor binary pattern histogram sequence (LGBPHS): a novel non-statistical model for face representation and recognition, in Proceedings of IEEE International Conference and Computer Vision, 2005, pp. 786–791
  10. 10. A. Alelaiwi et al., Steerable pyramid transform, and local binary pattern based robust face identification for e-health secured login, Computers and Electrical Engineering (2016),
  11. 11. L. Zhou, W. Liu, ZM. Lu, T. Nie, Face identification based on curvelets and local binary pattern features via using local property preservation, Journal of Systems and Software. 95: 209-216. DOI: 10.1016/j.jss.2014.04.037
  12. 12. H. Y. Patil, A. G. Kothari, K. M. Bhurchandi, Expression Invariant Face identification using Local Binary Patterns and Contourlet Transform, Optik, vol. 127, pp. 2670-2678, 2016
  13. 13. Y.Z. Goh, A.B.J. Teoh, M.K.O. Goh, Wavelet local binary patterns fusion as illuminated facial image preprocessing for face verification, Expert. Syst. Appl., vol. 38, pp. 3959-972, 2011
  14. 14. Z. Liu, X. Song, Z. Tang, A novel SRC fusion method using hierarchical multi-scale LBP and greedy search strategy, Neurocomputing, vol. 151, pp.1455-1467, 2015
  15. 15. X. Wang, Q. Zhu, J. Cui, Y. Wang, Sparse representation method based on Gabor and CLBP, Optik, vol. 124, pp. 5843-5850, 2013
  16. 16. Maleki, A., Rajaei, B, Pourreza, H.R., Rate-Distortion Analysis of Directional Wavelets, Image Processing, IEEE Transaction on Image Processing, vol.21, no.2, pp.588-600, Feb. 2012
  17. 17. M. A. Muqeet, R. S. Holambe, Local appearance-based face identification using adaptive directional wavelet transform, J. King Saud Univ.-Computer Inform. Sci., vol. 31, pp. 161-174. (2019)
  18. 18. Muqeet, Mohd. Abdul, Holambe, R.S., Local binary patterns based on directional wavelet transform for expression and pose-invariant face identification, Applied Computing and Informatics, vol.15, Issue. 2, July 2019, Pages 163-171
  19. 19. M. A. Muqeet, R. S. Holambe, A collaborative representation face classification on separable adaptive directional wavelet transform based completed local binary pattern features, Eng. Science and Tech., an Intern. Journ., vol. 21, no. 4, pp. 611-624
  20. 20. G.B. Huang, M. Ramesh, T. Berg., E. Learned-Miller, Labeled face in the wild: a database for studying face recognition in unconstrained environments, Technical Report, 07-49, Univ. of Massachusetts, Amherst, 2007
  21. 21. Wolf, L., Hassner, T., Taigman, Y., Similarity scores based on background samples, Computer Vision- ACCV 2009, pp. 88-97, 2010
  22. 22. D. Yi, Z. Lei, S. Liao, S.Z. Li, Learning face representation from scratch, arXiv preprint arXiv:1411.7923, 2014

Written By

Mohd. Abdul Muqeet and Qazi Mateenuddin Hameeduddin

Submitted: March 10th, 2020 Reviewed: July 22nd, 2020 Published: December 21st, 2020