Open access peer-reviewed chapter

Multimodal Biometrics for Person Authentication

By Ryszard S. Choras

Submitted: September 24th 2018Reviewed: February 6th 2019Published: March 14th 2019

DOI: 10.5772/intechopen.85003

Downloaded: 1061


Unimodal biometric systems have limited effectiveness in identifying people, mainly due to their susceptibility to changes in individual biometric features and presentation attacks. The identification of people using multimodal biometric systems attracts the attention of researchers due to their advantages, such as greater recognition efficiency and greater security compared to the unimodal biometric system. To break into the biometric multimodal system, the intruder would have to break into more than one unimodal biometric system. In multimodal biometric systems: The availability of many features means that the multimodal system becomes more reliable. A multimodal biometric system increases security and ensures confidentiality of user data. A multimodal biometric system realizes the merger of decisions taken under individual modalities. If one of the modalities is eliminated, the system can still ensure security, using the remaining. Multimodal systems provide information on the “liveness” of the sample being introduced. In a multimodal system, a fusion of feature vectors and/or decisions developed by each subsystem is carried out, and then the final decision on identification is made on the basis of the vector of features thus obtained. In this chapter, we consider a multimodal biometric system that uses three modalities: dorsal vein, palm print, and periocular.


  • feature transform
  • multimodal biometric recognition
  • levels of fusion
  • dorsal vein
  • periocular
  • palm print
  • PCA

1. Introduction

Biometrics is a technology that uses physical and/or behavioral characteristics of people to identify them. Systems of this type implement two processes (Figure 1) [1]:

  1. Enrollment

  2. Authentication

Figure 1.

Biometric recognition system.

The physical features are fingerprints, hand geometry, handprint, facial image, iris, retina, and ear. Behavioral features are signature, lip motion, speech, dynamics of typing, hand movements, and gait.

The characteristics of effective biometrics are:

  1. Unique features for each individual

  2. Invariant traits over time (e.g., due to the effect of aging)

  3. Features that are relatively easy to obtain (computational complexity small)

  4. Precise algorithms enabling classification

  5. Resistance to various types of attacks

  6. Low cost

  7. Ease of implementation

The security of the biometric system is usually assessed on the basis of some indicators. These are:

  • False match rate (FMR). It belongs to the group of matching errors. This indicator is defined as the expected probability that the downloaded sample will be falsely matched to the template in the database, but it will not be the test user pattern. If the indicator is high, it means that there is a risk that an unauthorized person will be recognized as a system user.

  • False rejection (FRR) is equivalent to the FMR. The difference between these indicators is that FMR refers to a single match, and the FRR refers to a situation where one or more attempts to match a sample to a template from the database may occur. The FRR error is referred to in the literature as type I error.

  • False discrepancy (FNMR). This is the coefficient determining the probability that the sample taken will not be matched to the pattern in the database belonging to the user from whom the sample was taken. In biometric verification (1:1) systems, the indicator means that the sample has not been identified by a specific pattern, while in biometric identification systems (1:N), this indicator determines the probability that a given pattern will not be found in the database.

  • The false acceptance factor (FAR) is equivalent to the FNMR indicator. The difference between him and FNMR is the same as between FRR and FMR.

  • Equal error rate (EER). It is defined as the intersection of the FAR and FRR characteristics in the graph of the dependence of these errors on the threshold of sensitivity (t). This factor indicates the optimal sensitivity threshold at which the same number of people is incorrectly rejected and incorrectly accepted. The lower the EER error value, the better the biometric system is.

The FMR (FRR) and FNMR (FAR) parameters can also be represented by graphs (Figure 2):

  • Receiver operating characteristic (ROC) curve showing the dependence of FNMR on FMR. You can use it to show the accuracy of the system.

  • Detection error trade-off (DET) showing error rates on both axes, most often on a logarithmic scale. This curve is plotted for both matching errors and decisions (Figure 2).

Figure 2.

The graph of FAR, FRR, and EER in receiver operating characteristic (ROC) curve.

If we use only one biometric authentication system, the results obtained are not always good enough. Unimodal biometric systems using a single sensor have many limitations, such as lack of uniqueness, universality, and lack of interference level associated with the acquired data, as a result of which they are unable to provide the required level of identification/verification efficiency (Table 1). This is due to the fact that the reliability of the biometric modality applied is affected by the precision of a single biometric system (Table 2).

Distortion of the input biometric dataDistorted biometric data may prevent the correct alignment process with database templates, as a result of which users are incorrectly rejected or identified
Intra class variationsBiometric data obtained from the person during authentication may differ from the data used to generate the template during registration, thus affecting the matching process. The biometric template should have a small intra-class variance
Interclass similaritiesBiometric features should be significantly different for different people and should ensure small similarities between classes in the feature space. There is an upper limit to the users who can be effectively distinguished by any biometric system. The capacity of the identification system cannot be arbitrarily increased for fixed sets of feature vectors and the matching algorithm. The biometric template should have large interclass variations
Non-universalityObtaining accurate (useful) biometric data from the users is not always possible
Intruder attacksAttacks of this type involve the manipulation of biometric features to avoid recognition. It is also possible to create artificial biometric patterns in order to accept the identity of another person

Table 1.

Limitations of unimodal biometrics.

Recognition accuracyThe multi-biometric system ensures greater accuracy and reliability thanks to many independent biometric features that are difficult to attack
Continuous monitoringIn case when one biometric modality is obstructed, other modalities of the multi-biometric system ensure correct user identification
PrivacyMulti-biometric systems provide greater resistance to certain types of loopholes and attacks. It is difficult and/or impossible to steal many biometric patterns (templates) stored in the biometric database
Biometric data enrollmentWhen biometric input data is unavailable or unacceptable by a biometric system, another biometric system modality may be used
Resistance on spoof attacksUsually the attacker is not able to use many relevant (accurate) spoofed biometrics

Table 2.

Advantages of multi-biometric systems.


2. Multi-biometric systems

2.1 Types of multi-biometric systems

The multi-biometric system can be (Figure 3) (a) a multi-sensor system that allows obtaining data from various sensors using one biometric feature, (b) a system with multiple algorithms processing a single biometric feature, (c) a system consolidating multiple occurrences of the same body trait, (d) a system using multiple templates of the same biometric method obtained with the help of a single sensor, and (e) a multimodal system combining information about the biometric features of the individual to establish his identity [2, 3, 4].

Figure 3.

Types of multi-biometric systems.

2.2 Fusion levels

In multimodal biometric systems, there are a number of strategies (scenarios) for the fusion of biometric information:

  • Data fusion from sensors. Data from various sensors form one vector. Fusion of information obtained from many different sensors for a single biometric feature.

  • The fusion of feature vectors extracted from various biometric modalities for further processing. A merger of information obtained from several unimodal biometric systems that process different body characteristics of the same person (Figure 4a).

  • Fusion at the decision level. The merger of decisions developed on the basis of information from different biometric modalities, and the resultant feature vector defines two main classes, i.e., rejection or acceptance (Figure 4b).

  • Rank level fusion. The classifier determines the rank of each registered biometric identity. A high position is a good indicator of a good fit (Figure 4b).

Figure 4.

Levels of fusion. (a) Feature level fusion and (b) score/rank level fusion.

2.3 Related work

The fusion of biometrics modalities on different levels of multi-biometrics system is extensively studied in the literature (Table 3). For all that the merger at the level of feature vectors is relatively poorly discussed. The merger at this level includes the integration of feature vectors corresponding to many sources of information. Because the feature vectors contain more elements than the input biometric data, it is obvious that the merger at the feature vectors level will provide better authentication results. However, mergers at this level are difficult to implement in practice because (i) sets of features of many modalities may be incompatible, (ii) the combination of two feature vectors may result in a vector of features with very large dimensionality, and (iii) a complex comparing system is required.

Biometrics traitsFusion methodsDescription of the implementation methodReferences
Fingerprint and faceFeatureIn [5], it was proposed to extract face and fingerprint characteristics invariant to the rotation and scaling of Zernike moments (ZM). On the basis of ZM, the fusion of facial features and fingerprints is realized. The RBF network implements the decision-making process. The accuracy rate is 96.55%. Testing result of authentication rate are FAR, 4.95%, and FRR, 1.12%[5]
ScoreIn [6], authors presented score level fusion technique using the SIFT features for the face and the minutiae features for fingerprint. Results are: FAR = 1.98%, FRR = 3.18%, and accuracy = 97.41[6]
finger knuckle print,
finger vein
Finger shape
FeatureThe multi-set canonical correlation analysis is used to fuse multiple feature sets. The feature based on MCCA achieves the recognition performance, with EER = 2.3900e-04[1]
With the help of the unified Gabor filter, fingerprint codes and finger vein codes are generated. The extraction of features is carried out by using a supervised local canonical correlation analysis (SLPCCA), and finally the NN-classifier is used[7]
Fingerprint and irisScoreIn [8], authors propose a frequency approach to generate a unified homogeneous template for fingerprint and iris features. Scores generated from these templates are fused using the sum rule[8]
Palm print and hand shapeFeatureInformation from the face image and gait image are combined at the function level. Using the principal component analysis (PCA) method, facial features were obtained The result of multiple discrimination analysis (MDA) is gait energy image (GEI) Recognition rate results are 91.3%[9]
Palm print and irisFeatureIn system described in [10], texture parameters are extracted based on Gabor filters.[10]
Fusion of the palm print features and iris features is based on the wavelets. Decision is obtained using kNN classifier. Recognition accuracy is 99.2% and FRR = 1.6%
In [11], fusion method for the information of phases about the iris and palm utilizes a Baud limited image product (BLIP)[11]
Finger knuckle and palm printFeatureIn this paper, feature extraction method for palm print is monogenic binary coding; for inner knuckle print recognition, two algorithms named ridgelet transform and SIFT are proposed. The extracted feature vectors are classified using SVM[12]
Palm print and faceFeatureThe PCA is used to extract features of palm and face images. Fusion technique concatenated the feature vectors of the face and palm modalities into one fused vector, and feature selection is performed.[13]
Face and gaitFeatureMethod is based on learning face and gait features in image transform spaces. Two methods are considered—PCA and LDA[14]
Face and irisScoreMulti-biometrics system using dual iris, visible and thermal face traits is considered. 1D Log-Gabor and Complex Gabor Jet Descriptor (CGJD) were used to extract feature vectors. Authors proposed a score level fusion algorithm[15]
The ordinal measures and local binary pattern (LBP) methods are proposed to extract features from iris and face regions, respectively[16]
FeaturePaper [17] presents the extractions of iris features based on 2D Gabor and facial features using the PCA method[17]
Face and hand geometryFeatureThe 2D DCT is used to extract discriminant face features which are concatenated with hand geometric features. The resultant feature vector is classified using SVM[18]
Face and earScoresTo match score level, fusion is proposed in [19]. Authors use Dempster-Shafer decision theory for each modality. Recognition rate is 95.53% with 4.47% EER[19]
Ocular—iris and conjunctival vascularScoreIn [4], authors presented fusion of both iris and conjunctival vascular information. A weighted fusion method is proposed for each modality. The fusion resulted in an EER of 2.83%[4]
Face, ear, and signatureRankIn [20], the PCA and Fisher’s linear discriminant (FLD) methods in the face, ear, and signature, multimodal biometrics system is proposed. Local features are extracted from face, ear, and signature data. Features are matched using Euclidean distance. This system is using rank level fusion[20]

Table 3.

Summary of works on multimodal biometric systems.

3. The proposed multi-biometric system

The multi-biometric system (dorsal vein + periocular + palm print) is presented in Figure 5.

Figure 5.

Considered multi-biometric system architecture.

In our proposed method, the first is preprocessing block including noise elimination, ROI detection and normalization, and contrast normalization. For all three modalities, noise elimination for an image fxyis performed using median filtering (2D MF) operation formulated as [21]:


where A1is the MFwindow.

Next step in preprocessing phase is ROI detection and normalization (Figure 6). This operation is quite different for dorsal vein images, palm print images, and periocular images. For dorsal vein images, we use distance transform to detect the dorsal image center and build square ROI based on this center coordinates [22, 23]. The ROI design for palm print images is based on hand-specific points (finger valleys) and two angles [24]. The periocular region is detected based on the center of the iris. Using the conventional algorithm for detecting the iris, we determine the center of the iris and its diameter. The periocular area is a rectangle centered in the iris center [25, 26].

Figure 6.

ROI area for dorsal vein images (a), palm print images (b), and periocular images (c).

After the ROI detection, we perform image size normalization and apply the contrast normalization by using CLAHE algorithm. The image is divided into non-overlapping areas of equal size, and the histograms of each region are calculated. Next, the cutoff threshold for histograms is obtained, and each histogram is processed in such a way that its height does not exceed the cutoff threshold [21].

The sample input images after normalization operations and operations using the CLAHE algorithm are shown in Figure 7. Next processing blocks include feature extraction, feature selection, fusion, and classification.

Figure 7.

Images after normalization (size 150 × 150 pixels) and after applying the CLAHE algorithm.

3.1 Gabor feature extraction

In biologically inspired vision models, receptor fields exist that are the primary aspect of early visual processing in mammalian vision systems. Gabor functions are widely used in image feature analysis because they are similar to receptive field profiles in mammalian cortical simple cells. These fields are modeled using Gabor filters [27].

Imitation of mammalian vision systems (or some of them) in object recognition systems leads to their increased efficiency and plausibility. Object recognition systems that are inspired by the biological approach use filter banks, in particular Gabor filters (Figure 8) [28, 29, 30, 31, 32].

Figure 8.

2D functions and 2D Gabor filter.

The 2D Gabor filter family can be represented as expressed in Eq. (2):


where Gθxy=excosθ+ysinθ22σx2+sinθ+ycosθ22σy2and Sω,θxy=eiωxcosθ+ωysinθeω2σ22.

The Gabω,θxycan be decomposed into a real RGabω,θxy=12πσ2GθxyRSω,θxyand an imaginary.

IGabω,θxy=12πσ2GθxyISω,θxyparts (for σx=σy=σ).

Gabor response images are obtained by convolution operation of multiscale and multi-orientation Gabor filters Gabω,θxywith the image f (x, y).


where andis the convolution operator.

The Gabor filter responses for palm print image and dorsal vein image are shown in Figures 9 and 10, respectively.

Figure 9.

Imaginary part of the Gabor filter responses of a palm print image.

Figure 10.

Imaginary part of the Gabor filter responses of a dorsal vein image.

3.2 Periocular feature extraction by LBP

The periocular area contains the iris, eyes, eyelids, eyelashes, and partially eyebrows. The LBPmethod can be used to describe the texture of the periocular area, and the feature vectors contain LBP features. The operator of local binary patterns (LBP) was proposed by Ojala [33] as a texture descriptor.

LBPdivides the image into non-overlapping blocks of the same size. Local image features are calculated for each block separately. For a set of pixels belonging to a given block, the LBPvalues are calculated and then a histogram is created. The feature vectors (histograms) of each block are combined to form a global vector of features of the entire image.

LBPanalyzes the local neighborhood consisting of gppoints located on a circle with radius Rand surrounding the center point of gcand checks whether the points of gpare greater or lesser than the gcpoint value.

The LBPvalue of thegcpoint is specified as follows:


wheregpand gcare the luminance values of the neighborhood and center point, respectively.

The idea of this operator is presented in Figure 11.

Figure 11.

The basic idea of LBP approach.

For an image size M×N, the image descriptor is a histogram created from the LBPvalues:


where kis one LBPpattern and Kis the maximal LBPpattern value (number bin of the histogram).

Using the LBPoperator, we obtain 2Pdifferent output values corresponding to 2Pdifferent binary patterns created by Pof neighboring pixels. Certain binary patterns contain more information than others, so we can only consider this subset of LBPvalues. Patterns of this subset are called uniform patterns. So we have a standard LBPP,Roperator and an LBPP,Ru2operator.

Typically image is divided into nblocks and histograms of each block are concatenate into feature vector [34].

In the case LBPP,Roperator, the histogram contains 256 bins. In the case ofLBPP,Ru2operator, the histogram contains 59 bins (Figures 12 and 13).

Figure 12.

The original image (a) and image as a result of the LBP operator (b).

Figure 13.

TheLBPP,Rhistogram (a), histograms of thenblocks (b), and theLBPP,Ru2histogram (c).

4. Feature reduction and data fusion

The multi-biometric system has been tested using certain parts of the following databases: PolyU palmprint [24], IIITD periocular database [25], and Bosphorus hand vein database [35]. We choose 20 subjects with 10 images per subject at random. From 10 images, 5 images are used for training and 5 for the testing.

The combination of feature vectors at this level is difficult to achieve in practice due to the combination of certain fundamentally different feature vectors that can result in a resulting vector of features with very large dimensionality. In a merger at the level of feature vectors, each individual modality process generates a feature vector. The fusion process combines these feature vectors into one vector.

For dorsal vein images and for palm print images, we perform the same image processing operations that the feature vectors have the same sizes. As a result of convolution operation of multiscale and multi-orientation Gabor filters with the input image, we get the Gabor response images. The feature vector has a very large size of (M x N x k x l)where M x Nis the image size, kis the number of scales, and lis the number of orientations. In our case, for both dorsal vein images and palm print images, we get a feature vector containing (150 × 150 × 3 × 6) = 405,000 items. The images subjected to the Gabor filtration are rescaled with a scale factor of 0.1, which allows obtaining a vector of features with a size of 1 × 4050 elements.

For periocular images, the feature vector has a size of 36 × 59 = 2124.

Next we reduce dimensionality of these vectors used in PCA method (Figure 14 and Table 4) [5]. Separated features are normalized using zero mean and unit variance as

Figure 14.

Steps to image processing using PCA.

PCA algorithm
Organizing the training set of images
where qis the number of images in the training set
Calculating the average of the set T
Calculating the covariance matrix C
The eigenvectors and corresponding eigenvalues are computed
The eigenvectors and their corresponding eigenvalues are paired and ordered from high to low. Approximated image is calculated as
for v=v1v2vk

Table 4.

PCA algorithm.


where μiand σiare the mean value and standard deviation of the i-th feature, f¯iis the normalized i-th feature vector.

Table 5 shows the recognition performance depending on the number of selected eigenvectors.

ModalityNumber of the eigenvectors
k = 40k = 60k = 80k = 100
Dorsal vein8889.391.492.6
Palm print88.789.390.692.8
Dorsal vein + palm print90.391.192.393.1
Dorsal vein + periocular91.19292.492.8
Palm print + periocular90.791.491.892.1
Dorsal vein + periocular + palm print93.29494.595.3

Table 5.

Recognition rates [%] for different modality.

5. Conclusion

In this chapter, Gabor’s functions and LBP features are proposed for recognition in a multi-biometric system that uses three modalities: dorsal vein, periocular, and palm print. Using PCA method dimensionality feature vectors from these modality are reduced. Feature vectors are normalized and fused using concatenation operation. Based on the results, we suggest that multi-biometric system using the fusion of dorsal vein, periocular, and palm print images can offer recognition rate which the unimodal biometric system cannot.

© 2019 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution 3.0 License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite and reference

Link to this chapter Copy to clipboard

Cite this chapter Copy to clipboard

Ryszard S. Choras (March 14th 2019). Multimodal Biometrics for Person Authentication, Security and Privacy From a Legal, Ethical, and Technical Perspective, Christos Kalloniatis and Carlos Travieso-Gonzalez, IntechOpen, DOI: 10.5772/intechopen.85003. Available from:

chapter statistics

1061total chapter downloads

1Crossref citations

More statistics for editors and authors

Login to your personal dashboard for more detailed statistics on your publications.

Access personal reporting

Related Content

This Book

Next chapter

Hardware Implementation of Audio Watermarking Based on DWT Transform

By Amit M. Joshi

Related Book

First chapter

Information Systems: From the Requirements to the Integrated Solution

By José Francisco Zelasco and Judith Donayo

We are IntechOpen, the world's leading publisher of Open Access books. Built by scientists, for scientists. Our readership spans scientists, professors, researchers, librarians, and students, as well as business professionals. We share our knowledge and peer-reveiwed research papers with libraries, scientific and engineering societies, and also work with corporate R&D departments and government entities.

More About Us