Multimodal Biometrics for Person Authentication

Unimodal biometric systems have limited effectiveness in identifying people, mainly due to their susceptibility to changes in individual biometric features and presentation attacks. The identification of people using multimodal biometric systems attracts the attention of researchers due to their advantages, such as greater recognition efficiency and greater security compared to the unimodal biometric system. To break into the biometric multimodal system, the intruder would have to break into more than one unimodal biometric system. In multimodal biometric systems: The availability of many features means that the multimodal system becomes more reliable. A multimodal biometric system increases security and ensures confidentiality of user data. A multimodal biometric system realizes the merger of decisions taken under individual modalities. If one of the modalities is eliminated, the system can still ensure security, using the remaining. Multimodal systems provide information on the “ liveness ” of the sample being introduced. In a multimodal system, a fusion of feature vectors and/or decisions developed by each subsystem is carried out, and then the final decision on identification is made on the basis of the vector of features thus obtained. In this chapter, we consider a multimodal biometric system that uses three modalities: dorsal vein, palm print, and periocular.


Introduction
Biometrics is a technology that uses physical and/or behavioral characteristics of people to identify them. Systems of this type implement two processes ( The security of the biometric system is usually assessed on the basis of some indicators. These are: • False match rate (FMR). It belongs to the group of matching errors. This indicator is defined as the expected probability that the downloaded sample will be falsely matched to the template in the database, but it will not be the test user pattern. If the indicator is high, it means that there is a risk that an unauthorized person will be recognized as a system user.
• False rejection (FRR) is equivalent to the FMR. The difference between these indicators is that FMR refers to a single match, and the FRR refers to a situation where one or more attempts to match a sample to a template from the database may occur. The FRR error is referred to in the literature as type I error.
• False discrepancy (FNMR). This is the coefficient determining the probability that the sample taken will not be matched to the pattern in the database belonging to the user from whom the sample was taken. In biometric verification (1:1) systems, the indicator means that the sample has not been identified by a specific pattern, while in biometric identification systems (1:N), this indicator determines the probability that a given pattern will not be found in the database.
• The false acceptance factor (FAR) is equivalent to the FNMR indicator. The difference between him and FNMR is the same as between FRR and FMR.
• Equal error rate (EER). It is defined as the intersection of the FAR and FRR characteristics in the graph of the dependence of these errors on the threshold of sensitivity (t). This factor indicates the optimal sensitivity threshold at which the same number of people is incorrectly rejected and incorrectly accepted. The lower the EER error value, the better the biometric system is.
The FMR (FRR) and FNMR (FAR) parameters can also be represented by graphs ( Figure 2): • Receiver operating characteristic (ROC) curve showing the dependence of FNMR on FMR. You can use it to show the accuracy of the system. The graph of FAR, FRR, and EER in receiver operating characteristic (ROC) curve.

Name Description
Distortion of the input biometric data Distorted biometric data may prevent the correct alignment process with database templates, as a result of which users are incorrectly rejected or identified Intra class variations Biometric data obtained from the person during authentication may differ from the data used to generate the template during registration, thus affecting the matching process. The biometric template should have a small intra-class variance Interclass similarities Biometric features should be significantly different for different people and should ensure small similarities between classes in the feature space. There is an upper limit to the users who can be effectively distinguished by any biometric system. The capacity of the identification system cannot be arbitrarily increased for fixed sets of feature vectors and the matching algorithm. The biometric template should have large interclass variations Non-universality Obtaining accurate (useful) biometric data from the users is not always possible Intruder attacks Attacks of this type involve the manipulation of biometric features to avoid recognition. It is also possible to create artificial biometric patterns in order to accept the identity of another person Table 1.
Limitations of unimodal biometrics.
• Detection error trade-off (DET) showing error rates on both axes, most often on a logarithmic scale. This curve is plotted for both matching errors and decisions ( Figure 2).
If we use only one biometric authentication system, the results obtained are not always good enough. Unimodal biometric systems using a single sensor have many limitations, such as lack of uniqueness, universality, and lack of interference level associated with the acquired data, as a result of which they are unable to provide the required level of identification/verification efficiency ( Table 1). This is due to the fact that the reliability of the biometric modality applied is affected by the precision of a single biometric system ( Table 2).

Multi-biometric systems 2.1 Types of multi-biometric systems
The multi-biometric system can be ( Figure 3) (a) a multi-sensor system that allows obtaining data from various sensors using one biometric feature, (b) a system with multiple algorithms processing a single biometric feature, (c) a system consolidating multiple occurrences of the same body trait, (d) a system using Name Description

Recognition accuracy
The multi-biometric system ensures greater accuracy and reliability thanks to many independent biometric features that are difficult to attack

Continuous monitoring
In case when one biometric modality is obstructed, other modalities of the multibiometric system ensure correct user identification Privacy Multi-biometric systems provide greater resistance to certain types of loopholes and attacks. It is difficult and/or impossible to steal many biometric patterns (templates) stored in the biometric database

Biometric data enrollment
When biometric input data is unavailable or unacceptable by a biometric system, another biometric system modality may be used

Resistance on spoof attacks
Usually the attacker is not able to use many relevant (accurate) spoofed biometrics Table 2.
Advantages of multi-biometric systems. multiple templates of the same biometric method obtained with the help of a single sensor, and (e) a multimodal system combining information about the biometric features of the individual to establish his identity [2][3][4].

Fusion levels
In multimodal biometric systems, there are a number of strategies (scenarios) for the fusion of biometric information: • Data fusion from sensors. Data from various sensors form one vector. Fusion of information obtained from many different sensors for a single biometric feature.
• The fusion of feature vectors extracted from various biometric modalities for further processing. A merger of information obtained from several unimodal biometric systems that process different body characteristics of the same person ( Figure 4a).  • Rank level fusion. The classifier determines the rank of each registered biometric identity. A high position is a good indicator of a good fit ( Figure 4b).

Related work
The fusion of biometrics modalities on different levels of multi-biometrics system is extensively studied in the literature ( Table 3). For all that the merger at the level of feature vectors is relatively poorly discussed. The merger at this level includes the integration of feature vectors corresponding to many sources of information. Because the feature vectors contain more elements than the input biometric

Biometrics traits
Fusion methods

Fingerprint and face
Feature In [5], it was proposed to extract face and fingerprint characteristics invariant to the rotation and scaling of Zernike moments (ZM With the help of the unified Gabor filter, fingerprint codes and finger vein codes are generated. The extraction of features is carried out by using a supervised local canonical correlation analysis (SLPCCA), and finally the NN-classifier is used [7] Fingerprint and iris Score In [8], authors propose a frequency approach to generate a unified homogeneous template for fingerprint and iris features. Scores generated from these templates are fused using the sum rule [8] Palm print and hand shape Feature Information from the face image and gait image are combined at the function level. Using the principal component analysis (PCA) method, facial features were obtained The result of multiple discrimination analysis (MDA) is gait energy image (GEI) Recognition rate results are 91.3% [9] Palm print and iris Feature In system described in [10], texture parameters are extracted based on Gabor filters. [10] Fusion of the palm print features and iris features is based on the wavelets. Decision is obtained using kNN classifier. Recognition accuracy is 99.2% and FRR = 1.6% In [11], fusion method for the information of phases about the iris and palm utilizes a Baud limited image product (BLIP) [11] Finger knuckle and palm print Feature In this paper, feature extraction method for palm print is monogenic binary coding; for inner knuckle print recognition, two algorithms named ridgelet transform [12] data, it is obvious that the merger at the feature vectors level will provide better authentication results. However, mergers at this level are difficult to implement in practice because (i) sets of features of many modalities may be incompatible, (ii) the combination of two feature vectors may result in a vector of features with very large dimensionality, and (iii) a complex comparing system is required.

The proposed multi-biometric system
The multi-biometric system (dorsal vein + periocular + palm print) is presented in Figure 5.
In our proposed method, the first is preprocessing block including noise elimination, ROI detection and normalization, and contrast normalization. For all three [13] Face and gait Feature Method is based on learning face and gait features in image transform spaces. Two methods are considered-PCA and LDA [14] Face and iris Score Multi-biometrics system using dual iris, visible and thermal face traits is considered. 1D Log-Gabor and Complex Gabor Jet Descriptor (CGJD) were used to extract feature vectors. Authors proposed a score level fusion algorithm [15] The ordinal measures and local binary pattern (LBP) methods are proposed to extract features from iris and face regions, respectively [16] Feature Paper [17] presents the extractions of iris features based on 2D Gabor and facial features using the PCA method [17] Face and hand geometry Feature The 2D DCT is used to extract discriminant face features which are concatenated with hand geometric features. The resultant feature vector is classified using SVM [18] Face and ear Scores To match score level, fusion is proposed in [19]. Authors use Dempster-Shafer decision theory for each modality. Recognition rate is 95.53% with 4.47% EER [19] Ocular-iris and conjunctival vascular Score In [4], authors presented fusion of both iris and conjunctival vascular information. A weighted fusion method is proposed for each modality. The fusion resulted in an EER of 2.83% [4] Face, ear, and signature

Rank
In [20], the PCA and Fisher's linear discriminant (FLD) methods in the face, ear, and signature, multimodal biometrics system is proposed. Local features are extracted from face, ear, and signature data. Features are matched using Euclidean distance. This system is using rank level fusion [20]  modalities, noise elimination for an image f x; y ð Þ is performed using median filtering (2D MF) operation formulated as [21]: where A 1 is the MF window. Next step in preprocessing phase is ROI detection and normalization ( Figure 6). This operation is quite different for dorsal vein images, palm print images, and periocular images. For dorsal vein images, we use distance transform to detect the dorsal image center and build square ROI based on this center coordinates [22,23]. The ROI design for palm print images is based on hand-specific points (finger valleys) and two angles [24]. The periocular region is detected based on the center of the iris. Using the conventional algorithm for detecting the iris, we determine the center of the iris and its diameter. The periocular area is a rectangle centered in the iris center [25,26].
After the ROI detection, we perform image size normalization and apply the contrast normalization by using CLAHE algorithm. The image is divided into nonoverlapping areas of equal size, and the histograms of each region are calculated. Next, the cutoff threshold for histograms is obtained, and each histogram is processed in such a way that its height does not exceed the cutoff threshold [21].
The sample input images after normalization operations and operations using the CLAHE algorithm are shown in Figure 7. Next processing blocks include feature extraction, feature selection, fusion, and classification.

Gabor feature extraction
In biologically inspired vision models, receptor fields exist that are the primary aspect of early visual processing in mammalian vision systems. Gabor functions are widely used in image feature analysis because they are similar to receptive field profiles in mammalian cortical simple cells. These fields are modeled using Gabor filters [27].
Imitation of mammalian vision systems (or some of them) in object recognition systems leads to their increased efficiency and plausibility. Object recognition systems that are inspired by the biological approach use filter banks, in particular Gabor filters (Figure 8) [28][29][30][31][32].
Ph ω, θ x; y ð Þ ¼ arctan where and * is the convolution operator. The Gabor filter responses for palm print image and dorsal vein image are shown in Figures 9 and 10, respectively.

Periocular feature extraction by LBP
The periocular area contains the iris, eyes, eyelids, eyelashes, and partially eyebrows. The LBP method can be used to describe the texture of the periocular  area, and the feature vectors contain LBP features. The operator of local binary patterns (LBP) was proposed by Ojala [33] as a texture descriptor.
LBP divides the image into non-overlapping blocks of the same size. Local image features are calculated for each block separately. For a set of pixels belonging to a given block, the LBP values are calculated and then a histogram is created. The feature vectors (histograms) of each block are combined to form a global vector of features of the entire image.
LBP analyzes the local neighborhood consisting of g p points located on a circle with radius R and surrounding the center point of g c and checks whether the points of g p are greater or lesser than the g c point value.
The LBP value of theg c point is specified as follows: where g p and g c are the luminance values of the neighborhood and center point, respectively.
The idea of this operator is presented in Figure 11.
For an image size M Â N, the image descriptor is a histogram created from the LBP values: where k is one LBP pattern and K is the maximal LBP pattern value (number bin of the histogram).
Using the LBP operator, we obtain 2 P different output values corresponding to 2 P different binary patterns created by P of neighboring pixels. Certain binary patterns contain more information than others, so we can only consider this subset of LBP values. Patterns of this subset are called uniform patterns. So we have a standard LBP P, R operator and an LBP u2 P, R operator. Typically image is divided into n blocks and histograms of each block are concatenate into feature vector [34].
In the case LBP P, R operator, the histogram contains 256 bins. In the case of LBP u2 P, R operator, the histogram contains 59 bins (Figures 12 and 13).

Feature reduction and data fusion
The multi-biometric system has been tested using certain parts of the following databases: PolyU palmprint [24], IIITD periocular database [25], and Bosphorus  hand vein database [35]. We choose 20 subjects with 10 images per subject at random. From 10 images, 5 images are used for training and 5 for the testing. The combination of feature vectors at this level is difficult to achieve in practice due to the combination of certain fundamentally different feature vectors that can result in a resulting vector of features with very large dimensionality. In a merger at the level of feature vectors, each individual modality process generates a feature vector. The fusion process combines these feature vectors into one vector.
For dorsal vein images and for palm print images, we perform the same image processing operations that the feature vectors have the same sizes. As a result of convolution operation of multiscale and multi-orientation Gabor filters with the input image, we get the Gabor response images. The feature vector has a very large size of (M x N x k x l) where M x N is the image size, k is the number of scales, and l is the number of orientations. In our case, for both dorsal vein images and palm print images, we get a feature vector containing (150 Â 150 Â 3 Â 6) = 405,000 items. The images subjected to the Gabor filtration are rescaled with a scale factor of 0.1, which allows obtaining a vector of features with a size of 1 Â 4050 elements.
For periocular images, the feature vector has a size of 36 Â 59 = 2124. Next we reduce dimensionality of these vectors used in PCA method ( Figure 14 and Table 4) [5]. Separated features are normalized using zero mean and unit variance as Figure 14.
Steps to image processing using PCA.

PCA algorithm
Organizing the training set of images where q is the number of images in the training set Calculating the average of the set T The eigenvectors and corresponding eigenvalues are computed The eigenvectors and their corresponding eigenvalues are paired and ordered from high to low. Approximated image is calculated as Table 4. PCA algorithm.
where μ i and σ i are the mean value and standard deviation of the i-th feature, f i is the normalized i-th feature vector. Table 5 shows the recognition performance depending on the number of selected eigenvectors.

Conclusion
In this chapter, Gabor's functions and LBP features are proposed for recognition in a multi-biometric system that uses three modalities: dorsal vein, periocular, and palm print. Using PCA method dimensionality feature vectors from these modality are reduced. Feature vectors are normalized and fused using concatenation operation. Based on the results, we suggest that multi-biometric system using the fusion of dorsal vein, periocular, and palm print images can offer recognition rate which the unimodal biometric system cannot.