Finger-Vein Image Restoration Based on a Biological Optical Model Finger-Vein Image Restoration Based on a Biological Optical Model

lowest MSSIM value and the highest CII value. This means the proposed method has better performance in ﬁnger-vein image enhancement.


Introduction
Finger-vein recognition, as a highly secure and convenient technique of personal identification, has been attracted much attention for years. In contrast to conventional appearance-based biometric traits such as face, fingerprint and palmprint, finger-vein patterns are hidden beneath the human skin and unnoticeable without the help of some specific viewing or imaging devices. This makes finger-vein trait resistant to steal or forgery, and thereby highly reliable for identity authentication.
Generally, in order to visualize finger-vein vessels inside the finger tissues, the near infrared (NIR) transillumination is often adopted in image acquisition system [1], as shown in Fig. 1. In this imaging manner, the image sensor placed under a finger is used to visualize the transmitted NIR lights, as shown in Fig. 1(a) and (b), here, Fig. 1(b) is a disassembled homemade imaging device. Then with the help of imaging software, the finger-vein images can be captured by the computer. Due to the interaction between NIR light and biological tissues, thus the captured images inevitably carry some important inner information of finger tissue. In blood vessels, the hemoglobin absorbs more NIR radiation than other substances in finger tissues [2], the intensity distribution of transmitted NIR rays therefore vary spatially in terms of the vessel distribution, and venous regions can cast darker "shadows" on imaging plane while the other tissues present a brighter background, as shown in Fig. 1(d). From Fig.  1(d), we can clearly see that, in a captured image, not all regions are useful for accuracy finger-vein recognition, so to eliminate some unwanted regions, a simple but effective method of region of interest (ROI) localization have been proposed in our previous work [5]. In Fig. 1(e), we list some ROI extraction results for illustration.
Unfortunately, the captured finger-vein images are always not good in quality due to lower cotrast such that the venous regions are not salient. This certainly makes finger-vein feature representation unreliable, and further impairs the accuracy of finger-vein recognition in practice. According to tissue optics, multiple light scattering predominates in lights that penetrate through the biological tissue layer [3] as the biological tissue is a kind of highly heterogeneous optical medium in imaging. Thus, the quality of finger-vein images is always not good because the scattering effects can greatly reduce the contrast between the venous and non-venous regions [4]. The basic concept of image degradation due to light scattering is illustrated in Fig. 3. If the incident light is not scattered in optical medium, a real shadow of an object must be casted on the imaging plane, as shown in Fig. 3(a), where the dark circle region represents the real shadow of an object. However, the object shadow is always blurred to a certain extent since light scattering is inevitable in real situations, as shown in Fig. 3(b). Hence, in practical scenario, the inherent advantage of finger-vein can not always be exploited effectively and reliably for finger-vein recognition due to the low contrast of venous regions. Therefore, to exploit the genuine characteristics in finger-vein images, the visibility of finger-vein patterns should be reliably improved in advance.
In this chapter, we first give an analysis of the intrinsic factors causing finger-vein image degradation, and then propose a simple but effective image restoration method based on scattering removal. To give a proper description of finger-vein image degradation, a biological optical model (BOM) specific to finger-vein imaging is proposed according to the principle of light propagation in biological tissues. Finally, based on BOM, the light scattering component is properly estimated and removed for finger-vein image restoration.
In the following sections, we first give a brief description of the related work in Section 2. Then, in Section 3 the traditional image dehazing model is presented, and the optical model used in this chapter is derived after discussing the difference and relationship between our model and the image dehazing model. In Section 4, the steps of scattering removal algorithm are detailed. For finger-vein image matching, the Phase-Only-Correlation measure is used in Section 5. The experimental results are reported in Section 6. Finally, in Section 7, we give some conclusions.

Related work
Traditionally, many image enhancement methods have been proposed to improve the quality of vein images. Histogram equalization based algorithms were used to enhance the contrast between the venous and background regions in [6,7]. Considering the variations of vein-coursing directions, different oriented filtering strategies were used to highlight the finger-vein texture. [8][9][10][11]. The retinex theory combined with the fuzzy was adopted to enhance the near-infrared vein images [12]. Pi et al. [13] used an edge-preserving filter and an elliptic high-pass filter together to denoise and enhance some small blurred finger veins. Gao et al. [14] combined the traditional high frequency emphasis filtering algorithm and the histogram equalization to sharpen the image contrast. Oh et al. [15] proposed a homomorphic filter incorporating morphological subband decomposition to enhance the dark blood vessels. Although these methods can respectively enhance vein images to some extent, their performances were considerably undesirable in practice since they all did not treat of the key issue of light scattering in degrading finger-vein images.
Strong scattering occurring in the biological tissue during vein imaging is the main reason causing contrast deterioration in finger-vein images [16]. Considering light transport in skin tissue, Lee and Park used an depth-dependent point spread function (D-PSF) to address the blurring issue in finger-vein imaging [29,30]. This method is encouraging in finger-vein visibility improvement, however, D-PSF is derived for handling degraded issues in transcutaneous fluorescent imaging manner but not in transillumination manner [31]. Hence, the performance of D-PSF on light scattering suppression is still unsatisfying for finger-vein images since, in transillumination, light attenuation (absorption and scattering) arises not only from the skin but from other tissues of the finger, such as bone, muscles, and blood vessels [33]. Moreover, estimating biological parameters properly is also a difficult task for D-PSF based image debluring technique in practice. Therefore, for reliable finger-vein image contrast improvement, this chapter aims to find a proper way of scattering removal according to tissue optics, especially skin optics. In computer vision, scattering removal has been a hot topic for reducing the atmospheric scattering effects on the images of outdoor scenes [17][18][19][20][21]. This technique often is termed as dehazing or de-weather, which is based on a physical model that describes the formation of hazing image. Inspired by image dehazing, we here propose an optical-model-based scattering removal algorithm for finger-vein image enhancement. The proposed optical model allows for the light propagation in finger-skin layer such that it is powerful in describing the effects of skin scattering on finger-vein images.

The optical model of atmospheric scattering
Light scattering is a physical phenomenon as light transports in turbid medium. In daily life, we are very familiar with light scattering, such as blue sky, fog and smoke. Therefore, the irradiance received by the camera is often attenuated due to medium absorption and scattering, which degrades the captured images and make them lose the contrast. Removing the scattering effect is certainly necessary for improving the scene visibility. In computer vision, the physical model widely used to image dehazing, also named Koschmieder model, is expressed as [22] (1) This model provides a very simple but elegant description for two main effects of atmospheric scattering on the observed intensity I d of an object at a distance d in a hazing or foggy day. Here, the intensity at close range (distance d = 0) I 0 is called the intrinsic intensity of the object, I ∞ is the intensity of environmental illumination (equivalent to an object at infinite distance), which is generally assumed to be globally constant, and K is the extinction coefficient of the atmosphere.
As illustrated in Fig. 4, the first effect of atmospheric scattering is called direct attenuation, and can be described by Beer-Lambert law, which results in an exponential attenuation of object intensity with the transmission distance through scattering media, i.e., the first term e −Kd I 0 on the right side of Eq. (1). The second effect, referred to as airlight in Koschmieder theory of horizontal visibility, is caused by the suspended particles in haze or fog that scatter the environmental illumination toward the observer. The airlight acts as an additional radiation superimposed on the image of the object, whose intensity is related to the environmental illumination I ∞ and increases with pathlength d from the observer to the object, as described by the term (1 − e −Kd )I ∞ . It is noticeable that, despite having not taken multiple scattering into account, the Koschmieder model is practicable for vision applications. In atmosphere, the distances between particles are usually large enough so that the particles can be viewed as independent scatterers, whose scattered intensities do not significantly interfere with each other, and thus the effect of multiple scattering is negligible [23]. Whereas, in the case of biological tissue, light propagation suffers a more complex process due to the complexity of tissue structure. Particularly, the scattering particles in biological tissue are so dense that the interaction of scattered intensities between neighboring particles cannot be ignored [24]. Hence, multiple scattering is said to be prevalent in the biological optical medium [1,[32][33][34][35].
From the biophotonic point of view, as the light propagates through a tissue, the transmitted light is composed of three components-the ballistic, the snake, and the diffuse photons [25], as shown in Fig. 5. Ballistic photons travel a straight, undeviated path in the medium. Snake photons experience some slight scattering events, but still propagate in the forward or near-forward direction. Diffuse photons undergo multiple scattering and emerge from random directions. Obviously, in transillumination imaging of objects embedded in the biological tissue, the ballistic photons with propagation direction preservation can form sharp shadows of objects on the imaging plane, whereas the multiple scattered diffuse photons can inevitably reduce the contrast of the shadows as well as giving rise to the unwanted, incoherent imaging background [26]. That is to say, the multiple scattering is the most unfavorable factor that contributes to diffuse photons and further leads to image blurring in optical transillumination imaging.
Based on the preceding analysis of image dehazing model and the associated knowledge about light propagation through biological tissue, we propose a simplified skin scattering model to characterize the effects of skin scattering on finger-vein imaging, as shown in Fig. 6. Before giving mathematical description of the proposed model, there are several points with respect to the optical model should be stated: • In a real finger-vein imaging system, the objects to be visualized are palm-side vein vessels which are mostly interspersed within the inner layer of the finger skin (see Fig. 7(a)). So, for the sake of simplicity, only the skin layer is considered as a reference optical medium regardless of the atmosphere between skin surface and camera, whose scattering effect is very small and negligible here.
• Human skin is known to be an inhomogeneous, multilayered tissue containing epidermis, dermis and subcutaneous layer, as shown in Fig. 7(a). But at the molecular level, skin tissues are composed of a limited number of basic molecular species, and these molecules are composed of optically similar chemical units [3]. Moreover, the ensemble of light-skin interaction homogenizes the optical behavior of biological structures. Thus, the skin can be viewed as a medium with a random but homogeneous distribution of scattering particles over its thickness [27], as shown in Fig. 7(b), and then the scattering coefficient of the skin tissue here can be assumed to be locally constant for a given finger subject.
• Different from the image dehazing techniques, we need not consider the effect of environmental illumination as well as the airlight indeed. Nevertheless, due to light interaction occurs among biological scatterers, the scattered radiation from both the object and the background will be partially re-scattered towards the observer, which approximately amounts to environmental illumination for finger-vein imaging.
In view of these points, the radiant intensity observed at skin surface corresponding to the object with a certain depth in the skin can be simply decomposed into the direct attenuation component and the scattering component, as shown in Fig. 6. The former, representing the unscattered of ballistic photons, is a reduction of the original radiation over the traversing medium, which obeys the Beer-Lambert law [16,36], while the latter represents the effect of snake and diffuse photons, which emerges randomly from the tissue surface. Especially, the proportion of scattered radiation enters into the direction of observer and interferes with the direct radiation of object, whose intensity increases with depth because a deeper object tends to suffer more influence of the scattered radiation.
For the direct attenuation component, its intensity on the imaging plane is mainly determined by the non-scattered light. So, assume that µ 1 and µ 2 denote the optical absorption and scattering coefficients, based on the Beer-Lambert law, we can obtain where I 0 represents an finger-vein image free of degradation, D is the object depth in a biological medium, µ = µ 1 + µ 2 is called transport attenuation coefficient, and I tr denotes the transmitted intensity after absorption and scattering. Noticeably, due to the heterogeneity of skin tissue and the spatial randomness of vein distribution, both the transport attenuation coefficient µ and the depth D vary spatially in tissue medium, that is, µ = µ(x, y) and D = D(x, y). So, for a given biological tissue, we can define T(x, y) is often called non-scattered transmission map [16], which describes the optical transmissivity of the given tissue medium.
For the scattering component, due to the randomicity of the scattered light, it can be regarded as the background illumination on the whole, and only a part of the background illumination can arrive at the imaging plane. For intuitively understanding this point, Fig. 8 gives a schematic illustration. In Fig. 8, s represents an original source in x-y coordinates, p is the observation of s on the imaging plane, H denotes a small column in the skin tissue corresponding to a beam from the object point s to a point p on the image plane (each pixel corresponds to a small column), the neighbor points (s ′ i , i = 1, 2, · · · , n) around s are viewed as the local background radiation sources, which would emit radiation and produce a scattering component along H.
where I 0 (s) still represents the intrinsic intensity of the object, that is veins, to be visualized, I r (s) denotes the intensity of scattered radiation, and I(p) is the observation of the vein object on the image plane. A key point needs to be noted that, different from the environmental illumination in atmosphere, I r (s) varies spatially because its value is associated to the intensities of the imaging background.
Let the original intensity of a neighbor point s ′ i be I 0 (s ′ i ), then the direct transmitted radiation of this point, that is the unscattered radiation, should be I 0 (s ′ i )T(s ′ i ). So, according to the energy conservation principle, the scattered radiation of this point should be 1 − is the depth of point s ′ i in the skin layer. Thus, we can obtain the scattered radiation I r (s) in H. Since the scattering directions are random, I r (s) here is considered as an average of total radiation from overall neighbor points and can be rewritten as where Ω(s) denotes the 2D neighborhood centered at point s, and Z Ω indicates the number of points in Ω(s). Given I r (s), µ(s) and D(s), we can obtain I 0 (s) which represents the intrinsic intensity of a finger-vein image without scattering corruption.
However, solving I 0 (s) from a single observed image I(p) with Eq. (4) is a very ill-posed problem. Not only is the extinct coefficient µ(s) of human skin tissue inconsistent, but the thickness D(s) also varies with different individuals. The values of I r (s), µ(s) and D(s) therefore can not be accurately evaluated in practice. This is due to that the light scattering phenomenon in tissues is very complex. Hence, we have to utilize the observation (or captured) image I(p) to estimate the scattering component for implementing scattering removal.

The proposed scattering removal algorithm for restoration
In the observation I(p), veins appear shadows due to light absorbtion, which makes vein information sensitive to illumination modification. Hence, finger-vein images should be transformed into their negative versions. In the negative versions, the venous regions turn brighter than their surroundings, veins thus can be regarded as luminous objects. Moreover, in this situation, the skin tissue can be approximately treated as the only opaque layer that blurs vein objects during imaging. This is beneficial for scattering illumination estimation. Thus, we can rewrite the proposed BOM aŝ whereÎ(p),Î 0 (s) andÎ r (s) represent the negative versions of I(p), I 0 (s) and I r (s), respectively. Referring to the image dehazing technique, we here introduce V(s) can be regarded as the total scattering component. Moreover, we can obtain the transmission map, T(s) describes the relative portion of light radiation surviving through a medium. Thus, the optical model can be rewritten asÎ Instead of directly computingÎ 0 (s), we first estimate the scattering component V(s), and then estimate the intensity of scattered radiationÎ r (s). Thus, the restored imageÎ 0 (s) can be obtained based on Eqs. (8) and (9).

Scattering component estimation
Generally, the distribution of scattering energy is not uniform in a local block Ω(s) since the skin medium is inhomogeneous. However, it is affirmable that (1) the directions of scattered light rays are random due to the high density of biological cells, and (2) the nearer s ′ i to s is, the higher the probability of the scattered light into column H is [40,41]. Although the direction of scattered light is highly random due to the heterogeneity of skin tissue, the multiple scattering is dominated by near forward scattering events in biological tissues [16,42]. Hence, it is reasonable that using the local observation to estimate the scattering component.
Here, unlike the solution of scattering component estimation described in [18], V(s) varies locally and spatially on finger-vein imaging plane due to the heterogeneousness of the human skin tissue. In this sense, three practical constraints should be introduced for V(s) estimation: • For each point s, the intensity V(s) is positive and cannot be higher than the final observed intensityÎ(p), that is, 0 ≤ V(s) ≤Î(p); • V(s) is smooth except the edges of venous regions since the points in Ω(s) approximate to be same in depth; •Î r (s) tends to be constant in Ω(s) and V(s) ≤Î r (s) ≤Î(p).
Based on these constraints, to estimate V(s), a fast algorithm described in [20] is modified as where B(s) = A(p) − median Ω(p) Î (p) − A(p) , A(p) = median Ω(p) Î (p) , Ω(p) denotes the 2D neighborhood centered at point p, w 1 (∈ [0, 1]) is a factor controlling the strength of the estimated scattering component. Next, for removing the estimated scattering effect, we also have to estimate T(s) based on the observation.

Scattering radiation estimation
To obtain the transmission map T(s), we should compute I r (s) in advance according to the Eq. 8. Intuitively, we can obtainÎ r (s) via Eq. (5) directly. However, it is a difficult task since the intrinsic intensityÎ 0 (s ′ i ) is unavailable in practice. Hence, considering the physical meaning that the scattered radiationÎ r (s) depends on the interaction among neighbor points in Ω(s), we here simply use a local statistic of Ω(p) to representÎ r (s), that is, where p i ∈ Ω(p), Z Ω(p) indicates the number of points in Ω(p), and w 2 (∈ [0, 1]) is a factor for making the constraint V(s) ≤Î r (s) ≤Î(p) satisfying. So, based on Eq. (8), we can estimate T(s) accordingly.

Finger-vein image restoration
Given the estimations of V(s) and T(s), we can approximately restore an original finger-vein image with scattering removal. That is, by solving Eq. (6) with respect toÎ 0 (s), we can obtain Thus, computing I 0 (s) pixelwise using Eq. (12) can generate an image I 0 (x, y) automatically and effectively. Here, I 0 (x, y) represents the restored finger-vein image which appears free of multiple light scattering.

Finger-vein image matching
In this section, the Phase-Only-Correlation (POC) measure proposed in [28] is used for simply handling the finger-vein matching problem based on the restored finger-vein images.
Assume that I 0 i (x, y) and I 0 j (x, y) are two restored images, and F i (u, v) and F j (u, v) represent their 2D DFT, respectively, according to the property of Fourier transform, that is, where " • " denotes a 2D correlation operator, we can compute the cross phase spectrum as Let r(x, y) = IDFT(R(u, v)), thus, r(x, y) is called the POC measure. The POC measure has a sharp peak when two restored finger-vein images are similar, whereas it will be near zero for those from different classes, as shown in Fig. 9. Moreover, the POC function is somewhat insensitive to image shifts and noises. This is helpful for accurately measuring the similarities in finger-vein image matching.
It is worth pointing out that, to robustly handle accurate image matching problem, band-limited phase-only-correlation (BLPOC) function has been also proposed in [28] and widely used for image matching in practice [37][38][39]. Compared with POC, BLPOC is more reliable in measuring the similarities between two images. However, traditional POC is yet more convincing than BLPOC in investigating the qualities of images. This is because the matching result based on POC is more sensitive to image quality than that of BLPOC. Hence, the POC function still can be used as a simple and effective measure to objectively evaluate the performance of the proposed method in scattering removal and venous region enhancement.

Experimental results
In this section, the used finger-vein images are captured by a homemade transillumination imaging system with a 760 nm NIR LED array source, and then extracted from raw images by the ROI localization and segmentation method proposed in [5]. The finger-vein image database contains 700 individual finger-vein images from 70 individuals. Each individual contributes 10 forefinger-vein images of the right hand. All cropped finger-vein images are 8-bit gray images with a resolution of 180 × 100.

Finger-vein image restoration
Here, some captured finger-vein image samples are collected to demonstrate the validity of the proposed method in finger-vein image restoration. Fig. 10 shows some examples of the estimated V(x, y), I r (x, y), T(x, y) and restored finger-vein images I 0 (x, y). After scattering removal, the contrast of finger-vein images is improved significantly, and the vein networks present in the restored images can be clearly distinguished from the background. This shows that the proposed optical model allowing for the effects of light scattering in skin layer, particularly the multiple scattering, is desirable for describing the mechanism of finger-vein image degradation.
Nevertheless, the proposed method is somewhat sensitive to image noises, as shown in Fig. 10(e). In fact, before lighting the palm-side veins, the NIR rays have been randomly diffused by finger dorsal tissues such as finger-back skin, bone, tendon, fatty tissue and so on. This inevitably gives rise to irregular shadows and noises in the captured finger-vein images, whereas the proposed optical model has not taken account of the effects of finger dorsal tissues except the palm-side skin. As a result, the spatial varied background noises are also strengthened when estimating the scattering components.
In Fig. 11, we compare our method with several common approaches for finger-vein image enhancement. Additionally, we treat the degraded finger-vein images as hazing images, and directly use dehazing method to restore them regardless of the mismatch between the Koschmieder model and the proposed model. Here, a method proposed in [20] is adopted to implement finger-vein image "dehazing", and the results are also shown in Fig. 11.  In order to evaluate the performance of the proposed method in terms of contrast improvement for finger-vein image, the mean structural similarity index (MSSIM) [13] and the contrast improvement index (CII) [15] are used as two common evaluation criterions. We first randomly choose 50 individual finger-vein images from database as samples, and use these enhancement methods in Fig. 11 to process the finger-vein image samples. Then, we obtain the average MSSIM and the average CII of every enhancement method.
In general, MSSIM is often used to measure the similarity between a processed image and a standard image with perfect quality ( i.e., a distortion-free image). The larger the value of MSSIM is, the better an image is improved. This makes a processed image more approximate to its standard quality. However, it is impossible for us to have standard or perfect finger-vein images since the captured images all are degraded due to light scattering. Therefore, we regard the degraded finger-vein images as standard references. Thus, the more the dissimilarity between a processed finger-vein image and its original version is, the better the finger-vein is improved. That is, the lower the value of MSSIM is, the better the quality of a restored image is. CII is often used to measure the improvement of contrast between a processed image and its original version, and the larger the value of CII is, the better the contrast of an improved image is.
Hence, the quality and the visibility of restored finger-vein images can be quantitatively evaluated using MSSIM and CII. In Table 1, we list the two values corresponding to different finger-vein enhancement methods. From Table 1, we can clearly see that the proposed method provides the lowest MSSIM value and the highest CII value. This means the proposed method has better performance in finger-vein image enhancement. The results from histogram template equalization (HTE) [6]. (c) The results from high frequency emphasis filtering (HFEF) [14]. (d) The results from circular Gabor filtering (CGF) [8]. (e) The results from image dehazing (ImD) [20]. (f) The results from the proposed method.

Finger-vein image matching
For finger-vein matching on this database, the number of genuine attempts is 3,150 (70C 2 10 ), and the number of impostor attempts is 241,500 (10 × 10C 2 70 ). By respectively using the original images, HTE-based images, HFEF-based images, CGF-based images and the proposed restored images for finger-vein matching under POC (Phase-Only-Correlation) measure, the ROC (receiver operating characteristic) curves are plotted in Fig. 12, where false non-match rates (FNMR) and false match rates (FMR) are shown in the same plot at different thresholds on the POC matching score, and EER (equal error rate) is the error rate where FNMR and FMR are equal.
From Fig. 12, we can clearly see that the proposed method has the best performance of ROC curves and makes the lowest EER. This indicates that the finger-vein images with scattering removal are more discriminative in inter-class. Hence, the proposed method is desirable for improving the accuracy of finger-vein image matching in practice.

Conclusions
In this chapter, a scattering removal method was introduced for finger-vein image restoration. The proposed method was based on a biological optical model which reasonably described the effects of skin scattering. In this model, the degradation of finger-vein images was viewed as a joint function of the direct light attenuation and multiple light scattering. By properly estimating the scattering components and transmission maps, the proposed method could effectively remove the effects of skin scattering effects from finger-vein images to obtain the restored results. The comparative experiments and quantitative evaluations demonstrated that the proposed method could provide better results compared to the common methods for finger-vein image enhancement and recognition.
Indeed, the proposed method also had its own drawbacks. First, the simplified model in our work did not take into account of the effects of background tissues, which made the proposed method somewhat sensitive to image noises while enhancing the vein patterns. Besides, the rough estimations of the scattering components as well as the scattered radiations could also decrease the performance of the proposed method to some extent. All these shortcomings will be of our further improvement in future work.