Open access peer-reviewed chapter
Abstract
Automated face recognition (AFR) aims to identify people in images or videos using pattern recognition techniques. Automated face recognition is widely used in applications ranging from social media to advanced authentication systems. Whilst techniques for face recognition are well established, the automatic recognition of faces captured by digital cameras in unconstrained, real‐world environment is still very challenging, since it involves important variations in both acquisition conditions as well as in facial expressions and in pose changes. Thus, this chapter introduces the topic of computer automated face recognition in light of the main challenges in that research field and the developed solutions and applications based on image processing and artificial intelligence methods.
Keywords
- face recognition
- face identification
- face verification
- face authentication
- face labelling in the wild
- computational face
1. Introduction
Automated face recognition (AFR) has received a lot of attention from both research and industry communities since three decades [1] due to its fascinating range of scientific challenges as well as rich possibilities of commercial applications [2], particularly in the context of biometrics/forensics/security [3] and, more recently, in the areas of multimedia and social media [4, 5].
Face recognition is the field trying to bring an answer to the question:
AFR could be distinguished further into the computer automated face identification and the computer automated face verification. Hence, on the one hand, automated face identification consists in a one‐to‐many (1:N) search of a face image among a database containing many different face images in order to answer questions such as ‘
Moreover, AFR could be the basis to the solution of the ‘
The general AFR process is illustrated in Figure 1. Usually, it first applies techniques addressing questions such as ‘
Figure 1.
Overview of the face detection and recognition processes.
In particular, this chapter is dedicated to the ‘why’ and ‘how’ of the computer‐automated face recognition in constrained and unconstrained environments. The remaining parts of this chapter are structured as follows: in Section 2, we describe AFR's today challenges, while corresponding scientific solutions and industrial applications are presented in Sections 3 and 4, respectively. Section 5 draws up new trends and future directions for automated face recognition performance improvements and evolution.
2. Challenges
The study and analysis of faces captured by digital cameras address a wide range of challenges, as detailed in Sections 2.1–2.7, which all have a direct impact on the computer automated face detection and recognition.
2.1. Pose variations
Head's movements, which can be described by the egocentric rotation angles, i.e. pitch, roll and yaw [11], or camera changing point of views [12] could lead to substantial changes in face appearance and/or shape and generate intra‐subject face's variations as illustrated in Figure 2, making automated face recognition across pose a difficult task [13].
Figure 2.
Illustration of pose variations around egocentric rotation angles, namely (a) pitch, (b) roll and (c) yaw.
Since AFR is highly sensitive to pose variations, pose correction is essential and could be achieved by means of efficient techniques aiming to rotate the face and/or to align it to the image's axis as detailed in reference [13].
2.2. Presence/absence of structuring elements/occlusions
The diversity in the intra‐subject face's images could also be due to the absence of structuring elements (see Figure 3a) or the presence of components such as beard and/or moustache (see Figure 3b), cap (see Figure 3c), sunglasses (see Figure 3d), etc. or occlusions of the face (see Figure 3e) by background or foreground objects [14].
Figure 3.
Illustration of (a) absence or (b‐d) presence of structuring elements, i.e. (b) beard and moustache, (c) cap,
Thus, face's images taken in an unconstrained environment often require effective recognition of faces with disguise or faces altered by accessories and/or by occlusions, as dealt by appropriate approaches such as texture‐based algorithms [15].
2.3. Facial expression changes
Some more variability in face appearance could be caused by changes of facial expressions induced by varying person's emotional states [16] which are displayed in Figure 4.
Hence, efficiently and automatically recognizing the different facial expressions is important for both the evaluation of emotional states and the automated face recognition. In particular, human expressions are composed of macro‐expressions, which could express, e.g., anger, disgust, fear, happiness, sadness or surprise, and other involuntary, rapid facial patterns, i.e. micro‐expressions; all these expressions generating non‐rigid motion of the face. Such facial dynamics can be computed, e.g., by means of the dense optical flow field [17].
Figure 4.
Illustration of varying facial expressions that reflect emotions such as (a) anger, (b) disgust, (c) sadness or
2.4. Ageing of the face
Another reason of face appearance's changes could be engendered by the ageing of the human face, and could impact on the entire AFR process if the time between each image capture is significant [18], as illustrated in Figure 5.
Figure 5.
Illustration of the human ageing process, where the same person has been photographed (a) at a younger age and (b) at an older age, respectively.
To overcome face ageing issue in AFR, methods need to take properly into account facial ageing patterns [18]. Indeed, over time, not only face characteristics such as its shape or lines are modified [19], but other aspects are changing as well, e.g. hairstyle [20].
2.5. Varying illumination conditions
Large variations of illuminations could degrade the performance of AFR systems. Indeed, for low levels of lighting of the background or foreground, face detection and recognition are much harder to perform [21], since shadows could appear on the face and/or facial patterns could be (partially) indiscernible. On the other hand, too high levels of lights could lead to over‐exposure of the face and (partially) indiscernible facial patterns (see Figure 6).
Robust automated face detection and recognition in the case of (close‐to‐) extreme or largely varying levels of lighting apply to image‐processing techniques such as illumination normalization, e.g. through histogram equalization [22]; or machine‐learning methods involving the actual image global image intensity average value [21].
Figure 6.
Illustration of camera lighting variations, leading to (a) over‐exposure of the face, (b) deep shadows on the face or (c) partial backlight.
2.6. Image resolution and modality
Other usual factors influencing AFR performance are related to the quality and resolution of the face image and/or to the set‐up and modalities of the digital equipment capturing the face [23]. For this purpose, ISO/IEC 19794‐5 standard [24] has been developed to specify scene and photographic requirements as well as face image format for AFR, especially in the context of biometrics. However, real‐world situations of face image acquisition imply the use of different photographic hardware, including one or several cameras which could be omnidirectional or pan‐tilt‐zoom [25], and which could include, e.g. wide‐field sensors [25], photometric stereo [26], etc. Cameras could work in the range of the visible light or use infra‐red sensors, leading to multiple modalities for AFR [6]. Hence, faces acquired in real‐world conditions lead to further AFR challenges.
Figure 7.
Illustration of variations of the image scale and resolution, with (a) a large‐scale picture, (b) a small‐scale picture and (c) a low‐resolution picture.
For example, as shown in Figure 7, in some situations, a face could be captured at distance resulting in a smaller face region image compared to the one in a large‐scale picture. On the other hand, some digital camera could have a low resolution [27] or even very low resolution [28], if the resolution is below 10 × 10, leading to poor quality face images, from which AFR is very difficult to perform. To deal with this limitation, solutions have been proposed to reconstruct a high‐resolution image based on the low‐resolution one [28] using the super‐resolution method [29, 30].
2.7. Availability and quality of face datasets
Each AFR technology requires an available, reliable and realistic face database in order to perform the 1:N or 1:1 face search within it (see Figure 1). Hence, the quality such as completeness (e.g. including variations in facial expressions, in facial details, in illuminations, etc.) as well as accuracy (e.g. containing ageing patterns, etc.) and the characteristics (e.g. varying image file format and colour/grey level, face resolution, constrained/unconstrained environment, etc.) of a face dataset are crucial to the AFR process [31]. Moreover, when dealing with face data, people's consent and privacy should be respected as AFR systems should comply with the Data Protection Act 2010 [32].
For research purpose, several face databases have been developed and are publicly available. Well‐established, online face databases are as follows:
ORL [33] is a 400‐picture dataset of 40 distinct subjects, in portable grey map (
pgm ) format and with a 92 × 112 pixel resolution, 8‐bit grey level. Men and women's faces are taken against a dark homogeneous background, under varying illumination conditions. The subjects are in up‐right, frontal position, with variations in face expressions, facial details and poses within ±20% in yaw and roll.Caltech Faces [34] dataset consists of 450
jpeg images with a resolution of 896 × 592 pixels. Each image shows the frontal view of a face (single pose) of one out of 27 unique persons, under different lighting, expressions and backgrounds.The Face Recognition Technology (FERET) [35] database has been built with 14,126 face images from 1199 individuals, defining sets of 5–11 greyscale images per person. Each set contains mugshots with different facial expressions and facial details, acquired using various cameras and varying lighting.
BioID Face database [36] has 1521 frontal face images of 23 people. Images of 384 × 286 pixel resolution are in
pgm format and have been captured in real‐world conditions, i.e. with a large variety of illumination, background and face size.Yale face database [37] has 165 greyscale,
gif images of 15 individuals. There are 11 images per subject, one per different facial expression or configuration, i.e. left/centre/right‐light, with or without glasses and with different expressions.Caltech 10,000 web faces [38] have collected 10,524 human faces of various resolutions and in different settings (e.g. portrait images, group of people, etc.) from
Google Image . Coordinates of eyes, nose and the centre of the mouth for each frontal face are provided in order to be used as ground truth for face detection algorithms, or to align and/or crop the human faces for AFR.
Some databases contain both 2D and 3D face data, e.g. Face Recognition Grand Challenge (FRGC) dataset [39] recorded such 50,000 un‐/controlled images from 4003 subject sessions.
Other datasets have multiple modalities such as XM2VTSDB multi‐modal face database [40] which is the Extended M2VTS database. It is a large, multi‐modal database captured onto high‐quality, digital video. It contains four recordings, each with a speaking head shot and a rotating head shot, of 295 subjects taken over a period of 4 months. This database includes high‐quality colour images, 32 kHz 16‐bit sound files, video sequences and also a 3D model.
Another multi‐modal database is the Surveillance Cameras Face (SCFace) [41] dataset. It has recorded 4160 static human faces of 130 subjects, in the visible and infrared spectrum, in an unconstrained indoor environment, using a multi‐camera set‐up consisting of five video‐surveillance cameras which various qualities mimic real‐world conditions.
Recent developments of face databases focus on capturing faces in the wild, i.e. in unconstrained environments. For example, Face Detection Data Set and Benchmark (FDDB) [42] is a dataset of 2845 images, both greyscale and colour ones, with 5171 faces in the wild, which could include occlusions, poses variations, low resolution and out‐of‐focus faces.
Labelled Faces in the Wild (LFW) [43] database is a popular dataset for studying multi‐view faces in an unconstrained environment. It has recorded 13,233 foreground face images; other faces in the images being assimilated to the background. It has targeted 5749 different individuals, which could have one or more images in the database, and presents variations in pose, lighting, expression, background, race, ethnicity, age, gender, clothing, hairstyles, camera quality, colour saturation, focus, etc. Images have a 250 × 250 pixels resolution and are in
Some other available face datasets have been designed for specific purposes. Hence, Spontaneous MICro‐expression database (SMIC) [44] is used for facial micro‐expressions recognition, while the Acted Facial Expression in the Wild (AFEW) database [45], which has semi‐automatically collected face images with acted emotions from movies, is dedicated to macro‐expression recognition in close‐to‐real conditions. On the other hand, FG‐NET Ageing database (FG‐NET) [46] could be applied for age estimation, age‐invariant face recognition and age progression.
3. Solutions
Major pattern recognition techniques as well as main machine‐learning methods used for AFR systems are presented in Section 3.1, while classic approaches for AFR in still images or video databases/live video streams are mentioned in Section 3.2.
3.1. Face recognition systems
Most of the AFR systems consist in a two‐step process (see Figure 8) based firstly on facial feature extraction, as explained in Section 3.1.1, and second, on facial feature classification/matching against an available face database, as mentioned in Section 3.1.2.
3.1.1. Feature extraction
Facial features are representing the face in a codified way which is computationally efficient for further processes such as matching, classification or other machine‐learning techniques, in order to perform AFR. On the other hand, computing facial features in an image could serve to detect a face and to locate it within the image, as illustrated in Figure 9.
Figure 8.
Schematic representation of the automated face recognition system.
Figure 9.
Face location via (a) a bounding box and (b) an ellipse.
Facial feature representations could be of different nature from sparse to dense ones, and could be focused on face appearance, face texture or face geometry [15].
Figure 10.
Results of facial feature modelling using different approaches, e.g. (a-b) Haar-like features; (c) Linear Binary Patterns (LBP); (d) Edge map; (e) Active shape; (f) SIFT points.
Commonly computed facial features are Haar‐like features [47] (Figure 10(a, b)); linear binary patterns (LBP) [48] (Figure 10(c)), which have been extended to local directional pattern (LDP) [49] for micro‐expressions recognition in particular; edge maps (Figure 10(d)) and their extension to line edge maps (LEM) [50]; active shape or active contours [51] (Figure 10(e)); SIFT points [52] (Figure 10(f)), etc.
The detected facial features, e.g. with SIFT points usually correspond to some or all elements of the set of facial anthropometric landmarks, i.e., facial fiducial points (FPs) (see Figure 11), which are defined as follows: FP1—top of the head, FP2—right eyebrow right corner, FP3—right eyebrow left corner, FP4—left eyebrow right corner, FP5—left eyebrow left corner, FP6—right eye right corner, FP7—right eye centre of pupil, FP8—right eye left corner, FP9—left eye right corner, FP10—left eye centre of pupil, FP11—left eye left corner, FP12—nose right corner, FP13—nose centre bottom, FP14—nose left corner, FP15—mouth right corner, FP16—mouth left corner, FP17—chin corner, FP18—right ear top corner, FP19—right ear bottom corner, FP20—left ear top corner and FP21—left ear bottom corner [53].
Figure 11.
Illustration of the 21 facial landmarks.
Computer automated face recognition relies on facial features, in the same way forensic examiners focus their attention not only on the overall similarity of two faces regarding their shape, size, etc. [54], but also on morphological comparisons region by region, e.g. nose, mouth, eyebrows, etc. [53]. Some AFR methods evaluate also discriminative characteristics such as the distance from people’s mouth to the nose, nose to eyes, mouth to eyes, etc. [55]. This adds robustness into AFR systems in the case of modification of some facial patterns over the course of time or occlusions.
Once the face is detected/located and the facial features are extracted, actions to crop the face, to correct its alignment by rotating it, etc., could be performed to address the challenges mentioned in Section 2, before passing the facial features into the next stage described in Section 3.1.2.
3.1.2. Feature classification/matching
For the recognition stage itself of the face recognition process, classification is often used as shown in Figure 12. Indeed, it is a machine‐learning technique [56] that has the task of first learning and then applying a function that maps the facial features of an individual to one of the predefined class labels, i.e. class 1 (face of the individual) or class 2 (not the face of the individual), leading in this case to a binary classifier. Classifiers could be applied to the entire set of the extracted facial features or to some specific face attributes, e.g. gender, age, race, etc. [57]. More recently, methods like neural networks are used as classifiers [58].
Figure 12.
Overview of the model computation.
On the other hand, some AFR systems use the matching technique that could be applied on facial geometric features or templates [59]. This approach is also useful for multimodal face data [60].
3.2. Examples of methods
Among hundreds of techniques developed in this field [1–10], Sections 3.2.1–3.2.4 explain briefly some well‐established methods for automated face recognition.
3.2.1. Eigenfaces
The eigenface approach [61] is a very successful AFR method. It involves pixel intensity features and uses the principal component analysis (PCA) of the distribution of faces, or
3.2.2. Active appearance models
The active appearance model (AAM) [64] combines shape and texture features; thus it is slower but more robust for AFR than active shape models (ASM). AAM is built as a multi‐resolution model based on a Gaussian‐image pyramid. For each level of the pyramid, a separate texture model is computed using 400 face images. Each face is labelled with 68 points around the main features, and the facial region is sampled by c. 10,000 intensity values. AFR is performed by matching the test face with the AAM, following a multi‐resolution approach that improves speed and robustness of this method [64].
3.2.3. Local binary patterns
In reference [48], local binary patterns (LBP), which are texture features, have been introduced for AFR. In particular, the face image is divided into independent regions where the LBP operator is applied to codify every pixel of each region by thresholding the 3 × 3‐neighbourhood of each pixel with the centre pixel value and by binarizing it, and then, creating a local texture descriptor with the histogram of the codes for each face region. A global description of the face is formed by concatenating the local descriptors. Next, the nearest‐neighbour classifier is used [48]. LBP approach has been widely adopted for AFR, and several enhancements have been proposed, e.g. the local directional patterns (LDP) [49].
3.2.4. SIFT
The discriminative deep metric‐learning (DDML) [52] approach for AFR in unconstrained environment uses facial features such as SIFT descriptors and trains a deep neural network as a classifier to learn a Mahalanobis distance metric in order to maximize face's inter‐class variations and minimize face's intra‐class variations, simultaneously [52].
4. Applications
Nowadays, industry integrates cutting‐edge, face recognition research into the development of the latest technologies for commercial applications such as mentioned in Sections 4.1–4.2.
4.1. Security
Face recognition is one of the most powerful processes in biometric systems [8] and is extensively used for security purpose in tracking and surveillance [65, 66], attendance monitoring, passenger management at airports, passport de‐duplication, border control and high security access control as developed by companies like
AFR is applied in forensics for face identification [68], face retrieval in still image databases or CCTV sequences [69], or for facial sketch recognition [70]. It could also help law enforcement through behaviour and facial expression observation [71], lie detection [72], lip tracking and reading [73].
Moreover, AFR is now used in the context of ‘Biometrics as a Service’ [74], within cloud‐based, online technologies requiring face authentication for trustworthy transactions. For example,
4.2. Multimedia
In our today's life, AFR engines are embedded in a number of multi‐modal applications such as aids for buying glasses or for digital make‐up and other face sculpting or skin smoothing technologies, e.g. designed by
In social media, many collaborative applications within
Other multimedia applications are using AFR, e.g. in face naming to generate automated headlines in
5. Conclusions
Since constraints shape the path for innovative solutions, we focused this chapter on scientific and technical challenges brought by computer automated face recognition, and we explained current solutions as well as potential applications. Moreover, there are a number of challenges ahead and plenty of room for innovations in this field of automated face recognition. In particular, three emerging directions are discussed in Sections 5.1–5.3.
5.1. Deep face
On the one hand, the proliferation of mobile devices such as smartphones and tablets, which are world‐widely available for consumers and which allow users to easily record digital pictures, and on the other hand, the outbreak of mobile and web applications, which manipulate and store thousands of pictures, have paved the way to the Big Data, and, among others, to the necessity to analysis large‐scale, face databases. This phenomenon has given rise to questions such as AFR technology scalability and computational power, and it has led to the development of a new AFR approach called deep face recognition [88], which involves deep‐learning techniques using convolutional neural networks [89], well fitted for big datasets [90]. Indeed, deep face methods are using large databases for training their models, as by biomimetics, they rely on the familiarity concept [91], which is based on the fact that more people are familiar with a person's face, more easily they recognized his/her face, even in complex situations like occlusions or low resolution. Moreover, the recent development of the deep face approach has benefited from progress in parallel computing tools for acceleration and enhancement of distributed computing techniques for scalability. In particular, for deep face recognition, graphics processing units (GPUs), which are specialized processors for real‐time, high‐resolution 3D graphics, are used as highly parallel multi‐core systems for big data [92], together with the Compute Unified Device Architecture (CUDA), which provides a simple and powerful platform [93], making easier for specialists in parallel programming to utilize GPU resources without advanced skills in graphics programming. Since the above‐mentioned, iterative computation consists of local parallel processing, CUDA implementation is employed for reducing the computation time of the AFR system [93]. However, deep face‐based methods generate themselves further challenges, e.g. face frontalization [94] that is the process of synthesizing frontal facing views of faces appearing in single unconstrained photos, in order to boost AFR performance within intelligent systems.
5.2. Wild face
Another challenge that has appeared with the generation of a large amount of visual data captured ‘in the wild’, i.e. in an unconstrained environment, by commercial cameras is the automated recognition of faces in the wild. It involves the enhancement of AFR methods [95] in order they efficiently deal with complex, real‐world backgrounds [96], multiple‐face scenes [51], skin‐colour variations [97], gender variety [98] and with inherent challenges such as image quality, resolution, illumination or facial pose correction [23, 27, 99].
5.3. Dynamic face
In the recent years, handling facial dynamics efficiently is crucial for AFR systems, because people have recorded a large amount of faces as still digital images, e.g. selfies or as video streams, e.g. CCTV sequences or online movies. Indeed, on the one hand, the different variations in facial micro/macro expressions [100], which generate fast, facial dynamics and the different processes such as ageing, which is an extremely slow, dynamic problem since the face evolves over large periods of time [18], have all an impact on AFR techniques. On the other hand, face acquisition in videos intrinsically creates facial dynamics due to camera motion, change of point of view, as well as head's movements or pose variations. Such situations require AFR engines perform in real time [84], apply image/frames pre‐processing such as face alignment [101], cope with intra‐class variations/inter‐class similarities [102] and are able to process single/multiple camera views [41] or synthesize a 3D face model from a single camera [103], leading to the wider study of the computational face.
References
- 1.
Sirovich L., Kirby M. Low‐dimensional procedure for the characterization of human faces. Journal of the Optical Society of America A ‐ Optics, Image Science and Vision. 1987. 4(3):519–524. - 2.
Zhao W., Chellappa R., Rosenfeld A., Phillips P.J. Face recognition: A literature survey. ACM Computing Surveys. 2003. 35(4):399–458. - 3.
Kisku D. R., Gupta P., Sing J. K., editors. Advances in Biometrics for Secure Human Authentication and Recognition . CRC Press, Taylor and Francis. 2013. 450 p. - 4.
Face Detection & Recognition Online Resources. Maintained by Dr. R. Frischholz [Internet]. 2016. Available from: https://facedetection.com/ [Accessed: 2016‐07‐07] - 5.
Face Recognition Online Resources. Information Pool for the Face Recognition Community. Maintained by Dr. M. Grgic [Internet]. 2016. Available from: http://www.face‐rec.org/ [Accessed: 2016‐07‐07] - 6.
Chellappa R., Wilson C.L., Sirohey S. Human and machine recognition of faces: A survey. Proceedings of the IEEE. 1995. 83(5):705–740. - 7.
Iosifidis, A., Gabbouj, M. Scaling-up class-specific kernel discriminant analysis for large-scale face verification. IEEE Transactions on Information Forensics and Security . 2016.11(11):2453–2465 - 8.
Almudhahka N., Nixon M., Hare J. Human face identification via comparative soft biometrics. In: Proceedings of the IEEE International Conference on Identity, Security and Behavior Analysis (ISBA). Sendai, JP, 29 Feb–02 Mar 2016. pp. 1–6. - 9.
Berg T.I., Berg A.C., Edwards J., Forsyth D.A. Who's in the Picture?. In: Proceedings of the Neural Information Processing Systems Conference (NIPS). Springer. 2004. pp. 137–144. - 10.
Torres L. Is there any hope for face recognition? In: Proceedings of the IEEE International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS) . 2004. - 11.
Murphy‐Chutorian E., Trivedi M.M. Head pose estimation in computer vision: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence . 2009. 31(4):607–626. - 12.
Zhang X., Gao, Y. Face recognition across pose: A review. Pattern Recognition . 2009. 42(11):2876–2896. - 13.
Nash S., Rhodes M., Olszewska J. I. iFR: Interactively‐pose‐corrected face recognition. In: Proceedings of the INSTICC International Conference on Bio‐Inspired Systems and Signal Processing (BIOSIGNALS). 2016. pp. 106–112. - 14.
Du S., Ward R. Face recognition under pose variations. Journal of the Franklin Institute. 2006. 343(6):596–613. - 15.
Singh R., Vatsa M., Noore A. Recognizing face images with disguise variations. In: Delac K., Grgic M., Bartlett M. S.; editors. Recent Advances in Face Recognition . Vienna: InTech; 2008. pp. 149–160. - 16.
Prikler F. Evaluation of emotional state of a person based on facial expression. In: Proceedings of the IEEE International Conference on Perspective Technologies and Methods in MEMS Design (MEMSTECH). 2016. pp. 161–163. - 17.
Shreve M., Godavarthy S., Goldgof D., Sarkar, S. Macro‐ and micro‐expression spotting in long videos using spatio‐temporal strain. In: Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition Workshops (AFGR). 2011. pp. 51–56. - 18.
Liu L., Xiong C., Zhang H., Niu Z., Wang M., Yan S. Deep aging face verification with large gaps. IEEE Transactions on Multimedia . 2016. 18 (1):64–75. - 19.
Iqbal M.T.B., Ryu B., Song G., Chae O. Positional Ternary Pattern (PTP): An edge based image descriptor for human age recognition. In: Proceedings of the IEEE International Conference on Consumer Electronics (ICCE). 2016. pp. 289–292. - 20.
Julian P., Dehais C., Lauze F., Charvillat V., Bartoli A., Choukroum A. Automatic hair detection in the wild. In: Proceedings of the IEEE International Conference on Pattern Recognition (ICPR). 2010. pp. 4617–4620. - 21.
Wood R., Olszewska J. I. Lighting‐variable AdaBoost based‐on system for robust face detection. In: Proceedings of the INSTICC International Conference on Bio‐Inspired Systems and Signal Processing (BIOSIGNALS). 2012. pp. 494–497. - 22.
Shan S., Gao W., Cao B., Zhao D. Illumination normalization for robust face recognition against varying lighting conditions. In: Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition (AFGR). 2003. pp. 157–164. - 23.
Abaza A., Harrison M.A., Bourlai T., Ross A. Design and evaluation of photometric image quality measures for effective face recognition. IET Biometrics. 2014. 3(4):314–324. - 24.
ISO/IEC 19794‐5:2011. The Face Image Format Standards. In: Information Technology—Biometric data interchange formats: Part 5: Face image data. 2nd Ed. 2011. - 25.
Prince S.J.D., Elder J., Hou Y., Sizinstev M., Olevskiy E. Towards face recognition at a distance. In: Proceedings of the IET Conference on Crime and Security. 2006. pp. 570–575. - 26.
Kautkar S.N., Atkinson G.A., Smith M.L. Face recognition in 2D and 2.5 D using ridgelets and photometric stereo. Pattern Recognition. 2012. 45(9):3317–3327. - 27.
Mudunuri S.P., Biswas S. Low resolution face recognition across variations in pose and illumination. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2016. 38(5):1034–1040. - 28.
Zou W.W.W., Yuen P.C. Very low resolution face recognition problem. IEEE Transaction on Image Processing . 2012. 21(1):327–340. - 29.
Huang H., He H. Super‐resolution method for face recognition using non‐linear mappings on coherent features. IEEE Transaction on Neural Networks . 2011. 22(1):121–130. - 30.
Li H., Lam K.M. Guided iterative back‐projection scheme for single image super‐resolution. In: Proceedings of the IEEE Global High Tech Congress on Electronics (GHTCE). 2013. pp. 175–180. - 31.
Gross R. Face databases. In: Li S.Z., Jain A.K., editors. Handbook of Face Recognition. Springer‐Verlag. 2005. pp. 301–327. - 32.
Senior A.W, Pankanti S. Privacy protection and face recognition. In: Li S.Z., Jain A.K., editors. Handbook of Face Recognition . 2nd ed. Springer. 2011. pp. II.5–II.21. - 33.
ORL. Face Database. AT&T Laboratories Cambridge [Internet]. 1994. Available from: www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html [Accessed: 2016‐07‐07] - 34.
Caltech Faces. A Public Dataset for Face Recognition. CalTech University. USA [Internet]. 1999. Available from: http://www.vision.caltech.edu/html‐files/archive.html [Accessed: 2016‐07‐07] - 35.
Phillips P.J., Moon H., Rizvi S.A., Rauss P.J. The FERET evaluation methodology for face‐recognition algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence . 2000. 22(10):1090–1104. - 36.
Jesorsky O., Kirchberg K., Frischholz R. Face detection using the Hausdorff distance. In Bigun J., Smeraldi F., editors. Audio and Video based Person Authentication . LNCS Springer. 2001. pp. 90–95. - 37.
Yale Face Database. Yale University. Connecticut, USA [Internet]. 2001. Available from: http://vision.ucsd.edu/yale_face_dataset_original/yalefaces.zip [Accessed: 2016‐07‐07] - 38.
Caltech 10000 Web Faces. Human Faces Collected from Google Image Search. CalTech University. Pasadena, California, USA [Internet]. 2005. Available from: www.vision.caltech.edu/Image_Datasets/Caltect_10K_WebFaces [Accessed: 2016‐07‐07] - 39.
Phillips P.J., Flynn P.J., Scruggs T., Bowyer K.W., Chang J., Hoffman K., Marques J., Min J., Worek W. Overview of the face recognition grand challenge. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2005. pp. I.947–I.954. - 40.
Matas J., Hamouz M., Jonsson K., Kittler J., Li Y., Kotroupolous C., Tefas A., Pitas I., Tan T., Yan H., Smeraldi F., Bigun J., Capdevielle N., Gerstner W., Ben‐Yacoub S., Abduljaoued Y., Mayoraz E. Comparison of face verification results on the XM2VTS database. In: Proceedings of the IEEE International Conference on Pattern Recognition (ICPR). 2000. pp. 858–863. - 41.
Grgic M., Delac K., Grgic S., Klimpak B. SCface ‐ Surveillance cameras face database. Multimedia Tools Applications. 2011. 51:863–879. - 42.
FDDB. A Benchmark for Face Detection in Unconstrained Settings. Jain V., Learned‐Miller E. Technical Report. UM‐CS‐2010‐009. University of Massachusetts, Amherst. USA [Internet]. 2010. Available from: http://vis‐www.cs.umass.edu/fddb/ [Accessed: 2016‐07‐07] - 43.
Learned‐Miller E., Huang G.B., Roy‐Chowdhury A., Li H., Hua G. Labeled Faces in the Wild: A Survey. In: Kawulok M., Emre Celebi M., Smolka B., editors. Advances in Face Detection and Facial Image Analysis . Springer. 2016. pp. 189–248. - 44.
Li X., Pfister T., Huang X., Zhao G., Pietikainen M. A spontaneous micro‐expression database: Inducement, collection and baseline. In: Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition (AFGR). 2013. pp. 1–6. - 45.
Dhall A., Goecke R., Joshi J., Wagner M., Gedeon T. Emotion recognition in the wild challenge 2013. In: Proceedings of the ACM International Conference on Multimodal Interaction (ICMI). 2013. pp. 509–516. - 46.
Panis G., Lanitis A., Tsapatsoulis N., Cootes T.F. Overview of research on facial ageing using the FG‐NET ageing database. IET Biometrics . 2016. 5(2):37–46. - 47.
Viola, P., Jones, M.J. Robust real‐time face detection. International Journal of Computer Vision . 2004. 57(2):137–154. - 48.
Ahonen T., Hadid A., Pietikainen M. Face description with local binary patterns: Application to face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence . 2006. 28(12):2037–2041. - 49.
Jabid T., Kabir M. H., Chae O. Local Directional Pattern (LDP) for Face Recognition . In:Proceedings of the IEEE International Conference on Consumer Electronics (ICCE) . 2010. pp. 329–330. - 50.
Gao Y., Leung M.K.H. Face recognition using line edge map. IEEE Transactions on Pattern Analysis and Machine Intelligence . 2002. 24(6):764–779. - 51.
Olszewska J. I. Multi‐scale, multi‐feature vector flow active contours for automatic multiple‐face detection. In: Proceedings of the INSTICC International Conference on Bio‐Inspired Systems and Signal Processing (BIOSIGNALS) . 2013. pp. 429–435. - 52.
Hu J., Lu J., Tan Y.P. Discriminative deep metric learning for face verification in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2014. pp. 1875–1882. - 53.
Tome P., Fierrez J., Vera‐Rodriguez R., Ramos D. Identification using face regions: Application and assessment in forensic scenarios. Forensic Science International. 2013. 233:75–83. - 54.
Tanaka J.W., Farah M.J. Parts and wholes in face recognition. Quarterly Journal of Experimental Psychology, Section A: Human Experimental Psychology. 1993. 46(2):225–245. - 55.
Guo J.M., Tseng S.H., Wong K.S. Accurate facial landmark extraction. IEEE Signal Processing Letters . 2016. 23(5):605–609. - 56.
Mitchell T. Machine Learning. McGraw Hill. 1997. - 57.
Kumar N., Berg A., Belhumeur P.N., Nayar S.K. Attribute and smile classifiers for face verification. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) . 2009. pp. 365–372. - 58.
Gallego‐Jutgla E., de Ipin K. L., Marti‐Puig P., Sole‐ Casals J. Empirical mode decomposition‐based face recognition system. In: Proceedings of the International Conference on Bio‐Inspired Systems and Signal Processing . 2013. pp. 445–450. - 59.
Brunelli R., Poggio T. Face recognition: Features versus Templates. IEEE Transactions on Pattern Analysis and Machine Intelligence . 1993. 15(10):1042–1052. - 60.
Sun Y., Nasrollahi K., Sun Z., Tan T. Complementary cohort strategy for multimodal face pair matching. IEEE Transactions on Information Forensics and Security . 2016. 11(5):937–950. - 61.
Turk J., Pentland A. Face recognition using eigenfaces. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 1991. pp. 586–591. - 62.
Ruiz‐del‐Solar J., Navarrete P. Eigenspace‐based face recognition: A comparative study of different approaches. IEEE Transactions on Systems, Man and Cybernetics, Part C . 2005. 35(3):315–325. - 63.
Belhumeur P.N., Hespanha J.P., Kriegman D.J. Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence . 1997. 19(7):711–720 - 64.
Cootes T.F., Edwards G.J., Taylor C.J. Active appearance model. IEEE Transactions on Pattern Analysis and Machine Intelligence . 2001. 23(6):681–685. - 65.
Olszewska J. I., De Vleeschouwer C., Macq B. Multi‐Feature Vector Flow for Active Contour Tracking. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . 2008. pp. 721–724. - 66.
Uiboupin T., Rasti P., Anbarjafari G., Demirel H. Facial image super resolution using sparse representation for improving face recognition in surveillance monitoring. In: Proceedings of the IEEE Signal Processing and Communication Application Conference (SIU). 2016. pp. 437–440. - 67.
Aurora Computer Services. Provider of Biometric Solutions [Internet]. 2016. Available from: http://auroracs.co.uk/ [Accessed: 2016‐07‐07] - 68.
Balasubramanian Y., Sivasankaran K., Krishraj S.P. Forensic video solution using facial feature‐based synoptic Video Footage Record. IET Computer Vision. 2016. 10(4):315–320. - 69.
Jain A.K., Klare B., Park U. Face matching and retrieval in Forensics applications. IEEE MultiMedia . 2012. 19(1):20–28. - 70.
Jain A.K., Klare B., Park U. Face recognition: Some challenges. In: Proceedings of the IEEE Conference on Automatic Face and Gesture Recognition (AFGR). 2011. pp. 726–733. - 71.
Baltrusaitis T., Robinson P., Morency L.P. OpenFace: An open source facial behavior analysis toolkit. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV). 2016. pp. 1–10. - 72.
Owayjan M., Kashour A., Al Haddad N., Fadel M., Al Souki G. The design and development of a lie detection system using facial micro‐expressions. In: Proceedings of the IEEE International Conference on Advances in Computational Tools for Engineering Applications (ACTEA) . 2012. pp. 33–38. - 73.
Jang M., Gan Z.H., Gao W.Y. Combining particle filter and active shape models for lip tracking. In: Proceedings of the IEEE World Congress on Intelligent Control and Automation (WCICA). 2006. pp. 9897–9901. - 74.
BioID Web Service (BWS). Biometrics as a Service [Internet]. 2016. Available from: https://playground.bioid.com/ [Accessed: 2016‐07‐07] - 75.
Mastercard App Using Selfies as a Payment Method [Internet]. 2016. Available from: http://newsroom.mastercard.com/videos/mastercard‐identity‐check‐facial‐recognition‐biometrics/ [Accessed: 2016‐07‐07] - 76.
Anthropics Technology. Provider of Face Control Solutions [Internet]. 2016. Available from: http://www.anthropics.com [Accessed: 2016‐07‐07] - 77.
Taigman Y., Yang M., Ranzato M.A., Wolf L. DeepFace: Closing the gap to human‐level performance in face verification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2014. pp. 1701–1708. - 78.
Schroff F., Kalenichenko D., Philbin J. FaceNet: A unified embedding for face recognition and clustering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2015. pp. 815–823. - 79.
Farfade S.S., Saberian M.J., Li L.J. Multi‐view face detection using deep convolutional neural networks. In: Proceedings of the ACM International Conference on Multimedia Retrieval (ICMR). 2015. pp. 643–650. - 80.
Sarkar S., Patel V. M., Chellappa R. Deep feature‐based face detection on mobile devices. In: Proceedings of the IEEE International Conference on Identity, Security and Behavior Analysis (ISBA). 2016. pp. 1–8. - 81.
Temelkov I. Why Facial Recognition will Change Marketing Forever [Internet]. 2016. Available from: http://curatti.com/facial‐recognition/ [Accessed: 2016‐07‐07] - 82.
Snapchat. Image Messaging & Multimedia Mobile Application [Internet]. 2016. Available from: https://www.snapchat.com [Accessed: 2016‐07‐07] - 83.
Everingham M., Sivic J., Zisserman A. “Hello! My name is … Buffy ‐ Automatic naming of characters in TV video”. In: Proceedings of the British Machine Vision Conference (BMVC). 2006. pp. 898–908. - 84.
Tasli H.E., den Uyl T.M., Boujut H., Zaharia T. Real‐time facial character animation. In: Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition (AFGR). 2015. - 85.
Han M.J., Lin C.H., Song K.T. Robotic emotional expression generation based on mood transition and personality model. IEEE Transactions on Cybernetics. 2013. 43(4):1290–1303. - 86.
Double Negative Visual Effects. Provider of Visual Effects for Films [Internet]. 2016. Available from: http://www.dneg.com/ [Accessed: 2016‐07‐07] - 87.
Disney Research. Provider of Advanced Visual Technologies [Internet]. 2016. Available from: https://www.disneyresearch.com/ [Accessed: 2016‐07‐07] - 88.
Parkhi O.M., Vedaldi A., Zisserman A. Deep Face recognition. In: Proceedings of the British Machine Vision Conference (BMVC). 2015. pp. 41.1–41.12. - 89.
Hu G., Yang Y., Yi D., Kittler J., Christmas W., Li S.Z., Hospedales T. When face recognition meets with deep learning: An evaluation of convolutional networks for face recognition. In: Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCV). 2015. - 90.
Sun Y., Wang X., Tang X. Deep learning face representation from predicting 10,000 classes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2014. pp. 1891–1898. - 91.
Sinha P., Balas B., Ostrovsky Y., Russell R. Face recognition by humans: 19 results all computer vision researchers should know about. Proceedings of the IEEE . 2006. 94(11):1948–1962. - 92.
Fung J., Mann S. Using graphics devices in reverse: GPU‐based image processing and compute revision. In: Proceedings of the IEEE International Conference on Multimedia & Expo (ICME) . 2008. pp. 9–12. - 93.
Owens J. D., Luebke D., Govindaraju N., Harris M., Kruger J., Lefohn A. E., Purcell T. J. A survey of general‐purpose computation on graphics hardware. Computer Graphics Forum . 2007. 26(1): 80–113. - 94.
Sagonas C., Panagakis Y., Zafeiriou S., Pantic M. Robust statistical face frontalization. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). 2015. pp. 3871–3879. - 95.
Simonyan K., Parkhi O.M., Vedaldi A., Zisserman A. Fisher vector faces in the wild. In: Proceedings of the British Machine Vision Conference (BMVC). 2013. pp. 8.1–8.12. - 96.
Wolf L., Hassner T., Taigman Y. Effective unconstrained face recognition by combining multiple descriptors and learned background statistics. IEEE Transactions on Pattern Analysis and Machine Intelligence . 2011. 33(10):1978–1990. - 97.
Rouge C., Shaikh S., Olszewska J. I. HD: Efficient hand detection and tracking. In: Proceedings of the IEEE International Conference on Computer Science and Information Science Workshop (FedCSIS) . 2016. - 98.
Prince S. J. D., Aghajanian J. Gender classification in uncontrolled settings using additive logistic models. In: Proceedings of the IEEE International Conference on Image Processing (ICIP). 2009. pp. 2557–2560. - 99.
Zhu X., Lei Z., Yan J., Yi D., Li S.Z. High‐fidelity pose and expression normalization for face recognition in the wild. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2015. pp. 787–796. - 100.
Khorsheed J.A., Yurtkan K. Analysis of Local Binary Patterns for face recognition under varying facial expressions. In: Proceedings of the IEEE Signal Processing and Communication Application Conference (SIU). 2016. pp. 2085–2088. - 101.
Travieso‐Gonzalez C.M., del Pozo‐Banos M., Alonso, J.B. Facial identification based on transform domains for images and videos. In: Riaz Z., editor. Biometric Systems, Design and Applications . InTech. 2011. pp. 293–316. - 102.
Du M., Chellappa R. Video‐based face recognition using the intra/extra‐personal difference dictionary. In: Proceedings of the British Machine Vision Conference (BMVC). 2014. - 103.
Hu X., Wang Y., Zhu F., Pan C. Learning‐based fully 3D face reconstruction from a single image. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2016. pp. 1651–1655.