Open access

Modelling and Simulation for the Recognition of Physiological and Behavioural Traits Through Human Gait and Face Images

Written By

Tilendra Shishir Sinha, Devanshu Chakravarty, Rajkumar Patra and Rohit Raja

Submitted: 25 February 2012 Published: 06 February 2013

DOI: 10.5772/52565

From the Edited Volume

Discrete Wavelet Transforms - A Compendium of New Approaches and Recent Applications

Edited by Awad Kh. Al - Asmari

Chapter metrics overview

1,841 Chapter Downloads

View Full Metrics

1. Introduction

In the present chapter the authors have ventured to explain the process of recognition of physiological and behavioural traits of human-gait and human-face images, where a trait signifies a character on a feature of the human subject. Recognizing physiological and behavioural traits is a knowledge intensive process, which must take into account all variable information of about human gait and human face patterns. Here the trained data consists of a vast corpus of human gait and human face images of subjects of varying ages. Recognition must be done in parallel with both test and trained data sets. The process of recognition of physiological and behavioural traits involves two basic processes: modelling and understanding. Recognition of human-gait images and human-face images has been done separately. Modelling involves formation of a noise-free artificial human gait model (AHGM) of human-gait images and formation of artificial human-face model (AHFM) of human-face images. Understanding involves utilization of the hence formed models for recognition of physiological and behavioural traits. Physiological traits of the subject are the measurement of the physical features of the subject for observation of characteristics. The observable characters may be categorized into four factors: built, height, complexion and hair. Behavioural traits of the subject involve the measurement of the characteristic behaviour of the subject with relevant to four factors: dominance, extroversion, patience and conformity. Recognition in this chapter has been done in two environments: open-air space and clear-under-water space. The current chapter presents a well defined application of high-end computing techniques like soft-computing, utility computing and also some concepts of cloud computing.

Figure 1.

A Schematic diagram for the formation of AHGM

Advertisement

2. Modelling of AHGM and AHFM

2.1. Illustration of an Artificial Human-Gait Model (AHGM)

Human-gait analysis is a systematic study and analysis of human walking. It is used for diagnosis and the planning of treatment in people with medical conditions that affect the way they walk. Biometrics such as automatic face and voice recognition continue to be a subject of great interest. Human-Gait is a new biometric aimed to recognize subjects by the way they walk (Cunado et al. 1997). However, functional assessment, or outcome measurement is one small role that quantitative human-gait analysis can play in the science of rehabilitation. If the expansion on human-gait analysis is made, then ultimate complex relationships between normal and abnormal human-gait can be easily understood (Huang et al. 1999; Huang et al. 1999). The use of quantitative human-gait analysis in the rehabilitation setting has increased only in recent times. Since past five decades, the work has been carried out for human-gait abnormality treatment. Many medical practitioners along with the help of scientists and engineers (Scholhorn et al. 2002) have carried out more experimental work in this area. It has been found from the literature that two major factors: time and effort, play a vital role. In the present chapter, a unique strategy has been adopted for further analysis of human-gait using above two factors for the recognition of physiological and behavioral traits of the subject. Many researchers from engineering field till 1980 have not carried out the work onhuman-gait analysis. In the year 1983, Garrett and Luckwill, carried the work for maintaining the style of walking through electromyography and human-gait analysis. In the year 1984, Berger and his colleagues, detected angle movement and disturbances during walking. In the year 1990, Yang and his colleagues, further carried the experimental work for the detection of short and long steps during walking. In the year 1993 Grabiner and his colleagues investigated that when an obstacle is placed in the path of a subject, how much time is taken by the subject to recover its normal walking after hitting an obstacle. In the year 1994, Eng and his colleagues, with little modifications and detection of angles, have carried out the same work. In the year 1996 again Schillings and his colleagues investigated similar type of work with little bit modifications in the mechanism as adopted by Grabiner and Eng in the year 1993 and 1994 respectively. In the year 1999 Schillings and his colleagues further carried the work that was done in the year 1996 with little modifications. In the year 2001 Smeesters and his colleagues calculated the trip duration and its threshold value by using human-gait analysis. From the literature, it has been also observed that very little amount of work has been carried out using high-end computing approach for the biometrical study through human-gait. The schematic diagram for the formation of knowledge-based model, that is, AHGM has been shown in figure 1.

Figure 1 gives an outline of the process of formation of AHGM. In this process a known human-gait image has to be fed as input. Then it has to be pre-processed for enhancement and segmentation. The enhancement is done for filtering any noise present in the image. Later on it is segmented using connected component method (Yang 1989; Lumia 1983). Discrete Cosine Transform (DCT) is employed for loss-less compression, because it has a strong energy compaction property. Another advantage in using DCT is that it considers real-values and provides better approximation of an image with fewer coefficients. Segmentation is carried out for the detection of the boundaries of the objects present in the image and also used in detecting the connected components between pixels. Hence the Region of Interest (ROI) is detected and the relevant human-gait features are extracted. The relevant features that have to be selected and extracted in the present chapter are based on the physical characteristics of human-gait of the subject. The physical characteristics that must be extracted are: foot-angle, step-length, knee-to-ankle (K-A) distance, foot-length and shank-width. These features are calculated using Euclidean distance measures. The speed of the human-gait can be calculated using Manhattan distance measures. Based on these features relevant parameters have to be extracted. The relevant parameters based on aforesaid geometrical features are: mean, median, standard deviation, range of parameter (lower and upper bound parameter), power spectral density (psd), auto-correlation and discrete wavelet transform (DWT) coefficient, eigen-vector and eigen-value. In this chapter of the book, the above parameters have been experimentally extracted after analyzing 10 frames of human-gait image of 100 different subjects of varying age groups. As the subject walks, the configuration of its motion repeats periodically. For this reason, images in a human-gait sequence tend to be similar to other images in the sequence when separated in time by the period of the human-gait. With a cyclic motion such as a human-gait, the self-similarity image has a repeating texture. The frequency of the human-gait determines the rate at which the texture repeats. Initially the subject is standing at standstill position. During this instance the features that have to be extracted are the foot-length, symmetrical measures of the knee- length, curvature measurement of the shank, maximum-shank-width and minimum-shank-width. Through the measurement of the foot-length of both the legs of the subject, the difference in the length of two feet can be detected. From the symmetrical measurement of the knee-length, the disparity in length of legs, if any, can be measured. Through curvature measurement of the shank, any departure from normal posture can be detected. Measurement of shank-width helps in predicting probable anomalies of the subject and also will show any history of injury or illness in the past. The relevant feature based parameters that have to be extracted are fed as input to an Artificial Neuron (AN) as depicted in figure 2. Each neuron has an input and output characteristics and performs a computation or function of the form, given in equation (1):

Oi= f(Si) and Si= WTXE1

where X = (x1,x2,x3,….,xm) is the vector input to the neuron and W is the weight matrix with wij being the weight (connection strength) of the connection between the jth element of the input vector and ith neuron. WT means the transpose of the weight matrix. The f (.) is an activation or nonlinear function (usually a sigmoid), Oi is the output of the ith neuron and Si is the weighted sum of the inputs.

Figure 2.

An artificial neuron

A single artificial neuron, as shown in figure 2, by itself is not a very useful tool for AHGM formation. The real power comes when a single neuron is combined into a multi-layer structure called artificial neural networks. The neuron has a set of nodes that connect it to the inputs, output or other neurons called synapses. A linear combiner is a function that takes all inputs and produces a single value. Let the input sequence be {X1,X2,…,XN} and the synaptic weight be {W1,W2,W3,….,WN}, so the output of the linear combiner, Y, yields to equation (2),

Y=i=1NXiWiE2

An activation function will take any input from minus infinity to infinity and squeeze it into the range –1 to +1 or between 0 to 1 intervals. Usually an activation function being treated as a sigmoid function that relates as given in equation (3), below:

f(Y)=11+eYE3

The threshold defines the internal activity of the neuron, which is fixed to –1. In general, for the neuron to fire or activate, the sum should be greater than the threshold value.

In the present chapter, feed-forward network has to be used as a topology and back propagation as a learning rule for the formation of corpus or knowledge-based model called AHGM. This model has to be optimized for the best match of features using genetic algorithm. The matching has to be done for the recognition of behavioural features of the subject, not only in open-air-space but also in under-water-space. The reason for adopting genetic algorithm is that, it is the best search algorithm based on the mechanics of natural selection, mutation, crossover, and reproduction. They combine survival of the fittest features with a randomized information exchange. In every generation, new sets of artificial features are created and are then tried for a new measure after best-fit matching. In other words, genetic algorithms are theoretically and computationally simple on fitness values. The crossover operation has to be performed by combining the information of the selected chromosomes (human-gait features) and generates the offspring. The mutation and reproduction operation has to be utilized by modifying the offspring values after selection and crossover for the optimal solution. Here in the present chapter, an AHGM signifies the population of genes or human-gait parameters.

2.1.1. Mathematical formulation for extraction of physiological traits from human-gait

Based on the assumption that the original image is additive with noise. To compute the approximate shape of the wavelet (that is, any real valued function of time, possessing a specific structure), in a noisy image and also to estimate its time of occurrence, two methods are generally used. The first one is simple-structural-analysis method and the second one is the template-matching method.Mathematically, for the detection of wavelets in noisy image, assume a class of wavelets, Si(t), I = 0,...N-1, all possess certain common structural features. Based on this assumption that noise is additive, then the corrupted image has to be modeled by the equation,

X(m,n) = i(m,n) + G d(m,n)E4

where i(m,n) is the clean image, d(m,n) is the noise and G is the term for signal-to-noise ratio control. Next windowing the image and assuming G = 1, equation (4) becomes:

xw(m,n) = iw(m,n) + dw(m,n)E5

Fourier transform of both sides of equation (5), yields:

Xw(ejω1,ejw2) = Iw(ejω1,ejw2) + Dw(ejω1,ejw2)E6

Where Xw(ejω1,ejw2), Iw(ejω1,ejw2) and Dw(ejω1,ejw2) are the Fourier transforms of windowed noisy, original-image and noisy-image respectively.

To de-noise this image, wavelet transform has to be applied. Let the mother wavelet or basic wavelet be ψ(t), which yields to,

y(t) = exp (j2pft – t2/2)E7

Further as per the definition of Continuous Wavelet Transform CWT (a,τ), the relation yields to,

CWT (a,t)=(1/ax(t)y{(t-t)/a} dtE8

The parameters obtained in equation (8) have to be discretized, using Discrete Parameter Wavelet Transform (DPWT).

This DPWT (m, n) is to be obtained by substituting a =a0m, τ = n τ0a0m. Thus equation (8) in discrete form results to equation (9),

DPWT (m, n) = 2m/2klx(k,l)Ψ(2-mk – n)E9

where ‘m’ and ‘n’ are the integers, a0 and τ0 are the sampling intervals for ‘a’ and ‘τ’, x(k,l) is the enhanced image. The wavelet coefficient has to be computed from equation (9) by substituting a0 = 2 and τ0 = 1.

Further the enhanced image has to be sampled at regular time interval ‘T’ to produce a sample sequence {i (mT, nT)}, for m = 0,1,2, M-1 and n=0,1,2,…N-1 of size M x N image. After employing Discrete Fourier Transformation (DFT) method, it yields to the equation of the form,

I(u,v)=m=0M1n=0N1i(m,n)exp(j2π(um/M+vn/N)E10

for u=0,1,2,…,M-1 and v = 0, 1, 2, ……..,N-1

In order to compute the magnitude and power spectrum along with phase-angle, conversion from time-domain to frequency-domain has to be done. Mathematically, this can be formulated as, let R(u,v) and A(u,v) represent the real and imaginary components of I(u,v) respectively.

The Fourier or magnitude spectrum, yields to,

|I(u,v)|=[R2(u,v)+A2(u,v)]1/2E11

The phase-angle of the transform is defined as,

φ(u,v)=tan1[A(u,v)R(u,v)]E12

The power-spectrum is defined as the square of the magnitude spectrum. Thus squaring equation (11) yields to,

P(u,v)=|I(u,v)|2=R2(u,v)+A2(u,v)E13

Due to squaring, the dynamic range of the values in the spectrum becomes very large. Thus to normalize this, logarithmic transformation has to be applied in equation (11). Thus it, yields,

|I(u,v)|normalize=log(1+|I(u,v)|)E14

The expectation value of the enhanced image has to be computed and it yields to the relation as,

E[I(u,v)] =1MNu=0M1v=0N1I(u,v)E15

where ‘E’ denotes expectation. The variance of the enhanced image has to be computed by using the relation given in equation (16),

Var[I(u,v)] = E{[I(u,v) – I’(u,v)]2E16

The auto-covariance of an enhanced image has to be also computed using the relation given in equation (17),

Cxx(u,v) = E{[I(u,v) – I’(u,v)][I(u,v) – I’(u,v)]}E17

Also the powerspectrumdensity has to be computed from equation (17),

PE(f)=m=0M1n=0N1Cxx(m,n)W(m,n)exp(j2πf(m+n))E18

where Cxx(m,n) is the auto-covariance function with ‘m’ and ‘n’ samples and W(m,n) is the Blackman-window function with ‘m’ and ‘n’ samples.

The datacompression has to be performed using Discrete Cosine Transform (DCT). The equation (19) is being used for the data compression.

DCTc(u,v)=m=0M1n=0N1I(m,n)cos(2πT(m+n)MN)E19

Further for the computation of principal components (that is, eigen-values and the corresponding eigen-vectors), a pattern vectorpn¯, which can be represented by another vector qn¯ of lower dimension, has to be formulated using (10) by linear transformation. Thus the resultant yields to equation (20),

pn¯=[M]qn¯E20

where[M]=[I(m,n)] for m= 0 to M-1 and n = 0 to N-1.

andqn¯ = min([M]), such that qn¯> 0

Taking the covariance of equation (20), it yields, the corresponding eigen-vector, given in equation (21),

P¯=cov(pn)¯E21

and thus

P¯.Mi=λi.MiE22

where ‘λi’are the corresponding eigen-values.

Segmentation of an image has to be performed using connected-component method. For mathematical formulation, let ‘pix’ at coordinates (x,y) has two horizontal and two vertical neighbours, whose coordinates are (x+1,y), (x-1,y), (x,y+1) and (x,y-1). This forms a set of 4-neighbors of ‘pix’, denoted as N4(pix). The four diagonal neighbours of ‘pix’ have coordinates (x+1,y+1),(x+1,y-1),(x-1,y+1) and (x-1,y-1), denoted as ND(pix). The union of N4(pix) and ND(pix), yields 8-neighbours of ‘pix’. Thus,

N8(pix) = N4(pix)ND(pix)E23

A path between pixels ‘pix1’ and ‘pixn’ is a sequence of pixels pix1, pix2, pix3,…..,pixn-1,pixn, such that pixk is adjacent to pk+1, for 1 ≤ k < n. Thus connected-component is defined, which has to be obtained from the path defined from a set of pixels and which in return depends upon the adjacency position of the pixel in that path.

From this the speed of walking has to be calculated. Mathematically, it has to be formulated as, let the source be ‘S’ and the destination be ‘D’. Also assume that normally this distance is to achieve in ‘T’ steps. So ‘T’ frames or samples of images are required.

Considering the first frame, with left-foot (FL) at the back and right-foot (FR) at the front, the coordinates with (x,y) for first frame, such that FL(x1,y1) and FR(x2,y2). Thus applying the Manhattan distance measures, the step-length has to be computed as,

|steplength|=|x2x1|+|y2y1|E24

Let normally, Tact steps are required to achieve the destination. From equation (24), T1 has to be calculated for the first frame. Similarly, for ‘nth’ frame, Tn has to be calculated. Thus total steps, calculated are,

Tcalc= T1+ T2+ T3+ ……+ TnE25

Thus walking-speed or walking-rate has to be calculated as,

walkingspeed={norm,ifTact=Tcalcfast,ifTact<Tcalcslow,ifTact>TcalcE26

2.1.2. Mathematical formulation for extraction of behavioral traits from human-gait

Next to compute the net input to the output units, the delta rule for pattern association has to be employed, which yields to the relation,

y-inj=i,j=1nxiwijE27

where ‘y-inj’ is the output pattern for the input pattern ‘xi’ and j = 1 to n.

Thus the weight matrix for the hetero-associative memory neural network has to be calculated from equation (27). For this, the activation of the output units has to be made conditional.

yi={+1ifyinj>00ifyinj=01ifyinj<0E28

The output vector ‘y’ gives the pattern associated with the input vector ‘x’. The other activation function may also be used in the case where the target response of the net is binary. Thus a suitable activation function has been proposed by,

f(x) ={1ifx>00ifx<=0E29

Considering two measures, Accuracy and Precision has been derived to access the performance of the system, which may be formulated as,

Accuracy =CorrectlyRecognizedfeatureTotalnumberoffeaturesE30
Precision =TPRTPR+FPRE31

where TPR = True positive recognition and FPR = False positive recognition.

Further the analysis has to be done for the recognition of behavioral traits with two target classes (normal and abnormal). It can be further illustrated that AHGM has various states, each of which corresponds to a segmental feature vector. In one state, the segmental feature vector is characterized by eleven parameters. Considering only three parameters: the step_length: distance, mean, and the standard deviation, the AHGM is composed of the following parameters

AHGM1={Ds1,μs1,σs1}E32

where AHGM1 means an artificial human-gait model of the first feature vector, Ds1 means the distance, μs1 means the mean and σs1 means the standard deviation based on step_length. Let wnorm and wabnorm be the two target classes representing normal foot and abnormal foot respectively. The clusters of features have been estimated by taking the probability distribution of these features. This has been achieved by employing Bayes decision theory. Let P(wi) be the probabilities of the classes, such that, i = 1,2,….M also let p(β/wi) be the conditional probability density. Assume an unknown gait image represented by the features, β. So, the conditional probability p(wj/β), which belongs to jth class, is given by Bayes rule as,

P(wj/β) =p(β/wj)P(wj)p(β)E33

So, for the class j = 1 to 2, the probability density function p(β), yields,

P(β) =j=12p(β/wj)P(wj)E34

Equation (33) gives a posteriori probability in terms of a priori probability P(wj). Hence it is quite logical to classify the signal, β, as follows,

If P(wnorm/ β) > P(wabnorm/ β), then the decision yields β Є wnorm means ‘normal behaviour’ else the decision yields β Є wabnorm means ‘abnormal behaviour’. If P(wnorm/ β) = P(wabnorm/ β), then it remains undecided or there may be 50% chance of being right decision making. The solution methodology with developed algorithm has been given below for the complete analysis through human-gait, made so far in the present chapter of the book.

Algorithm 1.

NGBBCR {Neuri-Genetic Based Behavioral Characteristics Recognition}

Next for the formation of an artificial human-face model (AHFM) the illustration has been thoroughly done in the next subsequent section of this chapter of the book.

2.2. Illustration of an Artificial Human-Face Model (AHFM)

In the recent times, frontal portion of the human-face images have been used for the biometrical authentication. The present section of the chapter incorporates the frontal-human-face images only for the formation of corpus. But for the recognition of physiological and behavioural traits of the subject (human-being), side-view of the human-face has to be analysed using hybrid approach, which means the combination of artificial neural network (ANN) and genetic algorithm (GA). The work has to be carried out in two stages. In the first stage, formation of the AHFM, as a corpus using frontal-human-face images of the different subjects have to be done. In the second stage, the model or the corpus has to be utilized at the back-end for the recognition of physiological and behavioural traits of the subject. An algorithm has to be developed that performs the above specified objective using neuro-genetic approach. The algorithm will be also helpful for the biometrical authentication. The algorithm has been called as HABBCR (Hybrid Approach Based Behavioural Characteristics Recognition). The recognition process has to be carried out with the help of test image of human-face captured at an angle of ninety-degree, such that the human-face is parallel to the surface of the image. Hence relevant geometrical features with reducing orientation in image from ninety-degree to lower degree with five-degree change have to be matched with the features stored in a database. The classification process of acceptance and rejection has to be done after best-fit matching process. The developed algorithm has to be tested with 100 subjects of varying age groups. The result has been found very satisfactory with the data sets and will be helpful in bridging the gap between computer and authorized subject for more system security. More illustrations through human-face are explained in the next subsection of this chapter of the book.

2.2.1. Mathematical formulation for extraction of physiological traits from human-face

The relevant physiological traits have to be extracted from the frontal-human-face images and the template matching has to be employed for the recognition of behavioural traits of the subject. Little work has been done in the area of human-face recognition by extracting features from the side-view of the human-face. When frontal images are tested for its recognition with minimum orientation in the face or the image boundaries, the performance of the recognition system degrades. In the present chapter, side-view of the face has to be considered with 90-degree orientation. After enhancement and segmentation of the image relevant physiological features have to be extracted. These features have to be matched using an evolutionary algorithm called genetic algorithm. Many researchers like Zhao and Chellappa, in the year 2000, proposed a shape from shading (SFS) method for pre-processing of 2D images. In the same year, that is, 2000, Hu et al. have modified the same work by proposing 3D model approach and creating synthetic images under different poses. In the same year, Lee et al. also has proposed a similar idea and given a method where edge model and colour region are combined for face recognition after synthetic image were created by a deformable 3D model. In the year 2004, Xu et al. proposed a surface based approach that uses Gaussian moments. A new strategy has been proposed (Chua et al. 1997 and Chua et al. 2000), with two zones of the frontal-face. They are forehead portion, nose and eyes portion. In the present work, the training of the system has to be carried out using frontal portion of the face, considering four zones of human-face for the recognition of physiological characteristics or traits or features. They are:

First head portion, second fore-head portion, third eyes and nose portion and fourth mouth and chin portion. From the literature survey it has been observed that still there is a scope in face recognition using ANN and GA (Hybrid approach). For the above discussion the mathematical formulation is same as done for the human-gait analysis.

2.2.2. Mathematical formulation for extraction of behavioral traits from human-face

A path between pixels ‘pix1’ and ‘pixn’ is a sequence of pixels pix1, pix2, pix3,…..,pixn-1,pixn, such that pixk is adjacent to pk+1, for 1 ≤ k < n. Thus connected component is defined, which has to be obtained from the path defined from a set of pixels and which in return depends upon the adjacency position of the pixel in that path. In order to compute the orientation using reducing strategy, phase-angle must be calculated first for an original image. Hence considering equation (12), it yields, to considerable mathematical modelling.

Let Ik be the side-view of an image with orientation ‘k’. If k = 90, then I90 is the image with actual side-view.If the real and imaginary component of this oriented image is Rk and Ak. For k = 90 degree orientation,

|Ik|=[Rk2+Ak2]1/2E35

For k = 900, orientation,

|I90|=[R902+A902]1/52E36

Thus phase angle of image with k = 90 orientations is

ϕk=tan1[AkRk]E37

If k = k-5, (applying reducing strategy), equation (37) yields,

φk5=tan1[Ak5Rk5]k5E38

From equation (37) and (38) there will be lot of variation in the output. Hence it has to be normalized, by imposing logarithmic to both equations (37) and (38)

φnormalize=log(1+(φkφk5))E39

Taking the covariance of (39), it yields to perfect orientation between two side-view of the images, that is, I90 and I85

Iperfectorientation=Cov(φnormalize)E40

The distances between the connected-components have to be computed using Euclidean distance method. A perfect matching has to be done with the corpus with best-fit measures using genetic algorithm. If the matching fails, then the orientation is to be reduced further by 50, that is k = k-5 and the process is repeated till k = 450.

The developed algorithm for the recognition of behavioural traits through human-face has been postulated below.

Algorithm 2.

HABBCR {Hybrid Approach Based Bihevioral Characteristics Recognition}

The understanding of the two formed models with mathematical analysis has been illustrated in the subsequent sections of this chapter of the book.

2.3. Understanding of AHGM and AHFM with mathematical analysis

The recognition of physiological and behavioral traits of the subject (human-being), a test image has to be fed as input. This has been shown in figure 3 below.

Figure 3.

General outline for the understanding of AHGM and AHFM models

From figure 3, first the test image has to be captured and hence to be filtered using DCT after proper conversion of original image to grayscale image. Later on it has to be segmented for further processing and hence to be normalized. Relevant physiological and behavioral features have to be extracted and proper matching has to be done using the developed algorithms named as NGBBCR and HABBCR. In the present chapter two environments: open-air space and under-water space have been considered.

2.3.1. How open-air space environment has been considered for recognition?

Simply a test image in an open-air space (either a human-gait or human-face) has to be captured. It has to be converted into a grayscale image. Later on it has to be filtered using DCT. Hence normalized and then localized for the region of interests (ROI) with object of interests (OOI). Hence a segmentation process has to be completed. Sufficient number of physiological and behavioral traits or features or characteristics has to be extracted from the test image and a template (say OATEMPLATE has to be formed. Using Euclidean distance measures, the differences have to be calculated from that stored in the corpus (AHGM and AHFM). This has to be carried out using lifting-scheme of wavelet transform (LSWT). The details can be explained through the figure 4, as shown below.

Figure 4.

Lifting-scheme of wavelet transforms

Figure 4, shows that the enhanced and segmented part of the test image, x(m, n) has to be split into two components: detail component and coarser component. Considering both the obtained components, additional parameters have to be computed using mathematical approximations. These parameters (referred as prediction (P) and updation(U) coefficients) have to be used for an optimal and robust recognition process.

With reference to figure 4, first the enhanced and segmented part of the test image x(m, n), has to be separated into disjoint subsets f sample, say xe(m, n) and xo(m, n). From these the detail value, d(m, n), has to be generated along with a prediction operator, P. Similarly, the coarser value, c(m, n), has also to be generated along with an updation operator, U, which has to be multiplied with the detail signal and added with the even components of the enhanced and segmented part of the test image. These lifting-scheme parameters have to be computed using Polar method or Box-Muller method.

Let us assume X(m, n) be the digitized speech signal after enhancement. Split this speech signal into two disjoint subsets of samples. Thus dividing the signal into even and odd component: Xe(m, n) and Xo(m, n) respectively. It simplifies to Xe(m, n) = X(2m, 2n) and Xo(m, n) = X(2m+1, 2n+1). From this simplification two new values have to be generated called detail value d(m, n) and coarser value c(m, n).

The detail value d(m, n) has to be generated using the prediction operator P, as depicted in figure 4. Thus it yields,

d(m, n) = Xo(m, n) – P (Xe(m, n))E41

Similarly, the coarser value c(m, n) has to be generated using the updation operator U and hence applied to d(m, n) and adding the result to Xe(m, n), it yields,

c(m, n) = Xe(m, n) + U (d(m, n))E42

After substituting equation (41) in equation (42), it yields,

c(m, n) = Xe(m, n) + U Xo(m, n) – UP (Xe(m, n))E43

The lifting – scheme parameters, P and U, has to be computed initially using simple iterative method of numerical computation, but it took lot of time to display the result. Hence to overcome such difficulty polar method or Box-Muller method has to be applied. The algorithm 3 has been depicted below for such computations.

Algorithm 3.

Computation of U and P values

As per the polar or Box-Muller method, rescaling of the un-standardized random variables can be standardized using the mean and variance of the test image. This can be more clearly expressed as follows: let Q be an un-standardized variable or parameter and Q’ be its standardized form. Thus Q’ = (Q – μ)/σ relation rescales the un-standardized parameters to standardized form Q = σ Q’ + μ, where μ is the mean and σ is the standard deviation.

Further the computed values of U and P has to be utilized for the application of inverse mechanism of lifting scheme of wavelet transform. Further for crosschecking the computed values of the lifting scheme parameters, inverse mechanism of lifting-scheme of wavelet transform (IALS-WT) has to be employed, that has been depicted in figure 5.

Figure 5.

Inverse-lifting-scheme of wavelet transforms

With reference to figure 5, the coarser and detail values have to be utilized along with the values of lifting scheme parameters. Later the resultant form has to be merged and the output, y(m, n) has to be obtained. Further a comparison has to be made with the output y(m, n) and the enhanced and segmented part of the image x(m, n). If this gets unmatched then using feedback mechanism the lifting – scheme parameters have to be calibrated and again the whole scenario has to be repeated till the values of x(m, n) and y(m, n) are matched or close-to-matching scores are generated.

From figure 5, further analysis has to be carried out by merging or adding the two signals c”(m, n) and d”(m, n) for the output signal Y(m, n). Thus it yields,

Y(m, n) = c”(m, n) + d”(m, n)E44

where c”(m, n) is the inverse of the coarser value c(m, n), and d”(m, n) is the inverse of the detail value d(m, n).From figure 5, it yields:

c”(m, n) = c(m, n) – U d(m, n)E45

and

d”(m, n) = d(m, n) + P c”(m, n)E46

On substituting equation (45) in equation (46), it gives:

d”(m, n) = (1 – UP)d(m, n) + P c(m, n)E47

On adding equations (46) and (47), it yields,

Y(m, n) = (1 + P) c(m, n) + (1 – U – UP) d(m, n)E48

To form a robust pattern matching model, assume the inputs to the node are the values x1, x2,…,xn, which typically takes the values of –1, 0, 1 or real values within the range (-1, 1). The weights w1, w2,…,wn, correspond to the synaptic strengths of the neuron. They serve to increase or decrease the effects of the corresponding ‘xi’ input values. The sum of the products xi * wi, i = 1 to n, serve as the total combined input to the node. So to perform the computation of the weights, assume the training input vector be ‘Gi’ and the testing vector be ‘Hi’ for i = 1 to n. The weights of the network have to be re-calculated iteratively comparing both the training and testing data sets so that the error is minimized.

If there results to zero errors a robust pattern matching model is formed. The process for the formation of this model is given in algorithm-4.

Algorithm 4.

Robust pattern matching model

Depending upon the distances, the best test-matching scores are mapped using unidirectional-temporary associated memory of artificial neural network (UTAM). The term unidirectional has to be used because each input component is mapped with the output component forming one-to-one relationship. Each component has to be designated with a unique codeword. The set of codewords is called a codebook. The concept of UTAM has to be employed in the present work, as mapping-function for two different cases:

  1. Distortion measure between unknown and known images

  2. Locating codeword between unknown and known image feature

To illustrate these cases mathematically, Let Kin = {I1,I2,….,In} and Kout = {O1,O2,……,Om} consisting of ‘n’ and ‘m’ input and output codeword respectively. The values of ‘n’ and ‘m’ are the maximum size of the corpus. In the recognition stage, a test image, represented by a sequence of feature vector, U = {U1,U2,….,Uu}, has to be compared with a trained image stored in the form of model (AHGM and AHFM), represented by a sequence of feature vector, Kdatabase = {K1,K2,….,Kq}. Hence to satisfy the unidirectional associatively condition, that is, Kout = Kin, AHGM and AHFM has to be utilized for proper matching of features. The matching of features, have to be performed on computing the distortion measure. The value with lowest distortion has to be chosen. This yield to, the relation,

Cfound=argmin1<=q<=n{S(Uu,Kq)}E49

The distortion measure has to be computed by taking the average of the Euclidean distance

S(U,Ki)=1Qi=1Qd(ui,Cmini,q)E50

where Cmini,q denotes the nearest value in the template or AHGM or AHFM and d(.) is the Euclidean distance. Thus, each feature vector in the sequence ‘U’ has to be compared with the codeword in AHGM and AHFM, and the minimum average distance has to be chosen as the best-match codeword. If the unknown vector is far from the other vectors, then it is very difficult to find the codeword from the AHGM and AHFM, resulting to out-of-corpus (OOC) problem. Assigning weights to all the codeword’s in the database (called weighting method) has eliminated the OOC problem. So instead of using a distortion measure a similarity measure that should be maximized are considered. Thus it yields,

Sw(U,Ki)=1Qi=1Q1d(ui,Cmini,q)w(Cmini,q)E51

Dividing equation (50) by equation (51), it yields,

γ= recognition rate =S(U,Ki)Sw(U,Ki)=unweightedweightedE52

The procedure for computing the weights, has been depicted in an algorithm – 5 below:

Algorithm 5.

Procedure to compute weight (S)

Next for locating the codeword, hybrid approach of soft computing has to be applied in the well-defined way in the present chapter.The hybrid approach of soft computing techniques utilizes some bit of concepts from forward-backward dynamic programming and some bit of neural-networks. From the literature it has been observed that, for an optimal solution, genetic algorithm is the best search algorithm based on the mechanics of natural selection, crossover, mutation and reproduction. It combines survival of the fittest among string structures with a structured yet randomized information exchange. In every generation, new sets of artificial strings are created and hence tried for a new measure. It efficiently exploits historical information to speculate on new search points with expected improved performance. In other words genetic algorithms are theoretically and computationally simple and thus provide robust and optimized search methods in complex spaces. The selection operation has to be performed by selecting the physiological and behavioral features of the human-gait and face images, as chromosomes from the population with respect to some probability distribution based on fitness values. The crossover operation has to be performed by combining the information of the selected chromosomes (human-gait and human-face image) and generates the offspring. The mutation operation has to be utilized by modifying the offspring values after selection and crossover for the optimal solution. Here in the present chapter, a robust pattern matching model signifies the population of genes or physiological and behavioral features. Using neuro-genetic approach a similar type of work has been done by TilendraShishir Sinha et al. (2010) for the recognition of anomalous in foot using a proposed algorithm NGBAFR (neuro-genetic based abnormal foot recognition). The methodology adopted was different in classification and recognition process and the work has been further modified by them and has been highlighted in the present part of the book using soft computing techniques of genetic and artificial neural network. Hence the classification and decision process are to be carried as per the algorithm discussed earlier in this chapter of the book.

2.3.2. How an underwater space environment has to be considered for recognition?

Simply a test image of a subject (either walking or swimming in water), has to be captured, keeping the camera in an open-air space at a ninety-degree angle to the surface of the water. It has to be converted into a grayscale image. Later on it has to be filtered using DCT. Hence normalized and then localized for the region of interests (ROI) with object of interests (OOI). Later on segmentation process has to be done using structural analysis method. Sufficient number of physiological and behavioral traits or features or characteristics has to be extracted from the test image and a template (say UWTEMPLATE) has to be formed. Using Euclidean distance measures, the differences between the test-data-set and the trained-data-set has to be calculated. Depending upon the distances obtained, the best test-matching scores are generated using genetic algorithm for an optimal classification process and finally the decision process are to be carried out.

2.4. Experimental results and discussions through case studies

In the present chapter, human-gait and human-face have been captured in an open-air space. The testing of the physiological and behavioural characteristics or features or traits of the subject is not only done in an open-air space but also in under-water space.

2.4.1. Case study: In an open-air space

In an open-air space, first a human-gait image has to be captured and fed as input. Next it has to be enhanced. Later on it is compressed for distortion removal with loss less information. Next it has to be segmented for contour detection and the relevant physiological features have to be extracted. All the features of the human-gait image are stored in a corpus called AHGM. Similarly, in an open-air space, human-face image has also to be captured and fed as input. Next it is enhanced, compressed, segmented and relevant physiological features are extracted. The extracted physiological features are stored in a corpus called AHFM. For the recognition of physiological and behavioural traits, test images of human-gait and human-face (from side-view) are fed as input. Both the images are enhanced and compressed for distortion removal. Then both are segmented for the extraction of relevant physiological and behavioural features. By using the computer algorithm’s(depicted in algorithm-2 and algorithm-3) the extracted features have to be compared with that stored in the corpus (AHGM and AHFM). Depending upon the result of comparison the classification has to be made. Relevant testing with necessary test data considering 10 frames of 100 different subjects of varying ages proves the developed algorithm. The efficiency of recognizing the physiological and behavioural traits have been kept at a threshold range of 90% to 100% and verified from the trained-data-set. Using genetic algorithm the best-fit scores have been achieved. Figure 6, shows the original image of one subject along with the segmented portion of the human-gait in standing mode.

Figure 6.

Segmented image in standing mode of human-gait

Figure 7.

a) Original Gait Image (b) ROI Foot Image (c) ROI Shank Image (d) ROI Leg Image (e) ROI Swing Foot Image

Figure 8.

Physiological feature of human-gait in walking mode of subject #1 with right leg at the front

After segmentation of human-gait image in walking mode, extraction of physiological features using relevant mathematical analysis has to be done. Some of the distance measures of subject #1 with right leg at the front have been shown in figure 8. Similarly, distance measures of subject #1 with left leg at the front have been shown in figure 9.

Figure 9.

Physiological feature of human-gait in walking mode of subject #1 with left leg at the front

Figure 10.

Step-length and Knee-to-ankle measure in walking mode of the subject

The relevant physiological feature, that is, step-length and knee-to-ankle distance has been also extracted that has been shown in figure 11.

As per the developed algorithm called NGBBCR, for most of the test cases, ‘NORMAL BEHAVIOUR’ has been achieved. Very few test cases for ‘ABNORMAL BEHAVIOUR’ have been achieved. Table 1, depicted below describes the physiological features extracted in standing and walking mode of subject #1.

Table 1.

Physiological features extracted in standing and walking mode of subject #1

From table 1, it has been observed that minimum variations have been found from one frame to other. This has been plotted in figure 11, below, for the graphical analysis. The extracted parameters with respect to physiological features that have been verified for best-fit scores using NGBBCR has been shown in table 2a and in table 2b. The graphical representation of table 2a and table 2b has been depicted in figure 12.

Figure 11.

Graphical representation of physiological features extracted in standing and walking mode of subject #1

2a
Image Files Gait Characteristics Mean LB UB SD Auto Corr.
IMG1 Standing
(Left Leg facing towards Camera)
10.152 -269.408 2620.94 156.513 30050
IMG2 Standing
(Right Leg facing towards Camera)
10.2679 -242.651 2584.98 156.12 28665.2
IMG3 Walking
(Left Leg Movement)
9.10686 -290.723 2637.0 151.059 36575.5
IMG4 Walking
(Right Leg Movement)
9.04764 -430.713 2548.37 148.452 41360
IMG5 Walking
(Left Leg Movement)
9.2831 -412.108 2658.3 152.113 43500.5
IMG6 Walking
(Right Leg Movement)
9.07875 -365.896 2650.96 150.842 41360
IMG7 Walking
(Left Leg Movement)
9.67294 -384.685 2544.82 149.573 36796.7
IMG8 Walking
(Right Leg Movement)
9.67004 -376.443 2612.81 152.036 39045
IMG9 Walking
(Left Leg Movement)
9.83117 -423.315 2702.1 155.715 41125.5
IMG10 Walking
(Right Leg Movement)
9.8643 -349.463 2486.73 147.951 35262.5
2b
Image Files Gait Characteristics Psd Approx. Coeff. Detail Coeff. Eigen Vector Eigen Value
IMG1 Standing
(Left Leg facing towards Camera)
6.1983e+008 28.974 -3.99699 0.000427322 130.005
IMG2 Standing
(Right Leg facing towards Camera)
6.1983e+008 15.9459 9.62817 0.00041897 127.639
IMG3 Walking
(Left Leg Movement)
6.1983e+008 14.5305 3.19103 0.000178086 69.244
IMG4 Walking
(Right Leg Movement)
6.1983e+008 25.63 -8.18713 0.000171731 62.8614
IMG5 Walking
(Left Leg Movement)
6.1983e+008 14.3655 3.43852 0.000154456 64.3655
IMG6 Walking
(Right Leg Movement)
6.1983e+008 24.8326 -8.01451 0.000174672 64.7545
IMG7 Walking
(Left Leg Movement)
6.1983e+008 27.8642 -9.02633 0.000184038 67.9991
IMG8 Walking
(Right Leg Movement)
6.1983e+008 26.4634 -8.32745 0.000172107 67.8427
IMG9 Walking
(Left Leg Movement)
6.1983e+008 14.376 3.38278 0.000160546 69.2694
IMG10 Walking
(Right Leg Movement)
6.1983e+008 13.9372 3.50317 0.000197328 67.6806

Table 2.

2a. The extracted parameters with respect to gait features of subject # 1; 2b. The extracted parameters with respect to gait features of subject # 1

Figure 12.

Graphical representation of physiological features of subject #1

From figure 12, it has been observed that the energy values (on Y-axis) are lying in between -10 to +15, for all the parameters. The parameters power spectral density (psd) and standard deviation (SD) have been found constant for any frame of the subject. The eigenvector and eigenvalues are also satisfying their mathematical properties. For the rest of the extracted parameters in the work, minimum variations have been observed.

Next the frontal part of human-face has been captured, with four zones that have been depicted in figure 13, below.

Figure 13.

a) Original image (b) ROI Eye Image (c) ROI Nose Image(d) ROI Forehead Image (e) ROI Lips Image (f) ROI Chin Image

The relevant physiological features measured from side-view have been shown in table 3 below.

Table 3.

Physiological features of human-face from side-view of subject #1

2.4.2. Case study: In under-water space

In under-water space, first a human-gait image has to be captured and fed as input. Next it has to be enhanced. Later on it is compressed for distortion removal with loss less information. Next it has to be segmented for contour detection and the relevant physiological features have to be extracted. All the features of the human-gait image are stored in a corpus called UWHGM (under water human-gait model). Similarly, in under-water space, human-face image has also to be captured and fed as input. Next it is enhanced, compressed, segmented and relevant physiological features are extracted. The extracted physiological features are stored in a corpus called UWHFM (under water human-face model). For the recognition of physiological and behavioural traits, test images of human-gait and human-face are fed as input. Both the images are enhanced and compressed for distortion removal. Then both are segmented for the extraction of relevant physiological and behavioural features. By using the computer algorithm’s(depicted in algorithm-2 and algorithm-3) the extracted features have to be compared with that stored in the corpus (AHGM and AHFM). Depending upon the result of comparison the classification has to be made. Relevant testing with necessary test data considering 10 frames of 100 different subjects of varying ages proves the developed algorithm. The efficiency of recognizing the physiological and behavioural traits have been kept at a threshold range of 90% to 100% and verified from the trained-data-set. Using genetic algorithm the best-fit scores have been achieved. Figure 14, shows the original image of one subject along with the segmented portion of the human-gait in walking-mode in under-water space.

Figure 14.

Testing of under-water human-gait image for subject #1

Advertisement

3. Conclusion and further scope

This chapter includes in-depth discussion of an algorithm developed for the formation of a noise-free AHGM and AHFM using relevant physiological and behavioural features or traits or characteristics of the subject using human-gait and human-face image. The algorithm has been named NGBBCR and HABBCR. It may be noted that the algorithms have been tested on a vast amount of data and have been found subject independent and environment independent. The algorithms have been tested not only in an open-air space but also in under-water space. A thorough case study has been also done. The trained-data set is matched with the test-data set for the best fit and this involves the application of artificial neural network, fuzzy set rules and genetic algorithm (GA). At every step in the chapter thorough mathematical formulations and derivations have been explained.

Human-gait analysis may be of immense use in medical field for recognition of anomalies and also to track the prosodic features like mood, age, gender of the subject. It may also be used to track the recovery of a patient from injury and operation. Further it will prove to be handy to spot and track individuals in a crowd and hence help investigation department.

Human-face analysis also will help in medical field to treat various diseases like squint, facial distortions and other problems which show their symptoms through facial anomalies. It will be of great help for plastic surgeons to rectify features and enhance the beauty of the subjects.

Underwater object recognition itself is a vast and challenging area and is proving to be of great help in fields of environmental-biology, geology, defence, oceanography and agriculture. Understanding the life and possible dangers of deep water animals, tracking submarines and destructive materials like explosives and harmful wastes are some areas of interest in this field.

References

  1. 1. CunadoD.NixonM. S.CarterJ. N.1997Using gait as a biometric, via phase-weighted magnitude spectrain the proceedings of First International Conference, AVBpA’97, Crans-Montana, Switzerland, 95102
  2. 2. HuangP. S.HarrisC. J.NixonM.M.S.,1999Recognizing humans by gait via parametric cannonical space, Artificial Intelligence in Engineering, 134359366
  3. 3. HuangP. S.HarrisC. J.NixonM. S.1999Human gait recognition in canonical space using temporal templatesIEEE Proceedings Vision Image and Signal Processing, 146293100
  4. 4. ScholhornW. I.NiggB. M.StephanshynD. J.LiuW.2002Identification of individual walking patterns using time discrete and time continuous data sets, Gait and Posture, 15180186
  5. 5. GarrettM.LuckwillE. G.1983Role of reflex responses of knee musculature during the swing phase of walking in man.European Journal Application Physical Occup Physiology, 5213641
  6. 6. BergerW.DietzV.QuinternJ.1984Corrective reactions to stumbling in man: neuronal co-ordination of bilateral leg muscle activity during gait,The Journal of Physiology357109125
  7. 7. YangJ. F.WinterD. A.WellsR. P.1990Postural dynamics of walking in humans., Biological Cybernetics, 624321330
  8. 8. GrabinerM. D.DavisB. L.1993Footwear and balance in older men.Journal of the American Geriatrics Society41910111012
  9. 9. EngJ. J.WinterD. A.PatlaA. E.1994Strategies for recovery from a trip in early and late swing during human walking.Experimentation Cerebrale, 2339349
  10. 10. SchillingsA. M.Van WezelB. M.DuysensJ.1996Mechanically induced stumbling during human treadmill walking.Journal of Neuro Science Methods, 6711117
  11. 11. SchillingsA. M.van WezelB. M.MulderT.DuysensJ.1999Widespread short-latency stretch reflexes and their modulation during stumbling over obstacles.Brain Research8162480486
  12. 12. SmeestersC.HayesW. C.Mc MahonT. A.2001The threshold trip duration for which recovery is no longer possible is associated with strength and reaction time,Journal of Biomechanics345589595
  13. 13. YangX. D.1989An improved Algorithm for Labeling connected Components in a Binary ImageTR 89981
  14. 14. LumiaR.1983A New Three-dimensional connected components AlgorithmComputer VisionGraphics, and Image Processing, 23207217
  15. 15. HarrisR. I.BeathT.1948Etiology of Personal Spatic Flat Foot, The Journal of Bone and Joint Surgery, 30B4624634
  16. 16. KoverT.VighD.VamossyZ.2006MYRA- Face Detection and Face Recognition system, in the proceedings of Fourth International Symposium on Applied Machine Intelligence, SAMI 2006, Herlany, Slovakia, 255265
  17. 17. Rein-LienHsu.Mohammad-MottalebAbdel.AnilK.Jain, 2002Face detection in color imagesin IEEE transactions of Pattern Analysis and Machine Intelligence, 245696706
  18. 18. ZhangJ.YanY.LadesM.1991997Face recognition: eigenface, elastic matching and neural nets, in the proceedings of IEEE, 85914931435
  19. 19. TurkM. A.PentlandA. P. 1991, Face recognition using eigenfaces, in the proceedings of IEEE Computer society conference on computer vision and pattern recognition, 586591 .
  20. 20. ZhaoW. Y.ChellapaR.2000SFS Based View Synthesis for Robust Face Recognition in the proceedings of IEEE international Automatic Face and Gesture recognition.
  21. 21. Hu., Y., Jaing, D., Yan, S., Zhang, L., and Zhang, H., Automatic 3d reconstruction for face recognition, in the proceedings of IEEE International Conferences on Automatic Face and Gesture Recognition 2000.
  22. 22. LeeC. H.ParkS. W.ChangW.ParkJ. W.Improving the performance of Multiclass SVMs in face recognition with nearest neighbours rule in the proceedings of IEEE International conference on tools with Artificial Intelligence 2000
  23. 23. Xu, C., Wang, Y., Tan, T., Quan, L., Automatic 3D Face recognition combining global geometric features with local shape variation information, in the proceedings of IEEE International Conference for Automatic Face and Gesture Recognition 2004.
  24. 24. Chua, C.S., Jarvia, R., Point Signature : A new Representation for 3D Object Recognition, International Journal on Computer Vision vol 25, 1997.
  25. 25. Chua, C.S., Han, F, Ho, Y.K., 3D Human face recognition using point signature, in the proceedings of IEEE International Conference on Automatic Face and Gesture Recognition 2000.
  26. 26. SinhaTilendra.ShishirPatra.RajkumarRajaRohit.2011A Comprehensive analysis for abnormal foot recognition of human-gait using neuro-genetic approachInternational Journal of Tomography and Statistics16W115673

Written By

Tilendra Shishir Sinha, Devanshu Chakravarty, Rajkumar Patra and Rohit Raja

Submitted: 25 February 2012 Published: 06 February 2013