InTech uses cookies to offer you the best online experience. By continuing to use our site, you agree to our Privacy Policy.

Computer and Information Science » Human-Computer Interaction » "Augmented Reality", book edited by Soha Maad, ISBN 978-953-7619-69-5, Published: January 1, 2010 under CC BY-NC-SA 3.0 license. © The Author(s).

Chapter 9

AAM and Non-rigid Registration in Augmented Reality

By Yuan Tian, Tao Guan and Cheng Wang
DOI: 10.5772/7132

Article top

AAM and Non-rigid Registrationin Augmented Reality

Yuan Tian1, Tao Guan1 and Cheng Wang1

1. Introduction

Augmented Reality copes with the problem of dynamically augmenting or enhancing the real world with computer generated virtual objects [Azuma, 1997; Azuma, 2001]. Registration is one of the most pivotal problems in augmented reality applications. Typical augmented reality applications track 2D patterns on rigid planar objects in order to acquire the pose of the camera in the scene. Although the problem of rigid registration has been widely studied [Yuan et al., 2005; Yuan et al., 2006; Guan et al., 2008a; Guan et al., 2008b; Li et al., 2008], non-rigid registration is recently receiving more and more attention. There are many non-rigid objects existing in the real world such as animated faces, deformable cloth, hand and so forth. How to overlaid virtual objects on the non-rigid objects is particular challenging.

Recently, many related non-rigid registration approaches have been reported. In many cases (e.g. human faces), only a few feature points can be reliably tracked. In [Bartoli et al., 2004], a non-rigid registration method using point and curve correspondences was proposed to solve this problem. They introduced curves into the non-rigid factorization framework because there are several curves (e.g. the hairline, the eyebows) can be used to determine the mapping. The mapping function is computed from both point and curve correspondences. This method can successfully augment the non-rigid object with a virtual object. In [Pilet et al., 2005; Pilet et al., 2007], they presented a real-time method for detecting deformable surfaces with no need whatsoever for a prior pose knowledge. The deformable 2D meshes are introduced. With the use of fast wide baseline matching algorithm, they can superimpose an appropriately deformed logo on the T-shirt. These methods are robust to large deformations, lighting changes, motion blur and occlusions. To align the virtual objects generated by computers with the real world seamlessly, the accurate registration data should be provided. In general, registration can be achieved by solving a point matching problem. The problem is to find the correspondence between two sets of tracked feature points. Therefore, the detection of feature points and the points tracking are the two main problems. Rigid object detection and tracking have been extensively studied and effective, robust, and real-time solutions proposed [Lowe, 2004; Lepetit & Fua, 2005; Lepetit et al., 2005; Rosten & Drummond, 2005]. Non-rigid object detection and tracking is far more complex because the object is deformable and not only the registration data but also a large number of deformation parameters must be estimated.

Active Appearance Models (AAM), introduced a few years ago [Cootes et al., 1998; Cootes et al., 2001], are commonly used to track non-rigid objects, such as faces and hands. There are many methods have been proposed to track non-rigid objects using AAM. A working system for finding and tracking a human face and its features using active appearance models was presented in [Ahlberg, 2001]. A wireframe model is adapted to the face in each image. Then the model is matched to the face in the input image using active appearance algorithm. In [Sung & Kim, 2004], the previous 2D AAM is extended to 3D shape model and modified model fitting algorithm was proposed. In [Markin & Prakash, 2006], occluded images are included into AAM training data to solve the occlusion and self-occlusion problem. This approach can improve the fitting quality of the algorithm.

With known coordinates of 3D points in the world coordinates and the corresponding 2D image points, the camera pose can be estimated. For non-rigid objects, AAM algorithm is a robust method to acquire the 2D image points. It has been proven to be a useful method for matching any of the statistical models to a new image rapidly. The 3D points of the non-rigid objects can be represented by a linear combination of a set of 3D basis shapes. By varying the configuration weights and camera pose parameters, the error between the estimated 2D points (projected by the estimated 3D shapes using estimated camera pose parameters) and 2D tracking points can be minimized.

Many methods have been proposed to recover 3D shape basis from 2D image sequences. In [Tomasi & Kanade, 1992], the factorization method is used to recover shape and motion from a sequence of images under orthographic projection. The image sequence is represented as a measurement matrix. It is proved that under orthography, the measurement matrix is of rank 3 and can be factored into 3D pose and 3D shape matrix. Unfortunately this technique can not be applied to non-rigid deforming objects, since they are based on the rigidity assumption. The technique based on a non-rigid model is proposed to recover 3D non-rigid shape models under scaled orthographic projection [Bregler et al., 2000]. The 3D shape in each frame can be expressed by a linear combination of a set of K basis shapes. Under this model, the 2D tracking matrix is of rank 3K and can be factored into 3D pose, object configuration and 3D basis shapes with the use of SVD.

In this chapter, a novel non-rigid registration method for augmented reality applications with the use of AAM and factorization method is introduced. We focus on AAM algorithm and factorization method which can obtain the 3D shape basis, object configuration and 3D pose simultaneously. The following demonstrations are mainly based on the researches presented in [Tomasi & Kanade, 1992; Cootes et al., 1998; Bregler et al., 2000; Cootes et al., 2001; Xiao et al., 2004; Zhu et al., 2006; Tian et al., 2008]. In section 2, we will have a detailed review of active appearance models and focus on the way of how to track non-rigid objects. In section 3, we will illustrate how to compute the 3D shape basis, the camera rotation matrix and configuration weights of each training frame simultaneously from the 2D tracking data using factorization algorithm. In section 4, we will introduce how to compute the precise configuration weights and camera rotation matrix by optimization method and the experimental results are also given.

2. Tracking non-rigid objects using AAM

Tracking non-rigid objects using AAM includes five main steps. The first step is to obtain landmarks in training image set. Then establish the shape model and texture model separately. These two models are unified into one appearance model in the next step. Finally the difference between the closest synthesized AAM model and input image is minimized to get the accurate tracking result. The flowchart is shown in Figure 1.

media/image1.png

Figure 1.

The flowchart of tracking non-rigid objects using AAM.

2.1. Obtain landmarks

media/image2.png

Figure 2.

Examples of traning images manually labeled with consistent landmarks.

Before establishing the models, we should place hundreds of points in each 2D training image. The landmarks in each image should be consistent. We usually choose the points of high curve or junctions as landmark points which will control the shape of the target object strictly. The acquisition process is cumbersome especially when it is done manually. Researchers have looked for different ways to reduce the burden. Ideally, one only need to place points in one image and the corresponding points in the remain traning images can be found automatically. Obviously, this is impossible. However many semi-automatic methods have been proposed and can successfully find the correspondences accross an image set. Here we focus on the tracking procedure, more details about semi-autometic placement of landmarks can be found in [Sclaroff & Pentland, 1995; Duta et al., 1999; Walker et al., 2000; Andresen & Nielsen, 2001]. The examples of training images manually labeled with consistent landmarks are shown in Figure 2. We use 7 training images taken from different viewpoints. Each image is labeled with 36 landmarks shown as the “ × ” in the figures.

Given a training image set with key landmark points are marked on each example object, we can establish the shape and texture variations. These approaches will be detailedly illustrated in the following sections.

2.2. Establish shape model

Suppose L denotes the number of shapes in a training image set and m is the number of key landmark points on each image. The vector representation for each shape would then be:

Xi=[x1,x2,,xm,y1,y2,,ym]T,   i=1,2,,L
(1)

Then the training image set Ω=(X1,X2,,XL) .

The shape and pose (include position, scale and rotation) of the object in each training image is different, so the shapes should be aligned to filter out the pose effects. We will first explain how to align two shapes and then extend it to a set of shapes.

If X and X’ are two shape vectors, the goal is to minimize the square sum of corresponding landmarks after alignment using similarity transformation technique. That is to minimize E:

E=|T(X)X|2
(2)

where

T(xy)=(uvvu)(xy)+(txty)
(3)

Choose three kinds of transformation factors: scale factor-s, rotation factor-θ and translation factor-t. And u=scosθ , v=ssinθ . Hence s2=u2+v2 θ=tan1(v/u) . Then equation (2) can be rewritten as:

E(u,v,tx,ty)=|T(X)X|2=i=1m(uxivyi+txxi)2+(vxi+uyi+tyyi)2
(4)

To solve the minimum value of equation (4) is equivalent to let the partial derivative equals to zero. Then the transformation parameters are:

u=(xx)|x|2,v=i=1m(xiyiyixi)|x|2,tx=1mi=1mxi,ty=1mi=1myi
(5)

The alignment of a set of shapes can be processed by iterative approach suggested by [Bookstein, 1996]. The detailed process is shown in Table 1.

So the new training image set Ω^=(X^1,X^2,,X^L) . The mean shape is calculated by:

Step 1: Choose one shape as the initial estimate of the mean shape.
Step 2: Scale the mean shape so that|X¯0|=1.
Step 3: Align all the remaining shapes to the mean shape using the method described above.
Step 4: Re-estimate the mean shape from aligned shapes.
Step 5: Scale the new mean shape so that|X¯new|=1.
Step 6: If the new calculated mean shape doesn’t change significantly, convergence is declared; else return to step 3.

Table 1.

The process of aligning a set of shapes

X¯=1Li=1LX^i
(6)

The covariance matrix can thus be given as:

s=1L1i=1L(X^iX¯)(X^iX¯)T
(7)

Calculate the eigenvalue λs,i of s and the corresponding eigenvector ηs,i

ηs,i=λs,iηs,i
(8)

Sort all the eigenvalues in descending order:

λs,iλs,i+1,i=1,2,,2m1
(9)

The corresponding set of eigenvectors is Ηs=[ηs,1,ηs,2,,ηs,2m]

To reduce the computational complexity, the principal component analysis (PCA) method is used to reduce the space dimensions. Choose t largest eigenvalues which satisfy the following condition:

i=1tλs,i0.98(i=12mλs,i)
(10)

Then any shape instance can be generated by deforming the mean shape by a linear combination of eigenvectors:

X^X¯+Φsbs
(11)

where Φs is a matrix of dimension 2m×t and contains t eigenvectors corresponding to the largest eigenvalues, Φs=(φs,1|φs,2||φs,t) . bs is a t dimensional vector and the eigenvectors are mutually orthogonal, so it can be given by:

bs=Φ1s(X^X¯)=ΦsT(X^X¯)
(12)

By varying the elements of bs , new shape instance can be generated using equation (11).

2.3. Establish texture model

To establish a complete appearance model, not only the shape but also the texture should be considered. We should firstly acquire the pixel information over the region covered by the landmarks on each training image. Then the image warping method is used to consistently collect the texture information between the landmarks on different training images. Finally establish the texture model using the PCA method.

The texture of a target object can be represented as:

g=[g1,g2,,gn]T
(13)

where n denotes the number of pixel samples over the object surface.

In an image, the target object may occupy a small region of the whole image. The pixel intensities and global changes in illumination are different in each training image. The most important information is the texture that can reflect the characteristic of the target object. Due to the number of pixels in different target region is different and it is difficult to acquire the accurate corresponding relationship between different images, the texture model can not be established directly. We need to obtain a texture vector with the same dimension and corresponding relationship. So we warp each example image so that its control points match the reference shape. The process of image warping is described as follows:

Firstly, apply Delaunay triangulation to the reference shape to obtain the reference mesh which is consisted of a set of triangles. We choose the mean shape as the reference shape.

Secondly, suppose v1 , v2 and v3 are three vertices of a triangle in the example mesh, any internal point v of the triangle can be written as:

v=v1+β(v2v1)+γ(v3v1)=αv1+βv2+γv3
(14)

where α+β+γ=1 . Constrain 0α,β,γ1 because v is inside the triangle. Given v=[x,y]T , v1=[x1,y1]T v2=[x2,y2]T v3=[x3,y3]T , then α,β,γ can be calculated by:

α=1(β+γ)
(15)
β=yx3x1yx3y1y3x+x1y3+xy1-x2y3+x2y1+x1y3+x3y2x3y1x1y2
(16)
γ=xy2xy1x1y2x2y+x2y1+x1y-x2y3+x2y1+x1y3+x3y2x3y1x1y2
(17)

Finally, the corresponding point v’ of the triangle in the reference mesh can be calculated by:

v=v1+β(v2v1)+γ(v3v1)=αv1+βv2+γv3
(18)

where v1 , v2 and v3 are the vertices of the corresponding triangle in the reference mesh.

After image warping process, each shape in the training set is warped to the reference shape and sampled. The influence from the global linear changes in pixel intensities is removed.

To reduce the effects that caused by global lighting variations, the example samples are normalized by applying a scaling a and offset b:

g=(g1,g2g3,,gn)=(g1ba,g2ba,g3ba,,gnba)
(19)

where

b=1ni=1ngi,a=σ,σ2=1ni=1n(gib)2
(20)

The establishment of the texture model is identical to the establishment of the shape model which is also analyzed by PCA approach. The mean texture is calculated by:

g¯=1Li=1Lgi
(21)

The covariance matrix can thus be given as:

g=1L1i=1L(gig¯)(gig¯)T
(22)

Calculate the eigenvalue λg,i of g and the corresponding eigenvector ηg,i :

ηg,i=λg,iηg,i
(23)

Sort all the eigenvalues in descending order:

λg,iλg,i+1,i=1,2,,2n1
(24)

The corresponding set of eigenvectors is Ηg=[ηg,1,ηg,2,,ηg,2n]

To reduce the computational complexity, the PCA method is used to reduce the space dimensions. Choose t largest eigenvalues which satisfy the following condition:

i=1tλg,i0.98(i=12nλg,i)
(25)

Then any texture instance can be generated by deforming the mean texture by a linear combination of eigenvectors:

gg¯+Φgbg
(26)

where Φg is a matrix of dimension 2n×t and contains t eigenvectors corresponding to the largest eigenvalues, Φg=(ϕg,1|ϕg,2||ϕg,t) . bg is a t dimensional vector and the eigenvectors are mutually orthogonal, so it can be given by:

bg=Φg1(gg¯)=ΦgT(gg¯)
(27)

By varying the elements of bg , new texture instance can be generated using equation (26).

2.4 Establish combined model

Since the shape model and texture model have been established, any input image can be represented using the shape parameter vector bs and texture parameter vector bg . Since there are some correlations between shape and texture variations, the new vector b can be generated by combining bs and bg :

b=(Wsbsbg)=(WsΦsT(X^X¯)ΦgT(gg¯))
(28)

where Ws is a diagonal matrix which adjust the weighting between pixel distances and pixel intensities. Ws is calculated by:

Ws=rI=[r00r]
(29)

where r 2 is the ratio of the total intensity variation to the total shape variation:

r=λgλs,λg=λg,i,λs=λs,i
(30)

Apply PCA on b, then

where Φc is the eigenvectors of covariance matrix corresponding to b:

Φc=(Φc,sΦc,g)
(32)

c is a vector of appearance model parameters controlling both the shape and texture of the models.

Using the linear nature of the model, the combined model including shape X and texture g can be expressed as:

X=X¯+ΦsW1Φc,sc
(33)
g=g¯+ΦgΦc,gc
(34)

Then a new image can be synthesised using equation (33) and (34) for a given c.

2.5 Fitting AAM to the input image

Fitting a AAMs to an image is considered to be a problem of minimizing the error between the input image and closest model instance [Wang et al., 2007]:

δI=IimageImodel
(35)
media/image82.png

Figure 3.

Results of tracking hand using AAM fitting.

where Iimage is the texture vector of the input image, and Imodel is the texture vector of the model instance. The goal is to adjust the appearance model parameters c to minimize |δI|2 . The simplest way is to construct a linear relationship:

where R can be computed by the following process:

  • Suppose c 0 is the model parameter of the current image, new model parameter c can be generated by variating c 0:

  • Generate new shape model X and normalized texture model gm according to equation (33) and (34).

  • Deform the current image and get the corresponding texture model gi , the difference vector can be written as:

δg will change along with the variation of δc and the shape model X

The fitting procedure is shown in Table 2 and the object tracking results are shown in Figure 3 and Figure 4. From the tracking results we can see that when the camera moves around the scene, the tracking results are satisfying.

3. Factorization algorithm

The flowchart of the factorization algorithm is shown in Figure 5, and the detailed demonstration will be given in the following sections.

3.1. Basic knowledge

The 3D shape of the non-rigid object can be described as a key frame basis set S1,S2,,SK . Each key frame basis Si is a 3×P matrix describing P points. The 3D shape of a specific configuration is a linear combination of the basis set:

media/image97.png

Figure 4.

Results of tracking face using AAM fitting.

Step 1: Generate the normalized texture vectorgm.
Step 2: Sample the image gi below the model shape.
Step 3: Evaluate the error vectorδg0=gigm.
Step 4: Evaluate the errorE0=|δg0|.
Step 5: Calculate the pose displacementδt=Rtδg0.
Step 6: Calculate the displacement in model parametersδc=Rcδg0.
Step 7: Seti=1.
Step 8: Update model parametersc=ckiδc.
Step 9: Transform the shape to invert the δt transformation.
Step 10: Repeat step 1-4 to form a new errorEi.
Step 11: If EiE0 set i=i+1 and go to step 8.
Step 12: Accept the new estimate.
where k=[1.5,0.5,0.125,0.0125,0.0625]T is the damping vector.

Table 2.

The procedure of fitting the input image to the model instance.

S=i=1KliSi   S,SiR3×P,liR
(39)

where

S=[x1x2...xPy1y2...yPz1z2...zP]
(40)
.

Under a scaled orthographic projection, the P points of S are projected into 2D image points (ui vi ):

[u1u2...uPv1v2...vP]=R(i=1KliSi)+T
(41)
media/image114.png

Figure 5.

The flowchart of the factorization algorithm.

R=[r1r2r3r4r5r6],T=[t1t2t3]
(42)

R contains the first two rows of the 3D camera rotation matrix. T is the camera translation matrix. As mentioned in [Tomasi & Kanade, 1992], we eliminate the camera translation matrix by subtracting the mean of all 2D points, and henceforth can assume that the 3D shape S is centred at the origin.

[u1u2...uPv1v2...vP]=[u1u¯u2u¯...uPu¯v1v¯v2v¯...vPv¯]
(43)

where

u¯=i=1Pui,v¯=i=1Pvi
(44)

Therefore, we can rewrite equation (40) as:

[u1u2...uPv1v2...vP]=R(i=1KliSi)
(45)

Rewrite the linear combination in equation (43) as a matrix-matrix multiplication:

[u1u2...uPv1v2...vP]=[l1Rl2R...lKR][S1S2...SK]
(46)

The 2D image points in each frame can be obtained using AAM. The tracked 2D points in frame t can be denoted as (uti,vti) . The 2D tracking matrix of N frames can be written as:

W=[u11u12...u1Pv11v12...v1Pu21u22...u2Pv21v22...v2P............uN1uN2uNPvN1vN2vNP]
(47)

Using equation (44) we can rewrite equation (45) as:

W=[l11R1l12R1l1KR1l21R2l22R2l2KR2lN1RNlN2RNlNKRN][S1S2SK]
(48)

where Rt denotes the camera rotation of frame t. lti denotes the shape parameter li of frame t.

3.2 Solving configuration weights using factorization

Equation (46) shows that the 2D tracking matrix W has rank 3K , and can be factored into the product of two matrixes: W=QB , where Q is a 2N×3K matrix, B is a 3K×P matrix. Q contains the camera rotation matrix Rt and configuration weights lt1,lt2,,ltK of each frame. B contains the information of shape basis S1,S2,,SK . The factorization can be done using singular value decomposition (SVD) method:

W2N×P=U˜D˜V˜T=Q˜2N×3KB˜3K×P
(49)

Then the camera rotation matrix Rt and shape basis weights lti of each frame can be extracted from the matrix Q˜

qt=[lt1Rtlt2RtltKRt]=[l1r1l1r2l1r3lKr1lKr2lKr3l1r4l1r5l1r6lKr4lKr5lKr6]
(50)

Transform qt into a new matrix q˜t :

q˜t=[l1r1l1r2l1r3l1r4l1r5l1r6l2r1l2r2l2r3l2r4l2r5l2r6lKr1lKr2lKr3lKr4lKr5lKr6]=[lt1lt2ltK][rt1rt2rt3rt4rt5rt6]
(51)

here, the q˜t can be factored using SVD method.

3.3 Solving true rotation matrix and shape basis

As mentioned in [Tomasi & Kanade, 1992], the matrix R˜t is a linear transformation of the true rotation matrix Rt . Likewise, S˜i is a linear transformation of the true shape matrix Si :

Rt=R˜tG,Si=G1S˜i
(52)

where G3×3 is found by solving a nonlinear data-fitting problem. In each frame we need to constrain the rotation matrix to be orthonormal. The constraints of frame t are:

[rt1rt2rt3]GG1[rt1rt2rt3]T=1
(53)
[rt4rt5rt6]GG1[rt4rt5rt6]T=1
(54)
[rt1rt2rt3]GG1[rt4rt5rt6]T=0
(55)

In summary, given 2D tracking data W , we can get the 3D shape basis S˜i , camera rotation matrix R˜t and configuration weights lti of each training frame simultaneously using factorization method.

4. Non-rigid registration method

The 2D tracking data can be obtained using the AAM algorithm mentioned in section 2. With the use of the factorization method mentioned in section 3, we can acquire the 3D shape basis. The 3D shape is represented as 3D points in the world coordinates. Given the configuration weights, the 3D shape can be recovered by linear combination of the 3D shape basis. By projecting the 3D points to the 2D image with known camera rotation matrix (suppose the intrinsic camera matrix has been calculated), the estimated 2D points can be acquired. If the error between the 2D tracking data and the estimated 2D points is small enough, we can accept the configuration weights and the rotation matrix. Finally, the virtual object can be overlaid to the real scene using the camera rotation matrix.

The initial configuration weights and camera rotation matrix can not be precise. Optimization of the configuration weights should be done to minimize the error between the 2D tracking points detected by AAM and the estimated 2D points which is projected by the 3D shape points. This is a non-linear optimization problem which can be successfully solved by the optimization methods. Different with [Zhu et al., 2006], we use the Levenberg-Marquardt algorithm. Equation (54) shows the cost function.

jsjsj2min=j=1Nsj(Ri=1KliSi)2min
(56)

where sj is the 2D tracking data j sj is the projected point.

The procedure of non-rigid registration is shown in Table 3.

Step 1: Track the 2D points sj using AAM.
Step 2: Initialize the configuration weightsli.
Step 3: Initialize the camera rotation matrixR.
Step 4: Calculate the 3D shapeS=i=1KliSi
Step 5: Project the 3D points S to 2D image:sj=A[R|T]S
Step 6: Evaluate the projection errorE=j=1Nsjsj2
Step 7: If E is not small enough, improve li andR, then repeat step 4-6.
Step 8: Accept the configuration weightsli, and the camera rotation matrixR
Step 9: Overlaid the virtual object to the scene.

Table 3.

The procedure of non-rigid registration

media/image171.png

Figure 6.

Examples of augmented images using our method.

Furthermore, we should take the orthonormality of rotation matrix into consideration. The proposed method has been implemented in C using OpenGL and OpenCV on a DELL workstation (CPU 1.7G×2, RAM 1G).

In offline stage, we construct the AAM hand models using 7 training images which are manually labelled with 36 landmarks as shown in Figure 1. We establish the hand shape basis using 300 training frames which is captured with a CCD camera. In online stage, the 2D points are tracked using AAM algorithm, and then Levenberg-Marquardt algorithm is used to optimize the parameters. Our experiment results are shown in Figure 6. From the results we can see that the virtual teapot can be overlaid on the hand accurately when the camera moves around the scene.

5. Conclusion

In this chapter, a non-rigid registration method for augmented reality applications using AAM and factorization method is proposed. The process is divided into two stages: offline stage and online stage. In the offline stage, the 3D shape basis is constructed. To obtain the shape basis of the object, we firstly factorize the 2D data matrix tracked by the AAM into the product of two matrixes. One matrix contains the camera rotation matrix and the configuration weights, and the other matrix contains the shape basis. Then the rotation matrix and the configuration weights can be separated using SVD method. Finally the orthonormality of the rotation matrix should be the constraints to get the true rotation matrix and configuration weights. In online stage, the 3D pose parameters and the shape coefficients are estimated. The purpose is to minimize the error between the 2D tracking points detected by AAM and the estimated 2D points which is projected by the 3D shape points. The Levenberg-Marquardt method is used to solve this problem. The rotation matrix and the configuration weights are optimized. Some experiments have been conducted to validate that the proposed method is effective and useful for non-rigid registration in augmented reality applications.

6. Acknowledgements

This work is supported in part by National Natural Science Foundation of China with project No. 60903095; in part by Postdoctoral Science Foundation of China with project No. 20080440941.

References

1 - J. Ahlberg, 2001 Using the active appearance algorithm for face and facial feature tracking. Proceedings of the 2nd International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, 68 72 , 0-76951-074-4 Canada, July 2001, IEEE Computer Society, Los Alamitos.
2 - P. R. Andresen, M. Nielsen, 2001 Non-rigid registration by geometry-constrained diffusion. Medical Image Analysis, 5 2 81-88, 1361-8415.
3 - R. T. Azuma, 1997 A survey of augmented reality. Presence-Teleoperators and Virtual Environments, 6 4 355-385, 1054-7460.
4 - R. T. Azuma, 2001 Recent advances in augmented reality. IEEE Computer Graphics and Applications, 21 6 34-47, 0272-1716.
5 - A. Bartoli, Tunzelmann. E. von, A. Zisserman, 2004 Augmenting images of non-rigid scenes using point and curve correspondences. Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 699 706 , 0-76952-158-4 DC, June-Junly 2004, IEEE Conputer Society, Los Alamitos.
6 - F. L. Bookstein, 1996 Landmark methods for forms without landmarks: localizing group differences in outline shape. Proceedings of the IEEE Workshop on Mathematical Methods in Biomedical Image Analysis, 25 244 , 0-81867-367-2 Francisco, CA, June 1996, IEEE Computer Society, Los Alamitos.
7 - C. Bregler, A. Hertzmann, H. Biermann, 2000 Recovering non-rigid 3d shape from image streams. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 690 696 , 1063-6919, Hilton Head Island, SC, June 2000, IEEE Computer Society, Los Alamitos.
8 - T. F. Cootes, G. J. Edwards, C. J. Taylor, 1998 Active appearance models. Lecture Notes in Computer Science, 1407 484-498, 0302-9743.
9 - T. F. Cootes, G. J. Edwards, C. J. Taylor, 2001 Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23 6 681-685, 0162-8828.
10 - T. F. Cootes, C. J. Taylor, 2001 Statistical models of appearance for medical image analysis and computer vision. Proceedings of the Society of Photo-optical Instrumentation Engineers, 236 248 , 0-81944-008-6 Diego, CA, February 2001, SPIE-INT Society Engeering, Bellingham.
11 - N. Duta, A. K. Jain, M. Dubuisson-Jolly, P. , 1999 Learning 2D shape models. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 8 14 , 1063-6919, Fort Collins, USA, June 1999, IEEE Computer Society, Los Alamitos.
12 - T. Guan, L. J. Li, C. Wang, 2008a Registration using multiplanar structures for augmented reality systems. Journal of Computing and Information Science in Engineering, 8 4 041002-1~041002-6, 1530-9827.
13 - T. Guan, L. J. Li, C. Wang, 2008b Robust estimation of trifocal tensors using natural features for augmented reality systems. Computing and Informatics, 27 6 891-911, 1335-9150.
14 - V. Lepetit, P. Fua, 2005 Monocular model-based 3D tracking of rigid objects: a survey. Foundations and Trends in Computer Graphics and Vision, 1 1 1-89, 1572-2740.
15 - V. Lepetit, P. Lagger, P. Fua, 2005 Randomized trees for real-time keypoint recognition. Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 775 781 , 0-76952-372-2 Diego, CA, June 2005, IEEE Computer Society, Los Alamitos.
16 - L. J. Li, T. Guan, B. Ren, et al. 2008 Registration based on Euclidean reconstruction and natural features tracking for augmented reality systems. Assembly Automation, 28 4 340-347, 0144-5154.
17 - D. G. Lowe, 2004 Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 20 2 91-110, 0920-5691.
18 - E. Markin, E. C. Prakash, 2006 Tracking facial features with occlusions. Journal of Zhejiang University: Science A, 7 7 1282-1288, 1009-3095.
19 - J. Pilet, V. Lepetit, P. Fua, 2005 Real-time non-rigid surface detection. Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 822 828 , 0-76952-372-2 Diego, CA, June 2005, IEEE Computer Society, Los Alamitos.
20 - J. Pilet, V. Lepetit, P. Fua, 2007 Fast non-rigid surface detection, registration and realistic augmentation. International Journal of Computer Vision, 76 2 109-122, 0920-5691.
21 - E. Rosten, T. Drummond, 2005 Fusing points and lines for high performance tracking. Proceedings of 10th IEEE International Conference on Computer Vision, 1508 1515 , 076952334 Beijing, China, October 2005, IEEE Computer Society, Los Alamitos.
22 - S. Sclaroff, A. P. Pentland, 1995 Modal matching for correspondence and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17 7 545-561, 0162-8828.
23 - J. Sung, D. J. Kim, 2004 Extension of AAM with 3D shape model for facial shape tracking. Proceedings of International Conference on Image Processing, 3363 3366 , 0-78038-554-3 October 2004, IEEE, New York.
24 - Y. Tian, Z. Y. Li, W. Liu, et al. 2008 Non-rigid registration using AAM and factorization method for augmented reality applications. Proceedings of the 2008 12th International Conference on Computer Supported Cooperative Work in Design, 705 709 , 978-1-42441-650-9 Xian, China, April 2008, IEEE, New York.
25 - C. Tomasi, T. Kanade, 1992 Shape and motion from image streams under orthography: a factorization method. International Journal of Computer Vision, 9 2 137-154, 0920-5691.
26 - K. N. Walker, T. F. Cootes, C. J. Taylor, 2000 Determining correspondences for statistical models of appearance. Proceedings of 6th European Conference on Computer Vision, 829 843 , 3-54067-685-6 Ireland, June 2000, Springer, Berlin.
27 - S. C. Wang, Y. S. Wang, X. L. Chen, 2007 Weighted active appearance models. 3rd Internatioanl Conference on Intelligent Computing, 1295 1304 , 978-3-54074-170-1 Qingdao, China, August 2007, Springer, Berlin.
28 - J. Xiao, S. Baker, I. Matthews, T. Kanade, 2004 Real-time combined 2D+3D active appearance models. Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 535 542 , 0-76952-158-4 DC, June-July 2004, IEEE Computer Society, Los Alamitos.
29 - M. L. Yuan, S. K. Ong, A. Y. C. Nee, 2005 Registration based on projective reconstruction technique for augmented reality systems. IEEE Transactions on Visualization and Computer Graphics, 11 3 254-264, 1077-2626.
30 - M. L. Yuan, S. K. Ong, A. Y. C. Nee, 2006 Registration using natural features for augmented reality systems. IEEE Transactions on Visualization and Computer Graphics, 12 4 569-580, 1077-2626.
31 - J. K. Zhu, S. C. H. Hoi, M. R. Lyu, 2006 Real-time non-rigid shape recovery via active appearance models for Augmented Reality. 9th European Conference on Computer Vision, 186 197 , 3-54033-832-2 Austria, May 2006, Springer, Berlin.