Behavior Recognition Using any Feature Space Representation of Motion Trajectories

In recent years, there has been a growth of research activity aimed at the development of sophisticated content-based video data management techniques. This development is now especially timely given an increasing number of systems that are able to capture and store data about object motion such as those of humans and vehicles. This has acted as a spur to the development of content-based visual data management techniques for tasks such as behavior classification and recognition, detection of anomalous behavior and object motion prediction. Behavior can obviously be categorized at different levels of granularity. In far-field surveillance, we are primarily interested in trajectory-based coarse motion description involving movement direction (right/left or up/down) and motion type (walking, running or stopping). These techniques are essential for the development of next generation ‘actionable intelligence’ surveillance systems.


Introduction
In recent years, there has been a growth of research activity aimed at the development of sophisticated content-based video data management techniques. This development is now especially timely given an increasing number of systems that are able to capture and store data about object motion such as those of humans and vehicles. This has acted as a spur to the development of content-based visual data management techniques for tasks such as behavior classification and recognition, detection of anomalous behavior and object motion prediction. Behavior can obviously be categorized at different levels of granularity. In far-field surveillance, we are primarily interested in trajectory-based coarse motion description involving movement direction (right/left or up/down) and motion type (walking, running or stopping). These techniques are essential for the development of next generation 'actionable intelligence' surveillance systems.
Processing of trajectory data for activity classification and recognition has gained significant interest quite recently.
Various techniques have been proposed for modeling of trajectory-based motion activity patterns and using the modeled patterns for classification and anomaly detection. Much of the earlier research focus in motion analysis has been on high-level object trajectory representation schemes that are able to produce compressed forms of motion data (Aghbari et al., 2003;Chang et al., 1998;Dagtas et al., 2000;Hsu & Teng, 2002;Jin & Mokhtarian, 2004;Khalid & Naftel, 2005;Shim & Chang, 2004). This work presupposes the existence of some low-level visual tracking scheme for reliably extracting object-based trajectories (Hu, Tan, Wang & Maybank, 2004;Vlachos et al., 2002). The literature on trajectory-based motion understanding and pattern discovery is less mature but advances using Learning Vector Quantization (LVQ) (Johnson & Hogg, 1995), Self-Organising Maps (SOMs) (Hu, Xiao, Xie, Tan & Maybank, 2004;Owens & Hunter, 2000), Hidden Markov Models (HMMs) (Bashir et al., 2006;2005b), and fuzzy neural networks  have all been reported. These approaches are broadly categorized into statistical and neural network based approaches.
In a development of trajectory-based motion event recognition systems, there are different questions that we need to answer before proposing or selecting a pattern modeling and recognition technique. These includes: 1. What is the feature space representation of trajectories? 6 www.intechopen.com 2. What is the distribution of trajectories in a given feature space representation? Do we need to cater for complex shape distributions that may exit in a given motion pattern? 3. Do we expect to have a multimodal distribution of trajectories within a given pattern?
Most of the trajectory-based motion recognition system, as proposed in relevant literature (Hu, Xiao, Xie, Tan & Maybank, 2004;Khalid & Naftel, 2005;Owens & Hunter, 2000) can operate only on feature space representation of trajectories that lies in Euclidean space with a computable mean. However, a survey of recent literature in the areas of motion feature computation for trajectory representation shows that most of the feature space representation are complex and do not lie in the Euclidean space (Bashir et al., 2006;2005a;b;Hamid et al., 2005;Keogh et al., 2001;Xiang & Gong, 2006;Zhong et al., 2004). It is not possible to compute a mean representation of different trajectories using such complex feature spaces. They can therefore not be applied to complex feature spaces with incalculable mean. These approaches expect that the trajectories in a given motion pattern follow certain standard distribution such as Gaussian. They can not cater for multimodal complex shape distribution of trajectories within a given motion pattern which is expected in the presence of complex feature space representation of trajectories. The research presented in this chapter focuses on presenting a trajectory-based behavior recognition and anomaly detection system that have an answer to all of the above raised questions. The proposed approach does not impose any limitation on the representation of trajectories. It can operate using any trajectory representation in any feature space with a given distance function. The proposed approach can perform modeling, classification and anomaly detection in the presence of multimodal distribution of trajectories within a given motion pattern.
The remainder of the chapter is organized as follows: We review some relevant background material in section 2. In section 3, we present a framework of multimodal modeling of activity patterns using any feature space with a computable similarity function. A soft classification and anomaly detection techniques using multimodal m-Medoids model is presented in section 4. Comparative evaluation of currently proposed multimodal m-Medoids and previously proposed localized m-Medoids (Khalid, 2010a) based appraoch for activity classification and anomaly detection is presented in section 5. Experiments have been performed to show the effectiveness of proposed system for trajectory-based modeling, classification and anomaly detection in the presence of multimodal distribution of trajectories within a pattern, as compared to competitors. These experiments are reported in section 6. The last section summarizes the paper.

Background and related work
Motion trajectory descriptors are known to be useful candidates for video indexing and retrieval schemes. Variety of trajectory modeling techniques have been proposed to compute the feature for trajectory representation. Most of the techniques for learning motion behaviour patterns and recognition from trajectories use discrete point sequence vectors as input to a machine learning algorithm. Related work within the data mining community on representation schemes for indexing time series data is also relevant to the parameterisation of object trajectories. An object trajectory can be defined as a set of points representing the ordered observations of the location of a moving object made at different points in time. A trajectory can therefore be represented as a time series implying that indexing techniques Behavior Recognition Using any Feature Space Representation of Motion Trajectories 3 for time series are also applicable to motion data. For example, Discrete Fourier Transforms (DFT) (Faloutsos et al., 1994), Discrete Wavelet Transforms (DWT) (Chan & Fu, 1999), Adaptive Piecewise Constant Approximations (APCA) (Keogh et al., 2001), and Chebyshev polynomials (Cai & Ng, 2004) have been used to conduct similarity search in time series data. Previous work has also sought to represent moving object trajectories through piecewise linear or quadratic interpolation functions (Chang et al., 1998;Jeanin & Divakaran, 2001), motion histograms (Aghbari et al., 2003) or discretised direction-based schemes (Dagtas et al., 2000;Shim & Chang, 2001;. Spatiotemporal representations using piecewise-defined polynomials were proposed by (Hsu & Teng, 2002), although consistency in applying a trajectory-splitting scheme across query and searched trajectories can be problematic. Affine and more general spatiotemporally invariant schemes for trajectory retrieval have also been presented (Bashir et al., 2003;Jin & Mokhtarian, 2004). The importance of selecting the most appropriate trajectory model and similarity search metric has received relatively scant attention (Khalid & Naftel, 2005).
Modeling of motion patterns using trajectory data to perform motion based behavior recognition and anomaly detection has gained significant interest recently. Various techniques have been proposed for modeling of motion activity patterns and using the modeled patterns for classification and anomaly detection. These approaches are broadly categorized into statistical and neural network based approaches. Almost all statistical approaches dealing with anomaly detection are based on modelling the density of training data and rejecting test patterns that fall in regions of low density. There are various approaches that use Gaussian mixture models to estimate the probability density of data (Brotherton et al., 1998;Roberts & Tarassenko, 1994;Yeung & Chow, 2002). Various techniques based on hidden Markov models (HMM) have also been proposed (Xiang & Gong, 2005;Zhang et al., 2005). (Yacoob & Black, 1999) and (Bashir et al., 2005b; have presented a framework for modeling and recognition of human motion based on a trajectory segmentation scheme. A framework is presented to estimate the multimodal probability density function (PDF), based on PCA coefficients of the sub-trajectories, using GMM. Different classes of object motion are modelled by a continuous HMM per class where the state PDFs are represented by GMMs. The proposed technique has been shown to work for sign language recognition. The proposed classification system can not handle anomalies in test data and can only classify samples from normal patterns. (Xiang & Gong, 2005; propose a framework for behavior classification and anomaly detection in video sequences. Natural grouping of behaviour patterns is learnt through unsupervised model selection and feature selection on the eigenvectors of a normalized affinity matrix. A Multi-Observation Hidden Markov Model is used for modelling the behaviour pattern. (Hu et al., 2006; and (Khalid & Naftel, 2006) models normal motion patterns by estimating single multimodal gaussian for each class. For anomaly detection in (Hu et al., 2006), the probability of a trajectory belonging to each motion pattern is calculated. If the probability of association of trajectory to the closest motion pattern is less then a threshold, the trajectory is treated as anomalous. In (Rea et al., 2004), a semantic event detection technique based on discrete HMMs is applied to snooker videos. (Zhang et al., 2005) propose a semi-supervised model using HMMs for anomaly detection. Temporal dependencies are modelled using HMMs. The probability density function of each HMM state is assumed to be a GMM. (Owens & Hunter, 2000) uses Self Organizing Feature Maps (SOFM) to learn normal trajectory patterns. While classifying trajectories, if the distance of 103 Behavior Recognition Using any Feature Space Representation of Motion Trajectories www.intechopen.com the trajectory to its allocated class exceeds a threshold value, the trajectory is identified as anomalous.
In our previous work (Khalid, 2010b), we have proposed m-Medoids based activity Modeling and Classification approach using low dimensional feature vector representation of trajectories in Euclidean Space (MC-ES). m-Medoids based approach models a pattern by a set of cluster centres of mutually disjunctive sub-classes (referred to as medoids) within the pattern. Once the m-Medoids model for all the classes have been learnt, the MC-ES approach performs classification of new trajectories and anomaly detection by checking the closeness of said trajectories to the models of different classes using hierarchical classifier. The anomaly detection module required specification of threshold which is used globally for all the patterns. However, this approach had unaddressed issues like manual specification of threshold for anomaly detection, identification of appropriate value of threshold for anomaly detection and anomaly detection of motion patterns with different scale and orientation which is used globally for all the patterns. These issues are addressed by a localized m-Medoids based approach (LMC-ES) as proposed in (Khalid, 2010a) which enables us to automatically select a local significance parameter for each pattern taking into consideration the distribution of individual patterns. LMC-ES can effectively handle patterns with different orientation and scale and has been shown to give superior performance than competitors including GMM, HMM and SVM based classifiers. However, there are still open issues (i) Modeling, classification and anomaly detection in the presence of multimodal distribution of trajectories within a pattern (ii) Soft classification in the presence of multimodal pattern distribution to minimize misclassification (iii) Modeling and classification in feature spaces for which we can not compute mean.
The contribution of this work is to present an extension of m-Medoids based modeling approach, wherein the multimodal distribution of samples in each pattern is represented using multimodal m-Medoids. An approach for multimodal model-based classification and anomaly detection is also presented. The presented mechanism is based on a soft classification approach which enables the proposed multimodal classifier to adapt to the multimodal distribution of samples within different patterns. The multimodal m-Medoids based modeling and classification is applicable to any feature spaces with a computable pairwise similarity measure.

Multimodal m-Medoids based modeling
Given a representation of trajectories in any feature space for a given motion pattern, we wish to model the underlying distribution of trajectories within a pattern using training data. A pattern is modeled by a set of cluster centers of mutually disjunctive sub-classes (referred to as medoids) within the pattern. The proposed modeling technique referred to as m-Medoids modeling, models the class containing n members with m medoids known a-priori. Modeling of pattern using multimodal m-Medoids approach in general feature space is a three step process, (i) identification of m medoids, (ii) computation of set of possible normality ranges for the pattern and (iii) selection of customized normality range for each medoid. The resulting models of identified patterns can then be used to classify new unseen trajectory data to one of the modeled classes or identify it as anomalous if it is significantly distant from all of the modeled pattern.

Step 1: Identification of m-Medoids
The algorithm for identification of medoids using finite dimensional features in general feature space with a computable similarity matrix is based on the affinity propagation based clustering algorithm (Frey & Dueck, 2007). Let DB (i) be the classified training samples associated to pattern i, the modeling algorithm comprises the following steps: 1. Form the affinity matrix A ∈ R n×n defined by Here s a , s b ∈ DB (i) , σ is the scaling parameter and P(a) is the preference parameter indicating the suitability of sample a to be selected as an exemplar (medoid). We set P(a) to the median of affinities of sample a with n samples. We use a dynamic value of σ which is set to be the 6 th nearest neighbor of s a to cater for variation in local distribution of trajectory samples. 2. Initialize availability matrix ℑ(a, b)=0 ∀a, b. The entry ℑ(a, b) in availability matrix stores the suitability of trajectory s b to be selected by trajectory s a as its exemplar. 3. Update responsibility matrix ℜ as The entry ℜ(a, b) in the responsibility matrix reflects the accumulated evidence for how well-suited trajectory s b to serve as an exemplar for trajectory s a while taking into account other potential exemplar for trajectory s a . 4. Update availability matrix ℑ as 5. Identify the exemplar for each sample as 6. Iterate through steps 3-5 till the algorithm is converged or maximum number of learning iterations (t max ) is exceeded. The algorithm is considered to have converged if there is no change in exemplar identification for certain number of iterations (t convergance ). 7. If the number of exemplars identified are smaller than the desired number of medoids, set higher values of preference and vice versa. The algorithm is repeated till the desired number of exemplars are identified. An appropriate value of preference parameter, for identification of desired number of medoids, is searched using a bisection method. 8. Append exemplars ξ a to the list of medoids M (i) modeling the pattern i.

Step 2: Computation of possible normality ranges
After the identification of medoids M (i) for pattern i, we intend to identify and pre-compute a set of possible normality ranges for a given pattern. Values of normality ranges for a given 105 Behavior Recognition Using any Feature Space Representation of Motion Trajectories www.intechopen.com pattern is determined by the inter-medoid distances within a given pattern. Hence, different patterns will have different set of possible normality ranges depending on the distribution of samples, and in turn medoids, within a pattern. In this step, a set of possible normality ranges D (c) for the pattern c is computed as follows: 1. Identify the closest pair of medoids (i, j) (indexed by (p, q))fromM (c) as follows: where dist(.,.) is the distance function for a given feature space representation of trajectories. 2. Set l = 1. 3. Populate the distance array at index l using 4. Remove the closest pair of medoids using 5. Set l = l + 1. 6. Iterate through steps 1-5 till there are no mediods left in M (c) .

Step 3: Selection of customized normality range for each medoid
After the identification of medoids and a set of possible normality ranges for a given pattern, we select different normality range for each medoid depending on the distribution of samples from the same and different patterns around a given medoid. The normality range is selected to minimize false positives (false identification of training samples from other patterns as a normal member of pattern that is being modeled) and false negatives (classification of normal samples of the pattern being modeled as anomalies). The algorithm for selection of customized normality range for each medoid, to enable multimodal m-Medoid based modeling of pattern, comprises of following steps: 1. Initialize significance parameter τ with the number of possible normality ranges for pattern c as computed in Step 2. 2. Sequentially input labeled training instances belonging to all classes and identify the closest medoid, indexed by r, using: where Q is the test sample. 3. Perform an anomaly test using the anomaly detection system, as proposed in section 4, assuming a one class classifier containing only pattern c represented by medoids set M (c) using the current value of τ. 4. Increment false positive count FP(r), corresponding to closest medoid M r , each time when the sample is a normal member of pattern c but is identified as anomalous.
where β is a scaling parameter to adjust the sensitivity of proposed classifier to false positives and false negatives according to specific requirements. 8. Set τ = τ − 1. 9. Iterate through step 2-8 till τ = 1. 10. Identify the value of significance parameter for a given medoid as: where τ (c,k) is the dynamic significance parameter that have a different normality range for each medoid depending on the local density.
The space complexity of the proposed modeling algorithm in general feature space is O(3 * n 2 ). The time complexity of our algorithm is the sum of time complexities of the three steps and is equivalent to O(ω * (n 2 + n 2 * log(n))) + O((# medoids * log(# medoids ))) + O(|DB| 2 * # medoids * log(# medoids )) where • O(n 2 ) is the time complexity of affinity matrix computation • O(n 2 * log(n)) is the time complexity of message passing to compute availability and responsibility matrix • ω is the number of times the modeling algorithm is repeated to identify m medoids. It has been observed that the value of ω normally lies in the range 3-10. • m * log(m) is the time complexity of computing possible normality range • |DB| * m is the time complexity for selecting customized normality range for each medoid where |DB| is the number of trajectories present in trajectory dataset DB.

Classification and anomaly detection
Once the m-Medoids based model for all the classes have been learnt, the classification of new trajectories is performed by checking the closeness of said trajectory to the models of different classes. The classification of unseen samples to known classes and anomaly detection is performed using following steps: 1. Identify k nearest medoids, from the entire set of medoids (M) belonging to different classes, to unseen sample Q as: where M is the set of all medoids from different classes and C is the ordered set of k closest medoids starting from the nearest medoid. 2. Initialize nearest medoid index ı to 1. 3. Set r to the id of ı th nearest medoid and c to the index of its corresponding class.
4. Set l = τ (c,r) . 5. Identify the normality threshold d w.r.t. the medoid r using D (c) as: 6. Test sample Q is considered to be a valid member of class c if: 7. If the condition specified in eq. (13) is not satisfied, increment the index ı by 1. 8. Iterate steps 3-7 till ı gets equivalent to k. If the test trajectory Q has not been identified as a valid member of any class, it is considered to be an outlier and deemed anomalous.
The time complexity of MMC-GFS based classification and anomaly detection algorithm is O(|M|)+O(k) for anomalous samples where |M| is the total number of medoids belonging to all classes. However, for most of the normal samples the time complexity is O(|M|). The time complexity can be further reduced by using efficient indexing structre like kd-trees to index |M| medoids for efficient k-NM search.

Relative merits of m-Medoids based modeling and classification algorithms
In this section, we provide a comparative evaluation of the proposed multimodal m-Medoids (MMC-GFS) and localized m-Medoids (Khalid, 2010a) based frameworks (LMC-ES) for modeling, classification and anomaly detection. These frameworks can be characterized in terms of the following attributes: • Ability to deal with multimodal distribution within a pattern • Ability to deal with variety of feature space representation of trajectories • Time complexity of generating m-Medoids based model of known patterns • Time complexity of classification and anomaly detection using learned models of normality • Scalability of modeling mechanism to cope with increasing number of training data For the ease in understanding of the comparative analysis, simulation of the working of proposed modeling and classification algorithms for arbitrary shaped patterns having multimodal distributions is presented in Fig. 1. In the left image of Fig. 1, each point represents the training sample and instances belonging to the same class are represented with same color and marker. Squares superimposed on each group of instances represent the medoids used for modeling the pattern. Normality region generated using different frameworks for classification and anomaly detection is depicted in the right image of Fig. 1. Test sample is considered to be a normal member of the class if it lies within the normality region, else it is marked as anomalous. Visualization of LMC-ES and MMC-GFS based modeling is provided in Fig. 1(a) and Fig. 1(b) respectively.
Multimodal modeling using MMC-GFS frameworks caters for the multimodal distribution within a pattern. On the other hand, LMC-ES framework always assumes a unimodal distribution within a pattern and hence can not cater for the dynamic distribution of samples within a pattern. It is apparent from Fig. 1 that MMC-GFS frameworks have generated more accurate models that have accommodated the variation in sample density within a given pattern. LMC-ES framework performs a hard classification of unseen sample. A sample is classified to a pattern represented by the majority of medoids from a set of k nearest medoids. The sample may not lie in the normality region of a pattern to which it is classified and hence 109 Behavior Recognition Using any Feature Space Representation of Motion Trajectories www.intechopen.com deemed anomalous. However, it is likely that it may still fall in the normality region of the second closest but less dense pattern having larger normality range. The hardness of LMC-ES based classification algorithm will result in the misclassification of such samples. However, MMC-GFS based classification and anomaly detection algorithm does not give a hard decision and checks for the membership of test trajectories w.r.t. different patterns until it is identified as a valid member of some pattern or it has been identified as anomalous w.r.t. k nearest medoids. This relatively softer approach enables the MMC-GFS based classification algorithm to adapt to the multimodal distribution of samples within different patterns. This phenomena is highlighted in Fig. 2. The samples, represented by 'x' marker, will be classified to blue pattern but is marked as anomalous using LMC-ES classifier as it falls outside the normality range of dense medoids belonging to the closest pattern. On the other hand, soft classification technique as proposed in MMC-GFS frameworks will correctly classify the sample as normal members of green pattern. Another benefit of MMC-GFS framework is that it can be applied to any feature space representation of trajectories with a given distance function. On the other hand, LMC-ES can only operate in feature spaces with a computable mean. Algorithms to generate m-Medoids model, as proposed in LMC-ES framework, is efficient and scalable to large datasets. On the other hand, the modeling algorithm of MMC-GFS is not scalable to very large datasets due to the requirement of affinity matrix computation. The space and time complexity is quadratic which is problematic for patterns with large number of training sample. However, this problem can be easily catered by splitting the training sample into subsets and selecting candidate medoids in each subset using algorithm as specified in section 3.1. The final selection of medoids can be done by applying the same algorithm again but now using the candidate medoids instead of all the training sample belonging to a given pattern. The classification algorithm of MMC-GFS framework is more efficient as compared to LMC-ES framework. This efficiency gain is due to the non-iterative unmerged anomaly detection with respect to a given medoid. The anomaly detection is done by applying a single threshold to the distance of the test sample from its ı th closest medoid as specified in eq. (13). On the other hand, LMC-ES implements iterative merged anomaly detection, which is more accurate but time consuming as compared to the modeling algorithm proposed in MMC-GFS framework. The time complexity of merged anomaly detection is O(m * log(m) − τ * log(τ)).

Experimental results
In this section, we present some results to analyze the performance of the proposed multimodal m-Medoids based modeling, classification and anomaly detection as compared to competitive techniques.

Experiment 1: Evaluation of m-Medoids based frameworks for classification and anomaly detection
The purpose of this experiment is to evaluate the performance of proposed MMC-GFS and LMC-ES based frameworks for classification of unseen data samples to one of the known patterns. The effectiveness of the proposed frameworks to perform anomaly detection is also demonstrated here. The experiment has been conducted on simulated SIM 2 dataset. Training data from simulated datasets is shown in Fig. 3. Test data for SIM 2 dataset is obtained by generating 500 samples from a uniform distribution such that (x, y) ∈ (U(1, 12), U(1, 12)).
We have used 50 medoids to model a class using its member samples. The classification and anomaly detection results for SIM 2 dataset, using LMC-ES and MMC-GFS frameworks are presented in Fig. 4(a) and Fig. 4(b) respectively. Training samples are represented using '+' marker whereas classified normal samples are represented by small circles. Data points belonging to same class are represented with same colour and marker. Samples from test data which have been identified as anomalous are represented with a black 'x' marker. It is apparent from Fig. 4 that multimodal m-Medoids based classification system as proposed in MMC-GFS framework performs better classification and anomaly detection while catering for multimodal distribution within the modeled pattern. On the other hand, LMC-ES based framework performs unimodal modeling of patterns and therefore the classification system does not adjust well to the variation of density within a pattern.
After demonstrating the efficacy of proposed classification and anomaly detection approach on synthetic data, the experiment is then repeated on real life LAB and HIGHWAY datasets. LAB and HIGHWAY datasets are classified motion datasets and contain anomalous trajectories within the datasets themselves. Classified training data for these datasets is obtained by randomly selecting half of the trajectories from each of the normal patterns in the dataset. The remaining half of the trajectories from the normal patterns along with anomalous trajectories are extracted and used as test data. Training samples from the LAB and HIGHWAY datasets are shown in Fig. 5 and Fig. 6 respectively. For ease of visualization, samples from each class are plotted separately on the background scene. The starting point of each trajectory is marked in green.
Trajectories from LAB and HIGHWAY datasets are modelled using DFT-MOD based coefficient feature vectors. (Khalid, 2010b). Patterns are modeled using 20 medoids per pattern. Once the multimodal m-Medoids based model for all the classes have been learnt, classification of samples from the test data is done using the classifier as proposed in section 112 Recent Developments in Video Surveillance www.intechopen.com  Fig. 7. The matching of classification obtained for each trajectory with its ground truth shows that no trajectory is misclassified. Trajectories identified as anomalous are shown in Fig. 8. It is clear from Fig. 8 that anomalous trajectories are significantly different from the normal motion patterns as shown in Fig. 7. The classification experiment is also conducted on the HIGHWAY dataset and the results obtained are shown in Fig. 9. Trajectories filtered as anomalous are shown in Fig. 10. These experimental results give evidence to the claim that MMC-GFS based classification and anomaly detection system is an effective and robust approach that works well with real life motion datasets.

Experiment 2: Comparison of proposed classifiers with competitive techniques
The purpose of this experiment is to compare the performance of classifiers as proposed in LMC-ES and MMC-GFS frameworks. For comparison of our results with competitive techniques, we establish a base case by implementing three different systems for comparison including Mahalanobis and GMM classifier. Real life ASL dataset is used for the experiment.

113
Behavior Recognition Using any Feature Space Representation of Motion Trajectories www.intechopen.com Signs from different number of word classes are selected. Classified training data is obtained by randomly selecting half of the trajectories from each word class leaving the other half to be used as test data. Trajectories from ASL dataset are represented using DFT-MOD based coefficient feature vectors (Khalid, 2010b). Patterns are modeled using 20 medoids per pattern. We have computed single multimodal Gaussian for modeling of patterns for Mahalanobis classifier. Modeling of patterns and classification of unseen samples using GMM is based on the approach as described in (Bashir et al., 2005a). Each class is modeled using a separate GMM. The number of modes to be used for GMM-based modeling is automatically estimated using a string of pruning, merging and mode-splitting processes as specified in (Bashir et al., 2005a). Once the models for all the classes have been learnt, the test data is passed to different classifiers and the class labels obtained are compared with the ground truth. The experiment is repeated with different numbers and combinations of word classes. Each classification experiment is averaged over 50 runs to reduce any bias resulting from favorable word selection.
The accuracy of different classifiers for wide range of word classes from ASL dataset is presented in Table 2. Based on these results, we can see that the multimodal m-Medoids based classifier as proposed in MMC-GFS framework yield a superior classification accuracy as compared to other classifiers closely followed by unimodal LMC-ES framework. GMM yields good results for lower number of classes but its performance deteriorates for higher number of word classes. It can also be observed from  Fig. 1 and Fig. 4. On the other hand, the competitive approaches impose assumptions on the PDF of patterns (normally gaussian). These approaches do not have the capacity to handle multimodal distribution within a pattern. As a result, the model generated by these approaches will not give an accurate representation of complex patterns and hence result in poor classification performance as compared to the proposed multimodal m-Medoids based approaches.

117
Behavior Recognition Using any Feature Space Representation of Motion Trajectories www.intechopen.com Similar experiment with ASL dataset (using similar experimental settings) has been conducted by (Bashir et al., 2007) using their proposed GMM and HMM-based classification system. They reported classification accuracies of 0.96, 0.92, 0.86 and 0.78 for 2, 4, 8 and 16 word classes respectively. A comparison of these classification accuracies with the results obtained using our approach reveals that classifiers from m-Medoids classifier family performs better than GMM and HMM-based recognition system (Bashir et al., 2007) despite the fact that our proposed classification approach is conceptually simpler and computationally less expensive.

Experiment 3: Quantitative evaluation of anomaly detection algorithms
Here we provide a quantitative evaluation and comparison of m-Medoids based anomaly detection algorithms, as proposed in MMC-GFS and LMC-ES frameworks, with competitors. We implemented three different anomaly detection techniques based on statistical test as proposed in (Khalid & Naftel, 2006), Grown When Required (GWR) novelty filter as proposed in (Marsland et al., 2002) and one-class classifier based anomaly detection as proposed in (Tax, 2001). (Khalid & Naftel, 2006) performs anomaly detection by using Mahalanobis classifier and conducting Hotelling's T 2 test. (Tax, 2001) perform anomaly detection by generating model of one class (referred to as target class) and distinguishing it from samples belonging to all other classes. There generation of model of the target class is done using SVM and GMM.
For SVM-based one class classifier (OCC-SVM), we have used RBF kernel for the modeing of target class. For GMM-based one class classifier (OCC-GMM), we have used the approach as specified in Experiment 2 to generate the GMM-based model.
The experiment has been conducted using different number of word classes from ASL dataset.
We have extracted half of the samples belonging to each word class for training purposes leaving the other half of the samples to be used as test data. DFT-MOD based coefficient feature vector representation of sign trajectories from training data is generated and used to generate models as required by the different classification approaches. MMC-GFS and LMC-ES framework based model of each class is generated using the algorithm as presented in section 3 and (Khalid, 2010a) respectively. Patterns are modeled using 20 medoids per pattern.
Once the model learning phase is over, anomaly detection using different techniques is carried out using test dataset. We would expect that few instances drawn from class X would be recorded as anomalous when tested against the same class, whereas nearly all instances would be detected as anomalous when tested against a different class Y. The experiment is repeated with different numbers and combinations of word classes. Each anomaly detection experiment is averaged over 50 runs to reduce any bias resulting from favorable word selection. Fig. 11 reports the result in terms of percentage of correct anomaly detection using various number of word classes from ASL dataset. The results demonstrate the superiority of anomaly detection using m-Medoids based MMC-GFS and LMC-ES frameworks. The anomaly detection accuracies obtained using MMC-GFS algorith are higher than unimodal LMC-ES based anomaly detection algorithm. MMC-GFS and LMC-ES performs better than OCC-SVM, OCC-GMM, GWR and Mahalanobis framework-based Naftel's method. The superior performance of proposed approach as compared to state-of-the-art techniques is due to the fact that our approach gives importance to correct classification of normal sample 118 Recent Developments in Video Surveillance www.intechopen.com and to the filtration of abnormal samples during the model generation phase. On the other hand, OCC-SVM generates good model of normal classes but classifies many of the abnormal samples as member of normal classes whereas GWR gives extra importance to filtering abnormal samples and in the process, identifies many normal samples as abnormal. Fig. 11. Percentage anomaly detection accuracies for different number of classes from ASL dataset

Discussion and conclusions
In this chapter, we have presented an extended m-Medoids based framework, referred to as MMC-GFS, for modeling of trajectory-based motion patterns. The strength of the proposed approach is its ability to model complex patterns without imposing any restriction on the distribution of samples within a given pattern. Once the multimodal m-Medoids model for all the classes have been learnt, the classification of new trajectories and anomaly detection is then performed using a proposed soft classification and anomaly detection algorithm which is adaptive to multimodal distributions of samples within a pattern. The strength of this technique is its ability to model complex patterns without imposing any restriction on the shape of patterns. MMC-GFS can be used for modeling, classification and anomaly detection in any feature space with a computable similarity function.
Experimental results are presented to show the effectiveness of proposed MMC-GFS classifier.
Modeling of pattern and classification using proposed frameworks is unaffected by variation of sample distribution within a pattern as demonstrated in Fig. 4. Quantitative comparison of MMC-GFS based classifiers with competitive techniques demonstrates the superiority of our multimodal approach as it performs consistently better than commonly used Mahalanobis, GMM and HMM-based classifiers.

119
Behavior Recognition Using any Feature Space Representation of Motion Trajectories www.intechopen.com Experiments are also conducted to show the effectiveness of anomaly detection capabilities of proposed frameworks. Anomaly detection results for different classes of ASL datasets, using different variants of proposed anomaly detection algorithm, are presented. It has been shown that anomaly detection using multimodal MMC-GFS frameworks gives better anomaly detection accuracies as compared to the unimodal LMC-ES approach. Although LMC-ES enables the anomaly detection system to adapt to the normality distribution of individual classes, it is insensitive to the variation of distributions within a pattern which results in degradation of its performance as compared to MMC-GFS frameworks. Comparison of proposed anomaly detection algorithms with an existing approach demonstrates the superiority of our approach approach as they consistently perform better for different number of classes.
The application of proposed MMC-GFS based modeling and recognition system is not only limited to trajectory-based behavior recognition, but can also be applied to other recognition tasks that are critical in video surveillance application. Some of the applications where MMC-GFS based modeling and classification system can be applied include but is not limited to object recognition in surveillance videos, gait recognition, scene recognition etc.