Open access

A Regime Recognition Algorithm for Helicopter Usage Monitoring

Written By

David He, Shenliang Wu and Eric Bechhoefer

Published: January 1st, 2010

DOI: 10.5772/7165

Chapter metrics overview

3,585 Chapter Downloads

View Full Metrics

1. Introduction

The importance of regime recognition for structural usage monitoring of helicopters cannot be overemphasized. Usage monitoring entails determining the actual usage of a component on the aircraft. This allows the actual usage/damage from a flight to be assigned to that component instead of the more conservative worst-case usage. By measuring the actual usage on the aircraft, the life of the components can be extended to their true lifetime. Usage monitoring requires an accurate recognition of regimes, where a regime is the flight profile of the aircraft at each instant of the flight. For each regime, a damage factor is assigned to each component that has usage. These damage factors are assigned by the original equipment manufactures (OEMs) based on measured stresses in the aircraft when undergoing a given maneuver. Therefore, it is important that the regimes can be recognized correctly during the flight of the aircraft to avoid either underestimated or overestimated damages for the aircraft. Another important aspect of regime recognition is related to the certification of health usage and monitoring system (HUMS). As outlined in a document of the Federal Aviation Administration (FAA) HUMS R&D Initiatives (Le, 2006), regime recognition and monitoring has been identified as a high priority HUMS R&D short-term task in the area of structural usage monitoring and credit validation. The certification readiness and the aircraft applicability of regime recognition and credit validation are lower in comparison to the overall HUMS assessment (12% to 18% and 59% to 82%, respectively). These cited assessment results clearly show the weakness of current regime recognition methods in HUMS.

Although important, not much work on regime recognition has been published. Two recent research papers are worth mentioning here. The first paper (Teal et al., 1997) described a methodology for mapping aircraft maneuver state into the MH-47E basic fatigue profile flight regimes in a manner which ensures a conservative, yet realistic, assessment of critical component life expenditure. They also presented the use of wind direction and magnitude estimation and inertial/air data blending to obtain high fidelity airspeed estimation at low speeds. An accuracy rate of 90% based on time was reported. This method basically is a logical test. The system firstly identifies the maneuver based on flight dynamic data and general principles of tandem rotor helicopter flight which are derived from flight experience and mathematical models correlated with flight test results, then the aircraft maneuver state is mapped directly into one of the basic fatigue profile flight regimes. The method is subject to the main weakness of logical test in dealing with the noisy measurement. If the measured parameters were free of noise, logical tests would give accurate results. The second paper (Berry et al., 2006) presented a regime recognition scheme implemented as a hierarchical set of elliptical function (EBF) neural networks. Motivated to develop an automatic regime recognition capability as an enhancement to the US Army’s Vibration Management Enhancement Program (VMEP), the EBF neural networks were devised to simplify neural net training and to improve the overall performance. The idea of using a hierarchical set of neural networks is to group individual regimes into regime groups, including an unknown regime group (regimes that cannot be classified as any regimes in one of the known regime groups). Regimes in each group are classified by an individual net in the hierarchical set. Regime recognition is carried out through a hierarchical process, e.g., if a regime cannot be classified as the first regime group by the top net in the hierarchy it will be passed to the lower level nets for further classification. In the paper, a total of 141 regimes of Sikorsky’s S-92 helicopter were grouped into 11 groups, including “level flight”, “auto”, “climb”, “dive”, and etc. As shown in the paper, the EBF neural network regime recognition scheme gave near perfect classification results for “level flight” regimes. However, the results for classification of all regime groups didn’t show a consistent effectiveness of the scheme. For example, for “level flight” group the classification rate is 97.85% but it is 33.18% for “turns” group. Because of the low classification rates for some groups, the scheme gave an overall rate of 76.21%. In addition to the requirement for a large amount of data to train a neural network, one variable that could also affect the performance of the scheme is the way by which the regimes are grouped. Another limitation of neural network is that as it is a black-box methodology, little analytical insights can be gained to enhance the regime recognition process.

Regime recognition is basically a data mining problem, i.e., mining the measured parameter data and mapping them to a defined flight profile. In this paper, the philosophy of data mining is adopted for regime recognition. In particular, a regime recognition algorithm developed based on hidden Markov model (HMM) is presented.

Advertisement

2. Regime recognition algorithm

Before presenting the data mining based approach for regime recognition, we first describe the regime recognition problem from a data mining perspective as following. Suppose we have Q regimes, denoted as ω = { ω 1 ω 2 ... ω i ... ω Q } By taking into account the time factor in regime recognition, each individual regime at time t is expressed as s ( t ) = ω i Given an observation sequence R = R 1 R 2 ... R t ... R T , where T is the length of observations in the sequence and each observation R t is a 1 × O vector, denoted as R t = { f t 1 f t 2 ... f t j ... f t O } T with f t j being the value of feature j of the tth observation and O the number of the features, the objective is to identify regime sequence denoted as Ω = { s ( 1 ) s ( 2 ) ... s ( T ) }

Accordingly, a hidden Markov model λ = ( π A B ) for regime recognition could be characterized as follows:

  1. The initial regime distribution π = { π i } where π i = P [ s ( 0 ) = ω i ]

    1 i Q E1

  2. The regime transition probability distribution A = { a i j } where a i j = P [ s ( t + 1 ) = ω j | s ( t ) = ω i ]

    1 i j Q E2

  3. The observation probability distribution in regime ω j B = { b j ( t ) } where b j ( t ) = P [ R t | s ( t ) = ω j ] 1 j Q

    1 t T E3

The estimation of λ = ( π A B ) is a crucial step if we want to compute the probability of a system in regime ω i based on the estimated HMM model λ . Generally, there are two main stages in regime recognition using HMM. The first stage is the training stage. The purpose of training stage is to estimate the three parameters of the HMM. The estimation of λ = ( π A B ) is carried out through an iterative learning process of adjusting the model parameters to maximize the probability P ( R t r a i n | λ ) of producing an observation sequence R t r a i n = R 1 R 2 ... R T 1 given model λ . Therefore, at the end of training process, we could obtain an estimated HMM model λ i = ( π i A i B i ) for each regime ω i . The second stage is the testing stage. The purpose of testing is to calculate the probability of generating the unknown observation sequence, given the estimated model λ i    1 i Q . Given a testing observation sequence R t e s t = R 1 R 2 ... R T 2 and a set of estimated models λ = { λ i = ( π i A i B i )    1 i Q } , log-likelihood LL of R t from the observation sequence R t e s t = R 1 R 2 ... R T 2 can be computed. Note that, in general, T 1 T 2 .

2.1. The training stage

The training stage is a process to determine model parameters from a set of training data. A priori values of π, A, and B are assumed and observations are presented iteratively to the model for estimation of parameters. Likelihood maximization is the basic concept behind this estimation procedure. In each iteration, the goal is to maximize the expected log-likelihood, i.e., logarithm of the probability that the model generates the observation sequence. This iterative process continues until the change in log-likelihood is less than some threshold and convergence is declared.

In an HMM, the observation probability is assumed to follow a Gaussian distribution. Although, all of the classical parametric densities are unimodal, many practical problems involve multimodal densities. In our algorithm, a Gaussian mixture model (GMM) is used. Let X = [ x 1 x 2 ... x k ... x T ] be the sample dataset and x k is a O × 1 vector. Let M be the number of mixture components. There is no definition for the number of mixture components per output distribution and there is no requirement for the number of mixture components to be the same in each distribution. If M = 1 , it is the unimodal density case. When M 1 , a mixture model can be expressed as:

p ( X ) = i = 1 M w i h ( X | μ i σ i ) E4

where p ( X ) is the modeled probability distribution function and w i is the mixture weight of component i. Clearly, w 1 + w 2 + ... + w M = 1 and 0 w i 1 for all i = 1 2 M h ( X | μ i σ i ) is a probability distribution parameterized by μ i σ i , and can be computed as:

h ( X | μ i σ i ) = e 1 2 d i 2 ( 2 π ) O 2 | σ i | E5

where d i 2 can be computed as:

d i 2 = [ d i 1 2 d i 2 2 ... d i k 2 ... d i T 2 ] t E6
d i k 2 = ( x k μ i ) t σ i 1 ( x k μ i )
k = 1 2 ... T E7
Once w , μ and σ are determined, p ( X ) is defined, i.e., the observation probability distribution B. So the estimation problem of an HMM model λ = ( π A B ) is converted to estimate λ = ( π A μ σ w ) , where μ = { μ 1 μ 2 ... μ M } , σ = { σ 1 σ 2 ... σ M } , w = { w 1 w 2 ... w M } . Here, the GMM parameters of each HMM model can be split into two groups: the untied parameters that are Gaussian-specific and the tied parameters that are shared among all the Gaussians of all the HMM states.

Although there is no optimal way of estimating the model parameters so far, local optimal is feasible using an iterative procedure such as the Baum-Welch method (or equivalently the expectation-modification method) (Rabiner, 1989, Levinson et al., 1983), or using gradient techniques (Dempster, 1977). In order to facilitate the computation of learning, three forward-backward variables are defined in the forward-backward algorithm:

  1. The probability of the partial observation sequence, R 1 R 2 ... R t , and regime ω i at time t, given model λ :

    α t ( i ) = P [ R 1 R 2 ... R t s ( t ) = ω i | λ ] E8

  2. The probability of the partial observation sequence from t+1 to the end, given regime ω i at time t and model λ :

    β t ( i ) = P [ R t + 1 R t + 2 ... R T | s ( t ) = ω i λ ] E9

  3. The probability of being in regime ω i at time t, and regime ω j at time t+1, given model λ and observation sequence R t , i.e.,

    ξ t ( i j ) = P [ s ( t ) = ω i s ( t + 1 ) = ω j | R t λ ] E10

We initialize forward variable as α t = 0 ( i ) = 0 , for all 1 i Q at time t = 0 then in the forward iteration, we calculate the forward variable α t ( j ) by the following equations from t = 1 to t = T:

α t + 1 ( j ) = [ i = 1 Q α t ( i ) a i j ] b j ( R t + 1 ) w h e r e : 1 t T 1 1 j Q E11

In the backward iteration, we compute the backward variable β t ( i ) and ξ t ( i j ) after initialization β T ( i ) = 1 at time t = T:

β t ( i ) = j = 1 Q a i j b j ( R t + 1 ) β t + 1 ( j ) w h e r e : t = T 1 T 2 ... 1 1 i Q E12
ξ t ( i j ) = α t ( i ) a i j b j ( R t + 1 ) β t + 1 ( j ) P ( R t r a i n | λ ) = α t ( i ) a i j b j ( R t + 1 ) β t + 1 ( j ) i = 1 Q j = 1 Q α t ( i ) a i j b j ( R t + 1 ) β t + 1 ( j ) w h e r e 1 i j Q E13

We define γ t ( i ) as the probability of being in regime ω i at time t, given the observation sequence and the model. Therefore, γ t ( i ) = j = 1 Q ξ t ( i j ) . So, a i j , the estimated probability of a regime transition from s ( t 1 ) = ω i to s ( t ) = ω j , can be calculated by taking the ratio between the expected number of transitions from ω i to ω j and the total expected number of any transitions from ω i can be computed as:

a i j = t = 1 T 1 ξ t ( i j ) t = 1 T 1 γ t ( i ) 1 i Q 1 j Q E14

Then the estimated observation probability distribution ( μ j σ j w j ) in regime ω j can be computed as:

μ j = t w t j E t j [ R t r a i n ] t w t j E15
σ j = t w t j E t j [ R t r a i n R t r a i n ' ] t w t j μ j μ j ' E16

Note in equations (8)- (9), weights w t j = t = 1 T 1 γ t ( j ) are posterior probabilities, E t j [ R t r a i n ] is the mathematical expectation of observation R t r a i n at time t under regime ω j , μ ^ j σ ^ j are both untied parameters of GMM, and the covariance type is full.

Similarly, the estimated initial regime distribution can be computed as:

π i = γ 1 ( i ) 1 i Q E1

2.2. The testing stage

As mentioned before, testing is a stage to evaluate the likelihood of an unknown observation belonging to a given regime. Since model λ ^ i = ( π i A i μ i σ i w i ) has been built up in the training process, it can be used to calculate the log-likelihood LL of a testing observation R t e s t based on model λ ^ i for regime ω i . This log-likelihood can be calculated efficiently using the forward algorithm.

The probability that model λ ^ i produces observation R t e s t is computed as:

P ( R t e s t | λ i ) = α ( i ) E2

By definition, α ( i ) is the probability of generating R t e s t and ending in regime ω i , therefore,

P ( R t e s t | λ i ) = α ( i ) = P [ s ( t ) = i ] P [ R t e s t | s ( t ) = i ] = π i p i ( R t e s t ) 1 i Q E3

In (11), p i ( R t e s t ) can be calculated from equation (1), and R t e s t in regime recognition is the unknown signal. So, the log-likelihood value is computed as:

L L i = Ln [ P ( R t e s t | λ i ) ] E4

To classify a testing observation into one of Q regimes, train Q HMMs, one per regime, and then compute the log-likelihood that each model gives to the testing observation, a set of log-likelihood value L L = { L L 1 L L 2 ... L L i ... L L Q } will be obtained.

Advertisement

3. Algorithm validation

In this section, the developed regime recognition algorithm was validated using the Army UH-60L flight card data.

The Army UH-60L flight card data was collected during a flight test and provided by Goodrich. The intent of the flight test was to provide flight data which could be used to refine and revise a preliminary set of regime recognition algorithms. The test pilots annotated detailed flight cards with actual event times as maneuvers were conducted during the UH-60L regime recognition flights. The on-board pilots maintained a detailed log of the maneuvers, flight conditions, and corresponding event times encountered during the mission flight. A total of 50 regimes were conducted with annotation in the flight test. A limited amount of non-annotated actual flight data was used prior to the flight test to check the functionality of the HUMS system. The recorded data was downloaded and processed after the flight test.

For the Army UH-60L helicopter, a total of 90 preliminary regimes were defined by original equipment manufacturer (OEM). Data of 22 basic aircraft parameters were collected from sensors mounted on the aircraft, or sensors added to the Goodrich IMD-HUMS system for regime recognition. These parameter data is used for the identification of events, control reversals, and regimes. The parameter monitoring is performed during the whole ground-air-ground (GAG) cycles, from rotor start to rotor shutdown, and takeoff to landing. Table 1 provides the list of parameters with their description collected from IMD-HUMS system for regime recognition.

During the validation process, the dataset was randomly divided into two subsets: 70% of data was used for training and 30% for testing. By using the training data, an HMM model was built for each regime. Then the testing data was input into the trained HMM models to compute the log-likelihood values. The maximum log-likelihood value indicates the identified regime. The confusion matrix generated during the testing is provided in Table 2. From the results in Table 2, we see that the overall accuracy of the regime recognition is 99%.

Note that in solving regime recognition using HMM, the training set is dependent on the time sequence of maneuvers. Thus, it is able to find the regime or very complex grouped maneuvers. On the down side, the training set is really too small to capture all of the various maneuvers sequences that could be encountered. For example, it likely that from straight and level, you could go in to a left turn, or a right turn. From a right turn, you can go back to level, climbing right turn, diving right turn, or a higher angle of back turn. In reality, a flight card should contain all of the mixed mode maneuvers.

Parameter No. Parameter Name Parameter Description
1 WowDly WOW Delayed
2 LngFlg Landing Flag
3 TkOFlg Takeoff Flag
4 RollAt Roll Attitude
5 PtchAt Pitch Attitude
6 RdAlt Radar Altitude
7 YawDt Yaw Rate
8 AltDt Altitude Rate
9 LatDt2 Lateral Acceleration
10 VertAccl Vertical Acceleration
11 MrRpm RPM of Main Rotor
12 CrNz Corrected Acceleration
13 CalSpd Calibrated Airspeed
14 Vh Airspeed Vh Fraction
15 TGT Turbine Gas Temperature
16 RMS_Nz RMS Normal Acceleration
17 TEngTrq Torque 1/Torque 2
18 AOB Angle of Bank
19 CR_Pedal Control Reversal Flag
20 Cr_Colct Corrected Collective Rate
21 Cr_Lat Corrected Latitude
22 Cr_Lon Corrected Longitude

Table 1.

Monitored parameters in IMD-HUMS system.

Regime 2 3 4 5 7 8 9 10 11 12 13 14 15 16
2 1 1 1 1 1 1 1 1 1 1 1 1 1 1
3 1 0.99 1 0.99 1 1 1 1 1 1 1 1 1 1
4 1 1 0.99 1 1 1 1 1 1 1 1 1 1 1
5 0.05 0.66 0.26 0.97 1 1 1 1 1 1 1 1 1 1
7 1 1 1 1 0.99 0.5 1 1 1 1 1 1 1 1
8 1 1 1 1 0.95 1 1 1 1 1 1 1 1 1
9 1 1 1 1 1 1 0.99 1 1 1 1 1 1 1
10 1 1 1 1 1 1 0.97 1 0.98 1 0.93 0.99 1 1
11 1 1 1 1 1 1 1 1 1 0.93 0.91 0.98 1 1
12 1 1 1 1 1 1 1 1 1 0.99 0.87 0.96 1 1
13 1 1 1 1 1 1 1 1 1 0.99 1 1 1 1
14 1 1 1 1 1 1 1 1 1 1 1 0.96 1 1
15 1 1 1 1 1 1 1 1 1 1 1 1 0.98 1
16 1 1 1 1 1 1 1 1 1 1 1 1 0.93 0.9
17 1 1 1 1 1 1 1 1 1 1 1 1 0.98 1
19 1 1 1 1 1 1 1 1 1 1 1 1 1 1
20 1 1 1 1 1 1 1 1 1 1 1 1 1 1
21 1 1 1 1 1 1 1 1 1 1 1 1 1 1
22 1 1 1 1 1 1 1 1 1 1 1 1 1 1
23 1 1 1 1 1 1 1 1 1 1 1 1 1 1
24 1 1 1 1 1 1 1 1 1 1 1 1 1 1
25 1 1 1 1 1 1 1 1 1 1 1 1 1 1
26 1 1 1 1 1 1 1 1 1 1 1 1 1 1
27 1 1 1 1 1 1 1 1 1 1 1 1 1 1
28 1 1 1 1 1 1 1 1 1 1 1 1 1 1
36 1 1 1 1 1 1 1 1 1 1 1 1 1 1
37 1 1 1 1 1 1 1 1 1 1 1 1 1 1
40 1 1 1 1 1 1 1 1 1 1 1 1 1 1
41 1 1 1 1 1 1 1 1 1 1 1 1 1 1
42 1 1 1 1 1 1 1 1 1 1 1 1 1 1
43 1 1 1 1 1 1 1 1 1 1 1 1 1 1
44 1 1 1 1 1 1 1 1 1 1 1 1 1 1
45 1 1 1 1 1 1 1 1 1 1 1 1 1 1
46 1 1 1 1 1 1 1 1 1 1 1 1 1 1
48 1 1 1 1 1 1 1 1 1 1 1 1 1 1
49 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Table 2.

The confusion matrix of validation test.

Regime 2 3 4 5 7 8 9 10 11 12 13 14 15 16
50 1 1 1 1 1 1 1 1 1 1 1 1 1 1
51 1 1 1 1 1 1 1 1 1 1 1 1 1 1
52 1 1 1 1 1 1 1 1 1 1 1 1 1 1
53 1 1 1 1 1 1 1 1 1 1 1 1 1 1
54 1 1 1 1 1 1 1 1 1 1 1 1 1 1
55 1 1 1 1 1 1 1 1 1 1 1 1 1 1
56 1 1 1 1 1 1 1 1 1 1 1 1 1 1
57 1 1 1 1 1 1 1 1 1 1 1 1 1 1
59 1 1 1 1 1 1 1 1 1 1 1 1 1 1
60 1 1 1 1 1 1 1 1 1 1 1 1 1 1
61 1 1 1 1 1 1 1 1 1 1 1 1 1 1
63 1 1 1 1 1 1 1 1 1 1 1 1 1 1
64 1 1 1 1 1 1 1 1 1 1 1 1 1 1
65 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Table 2.

The confusion matrix of validation test (Continued 1).

Regime 41 42 43 44 45 46 48 49 50 51 52 53 54 55
2 1 1 1 1 1 1 1 1 1 1 1 1 1 1
3 1 1 1 1 1 1 1 1 1 1 1 1 1 1
4 1 1 1 1 1 1 1 1 1 1 1 1 1 1
5 1 1 1 1 1 1 1 1 1 1 1 1 1 1
7 1 1 1 1 1 1 1 1 1 1 1 1 1 1
8 1 1 1 1 1 1 1 1 1 1 1 1 1 1
9 1 1 1 1 1 1 1 1 1 1 1 1 1 1
10 1 1 1 1 1 1 1 1 1 1 1 1 1 1
11 1 1 1 1 1 1 1 1 1 1 1 1 1 1
12 1 1 1 1 1 1 1 1 1 1 1 1 1 1
13 1 1 1 1 1 1 1 1 1 1 1 1 1 1
14 1 1 1 1 1 1 1 1 1 1 1 1 1 1
15 1 1 1 1 1 1 1 1 1 1 1 1 1 1
16 1 1 1 1 1 1 1 1 1 1 1 1 1 1
17 1 1 1 1 1 1 1 1 1 1 1 1 1 1
19 1 1 1 1 1 1 1 1 1 1 1 1 1 1
20 1 1 1 1 1 1 1 1 1 1 1 1 1 1
21 1 1 1 1 1 1 1 1 1 1 1 1 1 1
22 1 1 1 1 1 1 1 1 1 1 1 1 1 1
23 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Table 2.

The confusion matrix of validation test (Continued 2).

Regime 41 42 43 44 45 46 48 49 50 51 52 53 54 55
24 1 1 1 1 1 1 1 1 1 1 1 1 1 1
25 1 1 1 1 1 1 1 1 1 1 1 1 1 1
26 1 1 1 1 1 1 1 1 1 1 1 1 1 1
27 1 1 1 1 1 1 1 1 1 1 1 1 1 1
28 1 1 1 1 1 1 1 1 1 1 1 1 1 1
36 1 1 1 1 1 1 1 1 1 1 1 1 1 1
37 1 1 1 1 1 1 1 1 1 1 1 1 1 1
40 1 1 1 1 1 1 1 1 1 1 1 1 1 1
41 1 1 1 1 1 1 1 1 1 1 1 1 1 1
42 1 0.99 1 1 1 1 1 1 1 1 1 1 1 1
43 1 1 0.99 1 1 1 1 1 1 1 1 1 1 1
44 1 1 1 0.98 1 1 1 1 1 1 1 1 1 1
45 1 1 1 1 0.99 1 1 1 1 1 1 1 1 1
46 1 1 1 1 1 1.00 1 1 1 1 1 1 1 1
48 1 1 1 1 1 1 0.99 1 1 1 1 1 1 1
49 1 1 1 1 1 1 1 1 1 1 1 1 1 1
50 1 1 1 1 1 1 0.97 1 0.97 1 1 1 1 1
51 1 1 1 1 1 1 1 1 1 0.99 1 1 1 1
52 1 1 1 1 1 1 1 1 1 1 0.93 1 1 1
53 1 1 1 1 1 1 1 1 1 1 1 0.96 1 1
54 1 1 1 1 1 1 1 1 1 1 1 1 0.97 1
55 1 1 1 1 1 1 1 1 1 1 1 1 1 0.98
56 1 1 1 1 1 1 1 1 1 1 1 1 1 1
57 1 1 1 1 1 1 1 1 1 1 1 1 1 1
59 1 1 1 1 1 1 1 1 1 1 1 1 1 1
60 1 1 1 1 1 1 1 1 1 1 1 1 1 1
61 1 1 1 1 1 1 1 1 1 1 1 1 1 1
63 1 1 1 1 1 1 1 1 1 1 1 1 1 1
64 1 1 1 1 1 1 1 1 1 1 1 1 1 1
65 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Table 2.

The confusion matrix of validation test (Continued 3).

Regime 56 57 59 60 61 63 64 65 ?
2 1 1 1 1 1 1 1 1 1
3 1 1 1 1 1 1 1 1 1
4 1 1 1 1 1 1 1 1 1
5 1 1 1 1 1 1 1 1 1.00
7 1 1 1 1 1 1 1 1 1
8 1 1 1 1 1 1 1 1 1
9 1 1 1 1 1 1 1 1 1
10 1 1 1 1 1 1 1 1 1.00
11 1 1 1 1 1 1 1 1 1.00
12 1 1 1 1 1 1 1 1 1.00
13 1 1 1 1 1 1 1 1 1
14 1 1 1 1 1 1 1 1 1.00
15 1 1 1 1 1 1 1 1 1
16 1 1 1 1 1 1 1 1 1.00
17 1 1 1 1 1 1 1 1 1
19 1 1 1 1 1 1 1 1 1
20 1 1 1 1 1 1 1 1 1
21 1 1 1 1 1 1 1 1 1
22 1 1 1 1 1 1 1 1 1
23 1 1 1 1 1 1 1 1 1
24 1 1 1 1 1 1 1 1 1
25 1 1 1 1 1 1 1 1 1
26 1 1 1 1 1 1 1 1 1
27 1 1 1 1 1 1 1 1 1.00
28 1 1 1 1 1 1 1 1 1.00
36 1 1 1 1 1 1 1 1 1
37 1 1 1 1 1 1 1 1 1
40 1 1 1 1 1 1 1 1 1
41 1 1 1 1 1 1 1 1 1
42 1 1 1 1 1 1 1 1 1
43 1 1 1 1 1 1 1 1 1
44 1 1 1 1 1 1 1 1 1
45 1 1 1 1 1 1 1 1 1
46 1 1 1 1 1 1 1 1 1
48 1 1 1 1 1 1 1 1 1
49 1 1 1 1 1 1 1 1 1
50 1 1 1 1 1 1 1 1 1.00

Table 2.

The confusion matrix of validation test (Continued 4)

Regime 56 57 59 60 61 63 64 65 ?
51 1 1 1 1 1 1 1 1 1
52 1 1 1 1 1 1 1 1 1.00
53 1 1 1 1 1 1 1 1 1.00
54 1 1 1 1 1 1 1 1 1.00
55 1 1 1 1 1 1 1 1 1
56 0.97 1 1 1 1 1 1 1 1.00
57 1 0.98 1 1 1 1 1 1 1
59 1 1 1.00 1 1 1 1 1 1
60 1 1 1 0.99 1 1 1 1 1
61 1 1 1 1 0.99 1 1 1 1
63 1 1 1 1 1 0.99 1 1 1
64 1 1 1 1 1 1 0.98 1 1
65 1 1 1 1 1 1 1 0.98 1
Overall Accuracy 0.99

Table 2.

The confusion matrix of validation test (Continued 5)

In addition to the validation test, the performance of the HMM based regime recognition algorithm was compared with a number of data mining methods. These data mining methods included: neural network, discriminant analysis, K-nearest neighbor, regression tree, and naïve bayes. The results of the performance comparison test are provided in Table 3. In this test, to be consistent, data with the same regimes were used for all the data mining methods. From Table 3, we can see that the HMM based regime recognition algorithm outperforms all other data mining methods.

Note that in Table 3, the names of methods are defined as: HMM = hidden Markov model; NN = neural network (back propagation); DA = discriminant analysis; KNN = k-nearest neighbor; RT = regression tree; NB = naïve bayes.

Advertisement

4. Conclusion

In this paper, a data mining approach is adopted for regime recognition. In particular, a regime recognition algorithm developed based on HMM was presented. The HMM based regime recognition involves two major stages: model learning process and model testing process. The learning process could be implemented off-board. In this process, Gaussian mixture model (GMM) was used instead of unimodal density of Gaussian distribution in HMM. Once the learning process is completed, new incoming unknown signal could be tested and recognized on-board. The developed algorithm was validated using the flight card data of an Army UH-60L helicopter. The performance of this regime recognition algorithm was also compared with other data mining approaches using the same dataset. Using the flight card information and regime definitions, a dataset of about 56,000 data points labeled with 50 regimes recorded in the flight card were mapped to the health and usage monitoring parameters. The validation and performance comparison results have showed that the hidden Markov model based regime recognition algorithm was able to accurately recognize the regimes recorded in the flight card data and outperformed other data mining methods.

Regime No. HMM NN DA KNN RT NB
2 0.01% 32.00% 16.00% 0.00% 12.00% 46.00%
3 0.10% 17.65% 23.53% 41.18% 100.00% 94.12%
4 0.06% 0.00% 84.62% 69.23% 100.00% 84.62%
5 10.30% 3.03% 0.00% 0.00% 39.39% 75.76%
7 2.51% 10.20% 5.10% 10.20% 3.06% 0.00%
8 0.26% 10.00% 6.67% 43.33% 0.00% 53.33%
9 0.04% 18.75% 62.50% 56.25% 100.00% 100.00%
10 0.66% 50.00% 50.00% 35.71% 100.00% 100.00%
11 1.02% 0.00% 26.67% 20.00% 13.33% 66.67%
12 0.92% 0.00% 0.00% 0.00% 100.00% 88.89%
13 0.05% 0.00% 0.00% 3.45% 100.00% 68.97%
14 0.21% 13.33% 6.67% 46.67% 100.00% 80.00%
15 0.09% 33.33% 33.33% 66.67% 100.00% 50.00%
16 0.88% 40.00% 100.00% 100.00% 100.00% 80.00%
17 0.12% 100.00% 100.00% 100.00% 100.00% 100.00%
19 0.11% 10.00% 0.00% 50.00% 100.00% 80.00%
20 0.09% 0.00% 0.00% 0.00% 0.00% 70.83%
21 0.09% 9.09% 36.36% 36.36% 100.00% 100.00%
22 0.01% 21.43% 14.29% 50.00% 100.00% 100.00%
23 0.07% 0.00% 9.09% 9.09% 100.00% 90.91%
Overall 0.88% 12.79% 15.58% 21.16% 47.44% 57.44%

Table 3.

Results of performance comparison of various data mining methods (regime recognition error rate).

References

  1. 1. Berry J. Vaughan R. Keller J. Jacobs J. Grabill P. Johnson T. 2006 Automatic Regime Recognition using Neural Networks. Proceedings of American Helicopter Society 60th Annual Forum, Phoenix, AZ, May 9 11 .
  2. 2. Dempster A. P. Laird N. M. Rubin D. B. 1977 Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society, 39 1 1 38 .
  3. 3. Le D. 2006 Federal Aviation Administration (FAA) Health and Usage Monitoring System R&D Initiatives. http://aar400.tc.faa.gov/Programs/agingaircraft/rotorcraft/ index.htm.
  4. 4. Levinson S. E. Rabiner L. R. Sondhi M. M. 1983 An Introduction to the Application of the Theory of Probabilistic Functions of a Markov Process to Automatic Speech Recognition. Bell System Technical Journal, 62 4 1035 1074 .
  5. 5. Rabiner L. R. 1989 A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceedings of the IEEE, 77 2 257 286 .
  6. 6. Teal R. S. Evernham J. T. Larchuk T. J. Miller D. G. Marquith D. E. White F. Deibler D. T. 1997 Regime Recognition for MH-47E Structure Usage Monitoring. Proceedings of American Helicopter Society 53rd Annual Forum, Virginia Beach, VA, April 29 - May 1, 1997.

Written By

David He, Shenliang Wu and Eric Bechhoefer

Published: January 1st, 2010