## 1. Introduction

Cellular Automata (CA) are spatiotemporal discrete systems (Neumann, 1966) that can model dynamic complex systems. A variety of problem domains have been reported to date in successful CA applications. In this regard, digital image processing is one of those as reported by Wongthanavasu et. al. (Wongthanavasu et al., 2003; 2004; 2007) and Rosin (Rosin, 2006).

Generalized Multiple Attractor CA (GMACA) is introduced for elementary pattern recognition (Ganguly et al., 2002; Maji et al., 2003; 2008). It is a promising pattern classifier using a simple local network of Elementary Cellular Automata (ECA) (Wolfram, 1994), called attractor basin that is a reverse tree-graph. GMACA utilizes a reverse engineering technique and genetic algorithm in ordering the CA rules. This leads to a major drawback of computational complexity, as well as recognition performance. There are reports in successful applications of GMACA in error correcting problem with only one bit noise. It shows the promising results for the restricted one bit noise, but becomes combinatorial explosion in complexity, using associative memory, when a number of bit noises increases.

Due to the drawbacks of complexity and recognition performance stated previously, the binary CA-based classifier, called Two-class Classifier Generalized Multiple Attractor Cellular Automata with artificial point (2C2-GMACA), is presented. In this regard, a pattern recognition of error correcting capability is implemented comprehensively in comparison with GMACA. Following this, the basis on CA for pattern recognition and GMACA’s configuration are presented. Then, the 2C2-GMACA model and its performance evaluation in comparison with GMACA are provided. Finally, conclusions and discussions are given.

## 2. Cellular Automata for Pattern Recognition

Elementary Cellular Automata (ECA) (Wolfram, 1994) is generally utilized as a basis on pattern recognition. It is the simplest class of one dimension (1d) CA with * n* cells, 2 states and 3 neighbors. A state is changed in discrete time and space (

i

^{th }cell) by considering it nearest neighbor (

For * n*-cell ECA, the next state function (

*) with size |*M

*x8| and the nearest neighbour configuration (*n

n

*-cell ECA (*n

R

_{ i }; where

*=0, 1, 2…, n-1. Each rule is represented in binary numbers (*i

b

_{7 }

b

_{6 }

b

_{5 }

b

_{4}

b

_{3}

b

_{2}

b

_{1}

b

_{0}). If the binary numbers are decoded into decimal, it must equal to the number

R

_{i}such as ‘01011010’ for the rule-90. Simultaneously, A rule matrix (M) can also be represented the rule vector.

Let * M*(

*) be an element of the matrix at the*i,j

i

^{th}(

*) row and the*i=0,1,2,...,n-1

j

^{th}(

*) column. The*j=0,1,2,...,7

*(*M

*) is contained*i,j

b

_{j}of the rule-

R

_{i}. For example,

*(2,3) is*M

b

_{3}of the rule

R

_{2}(the rule-90) that is ‘1’. Consequently, the next state (

i

^{ th }cell is represented by the

*(*M

*) as the following:*i,j

where;

^{th
} cell.

_{i
} is the 3 neighbouring values (
^{th
} cell decoded in decimal.

The next state
* n*-cell ECA calculated is also defined by the rule matrix

*as following:*M

Suppose a system designed with a rule matrix (* M*) comprises a set of solutions

*=*Y

*. Consequently, the pattern classifiers based on the evolution of the ECA is defined as following*1, 2…, N

For an input
* Y* using the equation (3). Firstly, the present state (

*until it reaches some solution (*M

## 3. Generalized Multiple Attractor Cellular Automata

This section gives the detailed configuration of GMACA and its application in ECC. Suppose * an n*-bit pattern is sent in a communication system. Let

*be the sender‘s pattern and*X

*be the receiver’s pattern. Thus, the number of different bits between*Y

*and*X

*is determined by Hamming distance (*Y

*) defined as follows:*r

where

The number of possible error patterns (_{r
}) for a given * r* of

*-bit communication can be expressed as follow:*n

Then, the number of all possible error patterns (_{All
}) for a given _{max
}, where _{max
}

The maximum permissible noise (_{max
}) is the highest value of * r* allowed to occur in the communication system. The Hamming distance model of a message (pattern) and it errors are also represented by an attractor basin—that is, the messages is a pivotal point while the errors are transient states. Thus, the error correcting codes can be solved by the Generalized Multiple Attractor Cellular Automata (GMACA).

Suppose a communication system comprises * k* original messages of

*-bit data and the maximum permissible noise*n

r

_{max}. If error messages are corrected using the GMACA, thus a satisfied rule vector is required. The rule vector is a result of a reverse engineering technique. Firstly,

*attractor basins are randomly constructed with the number of nodes for each attractor basin equals*k

p

_{All }. Then original messages are randomly mapped into pivotal points while its possible errors are also randomly mapped into transient states at the same attract basin. Finally, the search heuristics, such as simulated annealing (SA) and genetic algorithm (GA) (Holland, 1992; Shuai, et al., 2007; Jie, et al., 2002) have been taken to explore the optimal structure. The search heuristics then iteratively changes directions and height of the attractor basins until the satisfied rule vector is acquired.

As reported in Ganguly, et al., 2002, Maji, et al., 2003 and Maji, et al., 2008, the GMACA provides the best performance of pattern recognition if it is trained with the _{max
} having a value of 1. Although percentage of recognition in testing is high when deals with the _{max
} equals 1, it sharply decreases the recognition performance when the _{max
} is greater than 1.

## 4. Proposed 2C2-GMACA Model

Due to the drawbacks of recognition performance resulting from the increasing _{max
} and search space complexity in rule ordering, the proposed method, called Two-class Classifier Generalized Multiple Attractor Cellular Automata with artificial point (2C2-GMACA) (Ponkaew, et al., 2011; Ponkaew, et al., 2011), is introduced. The 2C2-GMACA is designed based on two class classifier architecture basis. In this regard, two classes are taken to process at a time and a solution is binary answer +1 or -1, which is a pointer to the class label of solution. There are two kinds of attractor basins: a positive attractor basin that returns the +1 as the result and a negative attractor basin, otherwise.

Suppose a system consists of patterns
^{th
} pattern
^{th
} class label and * i*=

*Let*1,2,…N.

*as parameters as the equation (7) to assign the class.*A)

where

^{th
} cell.

^{th
} cell.

^{th
} cell.

(.) denotes “AND” logical operator.

Finally, the * x* is considered to be a member of the positive attractor basin and returns

L

^{+ }if

* Example 1*: Consider two attractor basins of

*-bit recognizer of 2C2-GMACA with periodic boundary condition given in Fig. 2, they are designed by a rule vector <232,212,178,142> representing in a matrix*4

*, and an artificial point (*M

*) of ‘0001’. Suppose a class label of the positive (*A

L

^{+ }) and the negative attractor basins (

where _{i
} is the 3 neighbour values (
^{th
} cell decoded in decimal. That is, _{0}= (011)_{2}=3, _{1
}= (110)_{2}=6, _{2}= (100)_{2}=4 and _{3}= (001)_{2}=1. Thus, the above equation is replaced with the _{i
} in decimal as following

Finally, the binary decision function will process the
* A*=0001 as co-parameters resulting in the following

The function returns 1 meaning that the input

### 4.1. 2C2-GMACA with Associative and Nonassociative Memories

Given a set of patterns
* i*=1,2…,

*. 2C2-GMACA takes two patterns {*k

y

_{i }and

y

_{j }are generated using the equation (6) with the maximum permissible noise (

r

_{max}), while all transient states are randomly generated

y

_{i }and

y

_{j }are mapped into the leaf nodes of the positive and negative attractor basins, respectively. After two attractor basins are completely constructed, it will be synthesized by a majority voting technique to arrive at the rule vector. In other word, the rule vector is determined in only one time step which is different from GMACA in that it is iteratively determined through the evolution of heuristic search. In this regard, complexity is the main drawback excluding recognition performance.

According to a binary classifier, 2C2-GMACA conducts multiclass classification by DDAG (Decision Directed Acyclic Graph), One-versus-All, One-versus-One, etc., for example. However, this paper focuses on DDAG approach [28]. Suppose that a set of three patterns {y_{1}, y_{2}, y_{3}}, where
* i*=1, 2, 3, is constructed using the DDAG scheme. Thus, total number of binary classifier is (

0

^{th }-level. Then, (1 vs 2) and (2 vs 3) are contained in the

1

^{st }-level. Finally, the solutions {3, 2, 1} are labeled in the leaf nodes of the

2

^{nd }-level. In order to assign a class label for an unknown input

*is evaluated until it reaches final level. At this point, a leaf node connecting to the edge of the binary decision function is assigned as the solution.*x

### 4.2. Design of Rule Vector

A majority voting rule is utilized to synthesize a rule vector for two attractor basins. It is one time step process which is different from a reverse engineering technique (Maji, et al., 2003; Maji, et al., 2008) using in GMACA. Reverse engineering technique continues reconstructing attractor basins randomly until arriving at the rule vector with the lowest collision. In this regard, 2C2-GMACA’s time complexity for ordering the rule is simply O(1). However, it must search for an optimal artificial point which applies evolutionary heuristic search. The 2C2-GMACA synthesis scheme comprises three phases as follows.

* Phase I*--- Two attractor basins, namely, positive and negative attractor basins, are generated. In this phase, two patterns {

L

^{ + }. Thus, the

L

^{ - }. Then, transient states of the

* Example 1*: Fig. 3(a) represents two attractor basins based on associative memory learning of 4 bit patterns with

r

_{max }=1. Suppose

*={1101, 0010} is a set of learnt patterns. The 2C2-GMACA takes two patterns {*Y

y

_{1}=1101,

y

_{2 }=0010} to process according to the multiclass classification algorithm. Let a class label of the positive

r

_{max }=1 are generated resulting in {1101, 0101, 1001, 1111, 1100} and {0010, 1010, 0110, 0000, 0011}, respectively. Then, all patterns are mapped into leaf nodes of attractor basins corresponding with its label as shown in Fig. 3(a).

* Phase II*--- Let

^{th}cell is decoded in decimal satisfying the j

^{th}column. The negative attractor basin considers the

* Example 2*: As shown in Fig. 3(b), two matrices

and

1

^{st}row and the

1

^{st}column; it is a total number of leaf nodes from the positive attractor basin where 3 neighbors (

1

^{st}cell decoded in decimal equal to

*, i.e.*1

*=1=001*j

_{2}=(

_{2}where

*=1.*i

* Phase III*--- Rule matrix

*is determined. The matrix*M

*with size |nx8| is the simplified form of the rule vector (*M

*), while an element*RV

*(*M

*,*i

*) represents the next state for the*j

i

^{ th }cell, where the 3 neighbor

*. The*j

*is designed by comparing between*M

*=0,1,2,...,*i

*-1 and*n

*=0,1,2,...,7, due to the following conditions:*j

1) if

2) if

Fig. 3(c) shows that a rule vector <232, 212, 178, 142> is obtained by the majority voting technique. The rule vector (matrix rule) is utilized to evolve the given pattern in one time step to the pattern at the next time step which becomes one of parameters of the binary decision function.

### 4.3. Design of Artificial Point

An artificial point (A) takes a major role in the binary decision function. It interprets the next state (

Selection is done by using a random pairing approach and a traditional single point crossover is also performed by random at the same point of the * n* element array of the selected two parents. Mutation makes a small change in the bits in the list of a chromosome with a small percentage. The fitness function is calculated as a cost for each chromosome. It is created from a true positive (

*) and a false positive (*TP

*) of the confusion matrix (Simon, et al., 2010) calculated by the below equation (8). The fitness function is given as following*FP

The search space complexity for rule ordering of the 2C2-GMACA is the all possible patterns of the artificial point, * 000…000 to 111….111*, which is

2

^{ n }, i.e. O(

2

^{ n }).

## 5. Performance Evaluation

This section reports performance evaluation of the proposed method in comparison with GMACA on a set of measured matrices consisting of search space and classification complexities, recognition percentage, evolution time for rule ordering, and effects of the number of pivotal point, permissible noises, p-parameter, pattern size on error correcting problem.

### 5.1. Reduction of Search Space

Given a set of learnt patterns
* i*=1,2…,

*, is original messages. The 2C2-GMACA and GMACA based associative memory learning will generate all transient states using the equation (6) with the maximum permissible noise (*k

r

_{ max }). Then, the transient states are constructed to be attractor basins.

* Theorem 1:* In training phase, a search space complexity of the GMACA (

*)*n

*the maximum permissible noise (*,

r

_{ max }) and the maximum permissible height (

h

_{ max }), while the search space complexity of 2C2-GMACA (

n.

* Proof:* From the set

*attractor basins randomly until a satisfied rule vector is acquired. Thus, the search space of the GMACA (*k

S

_{ GMACA }) is all possible patterns of

*attractor basins defined by*k

where * G* is the number of learnt patterns in each attractor basin previously defined by Cayley ‘s formula (Maji, et al., 2003) as follows:

where * p* is the number of possible transient states calculated from (6). Therefore, the above equation is defined following

It shows that search space complexity of GMACA is factorial growth
* n* and

r

_{max}. In real world application, it must face a severe search space in which the search heuristics cannot reach the optimal solution if

*or*n

r

_{max}is considered at a high number. In this regard, GMACA tries to examine the optimal values of the

r

_{max}and

h

_{max}. GMACA shows that the search space complexity can be reduced to O(

n

^{n}) if the

r

_{max}=1 as shown following

The search space complexity in Maji, et al., 2003 and Maji, et al., 2008 is examined under the _{max}* =2* and the

r

_{max}=

*as described below.*1

For the proposed 2C2-GMACA, the search space is the number of possible patterns (* G*) of artificial point:

*—that is; 2*000…000 to 111….111

^{n}. Due to DDAG approach for multiclass classification algorithm, the machine consists of

*(*k

*)*k-1

*binary classifier. Thus, the search space complexity of the 2C2-GMACA (*/2

S

_{2C2-GMACA}) is:

When comparing the search space complexity between GMACA and 2C2-GMACA, we found that GMACA can only be implemented if it is considered at the _{max}* =2* and

r

_{max}=

*while 2C2-GMACA can be implemented whatsoever with the exact solution through heuristic search. This corresponds to the reports in Maji, et al., 2003 and Maji, et al., 2008, the GMACA provides the best performance of pattern recognition when it is trained with the*1,

r

_{max}=1 and

h

_{max}

*. However, the percentage of recognition in testing is also high if the Hamming distance of patterns is less than or equal to 1 and it is decreased sharply when the Hamming distance is greater than 1.*=2

### 5.2. Reduction of Classification Complexity

* Theorem 2:* In worst case scenario of learning based on associative memory model, the classification complexity of

*-bit pattern for GMACA is O(*n

n

^{2}), while 2C2-GMACA is O(

*).*n

* Proof:* In general, time spent in classifying

*nodes of GMACA depends on an arrangement of nodes in attractor basins. At worst, the attractor basin is a linear tree. Thus, time for classifying*n

*nodes is the summation of the number of traversal paths from each node to a pivotal point. For example, the number of traversal paths of a pivotal point is 0 while the*n

n

^{th}-node is (

*1). This can be solved by arithmetic series (*n-

*is 1 and an initial term (*d

a

_{ 1 }) is 0, the equation in determining the summation is given as follows.

As being designed the height of attractor basis of 2C2-GMACA is limited to 1, the time of classifying * n* nodes is

n

### 5.3. Performance Analysis of 2C2-GMACA on Associative Memory

Pattern classifiers based on an associative memory is independent from the number of patterns to be learnt, because all possible distorted patterns are generated into learning system. Suppose a set of pivotal points
* i*=1, 2…,

*is original messages. 2C2-GMACA takes two pivotal points {*k,

*,*l

*=1, 2…,*m

*, to process at a time using the DDAG scheme. Thus, the number of classifiers of the 2C2-GMACA is*k

#### 5.3.1. Recognition and Evolution Time

This section reports recognition rate and evolution time for rule ordering between 2C2-GMACA and GMACA based on associative memory. Table 1 presents the recognition rate at different sizes of bit patterns (* n*) and the number of attractor basins (

*). It generates patterns with maximum permissible noise in training phase (*k

r

_{max}) and testing with different sizes of noise

r;

*and*n

*. The results show that 2C2-GMACA is superior to GMACA both recognition performance and times spent in rule ordering. This corresponds the previous mention that search space is the major problem of GMACA for ordering the rules when deals with high number of*k

r

_{max }.

#### 5.3.2. Effects of Number of Pivotal Points and Pattern Size

A pivotal point in 2C2-GMACA represents an original message in communication systems. Fig. 4 shows the effects of the number of pivotal points (* k*) in the recognition performance of the proposed 2C2-GMACA based on associative memory learning at a particular

r

_{max}and bit pattern. It shows that if is trained by

r

_{max}= 3 the recognition rate is almost 100% when the number of bit noises (

*) is not greater than 5 no matter of the number of classes (*r

*), and declined sharply when the number of bit noises increases. The less the number of classes, the better the recognition performance. Fig. 5 shows the effects of the number of bit pattern in recognition performance of the 2C2-GMACA based on associative memory learning by fixing*k

r

_{max}and the number of classes (

*). In this regard, when the number of bit noises in testing increases, the recognition of different number of bit patterns decreases in distinguishable manner. The more the number of bit patterns, the less the recognition performance.*k

### 5.4. Performance Analysis of 2C2-GMACA on Non-Associative Memory

The memory capacity becomes a serious problem of pattern classifier based on an associative memory learning if the classfier deal with the high values of _{max
} and * k.* It generates a large number of transient states. In ordet to solve this problem, the 2C2-GMACA based on non-associtive memory is presented. The transient states will be generated by randomly choosing bit noise

#### 5.4.1. Effects of Maximum Permissible Noise and P-Parameter

In order to examine the effects of the maximum permissible noise _{max
} on the error correcting problem of 2C2-GMACA based non-associative memory, two pivotal points are randomly generated and then the number of transient states is limited to some number
* p*. This method is called uniform distribution learning. Fig. 6 shows the effects of the

r

_{max }at

*=100 and*n

*is bits pattern. The number of pivotal points (*n

*) and transient states (*k

*) is fixed to 2 and 2000, respectively. Results are plotted in the inverted bell curve. It shows that the 2C2-GMACA has the lowest capability in range of*p

r

_{max }

r

_{ max }=

r

_{max }

The effects of the number of transient states (* k*=2) are examined and shown in Fig. 7. During the training phase, the number of bit pattern (

*) is set to 100, while the maximum permissible noise (*n

r

_{max }) is set nearly to

*---that is 2000, 4000 and 10000. The results show that the average percentage of recognition is highest if it is trained with the highest number of*p

*. However, it is memory consumptions as already mentioned.*p

## 6. Conclusions and Discussions

This chapter presents a non-uniform cellular automata-based algorithm with binary classifier, called Two-class Classifier Generalized Multiple Attractor Cellular Automata with artificial point (2C2-GMACA), for pattern recognition. The 2C2-GMACA is built around the simple structure of evolving non-uniform cellular automata called attractor basin, and classify the patterns on the basis of two-class classifier architecture similar to support vector machines. To reduce computational time complexity in ordering the rules, 2C2-GMACA is limited the height of attractor basin to 1, while GMACA can have its height to n, where n is a number of bit pattern. Genetic algorithm is utilized to determine the CA’s best rules for classification. In this regard, GMACA designs one chromosome consists of k-genes, where k is a number of classes (target patterns) to be classified. This leads to abundant state spaces and combinatorial explosion in computation, especially when a number of bit noises increases. For the design of 2C2-GMACA, a chromosome represents an artificial point which is consists of n-bit pattern. Consequently, the state space is minimal and feasible in computation in general pattern recognition problem. The 2C2-GMACA reduces search space for ordering a rule vector from GMACA which is O(^{n}) to O(* 1*)+O(

2

^{n}). In addition, multiple errors correcting problem is empirically experimented in comparison between the proposed method and GMACA based on associative and non-associative memories for performance evaluation. The results show that the proposed method provides the 99.98% recognition rate superior to GMACA which reports 72.50% when used associative memory, and 95.00% and 64.30% when used non-associative memory, respectively. For computational times in ordering the rules through genetic algorithm, the proposed method provides 7 to 14 times faster than GMACA. These results suggests the extension of 2C2-GMACA to other pattern recognition tasks. In this respect, we are improving and extending the 2C2-GMACA to cope with complicated patterns in which state of the art methods, SVM, ANN, etc., for example, poorly report the classification performance, and hope to report our findings soon.

## References

- 1.
Neumann J.V. Theory of Self-Reproducing Automata. University of Illinois Press 1966 - 2.
Wongthanavasu S. Sadananda R. A CA-based edge operator and its performance evaluation. - 3.
Wongthanavasu S. Lursinsap C. A 3D CA-based edge operator for 3D images - 4.
Wongthanavasu S. Tangvoraphonkchai V. Cellular Automata-Based Algorithm and its Application in Medical Image Processing - 5.
Rosin P.L. Training cellular automata for image processing. - 6.
Wolfram S. Cellular automata and complexity: collected papers. Addison-Wesley Pub. Co. 1994 - 7.
Delgado, O.G., Encinas, L.H., White, S.H., Rey, A.M.d. and Sanchez, G.R., Characterization of the reversibility of Wolfram cellular automata with rule number 150 and periodic boundary conditions. - 8.
Das S. Chowdhury D. R. An Efficient n x n Boolean Mapping Using Additive Cellular Automata - 9.
Maji P. Shaw C. Ganguly N. Sikdar B.K. Chaudhuri P.P. Theory and application of cellular automata for pattern classification. - 10.
Chady M. Chady a. M. Poli R. Evolution of Cellular-automaton-based Associative Memories - 11.
Ganguly N. Maji P. Sikdar B.K. Chaudhuri P.P. Generalized Multiple Attractor Cellular Automata (GMACA) For Associative Memory. - 12.
Maji P. Ganguly N. Chaudhuri P.P. Error correcting capability of cellular automata based associative memory. - 13.
Maji P. Chaudhuri P.P. Non-uniform cellular automata based associative memory: Evolutionary design and basins of attraction. - 14.
Niloy G. Maji P. Sikdar B.K. Chaudhuri P.P. Design and characterization of cellular automata based associative memory for pattern recognition. - 15.
Tan S.K. Guan S.U. Evolving Cellular Automata to Generate Nonlinear Sequences with Desirable Properties. - 16.
Shannon C.E. Weaver W. The Mathematical Theory of Communication. - 17.
de Oliveira P.P.B. Bortot J.C. Oliveira G.M.B. The best currently known class of dynamically equivalent cellular automata rules for density classification. - 18.
Sipper M. Co-evolving Non-Uniform Cellular Automata to Perform Computations 1996 - 19.
Holland J.H. Adaptation in natural and artificial systems. MIT Press 1992 - 20.
Oh H. Application of a Heuristic Search to the Deadlock Avoidance Algorithm that Achieves the High Concurrency. - 21.
Shuai D. Dong Y. Shuai Q. A new data clustering. approach Generalized cellular automata. - 22.
Jie Y. Binggang C. Study on an Improved Genetic Algorithm. - 23.
Ponkaew J. Wongthanavasu S. Lursinsap C. A nonlinear classifier using an evolution of Cellular Automata, - 24.
Ponkaew J. Wongthanavasu S. Lursinsap C. Two-class classifier cellular automata - 25.
Buhmann J. Divko R. Schulten K. Associative memory with high information content - 26.
Simon D. Simon D.L. Analytic Confusion Matrix Bounds for Fault Detection and Isolation Using a Sum-of-Squared-Residuals Approach