Fuzzy Perceptron Learning for Non-Linearly Separable Patterns

Abstract

Perceptron learning has its wide applications in identifying interesting patterns in the large data repositories. While iterating through their learning process perceptrons update the weights, which are associated with the input data objects or data vectors. Though perceptrons exhibit their robustness in learning about interesting patterns, they perform well in identifying the linearly separable patterns only. In the real world, however, we can find overlapping patterns, where objects may associate with multiple patterns. In such situations, a clear-cut identification of patterns is not possible in a linearly separable manner. On the other hand, fuzzy-based learning has its wide applications in identifying non-linearly separable patterns. The present work attempts to experiment with the algorithms for fuzzy perceptron learning, where perceptron learning and fuzzy-based learning techniques are implemented in an interfusion manner.

1. Introduction

A learning system could be thought as a collection of methods that are brought together in order to create an environment to facilitate different learning processes. The learning systems will provide various types of learning resources and descriptions of procedures for obtaining quality results [1]. The learning systems find their applications in the areas like, image recognition, speech recognition, traffic prediction, e-mail spam and malware filtering, automatic language translation, medical diagnosis, etc. [2].

As the data increases in large volumes in the digital repositories, it has become essential to look for alternative approaches to yield better results in extracting interesting patterns from the repositories. Intelligent learning systems are gaining attention from a wide range of researchers in the recent years in extracting patterns from the data repositories. The learning systems have three kinds of approaches. They are supervised, unsupervised, and semi-supervised learning approaches [3].

The concept of perceptron learning plays a critical role in pattern recognition, which has become a challenging problem in the data science research. In the recent years, perceptron learning algorithms are exhibiting their robust performance in identifying interesting patterns from large data repositories when compared to the traditional supervised learning approaches [4]. A perceptron can be thought as a computational prototype of a neuron. As a supervised learning approach, perceptron learning is used for linear classification of patterns. This learning approach uses the already available labelled data to classify the future data by predicting the class labels.

In the literature, it is studied that many researchers experimented with perceptron learning for identifying interesting patterns from the data. A novel autonomous perceptron model (APM) was proposed to address the issues of complexity of traditional perceptron architectures [4]. APM is a nonlinear supervised learning model, which has the architecture using the computational power of the quantum bits (qubits). The researchers [5], using biophysical perceptron (BP), tried to simulate the pyramidal cells in the brain with a wide variety of active dendritic channels. The BP, here, explores the ability of real neurons with extended non-linear dendritic trees to effectively perform the classification task in identifying interesting patterns from the data. Many researchers have experimented with perceptron learning in a wide variety of ways. However, the perceptron learning suffers several limitations. It works well for linearly separable patterns. Though some researchers experimented for identifying non-linearly separable patterns, the perceptron learning produced best results for binary separation of patterns only [5]. Also that perceptron learning suffers poor performance in case of overlapping patterns, that is, when patterns are not having sharp boundaries.

Fuzzy-based learning, on the other hand, is found to show its ability in performing well for overlapping patterns [6]. As a fuzzy-based learning approach, fuzzy C-means (FCM) is widely used by researchers for pattern recognition. A weighted local fuzzy regression model showed a better efficiency than the least squares regression for non-linear and high-dimensional pattern recognition of transport system in China [7]. The new kernelized fuzzy C-means clustering algorithm [8] uses a kernel-induced distance function as a similarity measure showed improved performance in identifying the patterns when compared to the conventional fuzzy C-means technique. In many research findings, it is observed that the fuzzy-based learning approach was used in a wide variety of ways to achieve better results in extracting non-linear and overlapping patterns.

The present work attempts to experiment with fuzzy perceptron learning, which implements the perceptron learning and fuzzy-based learning techniques in an interfusion manner. In the research literature, we can find a good amount of work related to the combination of fuzzy logic with perceptron learning. The fuzzy neural network (FNN) was proposed for pattern classification, which uses supervised fuzzy clustering and pruning algorithm to determine the precise number of clusters with proper centroids representing the patterns to be recognised [9]. In the fuzzy neural integrated networks [10], the researchers attempted to integrate the concept of fuzzy sets and neural networks to deal with pattern recognition problems. In an enhanced algorithm for fuzzy lattice reasoning (FLR) classifier, a new nonlinear positive valuation function was defined to produce better results for pattern classification [11]. Along with these, however, many other research experiments of fuzzy perceptron learning are supervised learning approaches only. Therefore, the present work focuses on experimenting with effective implementation of some techniques involved in the perceptron and fuzzy-based learning systems for unsupervised learning to identify interesting patterns in large datasets. As part of the present work, five algorithms are developed, two of which are related to perceptron learning, one is the standard fuzzy C-means (FCM) algorithm. The remaining two algorithms are proposed by the present work, which implement the perceptron learning and fuzzy-based learning in an interfusion manner using weights and weighted distances respectively. All the algorithms are implemented using three benchmark datasets. The CPU time, clustering fitness (CF), and sum of squared errors (SSE) are taken into consideration for performance evaluation of the algorithms.

2. Perceptron learning

Nowadays, the perceptron learning model can be thought as a more general computational model in identifying interesting patterns in a dataset. It takes an input, aggregates it along with the weights and produces the result. A perceptron is used to learn patterns and relationships in data. Patterns help us knowing about the interesting features around which objects may be grouped in a given population of data.

A perceptron may be configured for a specific application, such as pattern recognition and data classification through some learning process [12]. Perceptrons are information processing devices, which are built from interconnected elementary processing units. These units are called neurons. The perceptrons are robust in exhibiting their ability in distributed representation and computation, learning, generalisation, adaptivity, inherent contextual information processing, and fault tolerance [13].

The perceptron learning uses an iterative weight adjustment for the enhanced retrieval of patterns from a dataset. The iterative process converges to the weights, which produce the patterns that represent the different groups of data objects in the dataset uniquely. While operating for learning on patterns, the perceptrons use weights in connection to every input vector. A weight represents the information used by the perceptron to solve a problem [14].

The perceptron with multiple neurons is shown in Figure 1.

Figure 1.

A perceptron with a multiple neurons.

In Figure 1, X₁, X₂, …, X_n are the n input vectors and Y₁, Y₂, …, Y_m are the m neurons. The input vector X₁ is connected to neurons Y₁, Y₂, …, Y_m with weights W₁₁, W₁₂, …, W_1m, respectively, the input vector X₂ is connected to the neurons with weights W₂₁, W₂₂, …, W_2m, respectively so on and the input vector X_n is connected to the neurons with the weights W_n1, W_n2, …, W_nm, respectively. The weights of all input vectors for all neurons will be formulated as the weight matrix as shown below.

$$\left[\begin{array}{ccccc}{W}_{11}& W_{12}& \dots & \dots & W_{1m}\\ W_{21}& W_{22}& \dots & \dots & W_{2m}\\ \dots & \dots & \dots & \dots & \dots \\ \dots & \dots & \dots & \dots & \dots \\ \dots & \dots & \dots & \dots & \dots \\ W_{n1}& W_{n2}& \dots & \dots & W_{\mathit{nm}}\end{array}\right]$$

Though the perceptron learning exhibits its robustness in identifying the patterns in the data repositories, it works well for linearly separable patterns, that is, the patterns with sharp boundaries only. However, in the real time world, we may find overlapping patterns, that is, non-linearity in pattern associativity, where data objects may associate with multiple patterns. In such situations, the perceptron learning approach may suffer in identifying the patterns clearly. On the other hand, fuzzy-based learning has its wide applications in identifying patterns in the overlapping scenario. In the present work, two algorithms are implemented for perceptron learning. They are discussed in the following sub-sessions.

2.1 Perceptron learning using weights (PLW)

This algorithm implements the perceptron learning using weights [12]. With each input data vector, a weight is associated corresponding to each pattern. To generate the initial weights, one iteration of K-means algorithm is performed. The results of K-means iteration are used to compute the weight matrix. This weight matrix will be repeatedly updated in the subsequent iterations. For each input data vector weights are computed corresponding to every pattern. The input data vector is associated with the pattern corresponding to which the weight is maximum. This process is repeated for every iteration. The algorithm terminates when there is no change in the association of data vectors to the patterns. The algorithm for perceptron learning using weights is given below.

2.1.1 Algorithm PLW

Step 1: Determine the number of patterns, k, to be recognised from the dataset.

Step 2: Select k points randomly from the dataset and set them as cluster seeds to correspond the patterns to be recognised.

Step 3: Perform one iteration of K-means algorithm.

Step 4: Using the results of K-means iteration, compute cluster wise initial weights.

Step 5: Repeat steps 6–8 until the stopping condition.

Step 6: Generate weight matrix, where each element W_ij is computed as:

$${\mathrm{W}}_{\mathrm{ij}}\left(\mathrm{i}+1\right)={\mathrm{W}}_{\mathrm{ij}}\left(\mathrm{i}\right)+\mathrm{lr}\ast \left(\Vert {\mathrm{X}}_{\mathrm{i}}\Vert –{\mathrm{W}}_{\mathrm{ij}}\left(\mathrm{i}\right)\right)$$E1

Here, W_ij(i + 1) is the weight of ith data point X_i for jth cluster for the iteration (i + 1), W_ij(i) is the weight of ith data point for jth cluster for the iteration i, ||X_i|| is the norm of data point X_i, and lr is the learning rate. The lr may assume a value ranging between 0 and 1. To avoid possible biasedness in the computations, lr is assumed to be 0.5.

Step 7: Assign points to clusters using weights.

Step 8: Update cluster means, that is, refine patterns.

[End of step 5 loop]

Step 9: [End of algorithm]

2.2 Perceptron learning using weighted distances (PLWD)

This algorithm implements the perceptron learning using weighted distances [15]. With each input data vector, a weighted distance is associated corresponding to each pattern. To generate the initial weighted distances, one iteration of K-means algorithm is performed. Using the results of K-means the weight matrix is computed. This weight matrix is used to compute the weighted distances for each input data vector. The data vector is associated with the pattern corresponding to which the weighted distance is minimum. This weight matrix will be repeatedly updated in the subsequent iterations to compute the new weighted distances. This process repeats for every iteration. The algorithm terminates when there is no change in the association of data vectors to the patterns. The algorithm for perceptron learning using weighted distances is given below.

2.2.1 Algorithm PLWD

Step 1: Determine the number of patterns, k, to be recognised from the dataset.

Step 2: Select k points randomly from the dataset and set them as cluster seeds μ_j (j = 1, 2, …, m) to correspond the patterns to be recognised.

Step 3: Perform one iteration of K-means algorithm.

Step 4: Using the results of K-means iteration, compute cluster wise initial weights.

Step 5: Repeat steps 6–10 until the stopping condition.

Step 6: Generate weight matrix W using Eq. (1).

Step 7: For each data point X_i, compute the Euclidean distance d(X_i, μ_j) as follows:

$$d\left(X_i,{\mu }_j\right)=\sqrt{\sum _{l=1}^d{\left(x_{\mathit{il}}-{\mu }_{\mathit{jl}}\right)}^2}$$E2

Here, X_i is the ith data point, μ_j is the mean vector of the cluster j.

Step 8: For each data point compute the weighted distances as follows:

$${\mathit{Wd}}_j=W_{\mathit{ij}}\left(i+1\right)\cdot d\left(X_i,{\mu }_j\right)$$E3

Step 9: Assign points to clusters using weights.

Step 10: Update cluster means, that is, refine patterns.

[End of step 5 loop]

Step 11: [End of algorithm]

Though the perceptron learning algorithms are experimented widely by many researchers, they exhibit their robustness in identifying linearly separable patterns only.

3. Fuzzy-based learning

Fuzzy-based learning is used to handle the concept of partial truth, where the truth value may range between completely true and completely false [16]. It is an approach that allows for multiple possible truth values to be processed through the same data object. In fuzzy-based learning, the data objects are assumed being associated with multiple patterns. For each data object, the degree of association is measured in membership. This membership value may range between 0 and 1 (1 being high similarity and 0 being no similarity with the pattern).

Fuzzy-based learning techniques focus on modelling uncertain and vague information that is found in the real world situations. These techniques deal with the patterns whose boundaries cannot be defined sharply [17, 18]. By fuzzy-based learning, one can know if data objects fully or partially associate with the patterns that are under consideration based on their memberships of association [19]. Among the techniques of fuzzy-based learning, fuzzy C-means (FCM) is the most well-known one as it has the advantage of robustness for obscure information about the patterns [20, 21]. FCM is widely studied and applied in geological shape analysis [22], medical diagnosis [23], automatic target recognition [24], meteorological data [20], pattern recognition, image analysis, image segmentation and image clustering [25, 26, 27], agricultural engineering, astronomy, chemistry [28], detection of polluted sites [29], etc. The following section presents a brief discussion of FCM algorithm.

3.1 Fuzzy C-means (FCM)

The fuzzy C-means (FCM) is a technique that uses degree of membership for natural interpretation of patterns recognised [30]. The FCM associates the data vectors among k patterns. Each data vector may associate with each pattern with a membership degree. The membership of a data vector towards a pattern can range between 0 and 1.

The FCM algorithm is given below [31]. Here, U is the k × N membership matrix. While computing the cluster means and updating the membership matrix at each iteration, the FCM uses the fuzzifier factor, m. For most cases, m ranging between 1.5 and 3.0 gives good results [32]. In the present work, in all the experiments, m is set to 1.5.

3.1.1 Algorithm FCM

Step 1: Determine the number of patterns, k, to be recognised from the dataset.

Step 2: Select k points randomly from the dataset and set them as cluster seeds μ_j (j = 1, 2, …, m) to correspond the patterns to be recognised.

Step 3: Perform one iteration of K-means algorithm. Set t = 0.

Step 4: Using the results of K-means iteration, compute membership matrix $U_{k\mathrm{X}N}^{\left(0\right)}$_.

Step 5: Repeat steps 6–9 until the stopping condition.

Step 6: [Refine patterns] Update the mean of jth cluster μ_j as follows:

$${\mu }_j=\frac{{\sum }_{i=1}^N{\left(u_{\mathit{ij}}\right)}^m{X}_i}{{\sum }_{i=1}^N{\left(u_{\mathit{ij}}\right)}^m}$$E4

Here, u_ij is the membership degree of the data point X_i w.r.t. jth pattern and m is the fuzzifier factor.

Step 7: Compute the new membership matrix using:

$${u_{\mathit{ij}}}^{t+1}={\left[\sum _{l=1}^k{\left(\frac{{‖X_i-{{\mu }_j}^t‖}^2}{{‖X_i-{{\mu }_l}^t‖}^2}\right)}^{\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$m-1$}\right.}\right]}^{-1}$$E5

Step 9: Assign points to clusters using membership degrees. Set t = t + 1.

[End of step 5 loop]

Step 10: [End of algorithm]

4. Fuzzy perceptron learning

The fuzzy perceptron learning works in an interfusion manner, where the fuzzy logic is combined with perceptron learning for identifying non-linear and overlapping patterns. Much research work may be found in the literation where fuzzy perceptron learning is experimented in different applications [33, 34]. However, those experiments are confined to supervised learning only. The present work attempts to experiment with fuzzy perceptron learning for unsupervised cases. The present work proposes two algorithms, one is for fuzzy perceptron learning using weights and the other is for fuzzy perceptron learning using weighted distances.

4.1 Fuzzy perceptron learning using weights (FPLW)

This algorithm implements the perceptron learning using weights and FCM techniques in an interfusion manner. These techniques are performed in alternative iterations until the termination condition. Initially, one iteration of K-means algorithm is performed. Using the results of K-means, initial weights are computed as mentioned in the Section 2.1. Using these weights, weight matrix is generated to perform one iteration of perceptron learning algorithm to associate the input data vectors to the patterns. Using the results of perceptron learning step, membership matrix is computed to perform one iteration of FCM algorithm as mentioned in Section 3.1. The results of FCM step are used to update weight matrix to perform perceptron learning step. In this way the perceptron learning and FCM algorithms are repeated in alternative iterations until termination condition. The algorithm for fuzzy perceptron learning using weights (FPLW) is given below.

4.1.1 Algorithm FPLW

Step 1: Determine the number of patterns, k, to be recognised from the dataset.

Step 2: Select k points randomly from the dataset and set them as cluster seeds μ_j (j = 1, 2, …, m) to correspond the patterns to be recognised.

Step 3: Perform one iteration of K-means algorithm.

Step 4: Using the results of K-means iteration, compute cluster wise initial weights.

Step-5: Update cluster means μ_j (j = 1, 2, …, m).

Step 6: Repeat steps 7–13 until the stopping condition.

Step 7: Compute the weight matrix using Eq. (1).

Step 8: Assign points to clusters using weights.

Step 9: If there is no change in cluster assignment then go to step 14.

Step 10: Update cluster means using Eq. (4).

Step 11: Generate membership matrix $U_{k\mathrm{X}N}^{\left(0\right)}$ using Eq. (5).

Step 12: Assign points to clusters using membership matrix.

Step 13: If there is no change in cluster assignment then go to step 14.

[End of Step 6 loop]

Step 14: [End of Algorithm]

4.2 Fuzzy perceptron learning using weighted distances (FPLWD)

This algorithm implements the perceptron learning using weighted distances and FCM techniques in an interfusion manner. These techniques are performed in alternative iterations until the termination condition. Initially, one iteration of K-means technique is performed. Using the results of K-means, initial weights are computed as mentioned in the Section 2.2. Now, one iteration of perceptron learning algorithm is performed where the weight matrix is generated using the initial weights. Using this weight matrix, weighted distances are computed for every input vector X_i with respect to each pattern using the Eq. (3). The weighted distances are used to associate the input vectors to the patterns. Using the results of perceptron learning step, membership matrix is computed to perform one iteration of FCM algorithm as mentioned in Section 3.1. The results of FCM step are used to compute weight matrix for perceptron learning step. In this way the perceptron learning and FCM steps are repeated in alternative iterations until termination condition. The algorithm for fuzzy perceptron learning using weighted distances (FPLWD) is given below.

4.2.1 Algorithm FPLWD

Step 1: Determine the number of patterns, k, to be recognised from the dataset.

Step 2: Select k points randomly from the dataset and set them as cluster seeds μ_j (j = 1, 2, …, m) to correspond the patterns to be recognised.

Step 3: Perform one iteration of K-means algorithm.

Step 4: Using the results of K-means iteration, compute cluster wise initial weights.

Step-5: Update cluster means μ_j (j = 1, 2, …, m).

Step 6: Repeat steps 7–15 until the stopping condition.

Step 7: Compute the weight matrix using Eq. (1).

Step 8: For each data point compute the Euclidean distance using Eq. (2).

Step 9: For each data point compute weighted distances using Eq. (3).

Step 10: Assign points to clusters using weighted distances.

Step 11: If there is no change in cluster assignment then go to step 16.

Step 12: Update cluster means using Eq. (4).

Step 13: Generate membership matrix $U_{k\mathrm{X}N}^{\left(0\right)}$ using Eq. (5).

Step 14: Assign points to clusters using membership matrix.

Step 15: If there is no change in cluster assignment then go to step 16.

[End of Step 6 loop]

Step 16: [End of Algorithm]

5. Performance evaluation

For performance evaluation of algorithms, CPU time in seconds, sum of squared errors [35] and clustering fitness (CF) [36] are taken into consideration and are calculated for all the algorithms.

5.1 Sum of squared errors

The objective of pattern learning is to minimise the intra-cluster sum of squared errors (SSE). The lesser the SSE, the better the goodness of fit is. The SSE for the results of each algorithm is computed using Eq. (6).

$$\mathit{SSE}=\sum _{j=1}^k\sum _{X_i\in C_j}{\left(X_i-{\mu }_j\right)}^2$$E6

Here, X_i is the ith data point in the dataset, μ_j (j = 1, …, k) is the mean of the cluster C_j, and k is the number of patterns to be recognised.

5.2 Cluster fitness

While achieving high intra-cluster similarity, it is also important to achieve well separation of patterns.

So, it is also important to consider inter-cluster similarity while evaluating the performance of the algorithms. For this, the present work, computes the clustering fitness (CF) as a performance criterion, which requires the calculation of both intra-cluster similarity and inter-cluster similarity. The computation of CF also requires the experiential knowledge, λ. The computation of CF results in higher value when the inter-cluster similarity is low and results in lower value for when the inter-cluster similarity is high. Also that to make the computation of CF unbiased, the value of λ is taken as 0.5 [36].

5.2.1 Intra-cluster similarity for the cluster C_j

It can be quantified via a function of the reciprocals of intra-cluster radii within each of the resulting clusters. The intra-cluster similarity of a cluster C_j (1 = j = k), denoted as S_tra(C_j) [36], is defined by:

$$S_{\mathit{tra}}\left(C_j\right)=\frac{1+n}{1+{\sum }_1^n\mathit{\text{dist}}\left(I_l,\mathit{\text{Centroid}}\right)}$$E7

Here, n is the number of items in cluster C_j, I_j (1 = j = n) is the jth item in cluster C_j, and dist(I_j, Centroid) calculates the distance between I_j and the centroid of C_j, which is the intra-cluster radius of C_j. To smooth the value of S_tra(C_j) and allow for possible singleton clusters, 1 is added to the denominator and numerator.

5.2.2 Intra-cluster similarity for one clustering result C

It is denoted as S_tra(C) [36]. It is defined by:

$$S_{\mathit{tra}}\left(C\right)=\frac{{\sum }_1^k{S}_{\mathit{tra}}\left(C_j\right)}{k}$$E8

Here, k is the number of resulting clusters in C and S_tra(C_j) is the intra-cluster similarity for the cluster C_j.

5.2.3 Inter-cluster similarity

It can be quantified via a function of the reciprocals of inter-cluster radii of the clustering centroids. The inter-cluster similarity for one of the possible clustering results C, denoted as S_ter(C) [36] is defined by:

$$\begin{array}{l}{S}_{\mathit{ter}}\left(C\right)=\frac{1+k}{1+{\sum }_1^k\mathit{\text{dist}}\left({\mathit{\text{Centroid}}}_j,{\mathit{\text{Centroid}}}^2\right)}\\ \end{array}$$E9

Here, k is the number of resulting clusters in C, 1 = j = k, Centroid_j is the centroid of the jth cluster in C, Centroid² is the centroid of all centroids of clusters in C. We compute inter-cluster radius of Centroid_j by calculating dist(Centroid_j, Centroid²), which is distance between Centroid_j, and Centroid². To smooth the value of S_ter(C) and allow for possible all-inclusive clustering result, 1 is added to the denominator and the numerator.

5.2.4 Clustering fitness

The clustering fitness for one of the possible clustering results C, denoted as CF [36], is defined by:

$$\mathit{CF}=\lambda \mathrm{x}{S}_{\mathit{tra}}\left(C\right)+\frac{1-\lambda }{S_{\mathit{ter}}\left(C\right)}$$E10

Here, λ (0 < λ < 1) is an experiential weight, S_tra(C) is the intra-cluster similarity for the clustering result C and S_ter(C) is the inter-cluster similarity for the clustering result C.

6. Experiments and results

Experimental work has been carried out on the system with Intel(R) Core(TM) i3-5005 U CPU@2.00GHz processor speed, 4GB RAM, Windows 7 OS (64-bit) and using JDK1.7.0_45. Separate modules are written for each of the above discussed methods to observe the CPU time for clustering any dataset by keeping the cluster seeds same for all methods. I/O operations are eliminated and the CPU time observed is strictly for clustering of the data.

Along with the proposed algorithms FPLW and FPLWD for fuzzy perceptron learning, experiments are also conducted with the algorithms PLW, PLWD and FCM for performance comparison. All the algorithms are executed using the benchmark datasets with varying number of patterns to be recognised. In the present work, Magic Gamma, Letter Recognition and Intrusion datasets are used from UCI ML data repository [37]. All the developed algorithms, PLW, PLWD, FCM, FPLW and FPLWD, are executed using these datasets for varying number of patterns to be recognised (k = 10, 11, 12, 13, 14, 15).

All the algorithms operate in an iterative manner and terminate when a stopping condition is met. The stopping condition is when there is no change in the pattern associativity of the data vectors. The termination condition is the same for all the algorithms.

Details of the datasets are available in Table 1.

S. No.	Dataset	No. of points	No. of dimensions
1	Magic Gamma data	19,020	10
2	Letter Recognition data	20,000	16
3	Intrusion data	4,94,019	35

Table 1.

Details of datasets.

6.1 Observations with Magic Gamma dataset

The results of all algorithms, using Magic Gamma dataset, with respect to CPU time in seconds, clustering fitness and sum of squared errors are shown in Figures 2–4, respectively.

Figure 2.

CPU time of each clustering method (Magic Gamma dataset).

Figure 3.

Clustering fitness of each clustering method (Magic Gamma dataset).

Figure 4.

SSE of each clustering method (Magic Gamma dataset).

6.2 Observations with Letter Recognition dataset

The results of all algorithms, using Letter Recognition dataset, with respect to CPU time in seconds, clustering fitness and sum of squared errors are shown in Figures 5–7, respectively.

Figure 5.

CPU time of each clustering method (Letter Recognition dataset).

Figure 6.

Clustering fitness of each clustering method (Letter Recognition dataset).

Figure 7.

SSE of each clustering method (Letter Recognition dataset).

6.3 Observations with Intrusion dataset

The results of all algorithms, using Intrusion dataset, with respect to CPU time in seconds, clustering fitness and sum of squared errors are shown in Figures 8–10, respectively.

Figure 8.

CPU time of each clustering method (Intrusion dataset).

Figure 9.

Clustering fitness of each clustering method (Intrusion dataset).

Figure 10.

SSE of each clustering method (Intrusion dataset).

In all the experiments, it is observed that the algorithm FPLW, which implements the perceptron learning using weights and the FCM techniques in an interfusion manner, is showing consistently better performance in terms of clustering fitness (CF) and SSE than the other algorithms.

7. Conclusion

The present experiment mainly focuses on the study fuzzy perceptron learning for recognising non-linear patterns in the datasets. Many researchers contributed greatly towards fuzzy perceptron learning. However, their experiments are confined to supervised learning only. So, the present work experimented with the fuzzy perceptron learning approaches for unsupervised learning. The work proposes two new algorithms, that is, FPLW and FPLWD. These algorithms are implemented using three benchmark datasets. Along with these algorithms, the algorithms for standard FCM and perceptron learning using weights and weighted distances are also implemented for performance comparison. For all the algorithms the CPU time in seconds, clustering fitness (CF) and sum of squared errors (SSE) are taken into consideration for performance evaluation. All the developed algorithms are experimented with varying number of patterns (k) to be recognised.

In all the experiments, it is observed that the proposed algorithm for fuzzy perceptron learning using weights (FPLW) is consistently showing better performance with respect to clustering fitness and SSE. Of course, the algorithm FPLW is taking a little more time for its execution than the other algorithms. However, it could be negligible, as the main concern is for clearly recognising the non-linear patterns in the datasets.