Data Clustering for Fuzzyfier Value Derivation

Abstract

The fuzzifier value m is improving significant factor for achieving the accuracy of data. Therefore, in this chapter, various clustering method is introduced with the definition of important values for clustering. To adaptively calculate the appropriate purge value of the gap type −2 fuzzy c-means, two fuzzy values m1 and m2 are provided by extracting information from individual data points using a histogram scheme. Most of the clustering in this chapter automatically obtains determination of m1 and m2 values that depended on existent repeated experiments. Also, in order to increase efficiency on deriving valid fuzzifier value, we introduce the Interval type-2 possibilistic fuzzy C-means (IT2PFCM), as one of advanced fuzzy clustering method to classify a fixed pattern. In Efficient IT2PFCM method, proper fuzzifier values for each data is obtained from an algorithm including histogram analysis and Gaussian Curve Fitting method. Using the extracted information form fuzzifier values, two modified fuzzifier value m1 and m2 are determined. These updated fuzzifier values are used to calculated the new membership values. Determining these updated values improve not only the clustering accuracy rate of the measured sensor data, but also can be used without additional procedure such as data labeling. It is also efficient at monitoring numerous sensors, managing and verifying sensor data obtained in real time such as smart cities.

1. Introduction

In the majority of cases, fuzzy clustering algorithms have been verified to be a better method than hard clustering in dealing with discrimination of similar structures [1], dataset in dimensional spaces [2], and is more useful for unlabeled data with outliers [3]. Fuzzy C-means proved to offer better solutions in machine learning, and image processing than hard clustering such as Ward’s clustering and the k mean algorithm [4, 5, 6, 7, 8, 9]. Generally, fuzzy c-mean has 66% accuracy while Gustafson-Kessel scored 70% [10]. Fuzzy c-mean is one of the most largely applied and modified techniques in pattern recognition applications [11] even though the sensitivity of fuzzy C-means is counted as a weak point of outcome to the prototypes and also the optimizing process [12, 13, 14].

Classification algorithms are generally subject to various sources of uncertainty that should be appropriately managed. Fuzzy clustering can be used with datasets where the variables have a high level of overlap. Therefore, membership functions are represented as a fuzzy set which can be either Type-I, Type-II or Intuitionistic.

Data are generated by a possible distribution or collected from various resources; Since Euclidean distance leads to clustering outcomes of spherical shapes, which is suitable for most cases, it is a top choice for many applications, it is the measurement used in most clustering algorithms to decide new centers [15].

2. Basic notions

• Degree of membership: The degree of likelihood of one dataset belonging to several centers. The sum of membership degrees is equivalent to 1.

• Data: Data can be categories, compounded or numbers. Data in matrix form contains themes and features of various units. For instance, value and time.

• Clusters: Cluster is a group of data points or datasets that share similarities. Distance or distance norm is a mathematic interpretation of likeness. The point of the model clustering algorithms is the data structure.

• Fuzzifier value: The fuzzifier value is essential to find the clustering membership function when the density or volume of a given cluster is dissimilar to those of another cluster. It is assumed that all of the relative distances to the cluster center are equally 0.5, which implies that the fuzzifier value m is 1 and take account of a decision boundary. With these explained conditions, the fuzzy area does not exist.

Figure 1(a) the case where a small m value is set in two clusters with different volumes. Because the section with a fuzzy membership value extends to a bulky C₂ cluster, applying it to the C₁ cluster allot a lot of relatively unnecessary patterns. Figure 1(b) large m value is set. It seems to have good performance since similar membership values are assigned, but the center value of the C₁ cluster tends to move to the C₂ cluster, Figure 1(c) Fuzzy area in accordance with Interval type-2 m value. Instead of the fuzzy area according to the value of m₁ and m₂ using the characteristics of the Interval type-2 membership set, uncertainty can be reduced and a proper fuzzy area for the cluster volume can be formed.

Figure 1.

Fuzzy area between clusters according to m. (a) the case where a small m value, (b) large m value is set, (c) instance of appropriate fuzzy area using Interval type-2.

As presented above, deciding the lowest and highest boundary range values of the fuzzier value extracted from particular data has been suggested by some methods. The following is about PFCM membership function for deciding the fuzzifier value’s range. The membership function at k-th data point for cluster i is presented in Eq. (1). d_ik/d_ij signifies Euclidean distance value between cluster and data point.

$$u_{\mathit{ik}}=\frac{1}{{\sum }_{j=1}^c{\left(d_{\mathit{ik}}/d_{\mathit{ij}}\right)}^{2/\left(m-1\right)}}$$E1

The neighbor membership values are computed, employing the membership value presented in Eq. (1) in order to decide the fuzzifier value’s range. Summarization with an expression including fuzzifier value indicates Eq. (2). It obtains the lower and upper boundary values of the fuzzy constant which includes the number of clusters as C and the fuzzifier value as m.

$$1+\frac{C-1}{C}\cdot \frac{2}{\delta }\cdot \left|\Delta \right|\le m\le \frac{2logd}{log\left(\frac{\delta }{1-\delta }\cdot \frac{1}{c-1}\right)}+1\text{where}∆=\frac{d_i-d_i^{\ast }}{d_i^{\ast }}\text{and}\delta \text{is threshold}$$E2

3. Conventional fuzzy clustering algorithm

3.1 Fuzzy C- means (FCM)

FCM includes the concept of a fuzzifier m being used to determine the membership value of data X_k in a specific cluster with cluster prototype. Specifically, the equation of FCM is consist of the cluster center vi and the membership value of data X_k, representing k = 1, 2…n and i = 1, 2…c, where n indicates the number of patterns and c indicates the number of clusters. FCM requests the knowledge of the initial number of desired clusters. The membership value is by the relative distance between the pattern X_k and the cluster center V_i. However, one of the main weaknesses by using FCM is its noise sensitivity as well as its limited memberships. The weighting exponent m; is referred to the being effective on the clustering performance of FCM algorithm [16].

3.2 PCM

In order to solve problems of FCM method, PCM uses a parameter given by value estimated from the dataset itself. PCM applies the possibilistic approach which obviously means that the membership value of a point in a class represents the typicality of the point in the class. It also means the possibility of data X_k in the class with cluster prototype V_i where k = 1, 2…n and i = 1, 2…c. Then, the noise points are comparatively less typical, using typicality in PCM algorithm. Furthermore, noise sensitivity is significantly reduced [17, 18]. However, the PCM algorithm also has the problem that the clustering outcome is sensitively reacted according to the initial parameter value [19].

3.3 PFCM

The PFCM algorithm is a mixture of PCM algorithm and FCM algorithm [20]. Although the representative value limit (or constraint = 1) was mitigated, the heat constraints on the membership value were preserved, so the PFCM algorithm generated both membership and possibility, and solved the noise sensitivity problem as seen in the FCM [21]. The PFCM is based on the fuzzy value m, which determines the membership value, and the PFCM also uses constants to define the relative importance of fuzzy membership and typicality values in the objective function. The PFCM utilizes more parameters to determine the optimal solution for clustering, which increases the degree of freedom and thus controls better results than the above-mentioned study. However, when considering fuzzy sets and other parameters in certain algorithms, we face the potential for fuzzy of these parameters. In this paper, we describe the fuzziness of the fuzzy value m and the possible value of the bandwidth parameter and generate FOU of uncertainty for both considering the fuzzy m interval, i.e. the m₁ and m₂ intervals and the fuzzy interval. Existing studies have been implemented to measure the optimal range along the upper and lower bounds of fuzzy values through multiple iterations [22]. This study is ongoing, but the same fuzzy constant range cannot be applied to all data [23].

3.4 Type-1 fuzzy set (T1FS)

Type 1 fuzzy logic was first introduced by Jade (1965). Fuzzy logic systems are based on Type 1 fuzzy sets (T1FS), and have demonstrated their capabilities in many applications, especially for control of complex nonlinear systems that are difficult to model analytically [24, 25]. Since the Type 1 fuzzy logic system (T1FS) uses a clear and accurate type 1 fuzzy set, T1FS can be used to model user behavior under certain conditions. Type 1 fuzzy sets deal with uncertainty using precise membership functions that users think capture uncertainty [26, 27, 28, 29, 30]. When the Type 1 membership function is selected, all uncertainties disappear because the Type 1 membership function is completely accurate. The Type 2 fuzzy set concept was presented by Jade as an extension of the general fuzzy set concept., i.e. a type 1 fuzzy set [31]. All fuzzy sets are identified as membership functions. In a type 1 fuzzy set, each element is identified as a two-dimensional membership function. The membership rating for Type 1 fuzzy sets is [0, 1], which is an accurate number. The comparison of membership function and uncertainty extracted from the result of the conventional fuzzy clustering algorithm is shown as below [32].

FCM	$J_{\mathit{FCM}}\left(V,U,X\right)=\sum _{i=1}^c\sum _{k=1}^n{u}_{\mathit{ik}}^m‖x_k-v_i‖1<m<\infty$
PCM	$J_{\mathit{PCM}}\left(V,U,X\right)=\sum _{i=1}^c\sum _{k=1}^n{u}_{\mathit{ik}}^m{d}_{\mathit{ik}}^2+\sum _{i=1}^c{\eta }_i\sum _{k=1}^n{\left(1-u_{\mathit{ik}}\right)}^m$; $\eta :\mathit{\text{scale}},\mathit{\text{typicality η}}=\frac{{\sum }_{k=1}^n{u}_{\mathit{ik}}^m{‖x_k-v_i‖}^2}{{\sum }_{k=1}^n{u}_{\mathit{ik}}^m}$
FPCM	$J_{\mathit{\text{FPCM}}}\left(U,T,V\right)=\sum _{i=1}^c\sum _{k=1}^n\left(u_{\mathit{ik}}^m+t_{\mathit{ik}}^{\eta }\right){‖x_k-v_i‖}^2$
PFCM	$J_{\mathit{\text{PFCM}}}\left(U,T,V\right)=\sum _{i=1}^c\sum _{k=1}^n\left({\mathit{au}}_{\mathit{ik}}^m+{\mathit{bt}}_{\mathit{ik}}^{\eta }\right){‖x_k-v_i‖}^2+\sum _{i=1}^c{\delta }_i\sum _{k=1}^n\left(1+{\tau }_{\mathit{ik}}\right)\eta$
T1FC	$J_{T1\mathit{FC}}\left(X,U,C\right)=\sum _{i=1}^c\sum _{k=1}^n{u}_j{\left(x_i\right)}^m{d}_{\mathit{ij}}^2$

4. Advanced fuzzy clustering algorithm

Fuzzy c-means (FCM) is an unsupervised form of a clustering algorithm where unlabeled data X = {x₁, x₂…, x_N} is grouped together in accordance with their fuzzy membership values [33, 34]. Since, data analysis and computer vision problems, analyzing and dealing the uncertainties are a very important issue, FCM is being widely used in these fields. Several methods of other IT2 approach for pattern recognition algorithms have been successfully reported [35, 36, 37, 38, 39, 40, 41]. Type-1 fuzzy sets cannot deal uncertainties therefore; type-2 fuzzy sets were defined to represent the uncertainties associated with type-1 fuzzy sets. As shown in Figure 2, the type-reduction process in IT2 FSs requires a relatively large amount of computation as type-2 fuzzy methods increase the computational complexity due to the numerous combinations of embedded T2 FSs. Methods for reducing the computational complexity have been proposed, such as, the increase in computational complexity of T2 FSs may be less costly for improved performance by applying satisfactory results using T1 FSs. In [42], it was suggested that two Fuzzifier m values is used and the centroid type reduction algorithm for center update is incorporated for interval type-2 (IT2) fuzzy approach to FCM clustering. The IT2 FCM was suggested to clear up the complication with FCM for clusters with different number of volumes and patterns. Moreover, it was suggested that miscellaneous uncertainties were linked with clustering algorithms such as FCM and PCM [43]. Motivation of the success IT2 FSs has made on T1 FSs algorithms.

Figure 2.

(a) Cluster position uncertainty for T1FCM, (b) 1 T2 FCM, (c) QT2 FCM, (d) GT2 FCM algorithms.

4.1 Type-2 fuzzy set (T2 FS)

Due to their potential to model various uncertainties, Type-2 fuzzy sets (T2 FSs) have primarily received interest of increased research [44]. Type-2 fuzzy sets are characterized by a three-dimensional fuzzy membership function. The [0, 1] fuzzy set is the membership grade for each element of a type-2 fuzzy set. The extra third dimension provides extra degrees of freedom to get more information about the expressed term. Type-2 fuzzy sets are valuable in situations where it is difficult to resolve the exact membership function of the fuzzy set. This helps to incorporate uncertainty [45].

The computational complexity of the Type-2 fuzzy set is higher than that of the Type 1 fuzzy set. However, the results gained by the Type-2 fuzzy set are much better than those gained by the Type 1 fuzzy set. Therefore, if type-2 fuzzy sets can significantly improve performance (depending on the application), the increased computational complexity of the type-2 fuzzy sets can be an affordable price to pay [46].

4.2 Type-2 FCM (T2-FCM)

Type-2 FCM (T2-FCM), whose type-2 membership is promptly generated by extending a scalar membership degree to a T1-FS. When limiting the secondary fuzzy set to have a triangular membership function, T2-FCM extends the scalar membership u_ij to a triangular secondary membership function [47, 48].

4.3 General type-2 FCM

The GT2 FCM algorithm accepts a linguistic description of the fuzzifier value expressed as a set of T1 fuzzy- upper and lower value [49]. The linguistic fuzzifier value is denoted as a T1 fuzzy set of m. Figure 3 is shown as two examples of encoding the linguistic nation of the appropriate Fuzzifier value for the GT2 FCM algorithm using three linguistic terms.

Figure 3.

Two possible linguistic representation of the Fuzzifier M using T1 fuzzy sets. (a) membership value for a sample x′ (b) vertical slice x′.

4.4 Interval type 2 fuzzy sets (IT2 FSs)

In order to model uncertainty associated to a type-1 fuzzy set with an interval type 2 fuzzy set, a membership interval with all secondary grades of the primary memberships equaling to one can represent the primary membership J_x′ of a sample point x′ [18, 50].

Figure 3(a) represents an instance of an interval type 2 fuzzy set where the gray shaded region indicates FOU. In the figure, the membership value for a sample x’ is represented by the interval between upper ${\overline{\mu }}_{\tilde{A}}\left(x\text{'}\right)$, and lower ${\underset{¯}{\mu }}_{\tilde{A}}\left(x\text{'}\right)$ membership. Therefore, each x’ has a primary membership interval as

$$J_{x\text{'}}=\left[{\underset{¯}{\mu }}_{\tilde{A}}\left(x\text{'}\right),{\overline{\mu }}_{\tilde{A}}\left(x\text{'}\right)\right]$$E3

In the Figure 3(b) shown as the vertical slice x′, where the secondary grade for the primary membership of each x′ equals one, in accordance with the property of interval type-2 fuzzy sets. This interval is defined as the FOU. An interval type 2 fuzzy set A can be expressed as

$$\tilde{A}=\left\{\left.\left(\left(x,u\right),{\mu }_{\tilde{A}}(x,u)\right)\right|\forall x\in A,\forall u\in J_x\subseteq \left[0,1\right],\left(\left(x,u\right),{\mu }_{\tilde{A}}(x,u)\right)=1\right\}$$E4

4.5 Interval type-2 FCM (IT2-FCM)

In fuzzy clustering algorithms such as FCM, the fuzzy fire value m plays a significant [50] role in determining clustering uncertainty. However, it is generally difficult to properly determine the value of m. IT2-FCM regards fuzzy fire values as intervals [m₁, m₂] and settles two optimization matters [51].

First, an interval type 2 FCM is used to obtain a rough estimate of which data points belong to which cluster.

In Eq. (3) is minimized with respect to u_ij to provide upper and lower membership values.

$${\overline{\mathrm{u}}}_{\mathrm{j}}\left({\mathrm{x}}_{\mathrm{i}}\right)=\left\{\begin{array}{c}\frac{1}{{\sum }_{\mathrm{k}=1}^{\mathrm{c}}{\left({\mathrm{d}}_{\mathrm{ij}}/{\mathrm{d}}_{\mathrm{ik}}\right)}^{2/\left({\mathrm{m}}_1-1\right)}},\text{if}1/{\sum }_{\mathrm{k}=1}^{\mathrm{c}}\left({\mathrm{d}}_{\mathrm{ij}}/{\mathrm{d}}_{\mathrm{ik}}\right)<\frac{1}{\mathrm{c}}\\ \frac{1}{{\sum }_{\mathrm{k}=1}^{\mathrm{c}}{\left({\mathrm{d}}_{\mathrm{ij}}/{\mathrm{d}}_{\mathrm{ik}}\right)}^{2/\left({\mathrm{m}}_2-1\right)}},\text{otherwise}\end{array}\right.$$E5

$${\underset{¯}{\mathrm{u}}}_{\mathrm{j}}\left({\mathrm{x}}_{\mathrm{i}}\right)=\left\{\begin{array}{c}\frac{1}{{\sum }_{\mathrm{k}=1}^{\mathrm{c}}{\left({\mathrm{d}}_{\mathrm{ij}}/{\mathrm{d}}_{\mathrm{ik}}\right)}^{2/\left({\mathrm{m}}_1-1\right)}},\text{if}1/{\sum }_{\mathrm{k}=1}^{\mathrm{c}}\left({\mathrm{d}}_{\mathrm{ij}}/{\mathrm{d}}_{\mathrm{ik}}\right)\ge \frac{1}{\mathrm{c}}\\ \frac{1}{{\sum }_{\mathrm{k}=1}^{\mathrm{c}}{\left({\mathrm{d}}_{\mathrm{ij}}/{\mathrm{d}}_{\mathrm{ik}}\right)}^{2/\left({\mathrm{m}}_2-1\right)}},\text{otherwise}\end{array}\right.$$E6

After this cluster prototypes are calculated, then type reduction and then classification is done. Qiu et al. (2014) proposed this complete method of interval type-2 FCM for finding the clusters in each class of the histogram in individual dimensions is acquired with these labeled clusters. This histogram is smoothed by the mean of moving window (using a triangular window in my case). The curve fitting of this smoothed histogram gets the membership function. Histograms with values greater than the membership value are assigned as histograms for higher membership, and histograms for values less than membership value are saved as histograms for lower membership. Curve fitting is carried out severally in the top and bottom histograms to supply the top and bottom member values [52]. This membership value is suggested to estimate the values of fuzzifiers m₁ and m₂. Fixed-point iteration is a method of expressing the transcendental equation f(x) = 0 in the form of x = g(x) and then solving this expression iteratively for x in iterative relationship.

$$x_{i+1}=g\left(x_i\right),I=\mathit{0},\mathit{1},\mathit{2},\dots$$E7

where x₀ being some initial guess. Rewriting the equation to express Eq. (5) and (6) in the form of (7) and dropping the upper and lower term,

$$u_j=\frac{1}{{\sum }_{k=1}^c{\left(d_{\mathit{ij}}/d_{\mathit{ik}}\right)}^{2/\left(m-1\right)}}$$E8

$$\Rightarrow \frac{1}{u_j}={\sum }_{k=1}^c{\left(d_{\mathit{ij}}/d_{\mathit{ik}}\right)}^{2/\left(m-1\right)}$$

log on both sides, Eq. (8) can be rewritten as

$$\mathit{log}\left(\frac{1}{u_j}\right)=\mathit{log}\left({\sum }_{k=1}^c{\left(d_{\mathit{ij}}/d_{\mathit{ik}}\right)}^{2/\left(m-1\right)}\right)$$E9

$$\because \mathit{log}\left(a+c\right)=\mathit{log}a+\mathit{log}\left(1+\frac{c}{a}\right)$$

Extending this logarithmic identity to the sum of N elements,

$$\Rightarrow \mathit{log}\left(a_0+\sum _{k=1}^N{a}_k\right)=\mathit{log}{a}_0+\mathit{log}\left(1+\sum _{k=1}^N\frac{a_k}{a_0}\right)$$E10

$$log\left(\frac{1}{{\mathrm{u}}_{\mathrm{j}}}\right)=\frac{2}{\left(\mathrm{m}-1\right)}log\left(\frac{{\mathrm{d}}_{\mathrm{ij}}}{{\mathrm{d}}_{1\mathrm{j}}}\right)+log\left(1+\sum _{\mathrm{k}=2}^{\mathrm{c}}{\left(\frac{{\mathrm{d}}_{\mathrm{ij}}}{{\mathrm{d}}_{\mathrm{ik}}}\right)}^{2/\left({\mathrm{m}}_{{\mathrm{i}}_{\mathrm{old}}}-1\right)}\right)$$E11

Rearranging Eq. (11) and expressing it in terms of m, gives us Eq. (12).

$$\gamma =\frac{log\left(\frac{1}{{\mathrm{u}}_{\mathrm{j}}}\right)-log\left(1+{\sum }_{\mathrm{k}=2}^{\mathrm{c}}{\left(\frac{{\mathrm{d}}_{\mathrm{ij}}}{{\mathrm{d}}_{\mathrm{ik}}}\right)}^{2/\left({\mathrm{m}}_{{\mathrm{i}}_{\mathrm{old}}}-1\right)}\right)}{log\left(\frac{{\mathrm{d}}_{\mathrm{ij}}}{{\mathrm{d}}_{1\mathrm{j}}}\right)}$$E12

$$m_{\mathit{\text{jnew}}}=1+\frac{2}{\gamma }$$

So, Eq. (13) gives m_1jnew and m_2jnew, where m_{1j new} ≥ m_{2j new}. Eq. (12) is used to calculate fuzzifier values of each data. In some cases, the value of fuzzifier of particular data shows relatively large variation. Here, upper (m_upper) and a lower (m_lower) fuzzifier is necessary, using Eq. (2). If the curtain data point has a fuzzy fire value below the lower bound, the fuzzy fire value is set to the m_lower bound, and if it exceeds the upper bound, the fuzzy fire value is set to the m_upper bound. In the end, a mean of these fuzzifiers is taken to get the last fuzzifier values m₁ and m₂.

4.6 Multiple kernels PFCM algorithm

Typically, the kernel method uses a spatial conversion function to convert input data from input property space to kernel property space [53]. This is to change the kernel property space to a kernel property space so that it is easy to distinguish between overlapping data and having a nonlinear boundary surface in the input property space. If the data in the input space is $X_i,i=1,\dots ,N$, the data converted to the kernel property space through the function is represented by $\mathrm{\Phi }\left(X_j\right),j=1\dots N$. Alike as general PFCM, in the case of Kernels-PFCM, the goal is to minimize the following objective function.

$$J^{\varphi }=\sum _{k=1}^n\sum _{i=1}^c\left({\mathit{au}}_{\mathit{ik}}^m+{\mathit{bt}}_{\mathit{ik}}^{\eta }\right)\times d_{\mathit{ij}}^2+\sum _{i=1}^c\gamma \sum _{k=1}^n{\left(1-t_{\mathit{ik}}\right)}^{\eta }$$E13

In the input space for kernel K, the pattern x_i and the distance d_ij in the kernel attribute space of cluster prototype v_j are expressed as Eq. (14) by the kernel function.

$$\begin{array}{cc}{d}_{\mathit{ij}}& ={‖\mathrm{\Phi }\left(x_j\right)-\mathrm{\Phi }\left(v_j\right)‖}^2\\ & =\mathrm{\Phi }\left(x_j\right)\mathrm{\Phi }\left(x_j\right)+\mathrm{\Phi }\left(v_j\right)\mathrm{\Phi }\left(v_j\right)-2\mathrm{\Phi }\left(x_j\right)\mathrm{\Phi }\left(v_j\right)\\ & =K\left(x_j,x_j\right)+K\left(v_j,v_j\right)-2k\left(x_j,v_j\right)\end{array}$$E14

Commonly, the new Gaussian multi-kernel $\stackrel{\sim }{k}$ using a Gaussian kernel assumes a multi-kernel with the number of kernels S, and the formula is as follows [54].

$${\tilde{k}}^{\left(j\right)}=\left(x_j,v_j\right)=\sum _{l=1}^s\frac{w_{\mathit{il}}}{{\sigma }_l}\frac{exp\left(-\frac{{‖x_j-v_j‖}^2}{2{{\sigma }_l}^2}\right)}{{\sum }_{t=1}^s\frac{w}{{\sigma }_t}}$$E15

From [55] way, using e FCM-MK, normalized kernel is defined to recognize weights by cluster prototypes, resolution and membership values. Using this optimization way, following PFCM objective equation should be minimized. By minimizing the objective function, cluster prototype v_i, resolution-specific weight w_il, and membership value u_ij are defined.

$$\begin{array}{c}{J}_{m,\eta }\left(U,T,V;X\right)=2\sum _{k=1}^n\sum _{i=1}^c({\mathit{au}}_{\mathit{ik}}^m+{\mathit{bt}}_{\mathit{ik}}^{\eta }\times \\ \left(1-\sum _{l=1}^s\frac{w_{\mathit{il}}}{{\sigma }^2}exp\left(-\frac{{‖x_j-v_i‖}^2}{2{{\sigma }_l}^2}\right)\times \frac{1}{{\sum }_{t=1}^s\frac{w}{{\sigma }_t}}\right)\\ +\sum _{i=1}^c{\gamma }_i\sum _{k=1}^n\left({\left(1-t_{\mathit{ik}}\right)}^{\eta }\right)\end{array}$$E16

Here, ρ is a gradient descent way to learn rate parameter. Finally, using type reduction and hard partitioning, clustering is performed as described in the Interval Type-2 PFCM [56].

4.7 Interval type-2 fuzzy c-regression clustering

Let the regression function be represented by Eq. (17)

$$y_i=f^z\left(x_i,{\alpha }_j\right)=a_1^z{x}_{1i}+a_2^z{x}_{2i}+\cdots +a_M^z{x}_{\mathit{Mi}}+b_0^z$$E17

where, x_i = [x_1i,x_2i,. ..,x_Mi] represents points of data, the number of data indicates _i = 1,.. .,n, the number of clusters (or rules) indicates j = 1,.. ., c, the number of variables in each regression indicates q = 1,.. .,M and the number of regression functions indicates z = 1,.. ., r. By a_j, regression coefficients are denoted. We use weighted least square method (WLS) for calculating regression coefficients a_j, In this way, membership grades of partition matrix P are worked for weights. In Eq. (18), Xi is a data point matrix with inputs, y is a data point matrix with outputs.

$$x_i={\left[\begin{array}{c}{x}_{1,i}\\ x_{2,i}\\ \begin{array}{c}⋮\\ x_{M,i}\end{array}\end{array}\right]}^T,y={\left[\begin{array}{c}{y}_{1,}\\ y_2\\ \begin{array}{c}⋮\\ y_M\end{array}\end{array}\right]}^T,P_j=\left[\begin{array}{cc}\begin{array}{c}{u}_j\left(x_1\right)0\\ \begin{array}{cc}0& u_j\left(x_1\right)\end{array}\end{array}& \begin{array}{c}\begin{array}{cc}\dots & 0\end{array}\\ \begin{array}{cc}\dots & 0\end{array}\end{array}\\ \begin{array}{c}\begin{array}{cc}⋮& ⋮\end{array}\\ \begin{array}{cc}0& 0\end{array}\end{array}& \begin{array}{c}\begin{array}{cc}\ddots & ⋮\end{array}\\ \begin{array}{cc}\dots & u_j\left(x_1\right)\end{array}\end{array}\end{array}\right]$$E18

$${\alpha }_j={\left[X^t{P}_{j}X\right]}^{-1}{X}^T{P}_{j}y$$

The partition matrix P is acquired through Gaussian mixture distribution which is the first stage for computing regression coefficients. We consider two fuzzifiers or weighting exponent m₁ and m₂ for indicating the problem into IT2F. However, there is a difference that this model is FCM although our model is FCRM. These two fuzzy fires divide the objective function into two separate functions. The aim is to minimize the total error from Eq. (19) shows these two objective functions. It should be mentioned that the following proof is an extended and modified version of’type-1, which has been presented in [57].

$$\left\{\begin{array}{c}{J}_{m_1}\left(U,\upsilon \right)=\sum _{i=1}^n\sum _{j=1}^C{u}_j{\left(x_i\right)}^{m_1}{E}_{\mathit{ji}}\left({\alpha }_j\right)\\ J_{m_2}\left(U,\upsilon \right)=\sum _{i=1}^n\sum _{j=1}^C{u}_j{\left(x_i\right)}^{m_2}{E}_{\mathit{ji}}\left({\alpha }_j\right)\end{array}\right.$$E19

Where type-1 FCRM, E_ji is the total error, which indicates the distance between actual output and estimated regression equation, and it is presented by Eq. (20).

$$E_{\mathit{ji}}\left({\alpha }_j\right)={\left(y_i-f_j\left(x_i,{\alpha }_j\right)\right)}^2$$E20

Eq. (21) represents the Lagrangian of the objective functions of IT2 FCRM model. We expend the type-1 NFCRM algorithm to interval type-2 NFCRM.

$$\left\{\begin{array}{c}{L}_1\left({\lambda }_1,u_j\right)=\sum _{i=1}^n\sum _{j=1}^C{u}_j{\left(x_i\right)}^{m_1}{E}_{\mathit{ji}}\left({\alpha }_j\right)-{\lambda }_1\left(\sum _{j=1}^c{u}_j-1\right)\\ L_2\left({\lambda }_2,u_j\right)=\sum _{i=1}^n\sum _{j=1}^C{u}_j{\left(x_i\right)}^{m_2}{E}_{\mathit{ji}}\left({\alpha }_j\right)-{\lambda }_2\left(\sum _{j=1}^c{u}_j-1\right)\end{array}\right.$$E21

The partial derivatives with respect to u_j of Eq. (21) are set to 0 in Eq. (22) and (23) for minimizing the objective function.

$$\begin{array}{l}\left\{\begin{array}{l}\frac{\partial L_1}{\partial u_1\left(x_i\right)}=m_1{u}_1{\left(x_i\right)}^{m_{1-1}}{E}_{1i}\left({\alpha }_1\right)-{\lambda }_1=0\\ ⋮\\ \frac{\partial L_1}{\partial u_C\left(x_i\right)}=m_1{u}_C{\left(x_i\right)}^{m_{1-1}}{E}_{\mathit{Ci}}\left({\alpha }_C\right)-{\lambda }_1=0\end{array}\right\}\\ \end{array}$$E22

$$\left\{\begin{array}{l}\frac{\partial L_2}{\partial u_1\left(x_i\right)}=m_2{u}_1{\left(x_i\right)}^{m_{2-1}}{E}_{1i}\left({\alpha }_1\right)-{\lambda }_2=0\\ ⋮\\ \frac{\partial L_2}{\partial u_C\left(x_i\right)}=m_2{u}_C{\left(x_i\right)}^{m_{1-1}}{E}_{\mathit{Ci}}\left({\alpha }_C\right)-{\lambda }_2=0\end{array}\right\}$$E23

Next, the partial derivatives with respect to k₁ and k₂ are performed.

$$\frac{\partial L_1}{\partial {\lambda }_1}=-\left(\sum _{j=1}^c{u}_j\left(x_i\right)-1\right)=0$$E24

To adapt KPCM to IT2 KPCM, three steps are included. In other words, we update the prototype location via initialization, two different fuzzy devices, high and low membership or typicality value calculation, format reduction, and de-fuzzing for data patterns. In the way we propose, by using IT2FS, our point lies in the development of a prototype update process that can solve the cluster matching problem caused by KPCM. Cluster matching usually results in a set of patterns containing clusters that are relatively close to each other. This allows by definition a type 1 fuzzy set to obtain a type reduction via an embedded fuzzy set, but a type-reduced fuzzy set can be obtained by a combination of central intervals estimated from the embedded fuzzy set. This approach is a standard method for obtaining reduced fuzzy set types from IT2FS. However, this approach avoids due to its huge computational requirements, which include a number of embedded fuzzy sets. Therefore, we consider the KM algorithm as an alternative type reduction method. Since KM is an iterative algorithm which estimates both ends of an interval, calculating the left (right) interval $v_L$ ($v_R$) can be found without using all of the embedded fuzzy sets.

Form KERNELS SFCM ALGORITHM in Figure 4,

Figure 4.

FOU representation for our proposed IT2 KPCM approach with m₁ = 2, m₂ = 5 and variance = 0.5; (a) FOU of cluster 1 (b) FOU of cluster 2 [58].

The kernel distance,

$${‖\Phi \left(x_k\right)-v_i‖}^2$$E25

can be derived using the kernel way as

$${‖\mathrm{\Phi }\left(x_k\right)-v_i‖}^2=K\left(x_k,x_k\right)-2\frac{{\sum }_{j=1}^N{u}_{\mathit{ij}}^{m}K\left(x_k,x_j\right)}{{\sum }_{j=1}^N{u}_{\mathit{ij}}^m}+\frac{{\sum }_{j=1}^N{\sum }_{l=1}^N{u}_{\mathit{ij}}^m{u}_{\mathit{il}}^{m}K\left(x_j,x_l\right)}{{\left({\sum }_{j=1}^N{u}_{\mathit{ij}}^m\right)}^2}$$E26

The inverse mapping of prototypes is also needed to approximate the prototypes expressions $v_i$ in the feature space. The objective equation can be written as

$$V\left({\widehat{v}}_i,v_i\right)=\sum _{i=1}^C‖\mathrm{\Phi }\left({\widehat{v}}_i\right)-v_i‖=\sum _{i=1}^C(\mathrm{\Phi }{\left({\widehat{v}}_i\right)}^T\mathrm{\Phi }\left({\widehat{v}}_i\right)-2\mathrm{\Phi }\left({\widehat{v}}_i\right)v_i+v_i^T{v}_i$$E27

While, the final location for ${\widehat{v}}_i$ in the KPCM algorithm becomes,

$${\widehat{v}}_i=\frac{{\sum }_{k=1}^N{u}_{\mathit{ik}}^{m}K\left(x_k,{\widehat{v}}_i\right)x_k}{{\sum }_{k=1}^N{u}_{\mathit{ik}}^{m}K\left(x_k,{\widehat{v}}_i\right)}$$E28

The left (right) interval of the centroids can be found by employing the KM algorithm on the ascending order of a pattern set and its associated interval memberships. The result of the KM algorithm can be expressed as,

$$v_i=1.0/\left[v_L{v}_R\right]$$E29

While the procedure to calculate the left value of interval set v_L and right value v_R, defuzzification is used next to calculate the crisp centers and is defined as the midpoint between v_L, v_R. We can now compute the defuzzified output that is a crisp value of the prototypes by using the expression.

$$v_i=\frac{{\sum }_{v\in J_{Y_i}}\left(u\left(v\right)\right)v}{{\sum }_{v\in J_{Y_i}}\left(u\left(v\right)\right)}=\frac{v_L+v_R}{2}$$E30

Hard partitioning is used to classify test patterns using the resulting prototype of the procedure above. Euclidian distance is now used to hard partition patterns because the prototype is in feature space. The pattern is assigned to a cluster prototype with a minimum Euclidean distance. Experimental results presented in the following sections will demonstrate the validity of the proposed IT2 approach to KPCM clustering.

4.8 Interval type-2 possibilistic fuzzy C-means (IT2PFCM)

In order to solve the uncertainty existing in the fuzzifier value m in the general PFCM algorithm, Multiple Kernels PFCM algorithm should be extended to the Interval Type-2 fuzzy set. If there are N data, W set of resolution-specific weight, U partition matric, C clusters, V set of cluster prototype and S kernels, the cluster prototype can be obtained from minimizing the Gaussian kernel objective function as follows.

$$w_{\mathit{il}}^{\left(\mathit{new}\right)}=w_{\mathit{il}}^{\left(\mathit{old}\right)}-\rho \frac{\partial J}{\partial w_{\mathit{il}}}$$E31

$$d_{\mathit{ij}}^2=\left(2-2\sum _{i=1}^S\frac{w_{\mathit{il}}}{{\sigma }_l}\frac{exp\left(-\frac{{‖x_j-v_j‖}^2}{2{\sigma }_l^2}\right)}{{\sum }_{t=1}^s\frac{w}{{\sigma }_t}}\right)$$E32

Where,

$$v_i=\left(2-2\sum _{i=1}^S\frac{w_{\mathit{il}}}{{\sigma }_l}\frac{exp\left(-\frac{{‖x_j-v_j‖}^2}{2{\sigma }_l^2}\right)}{{\sum }_{t=1}^s\frac{w}{{\sigma }_t}}\right)$$E33

The cluster prototype is calculated to optimize the objective function for the center v_i of each cluster [23].

Where,

$${\overline{K}}^{\left(i\right)}\left(x_j,v_i\right)=\left(\sum _{i=1}^S\frac{w_{\mathit{jl}}}{{\sigma }_l^3}\frac{exp\left({‖x_j-v‖}^2\right)}{{\sum }_{t=1}^S\frac{w}{{\sigma }_t}}\right)$$E34

optimized membership value- the smallest membership value and the largest membership value for each pattern using the Interval Type-2 fuzzy set- is used for calculating the crisp value v_i. In order to compute v_R and v_L, determination of the upper or lower bound of fuzzifier is essential. It is organized as follows by given Eq. (38) [59] .

$$J\left(U,V,W\right)=2\sum _{i=1}^C\sum _{j=1}^N{u}_{\mathit{ij}}^m{d}_{\mathit{ij}}^2$$E35

Using the final v_R and v_L, the crisp center value is obtained from defuzzification as follows.

$$\begin{array}{l}\text{For}{v}_R,\\ \text{if}\left(v\left(i<k\right)\right)\text{then}{u}_{\mathit{ij}}={\overline{u}}_{\mathit{ij}}\\ \text{else}{u}_{\mathit{ij}}={\underset{¯}{u}}_{\mathit{ij}}\end{array}$$E36

Using the cluster Prototype v_i, obtained through the optimization function and the membership value u_ij, the resolution-specific weight value w_il is re-obtain as follows.

$$\frac{\partial J}{\partial w_{\mathit{il}}}=-2\sum _{i=1}^N\frac{u_{\mathit{ij}}^m}{{\sum }_{t=1}^S\frac{w_t}{{\sigma }_t}}\left(K(x_j,v_i-{\overline{K}}^{\left(i\right)}\left(x_i,v_j\right)\right)$$E37

Where

$$v_{\mathit{iR}}=\frac{{\sum }_{j=1}^N{u}_{\mathit{ij}}^m{\overline{K}}^{\left(i\right)}\left(x_{j,}{v}_i\right)x_j}{{\sum }_{j=1}^N{u}_{\mathit{ij}}^m{\overline{K}}^{\left(i\right)}\left(x_j,v_i\right)}$$E38

To define the Interval Type-2 fuzzy set and calculate uncertainty for membership, the input data, the primary fuzzy set, is needed to assign into the Interval Type-2 fuzzy set. Eventually, the upper and lower membership function are created from the primary membership functions.

After calculating the upper and lower membership for each cluster, we need to update the new center values. The membership is obtained from the Type-2 fuzzy set, however, the center value is a crisp value, the value cannot be calculated from the above method. Therefore, in order to compute the center value, type reduction is performed by the Type-1 fuzzy set. In addition, defuzzification is accomplished to change the value of Type-1 to a crisp value.

5. Heuristic method: histogram analysis

The goal of heuristic method is to extract information from data, and then adaptively calculates the fuzzifier value. In this approach, some heuristic type- 1 membership function is used appropriately for given dataset. The parameters are defined as the upper and lower membership is decided according to following rules. First, given that the membership values are determined, the IT2 PFCM algorithm calculates roughly in which cluster the data belongs to and then secure a histogram based on the classified clusters. The histogram from IT2 PFCM tends to be gentler and smoother through the membership function by curve fitting of the same histogram. Curve fitting is enforced separately on upper and lower histograms to obtain upper and lower membership values. In order to reach to the IT2 FS, determination of FOU is necessary, which is generally the set of membership values of the T2 FS. Given that, the greater values of the histogram than the membership value are allocated as the highest membership histogram while the opposite case is calculated. Figure 5 shows histograms and FOU determined by classification and dimensional calculation. To find X, satisfying f (X) = 0, it can be expressed as X = g(X) using fixed-point iteration, where X is,

Figure 5.

FOU obtained for individual class and dimension updated fuzzifier value m1 and m2 are obtained (a) class 1 dimension 1, and (b) class 2 dimension 1.

$$X_{i+1}=g\left(X\right),i=0,1,\dots ,N$$E39

Eq. (7) and (8) of the membership function u_i can be shown in the form of Eq. (38) as follows.

$$u_i=\frac{1}{\sum {\left(\frac{d_{\mathit{ik}}}{d_{\mathit{ij}}}\right)}^{\frac{2}{m-1}}}$$E40

Where fuzzifier value m is a value that determines the degree of final clustering fuzzifier as the value of the fuzzy parameter. This value of m₁ and m₂ is then applied into the algorithm for calculate updated clusters and this routine is repeated repeatedly. The detailed algorithm is as follows:

1. Set the initial fuzzifier value of m₁ and m₂.

2. Apply m₁ and m₂ to interval type-2 FCM and obtain the membership of data.

3. Generate a histogram of each cluster from the membership.

4. Curve fit the histogram to get primary memberships.

5. Create histogram of upper and lower membership.

6. Use curve fitting over upper and lower histograms to calculate upper and lower memberships.

7. Normalize the memberships according to upper membership.

8. Fuzzifier m_1i and m_2i are obtained using Eq. (13).

9. Average m_1i and m_2i and update m₁ and m₂ from the average.

10. The algorithm is iteratively performed using updated m₁ and m₂.

The Upper Membership Function (UMF) Histogram and Lower Membership Function (LMF) Histogram are drawn in Figure 5. A new membership function obtained from the Gaussian Curve Fitting (GF-F) method as.

From simply log process on both sides in Eq. (39), Eq. (40) can be expressed as follows:

$$log\left(\frac{1}{u_1}\right)=\frac{2}{m-1}log\left(\frac{d_{\mathit{ki}}}{d_{1i}}\right)+log\left(1+\sum _{j=2}^c{\left(\frac{d_{\mathit{ki}}}{d_{\mathit{ji}}}\right)}^{\frac{2}{m_{\mathit{dd}}-1}}\right).$$E41

Rearranging Eq. (40) and calculate it in terms of m, gives us Eq. (41), (42).

$$\gamma =\frac{log\left(\frac{1}{u_j}\right)-log\left(1+\sum _{k=2}^C{\left(\frac{d_{\mathit{ij}}}{d_{\mathit{ik}}}\right)}^{2/m_{\mathit{old}}-1}\right)}{log\left(\frac{d_{\mathit{ij}}}{d_{\mathit{ik}}}\right)}$$E42

$$m_{\mathit{\text{jnew}}}=1+\frac{2}{\gamma }$$E43

As in the above process, the membership value u_i ∈ {u_i(X_k)} and mj_new is used as a function to get the u_i. Where Eq. (9) is applied to each clustered data and updated, m_1inew and m_2inew values is easily calculated, averaging the fuzzifier value by Eq. (42), the new fuzzifier value m₁ and m₂ are finally calculated as follow

$$m_1=\left(\sum _{i=1}^N{m}_{1i}\right)/N,m_2=\left(\sum _{i=1}^N{m}_{2i}\right)/N$$E44

6. Comparing performances algorithms

Algorithms can be compared in previous experiences using the following criteria:

Root Mean Squared Error (RMSE): The evaluation metric used by all algorithms of clustering is RMSE. RMSE is calculated by the root of the averaging all squared errors between the original data (X) and the corresponding predicted values data (X̅).

$$\mathit{\text{RMSE}}=\sqrt{\frac{{\sum }_{\mathit{k}=\mathbf{1}}^{\mathit{n}}{\sum }_{\mathit{i}=\mathbf{1}}^{\mathit{c}}({\mathit{x}}_{\mathit{ik}}-{\overline{\mathit{x}}}_{\mathit{ik}}}{\mathit{n}}}$$E45

where n is the total number of patterns in a given data set and c is the number of clusters; x_ikand${\overline{\mathit{x}}}_{\mathit{ik}}$ the actual and predicted rating values data respectively.

Accuracy is one metric for evaluating classification models. Informally, accuracy is the fraction of predictions the model got right. Formally, accuracy has the following definition:

$$\mathit{\text{Accuracy}}=\frac{\mathit{\text{number of correct samples}}}{\mathit{\text{total number of samples}}}\ast 100$$E46

Acknowledgments

This work was supported by Institute of Korea Evaluation Institute of Industrial Technology (KEIT, Next Generation Artificial Intelligence Semiconductor R&D Program) grant funded by the Korea government (Ministry of Trade, Industry & Energy, MOTIE) (Project No. 20010098, Development of Mixed Signal SoC with complex sensor for Smart Home Appliances).