A Clustering Approach Based on Charged Particles

In pattern recognition, clustering is a powerful technique that can be used to find the identical group of objects from a given dataset. It has proven its importance in various domains such as bioinformatics, machine learning, pattern recognition, document clustering, and so on. But, in clustering, it is difficult to determine the optimal cluster centers in a given set of data. So, in this paper, a new method called magnetic charged system search (MCSS) is applied to determine the optimal cluster centers. This method is based on the behavior of charged particles. The proposed method employs the electric force and magnetic force to initiate the local search while Newton second law of motion is employed for global search. The performance of the proposed algorithm is tested on several datasets which are taken from UCI repository and compared with the other existing methods like K-Means, GA, PSO, ACO, and CSS. The experimental results prove the applicability of the proposed method in clustering domain.


Introduction
Clustering is an unsupervised technique which can be applied to understand the organization of data. The basic principle of clustering is to partition a set of objects into a set of clusters such that the objects within a cluster share more similar characteristics in comparison to the other clusters. A pre-specified criterion has been used to measure the similarity between the objects. In clustering, there is no need to train the data, it only deals with the internal structure of data and used a similarity criterion to group the objects into different clusters. Due to this, it is also known as unsupervised classification technique. It becomes a NP hard problem when number of clusters is greater than three. Consider a set S = [A 1 , A 2 , A 3 … A N ] such that A i S, consist of N number of data objects and another set P = [B 1 , B 2 … B K ] consist of K cluster centers. The objective of clustering is to arrange each data object from the set S with one of the cluster center j of set P such that the value of objective function is minimized. The objective function is defined as sum of squared Euclidean distance between the data object A i and cluster center B j and it can be described as follows: where A i denotes the i th data objects,B j denotes the j th cluster center, and D denotes the distance between i th data objects from the j th cluster center. It is also noted that: • Each cluster at least consists of one data object.
B j ≠ Ø, for all j {1, 2, 3 … K}, where B j represents the j th cluster and K denotes the total number of clusters.
• Each data object is allotted to only one cluster.
B i ⋂ B j , for all i ≠ j and i, j {1, 2, 3 …. K} such that i th and j th clusters do not consists same data objects.
• Each data objects should be allocated to a cluster. U j=1 K B j = S where S represents the set of data objects.
Hence, the aim of the partition based clustering algorithm is to determine the K number of cluster centers in a given dataset. Here, the MCSS algorithm is applied for determining the optimal cluster centers in a dataset.
Clustering has proven its importance in many applications successfully. Some of these are pattern recognition [1,2], image processing [3][4][5][6], process monitoring [7], machine learning [8,9], quantitative structure activity relationship [10], document retrieval [11], bioinformatics [12], image segmentation [13], construction management [14], marketing [15,16] and healthcare [17,18]. Broadly, clustering algorithms can be divided into two categories -partition based clustering algorithms and hierarchal clustering algorithms. In partition based clustering algorithms, partition a dataset into k clusters on the basis of some fitness functions [19]. While in hierarchical clustering algorithms, clustering of data occurs in the form of tree representation and this representation is known as dendrogram. Hierarchical clustering algorithms do not require any prerequisite knowledge about number of clusters in a dataset but its only drawback is lacking of dynamism as the objects are tightly bound with respective clusters [20][21][22][23]. However, our research is focused on partition clustering, which decomposes the data into several disjoint clusters that are optimal in terms of some predefined criteria. From the literature, it is found that K-means algorithm is the oldest, most popular, and extensively used partition based algorithm for data clustering. It is easy, fast, simple structure, and having linear time complexity [24,25]. In K-means algorithm, a dataset is decomposed into a predefined number of clusters and the data into distinct clusters based on the euclidean distance [25].

Magnetic charge system search (MCSS) algorithm
The magnetic charged system search (MCSS) algorithm is a recent meta-heuristic algorithm based on electromagnetic theory [48]. According to electromagnetic theory, moving charged particles produce an electric field as well as a magnetic field. Movement of the charged particles in a magnetic field enforces a magnetic force on the other charged particles and the resultant force is proportional to the charge (mass) and speed of the charged particles. The magnitude and direction of the resultant force depend on the two factors: first, velocity of the charged particles, and secondly, magnitude and direction of the magnetic field. Thus, the MCSS algorithm is further advancement in the charge system search (CSS) algorithm using the concept of electromagnetic theory. The difference between the CSS and MCSS is that the CSS algorithm considers only the electric force to determine the movement of CPs while the MCSS utilizes both the forces (electric and magnetic) to determine the same. Along this, MCSS can be either attractive or repulsive in nature. This nature of MCSS algorithm generates more promising solutions in random space. On the other hand, CSS algorithm is attractive by nature. Thus, the performance of the algorithm can be affected with small number of CPs. So, the addition of the magnetic force to the already existing electric force, results in enhancement of both the exploration and exploitation capabilities of CSS and this makes the algorithm more realistic one. Hence, the inclusion of magnetic force in the charge system search (CSS) algorithm results in the formation of a new algorithm known as magnetic charge system search (MCSS). The main steps of the MCSS algorithm are as follows.

Step 1: Initialization
Algorithm starts by identifying the initial positions of charged particles (CPs) in d-dimensional space in random order and set the initial velocities of CPs is zero. To determine the initial positions of CPs, equation 1 is used. A variable charge memory (CM) is used to store the best results.
( ) k j min j jmax jmin C X r (X X ) / K , where j 1, 2 ..d and k 1, 2 ..K = + * - where, C k denotes the k th cluster center for a given dataset, r j is a random number in the range of 0 and 1, X jmin and X j max denote the minimum and maximum value of the j th attribute of the dataset, and K represents the total number of clusters in a dataset.
Step 2: Compute the total force (F total ) acts on CPs.
The total force is the combination of the electric force and magnetic force, and this force influences the movement of CPs in d-dimensional space. It can be computed as follows: • Determine the electric force -when CPs move in d-dimensional space, an electric field is produced surrounding it, and exerted an electric field on the other CPs. This electric force is directly proportional to the magnitude of its charge and the distance between CPs. The magnitude of an electric force generated by a charge particle is enforced on another charge particle and it can be measured using equation 2.
( ) In equation 2, q i and q k represents the fitness values of i th and k th CP, r ik denotes the separation distance between i th and k th CPs, w 1 and w 2 are the two variables whose values are either 0 or 1, R represents the radius of CPs which is set to unity and it is assumed that each CPs has uniform volume charge density but changes in every iteration, and P ik denotes the moving probability of each CPs.
• Determine the magnetic force -The movement of CPs also produce magnetic field along with the electric field. As a result of this, a magnetic force is imposed on the other CPs and equation 3 is utilized to compute the magnitude of magnetic force exerted by a CP on other CPs. It can be either positive or negative depending on the value of average electric current of the previous iteration.
In equation 3, q k represents the fitness values of the k th CP, I i is the average electric current, r ik denotes the separation distance between i th data instance and k th CPs, w 1 and w 2 are the two variables whose values are either 0 or 1, R represents the radius of CPs which is set to unity, and PM ik denotes the probability of magnetic influence between i th data instance and k th CP. In other words, it can be summarized that the magnetic force can be either attractive or repulsive in nature. As a result of this, more promising solutions can be generated during the search. Whereas, the electric force is always attractive in nature. Therefore, this nature of electric force may influence the performance of the algorithm. Hence, to overcome the repulsive nature, a probability function is added with the electric force and finally, the total force acting on other CPs can be computed using equation 4.
Where, p r denotes a probability value to determine either the electric force (E ik ) repelling or attracting, E ik and M ik present the electric and magnetic forces exerted by the k th CP to i th data instance.
Step 3: Determine the new positions and velocities of CPs.
Newton second law of motion is applied to determine the movement of CPs. The magnitude of the total force with Newtonian laws is used to produce the next positions and velocities of CPs. The new positions and velocities of CPs can be computed using equation 5 and 6.
Where, rand 1 and rand 2 are the two random variable in the range of 0 and 1, Z a and Z v act as control parameters to control the influence of total force (F total ), and V k old denotes the velocity of k th CPs,m k is the mass of k th CPs which is equal to the q k ,Δt represents the time step which is set to 1, and C k old denotes the position of k th current CP.
where V k old denotes the new velocity of k th CP, C k old and C k new represents the old and new position of k th CP, and Δt represents the time stamp.
Step 4: Update charge memory (CM) CPs with better objective function values replace the worst CPs from the CM and store the positions of new CPs in CM.

Step 5: Termination condition
If the maximum iterations is reached and condition is satisfied, then stop the algorithm and obtain the optimal cluster centers. Otherwise repeat steps 2-4.

Pseudo code of MCSS algorithm for clustering
This section summarizes the pseudo code of the MCSS algorithm for clustering tasks.
Step 1: Load the dataset and initialize the parameters of MCCS algorithm.
Step 2: Initialize the initial positions and velocities of Charged Particles (CPs).
Step 3: Compute the value of objective function using equation 7 and arrange the data instances to the clusters using minimum value of objective function.
where, X ji denotes the j th attribute of the i th data instance, C jk represents the j th attribute of the k th CP, and d ik denotes Euclidean distance between i th data instance from the k th CP.
Step 4: Compute the mass of initial positioned CPs.
Step 5: Store the positions of initial CPs (C k ) into a variable, called charge memory (CM).
Step 6: While the termination conditions are not met, compute the value of Electric Force (E ik ) for each CPs as follows: Step 6.1: Calculate the value of moving probability (P ik ) for each charged particle C k .
Step 6.2: Compute the fitness of each instance q i .
Step 6.3: Compute the separation distance (r ik ) of CPs.
Step 6.4: Compute the value of (X i -C k ).
Step 6.5: Compute the value of Electric Force (E ik ) for each CPs Step 7: Determine the value of Magnetic Force (M ik ) for each CPs.
Step 7.1: Compute the value of average electric current (I i ).
Step 7.2: Compute the probability of magnetic influence (PM ik ).
Step 7.3: Compute the value of Magnetic Force (M ik ) for each CPs.
Step 8: Compute the total force (F total ) act on each CPs.
Step 9: Calculate the new positions and velocities of charged particles using equation 5 and 6.
Step 10: Recalculate the value of objective function using new positions of charge particles.
Step 11: Compare the newly generated charge particles to the charge particles reside in CM.
Step 12: Memorize the best solution achieved so far and Iteration= Iteration +1; Step 13: Output the best solution obtained.

Experimental results
This section deals with the experimental setup of our study. It includes the performance measures, parameters settings, datasets to be used, experiment results, and statistical analysis.
To prove the effectiveness of the MCSS algorithm, 10 datasets are applied in which two datasets are artificial ones and the rest are taken from UCI repository. The proposed algorithm is implemented in MATLAB 2010a using a computer with window operating, corei3 processor, 3.4 GHz and 4 GB RAM. Experimental outcomes of MCSS algorithm are compared with other clustering algorithms like K-means, GA [30], PSO [49], ACO [35], and CSS [38].

Performance measures
The performance of MCSS algorithm is examined over the sum of intra cluster distance and F-measure parameters. The sum of intra cluster distance can be measured in terms of best case, average case, and worst case solutions including standard deviation parameter which shows the dispersion of the data. F-measure parameter is used to measure the accuracy of proposed method. Performance measures are described as follows:

Intra cluster distances
Intra cluster distance can be used to measure the quality of clustering [35,36]. It indicates the distance between the data objects within a cluster and its cluster center. This parameter also highlights the quality of clustering i.e. minimum is the intra cluster distance, better will be the quality of the solution. The results are measured in terms of best, average, and worst solutions.

Standard Deviation (Std.)
Standard deviation gives the information about the scattering of data within a cluster [47,49]. Lower value of standard deviation indicates that the data objects are scattered near its center, while high value indicates that the data is dispersed away from its center point.

F-Measure
This parameter is measured in terms of recall and precision of an information retrieval system [50,51]. It is also described in terms of weighted harmonic mean of recall and precision. Recall and precision of an information retrieval system is computed using equation 8 which can be described as follows: The value of F-measure (F (i, j)) can be computed using equation 9.

(Recall Precision) F i, j (Recall Precision)
Finally, the value of F-measure for a given clustering algorithm which consists of n number of data instances is calculated using equation 10.

Parameters settings
In order to evaluate the performance of the proposed algorithm, user defined parameters are to be used prior to the process. In MCSS, there are four user defined parameters such as number of CPs, rand, R and ∈. The details of the parameters as follows: the number of CPs is equal number of clusters present in a dataset, rand is a random function that provides a value in the range of 0 and 1, R denotes the radius of CPs and it is set as 1, ∈ is also a user defined parameter which is used to prevent the singularity and it is set to 0.001. In addition to it, number of iterations for algorithm must be specified. Therefore, maximum iteration number is set to 100 and results are summed over 10 runs of the algorithm using different initial cluster centers for each dataset. Table 1 summarizes the parameters setting of MCSS algorithm. It is also mentioned that the performance of the proposed algorithm is compared with the K-means, GA, PSO, ACO, and CSS. The parameter settings of these algorithms are set accordingly as reported (Figures 1 and 2

Experiment results
This subsection demonstrates the results of the proposed algorithm. The results of the proposed algorithm are compared with other existing techniques like K-means, GA, PSO, ACO, and CSS using a mixture of datasets [53][54][55].    Table 4 summarizes the results of all the techniques for artificial dataset ART2. From the results, it is clearly shown that a significant difference occurred between the results of the proposed algorithm and other algorithms. The proposed algorithm outperforms using all of the parameters. Again, it is observed that the performance of the K-means algorithm is poor among all the methods. It is also stated that the results of the CSS and PSO algorithms are close to the optimal solution, but with slightly high value of standard deviation parameter. Table 5 displays the results of the proposed algorithm and other algorithms for iris dataset. From this table, it came to notice that results obtained using GA is far from the optimal solutions, while the proposed method again gives the superior results. It is also observed that K-means algorithm gives the better results with iris dataset; especially it performs well over GA and ACO algorithms. It is also noticed that results of the K-means algorithm is very close to the PSO algorithm (in terms of F-measure parameter).
Results of all six methods for wine dataset are listed in Table 6. It demonstrates that MCSS algorithm obtains good results (in terms of intra cluster distance and F-measure) in comparison to others, but slightly large value of standard deviation parameter. On the other hand, it is also stated that the performance of GA, PSO, and ACO are nearly same except some variation between F-measure parameter. Again, the performance of the K-means algorithm is better than the GA, PSO, and ACO in terms of F-measure parameter but with a large value of standard deviation parameter. CSS algorithm also gives good performance with wine dataset except MCSS, and obtains low value for standard deviation parameter which shows that in each iteration, a near optimal solution is generated.  Table 6. Comparison of the proposed MCSS algorithm with other clustering algorithms using wine dataset. Table 7 describes the results of all the six algorithms using LD dataset. From the results, it is clearly seen that the outcomes of the proposed algorithm is better in comparison to the other algorithms. It is also noted that K-means algorithm does not perform well with LD dataset, and results obtained using K-means are far-far away from the optimal ones. Again, it came into revelation that the performance of PSO algorithm is better except MCSS algorithm and its results are close to the optimal solutions. Results of all the six algorithm for cancer dataset is listed in Table 8. As it indicates, the performance of the GA and PSO algorithms are not so good with cancer dataset. But the proposed algorithm works well and achieves respectable results as compared to others. Kmeans algorithm also achieves good results over GA, ACO, and PSO algorithms.  Table 9. Comparison of the proposed MCSS algorithm with other clustering algorithms using CMC dataset. Table 9 demonstrates the results of all the six algorithms for CMC dataset. As can be seen clearly, the proposed method achieves better results in comparison to the other algorithms using all the parameters. It is also stated that the performance of the GA is found to be poor among all the algorithms (in terms of standard deviation and F-measure parameters). Along this, it is found that the K-means algorithm obtains maximum intra cluster distance amongst all. Table 10 illustrates the results of the proposed algorithm and all other algorithms for thyroid dataset. From the results, again it is observed that proposed algorithm obtains superior results in comparison to other algorithms, but gets the marginally high value of standard deviation parameter in comparison to GA, PSO, and ACO algorithms. Again, K-means results are far away from the optimal ones and CSS algorithm also obtain high value of standard deviation parameter except K-means. In the rest of the algorithms, performance of ACO algorithm is better. Results of all the six algorithm for glass dataset is summarized in Table 11. It indicates that CSS algorithm gives the better results for glass dataset in comparison to the other algorithms (in terms of intra cluster distance and standard deviation parameters). From the analysis of Fmeasure parameter, it is found that the performance of the MCSS is better than CSS. It is worthy to be noted that both the CSS and MCSS achieve good results on the cost of standard deviation parameter. Along this, it is also noticed that the GA exhibits weak performance.  In addition to it, it is also observed that both the MCSS and K-means algorithms exhibit similar performance in terms of F-measure parameters, but K-means obtains minimum value for standard deviation parameter. It is also stated that the quality of clustering is measured in terms of intra cluster distance. Therefore, MCSS algorithm provides good quality results in terms of intra cluster distance. Whereas, ACO method gets maximum intra cluster distance among all the methods. Over all, it is concluded that the proposed algorithm gives better performance with most of datasets in comparison to the other algorithms and quality of solutions is obtained. A statistical analysis is also carried out to prove the same.

Conclusion
In this chapter, a magnetic charged system search algorithm is applied to solve the clustering problems. The idea of proposed algorithm came from the electromagnetic theory and it is based on the behavior of moving charged particles. A moving charged particle exerts both the forces (electric force and magnetic force) on other charged particles and in turn altered the positions of charged particles. Therefore, in MCSS algorithm, initial population is presented in the form of charged particles. It utilizes the concept of electric and magnetic forces along with newton second law of motion to obtain the updated positions of charged particles. In MCSS, both the electric force (E k ) and magnetic force (M k ) correspond to the local search for the solution, while the global solution is exploited using newton second law of motion. The aim of this research is to investigate the applicability of MCSS algorithm for clustering problems. To achieve the same, performance of the MCSS algorithm is evaluated on variety of datasets and compared with K-Means, GA, PSO, ACO, and CSS using intra cluster distance, standard deviation, and F-measure parameters. Experiment results support the applicability of proposed algorithm in clustering field as well as the proposed method provides good results with most datasets in comparison to the other methods. Finally, it is concluded that proposed method not only gives good results but also improves the quality of solutions.