Enhanced Image Segmentation: Merging Fuzzy K-Means and Fuzzy C-Means Clustering Algorithms for Medical Applications

Separating an image into regions according to some criterion is called image segmentation. This paper presents an algorithm that combines the fuzzy k-means (FKM) and fuzzy c-means (FCM) clustering strategies. The proposed algorithm combines the FKM and the FCM algorithms mathematical features, which is referred to as (CFKCM). The FKM and FCM clustering algorithms are adopted to compare the performance and hence evaluate the proposed clustering algorithm. Tests are conducted, and performance parameters are calculated for validation. The comparison and assessment analysis are based on metrics related to the image clustering process, such as the Segmentation Accuracy (SA), Clustering Fitness (CF), and cluster Validity function (V pc and V pe ). A dataset of MR images is used by this research for the application, test, and evaluation of the image clustering. The results for clustering backbone MRI images show that CFKCM algorithm is featured by being more effective, and comparatively independent of the noise, where it can process the “clean” images and noisy images without knowing the type of the noise, which is the most difficult task in image segmentation.


Introduction
The field of image processing involves a number of major topics that are considered as hot research areas. Image segmentation is a clear example of these research topics, and few motivations are lying behind such research interests. Every stage of image segmentation is required and has some role in some of the boundless applications in the field of computer vision and image analysis with respect to machine learning strategies.
Image data is continuously increasing in both size and importance throughout the present time and in the near future. Working on image data is normally conducted using the image processing techniques, which are considered as necessary tools. Such algorithms are developed to simplify the work and make it much easier, while dealing with large amounts of data. Clustering techniques, for instance are based on the concept of breaking a set of data into some predefined subsets, such that the partition optimizes a given criterion. Clustering approaches are normally classified as being fuzzy or crisp (hard), according to the clustered subsets.
In hard clustering approaches, the data are divided up into a stated number of subsets, that are related such that each subset excludes the other. While, in fuzzy based clustering, the targeted items to cluster, may fit in to quite a lot of clusters with different degrees of membership.
With reference to the related published work, many researches in literature investigated with the Fuzzy C-Means (FCM) clustering algorithm in an extensive variety of ways aiming to achieve improved image segmentation results [1][2][3]. The FCM is an unsupervised method, which has been applied successfully to perform operations such as feature analysis [4], clustering [5][6][7][8][9], and classifier designs in fields such as astronomy [10], geology [11,12], medical imaging, target recognition [13], and image segmentation.
In [14], Aljebory, and Mohammed presented an exhaustive investigation of the topic, where a technique for segmenting noisy medical images is proposed. The suggested modified fuzzy c-means clustering (MFCM) improves the Spatial Fuzzy c-means clustering (SFCM) algorithm performance. We would like to acknowledge that the work in our paper is an extension to the modified c-means clustering algorithm that was developed and presented in [14].
Fuzzy C-Means (FCM) assigns pixels to each class by means of fuzzy membership function. Assuming X=(x 1 , x 2 , x 3 …x N ) represents an image with N pixels to be segmented into c clusters. FCM represents the minimization of the objective function expressed in (1) in an iterative manner: (1) Where is the membership of pixel in the cluster, is the cluster center, m is the fuzzifier that controls the fuzziness of partitions and lies within the range (1< m ≤ ∞), and ||.|| is a norm metric. The center of each cluster may either be approximated or randomly initialized [15 -17].
Another famous method is the k-means clustering, in which a set of n objects is partitioned into k clusters so that the resulting intra-cluster similarity is high but the inter-cluster similarity is low. In k-means algorithm, k data values are assigned as initial cluster centers, and a set of n objects is divided up into k clusters such that the resulting inter-cluster similarity is high but the inter-cluster similarity is low. Eventually, the k-means algorithm minimizes an objective function J, which is expressed in (2).
In equation (2), the symbol ‖.‖ refers to a distance measure (intra) between a pixel and the center of the cluster, it indicates the distance of cluster center from their n data points. K-means algorithm aims to minimize the sum of distances from each object to its cluster centroid. The algorithm iterates over all clusters and can be abstracted by the following steps: 1. Set k points into the space of the objects to be clustered. The k points represent initial k group centroids. 2. Based on minimum distance between the center and each pixel d = cj, make assignment for objects (pixels) and the closest centroid (assign pixels to the nearest center) 3. Recalculate the optimal new positions of the centers using the average relation and assign them the k centroids. 4. Repeat Step 2 and 3 until the centroids values stabilized with no change (i.e. it satisfies the tolerance or error value).
Despite their popularity in application, a disadvantage of both clustering methods (i.e. k-means, and c-means) is noticed. In both methods, it is not possible to find the appropriate number of clusters, and hence the application of the algorithms depends upon the clusters to be identified in advance [18]. In this paper, a new technique is proposed to estimate the optimal number of clusters by linking the k-means algorithm results with the c-means algorithm.
This paper structured as follows. Section 1 introduces the notions and importance of image processing and clustering. The concepts of the various image segmentation methods are also presented. In section 2, the FKM and FCM algorithms are compared, as a scientific foundation, to introduce the new algorithm. Section 3 presents the implementation of the proposed CFKCM algorithm together with an application example. Section 4 illustrates the experimental results to evaluate the performance of the proposed algorithm with respect to both FKM, and FCM techniques. In section 5, the analysis of the obtained results is presented. Finally, section 6 contains the conclusions and future work.

Fuzzy K-Means /C-Means Clustering
Clustering, as a term denotes the assignment of points to clusters such that in the same cluster they are as similar as possible, while points belonging to different clusters are not similar. In Fuzzy clustering, however, a data point can belong to more than one cluster.
As a well-known and widely used clustering algorithm, the standard FCM was primarily presented by Bezdek et al. [16], and its model was described by Ahmed et al. [19]. This algorithm is an iterative, unsupervised clustering algorithm, which can be used for image segmentation, where it considers the image as a dataset X; X={x 1 , x 2 … x N }. Then the dataset X is subdivided into clusters that produce an optimal number of c partitions with centers vector V= {v 1 , v 2 … v c } depending on memberships and satisfying the objective function in (1). So, the relation for is deduced as per (3).
The FCM algorithm iteratively optimizes (given in (1)) by continuously updating the values of and , until the difference in consecutive values is very small. As a performance evaluation, FCM works efficiently for the data sets which are not corrupted with noise, while it is not reliable and may wrongly classify noisy data. Furthermore, the standard FCM computes the memberships based on the Euclidean distance between pixels. This affects the clustering performance especially when neighboring pixels are having strong correlation such as the medical MR images. While the k-Means clustering algorithm determines hard (crisp) clusters, where a point belongs to only one cluster, Fuzzy K-Means (FKM) discovers soft clusters, where a particular point can belong to more than one cluster, considering a certain likelihood. The functions involved when applying FKM are the mapping of a given set of representative vectors into an improved one through partitioning data points. The algorithm starts with a set of initial cluster centers and repeatedly performs the mapping process until a stopping criterion is satisfied, assuming that no two clusters are having the same cluster representative. If two cluster centers agree, one should be perturbed to avoid agreement in the iterative process. Similar to what we have introduced about the FCM algorithm, the FKM aims to optimize Jk in (4).
∑ ∑ In equation (4), d denotes the Euclidean distance between data sample, V i and the j th center Cj. U ji is the membership that determines the degree of belongingness of each data sample to all the centers. m on the other hand, determines the degree of fuzziness, when m=1 the fuzzy clustering is reduced to the hard clustering algorithm.
In order to realize the motives behind the objective of this paper, a case of MRI image segmentation is considered; when applying the K-means, it is possible to detect a tumor or abnormality. While, by using C-means we can predict abnormality and tumor cells being either missed or not predicted by K-means algorithm. However, adopting C-means algorithm for clustering will reveal good results in segmenting noiseless images, but not with noisy images. Considering these facts, benefits may be obtained from integrating the two algorithms and having the number of iterations reduced, which affects the execution time and give an accurate detection. Both FCM and FKM algorithms perform the steps illustrated in the flowchart of Figure. 1. We have developed an algorithm that integrates both algorithms (FKM and KCM). The proposed approach, which we referred to as Combined Fuzzy k-means/c-means (CFKCM) algorithm, aims to emphasize the features and utilize them, while and overcoming the limitations of FKM and FCM techniques. The CFKCM approach, as a clustering algorithm, is used, and its performance is evaluated in the process of clustering MR medical images. The steps that the CFKCM algorithm follows are: pre-processing, clustering, object extraction and contouring, and validation stages.
By integrating the FKM and the FCM algorithms, the CFKCM is developed and the following two important drawbacks are targeted.  Fuzzy C-means clustering techniques need to go through large number of iterations before reaching to the right cluster centers and having qualitative results level for clustering.  Despite the certain fact that the Fuzzy K-means algorithm, will converge, optimal results may not be obtained.
The initial set of clusters and the number of clusters generated by the FCM clustering, are two issues on which the quality of the segmented image depends. To improve the clustering process, individual clusters should each have members with a strong relationship formed between them and the difference of belongingness among the clusters being minimized. The proposed algorithm performs the FKM clustering on the given dataset first. Resulted FKM centers are then used as cluster seeds, the FCM are performed on the given dataset, and clusters centers until reaching the termination conditions. The schematic diagram of the algorithm is as shown in Figure  2. It is illustrated in Figure 2 that the CFKCM algorithm starts by eliminating the noise from the MRI. The FKM algorithm is then adopted, where an initial value of clusters k, the total iterations limit, and the stopping criterion, are initialized. In the succeeding stage, the cluster centers {C } computed by FKM, are injected into the FCM algorithm to optimize its performance.
The second step is assigning each pixel value to the closest cluster focus based on a base distance between the pixel and the center of the cluster. Then the next stage starts by re-calculating the new centers by FCM. This continues until a pre-defined convergence rule is met. The memberships, distances and mean parameters are refreshed, then pixels in cluster and that scattered are reassigned according to the new parameters. Although, pixels are clustered once again by FCM depending on its altered cluster membership, there is no deduction by pixels in their groups.
With reference to the paper by Aljebory, and Mohammed [14], the membership function is defined by; The value of clustering fitness is calculated by substituting (5) in the following identity constraint ∑ So, we get; Where m ∈ (1, α) is a weighting exponent which determines the fuzziness of the cluster. The main objective of the algorithm is to make best use and maximize the intra-cluster similarity and minimize the inter-cluster similarity. In this work, the clustering fitness λ is considered as a performance measure, which depends on both similarities (0 < λ < 1). The clustering fitness has a higher value when the inter-cluster similarity is low and vice-versa.

CFKCM Algorithm Flow
At the beginning, the proposed CFKCM clustering algorithm applies the FKM technique on a given dataset. Resulting cluster centers are then used as cluster seeds to the FCM.
Assuming an object X={x 1 , x 2 ,…,x n }, where x i is an element point. Then the fuzzy clustering is making a collection of k clusters, C 1 ,C 2 …,C k , and a membership matrix U in domain [0,1], for i = 1… n and j = 1, 2,…, k, where each element of the matrix U has a weight representing the degree of membership of object i in cluster Cj . The algorithm uses the notations; -The distance between i th data and j th cluster center. -The number of clusters. m -The fuzziness index.
-The membership for i th data and j th cluster center. n -The number of data points.
-The j th cluster center.
The condition to satisfy is that; the number of points in a cluster is at least one, and not all of the points, and the constraint given by: , for i=1 to n (8)
Step 2: Use Euclidean distance ( ) to assign each data object to the closest cluster center.
The k-means (KM) algorithm is modified into fuzzy k-means (FKM) clustering algorithm as follows; Step 1: Set the values for the initial cluster centers C 0 = {C j (0)} and for ε.
Step 2: Use Euclidean distance ( ) to assign each data object to the closest cluster center.
Step 3: Based on the cluster centers update memberships Ui,j using the following equation (given m is the fuzzy index): Step 4: Use the following formula to compute the new center for each cluster ∑ ∑ (10)

FCM-Algorithm Steps
Step 1: Parameters Initialization: set k vectors randomly as cluster means; set the initial membership matrix U, and also set the current iteration t = 0.
Step 2: Assign each data object X i to clusters using the membership matrix.
Step 3: Calculate the j th cluster center based on (11).
Step 4: Use (12) to compute the new membership matrix, Step 5: Based on the resulted membership matrix assign each data object Xi to a cluster.
Step 6: Stop the process, if the numbers of vectors assigned to each cluster are not changing, if not, set t = t + 1 and repeat steps 3-5.

CFKCM-Algorithm Steps
In our current research, better clustering features are achieved by adopting the proposed concept of hybridization of FKM and FCM techniques. The CFKCM algorithm optimizes the segmentation algorithm by optimizing the initial cluster centers. The steps of the CFKCM algorithm are derived by combining the two algorithms that are based on fuzzy membership matrix. CFKCM first performs the FKM technique completely on the given dataset. The FCM will then adopt the given dataset based on the cluster centers resulted from the FKM as cluster seeds. This will continue until termination condition is reached. We emphasize that for running the first iteration of the FCM, the membership matrix, as well as the cluster centers are considered and calculated based on the results of FKM. Other iterations, that are remaining, continue as in the FCM algorithm.
The following steps describe the CFKCM algorithm for image segmentation; Step 1: Set initial cluster centers C j (j = 1… k) randomly from dataset, and set t = 0.
Step 2: Start the FKM algorithm steps and continue processing through its steps.
Step 3: End processing if the stopping criterion is reached, else repeat step 2.
Step 4: Using the set of k vectors generated by FKM algorithm start the FCM clustering.
Step 5: Compute the new center and membership matrix. Use the membership matrix to allocate data objects to clusters, and continue processing through its steps.
Step 6: Compute the percentage of change in centers and stop if the change < η, η is a small constant. Or else, set t = t + 1, and repeat steps 3-5.
For all of the algorithms FKM, FCM, and CFKCM the same termination criterion is used with different values (α, ε, η). The CFKCM algorithm flowchart is illustrated in Figure 3.

CFKCM Algorithm Application
In this section, the application of the CFKCM algorithm for spine MR images is presented. A total of 20 datasets for similar patients are used for testing the algorithm performance. Samples of the images are shown in Figure 4.
The clearest images in spinal cord and vertebra disks are selected for study. All MRIs are in DICOM format of dimensions (512 by 512) and (768 by 768) pixels. In order to study the consequence due to noise filtering on segmentation, and to assure the preprocessing importance, the MR image is subjected to a uniformly distributed random noise with variable magnitude before testing.
The spine consists of 33 individual bones stacked on each other. More than 120 MR images are taken for each case distributed by the main regions; neck (cervical), low back (lumbar), thoracic and sacral. These images presenting the anatomical structures on spine MRI (weighted T1, T2 axial and sagittal views) they are different in signal types; T1 weighted image (also referred to as T1WI or the "spin-lattice" relaxation time) is one of the basic pulse sequences in MRI and demonstrates differences in the T1 relaxation times of tissues [20]. The T2 relaxation refers to decline of transverse magnetization due to a combination of spin-spin relaxation and magnetic field inhomogeneity.
We focus on the applicability towards a large variety of clinical MR imaging sequences, dealing with low image quality, high anisotropy and spine pathologies. The algorithm can be switched into any one of the three modes; FKM, FCM, and the combined CFKCM mode. It was tested on segmenting spine section MRI images corrupted by different types of noise; zero centered symmetric α-stable noise, white noise, salt and pepper noise, and mixed noise. In this experiment, the mixed noise used is the blend of Gaussian white noise N (0,100) and unit dispersion, zero centered symmetric α-stable (SαS) noise. Variable amounts of noise and noise parameters are intentionally used in order to ensure the robustness of the algorithm. During experimentation, three different levels for the parameters controlling the intensity of the noise are adopted: Gaussian white noise (3%, 6%, and 9%), salt and pepper (4%, 8%, and 12%), while the parameter for the mixed noise α-stable, is set to (α=0.3, 0.6, and 0.9).
One more evaluation test of the performance of the three algorithms under different level of noises is also performed. This test is the Segmentation Accuracy (SA) test, in which the quality of the algorithms is measured by Segmentation Accuracy (SA) metric. SA is the percentage ratio of correctly classified pixels number to the total number of pixels, and is computed as per (13), by dividing the sum of pixels correctly assigned to a particular cluster divided by the sum of the total number of pixels in an image of "c' clusters.
The Segmentation Accuracy (SA) determines the fraction of correctly assigned pixels to a particular cluster, and reveals a clear idea about the noise removing capabilities of different algorithms. Higher value of SA is required for appropriate clustering.

CFKCM Algorithm Implementation
Let"s consider the image I(R×S ) with R columns and S rows to be clustered, p(x,y) the pixel in consideration, where x = 1, 2, ... R, and y =1, 2, ....S. Based on the analysis and algorithm steps, the position for each cluster center is calculated using (14): Moreover, the assignment of every pixel to more than one cluster is found through calculating the membership function, according to equation (15): Where the distance from point (x,y) to the cluster j center is d jp and d kp is the distance from point p(x,y) to other cluster k center, c is the number of clusters and m is the fuzziness index, m >1, which regulate the degree of fuzziness. All pixels are assigned to a specific cluster depending on the belonging that based on the distance.
Normally, the closer the pixel, the maximum is the membership effect on the result of clustering process. The membership value is effectively based on amount of belonging. The normalized belongingness degree value Mj for every cluster is defined by; (16) High relation between the center and its pixels in the same cluster happens, when maximum degree of belongingness is achieved, thus we have the best data clustering results. In fact, the double refinement criterion in CFKCM algorithm (shown in Figure 3) helps each cluster to have pixels with a strong relationship formed between them, and the difference of belongingness between the clusters should be reduced [21].

Experimental Results
The proposed algorithm is implemented using Matlab version R2015a. Different types of medical images for spine are taken as source images and used for algorithm analysis and evaluation. In this section, we introduce results by which the performance of the three algorithms (FKM, FCM, and proposed CFKCM) is presented and compared. Several sets of real images have been used with the values of (α, ε, η) are set to be small values; α = 0.0001, ε = 0.00005, and η = 0.00001 A main metric that is used as a performance comparison indicator of the algorithms under experimentation is the capability of the specific algorithm for noise removal. The algorithms (FKM and FCM) are tested to remove different types of noise (Gaussian, Salt and pepper, and mixed noise), while varying the noise level. It tests and assigns the pixels to proper clusters as indicated by the maximum values of SA.

Segmentation Accuracy (SA) Test
The SA metric, which is expressed in the formula of (13), is practically measured by conducting experiments on a dataset of 40 real MRIs. The MR images dataset is composed of cervical (neck) vertebrae numbered as C1 to C7, thoracic vertebrae numbered T1 to T12, lumber vertebrae numbered L1 to L5, and four fused bones of the coccyx. An additive noise with different types and variable levels is added to measure the algorithm sensitivity performance as in the following four experiments.

Experiment 1
Test data: neck vertebrae C1-C7 MR images, dimensions (512 by 512). The Segmentation Accuracy (SA) values, tabulated in Table 1, are plotted in figure 5. From this figure, it"s visually clear that the SA values for the CFKCM algorithm compared with the other algorithms are the highest.  Table 2, and plotted in Figure 6.  Test data: lumbar vertebrae numbered L1 to L5, dimension (768 by 768). For this set of images, SA values are calculated and presented in Table 3, and plotted in Figure 7. Test data: four fused bones of the coccyx, dimensions (512 by 512). The SA values that are related to these images are shown in Table 4, while Figure 8, illustrating them graphically.

Clustering Fitness Test
A large number of clusters lead to the involvement of more details which can eventually help in minute object detection. To show this, we choose a diversely illuminated images "neck, mid back, low back, and tailbone" from the real dataset, which contains differently illuminated layers. All the algorithms; FKM, FCM, and the proposed CFKCM are executed on each image data with variable number of clusters (c = 6,7,8,9,10,11,12,13,14). For all algorithms, the same cluster seed and termination conditions are used. The Clustering Fitness (CF) for one of the possible clustering results C, is calculated according to (17) [22]. (17) Where λ is the experiential weight, (0 < λ < 1), S ra (c) is the intra-cluster and Ser(c) is the inter-cluster similarity for the cluster c i . To avoid any bias, λ is supposed to be 0.5. The CF for each algorithm is tabulated in Table 5. In Tables (5a-5d) are the Clustering Fitness values (CF) being calculated for the four adopted categories of images (i.e. the cervical (neck) vertebrae, the thoracic vertebrae, the lumber vertebrae, and the fused bones of the coccyx) respectively. In these results, we can notice that the CF value is almost constant and not related to the increase of the number clusters. Further analysis of these result, illustrates that the proposed CFKCM algorithm outperforms both other two algorithms. With CFKCM, that is having the highest CF value, it will simply enhance the validity of segmentation performance.
The results for the Vertebrae T1-T12 MR images are elected to visually illustrate the behavior of the three clustering algorithm in terms of the clustering fitness, and the values given in Table 5-b are plotted in Figure 9.

Results and Analysis
The experimental results show that the CFKCM enhanced algorithmic segmentation in the following attributes; a-the starting number of clusters is almost optimal, b-higher SA even with noisy images, c-robust algorithm that, maintains the structural characteristics of the segmented image, d-higher CF, where the proposed algorithm shows appreciable performance for all sorts of noises, e-the center of the cluster is located in the optimal gravity center of all elements that belongs to the cluster.
The performance of clustering is evaluated by applying fuzzy partition method. The partition coefficient Vpc and entropy Vpe are defined as follows [23,24]: When Vpc maximum or Vpe minimum, the best performance is achieved. This test function is applied to the four types of images corrupted by different levels of Gaussian noise (5%, 9%, and 13%). The results for CFKCM and FCM are depicted as in Table 6.
For further illustration, part of the results is plotted in Figure 10, where the obtained Cluster validity function values for noise level = 13%, are emphasizing the expectation and telling that the performance of the CFKCM algorithm beats that of the FCM. The FCM algorithm is sensitive to noise; noisy pixels are almost wrongly classified. This is because the objective function does not consider the spatial information, but deal with images as the same as separate points. The integration of more than one constraint by combined in CFKCM enhances the validity of segmentation performance. This can be seen, clearly, in the visual results for both the FCM and the CFKCM algorithm.

Conclusions
The paper presented a study of merging two of the popular clustering algorithms, namely the fuzzy k-means (FKM) and fuzzy c-means (FCM) and identifying a proposed strategy out of this combination process. The resulted algorithm is referred to as a Combined Fuzzy K-means, C-Means (CFKCM). The application steps of the CFKCM algorithm were explained and the performance, in terms of image clustering, has been studied. A dataset of MR images was used for experimentation and performance evaluation. The study is based on comparing the modified algorithm with the two other algorithms (i.e. fuzzy k-means, and fuzzy c-means). We took into consideration a number of factors that are adopted in the field for performance measurement. Four distinct experiments were carried out for segmenting a dataset composed of 40 MR images, and the results of clustering backbone MR images including seven cervical (neck) vertebrae, twelve thoracic vertebrae, five lumber vertebrae, four fused bones of the coccyx, were obtained. Experimentation results showed that the proposed algorithm outperforms the other two algorithms based on the values of the Segmentation Accuracy (SA) as measuring metric. During the experiments an additive noise with different types, and variable levels is added to measure the algorithm sensitivity performance as in the following four experiments. The results showed that the noise effect with the MFCM was significantly less than with other algorithms. The clustering Fitness (CF) was also used as performance measurement metric, and CF tests were performed. Again, the three algorithms were tested, and the values of CF were calculated for MR images from the dataset with varying number of clusters (c = 6,7,8,9,10,11,12,13,14). The same cluster seed and termination conditions are used for all tests. Calculations resulted in the CFKCM algorithm having the highest CF value, and this result is an indicator to an enhanced validity of segmentation performance. A third test for performance evaluation was the calculation of clustering validity functions (terms V pc and V pe ). With higher values of V pc , and lower values of V pe , the CFKCM algorithm exhibits better performance than FCM.