Great effort has been put into the development of diagnosis methods for the most dangerous type of skin diseases—melanoma. This paper aims to develop a prototype capable of segment and classify skin lesions in dermoscopy images based on ABCD rule. The proposed work is divided into four distinct stages: (1) pre-processing, consists of filtering and contrast enhancing techniques, (2) segmentation, thresholding, and statistical properties are computed to localize the lesion, (3) features extraction, asymmetry is calculated by averaging the calculated results of the two methods: entropy and bi-fold. Border irregularity is calculated by accumulate the statistical scores of the eight segments of the segmented lesion. Color feature is calculated among the existence of six candidate colors: white, black, red, light-brown, dark-brown, and blue-gray. Diameter is measured by the conversion operation from the total number of pixels in the greatest diameter into millimeter (mm), and (4) classification, the summation of the four extracted feature scores multiplied by their weights to yield a total dermoscopy score (TDS); hence, the lesion is classified into benign, suspicious, or malignant. The prototype is implemented in MATLAB and the dataset used consists of 200 dermoscopic images from Hospital Pedro Hispano, Matosinhos. The achieved results show an acceptable performance rates, an accuracy 90%, sensitivity 85%, and specificity 92.22%.
- ABCD rule
- feature extraction
Melanoma, also known as malignant melanoma, is the most dangerous type of skin cancer that progresses from the pigment-containing cells known as melanocytes. Sometimes they progress from a mole with concerning changes including an increase in size, irregular edges, and change in color, itchiness, or skin breakdown . Melanomas may rarely occur in the mouth, intestines, or eye but typically occur in the skin [1, 2]. In men, they most commonly occur on the back, while in women, they are most common on the legs .
Authors of  also mentioned that the ultraviolet light (UV) exposure from either the sun or from other sources, such as tanning devices is the primary cause of melanoma, while about 25% develop from moles. Worldwide, in 2012, it registered 55,000 death cases in 232,000 people. North America, Europe, Australia, and New Zealand have the highest rates of melanoma in the world while it is less common in Latin America, Asia, and Africa.
One of the widely used methods by dermatologists to classify the cancerous skin—melanoma from normal skin is the ABCD rule. It is proved that it can be easily learned and rapidly calculated and has been proven to be a reliable method providing a more objective and reproducible diagnosis of melanoma [3, 4, 5]. To calculate the ABCD score, the “asymmetry, border, colors, and diameter” criteria are approximately estimated (semi-quantitatively). Each of the criteria is then multiplied by a given weight factor to yield a total dermoscopy score (TDS). TDS values less than 4.75 indicate a benign melanocytic lesion, values between 4.8 and 5.45 indicate a suspicious lesion, and values of 5.45 or greater are highly suggestive of melanoma.
To calculate the Asymmetry, the melanocytic lesion is bisected by two 90° axes. If both axes dermoscopically show asymmetric contours with regard to shape, the asymmetry score is 2. If there is asymmetry on one axis only, the score is 1. If asymmetry is absent with regard to both axes the score is 0. The border is calculated by dividing the lesion into eighths. Within each one-eighth, a sharp, abrupt cut-off of pigment pattern at the periphery receives a score 1, otherwise receives a score 0. Color feature is calculated by counting the existence of six different colors: white, red, light brown, dark brown, blue-gray, and black. The Diameter of melanomas is usually greater than 6 mm.
The proposed work relies on extracting and selecting specific information features that can be used to distinguish malignant, suspicious, and benign lesions by setting an automated cancer diagnosis using image processing techniques. More details on the image processing techniques used in this research exist in .
To achieve the aim of this research, four stages are implemented sequentially:
Pre-processing stage: it consists of filtering and contrast enhancement techniques to remove any unwanted structures (i.e., hair) that might corrupt the image. Also the aim of this stage is to eliminate the background noise and improve the image quality for the purpose of determining the focal areas in the image.
Segmentation stage: thresholding for binarization and statistical properties are computed such as, area and center of mass to localize the lesion.
Feature extraction/selection stage: this stage quantifies the ABCD rule. Asymmetry feature is calculated by averaging produced results from the two methods: Entropy and Bi-fold. Border irregularity is measured by the partition of the skin’s lesion into eight equal segments, the segment’s perimeter and area is calculated for each segment, and then accumulating their scores. Color feature is calculated among the existence of six candidate colors: white, black, red, light-brown, dark-brown, and blue-gray. Diameter is measured by the conversion operation from the total number of pixels in the greatest diameter into millimeter (mm).
The classification of image: this stage depends on the summation of the four extracted feature scores multiplied by their weights to yield a total dermoscopy score (TDS). Based on the TDS, the lesion is classified into benign, suspicious, or malignant.
Section 2 describes an overview of several systems proposed in the literature. Section 3 describes the research methodology. The experimental study and discussion are described in Section 4. Section 5 concludes this paper with some remarks on future work.
2. Related works
Several diagnostic systems for melanoma detection have been proposed. Some systems try to imitate the performance of dermatologists by detecting and extracting several dermoscopic features. These features can then be used to score a lesion in a similar way to the one adopted by dermatologists. In , general clinical principles of early melanoma detection are reviewed, providing the clinician with an up-to-date understanding of management strategies for their patients with numerous or atypical nevi. Many researchers have been working on the image processing and computer vision techniques for skin cancer detection. The most probably features to perform skin lesion segmentation used in various papers are shape, color, texture, and luminance.
Three methods of segmentation have been discussed by . The methods are: Otsu’s method, gradient vector flow (GVF), and color based using K-mean clustering. Feature extraction is based on the so-called ABCD-rule of dermatoscopy. While , a watershed segmentation is the proposed scheme used for image segmentation, border detection and decision related with structural nature of lesion. For more details of study, the segmentation methods used, a survey work on skin lesion segmentation problem implemented by image processing techniques are described by [10, 11, 12].
Based on a qualitative assessment of asymmetry (of boundary, color, and mass distribution), size functions (SFs) and support vector machine (SVM) are used to implement a new automatic classifier of melanocytic lesions . An automatic identification of asymmetry in digital images containing melanocytic skin lesion using Stolz strategy, based on the ABCD rule is proposed by . A survey on asymmetry analysis of malignant melanoma using image processing techniques to identify the asymmetricity of the melanoma skin lesions was presented by .
Several researchers proposed an image analysis tools to check for the various melanoma parameters like asymmetry, border, color, diameter, in terms of texture, size, and shape analysis for image segmentation and feature stages. The extracted feature parameters are used to classify the image as normal skin and melanoma cancer lesion [16, 17, 18, 19, 20, 21, 22].
In , they applied a Bag-of-features approach to malignant melanoma detection based on epiluminescence microscopy imaging (low-power microscopy (×50–100), commonly a television microscope applied to a glass slide covering mineral oil on the surface of a skin lesion, to determine malignancy in pigmented lesions). Each skin lesion is represented by a histogram of code words or clusters identified from a training data set. Classification results are achieved based on the implementation of naive Bayes and support vector machine classifiers. Other work utilized the Bag-of-feature model for the detection of melanomas in dermoscopy images and aimed at identifying the role of different local texture and color descriptors . The reported results show that the sensitivity is 93% and specificity is 85%.
The extracted features of segmented lesions used as inputs to the input layer of the artificial neural network. Different configurations of ANNs were implemented by the researchers for classification [25, 26, 27, 28]. In , they attached the Dermlite® DL1 dermatoscope to the iPhone. A new method called elliptical symmetry was proposed for quantifying asymmetry. Gaussian smoothing and lacunarity analysis to measure border irregularity were proposed. In Gaussian smoothing, the contour was smoothed and compared with the perimeter of the original lesion. The lacunarity was used to analysis the borders of the image. Finally, the extracted features were fed to input layer of the multi-stage neural network classifier. While , 2D-Wavelet transform is the feature extraction method used. These features are given as the input to the artificial neural network classifier. An unsupervised approach for lesion segmentation is proposed by . Iterative thresholding is applied to initialize level set automatically. The accuracy of detected border is compared with Growcut and mean-shift algorithms. Four features relying on visual diagnosis: asymmetry (A), border (B), color (C), and diameter (D) are computed and used to construct a classification module based on artificial neural network for the recognition of malignant melanoma. The authors of  used a hybrid algorithm combining a region-oriented and a thresholding method to segment the lesion. A multilayer perceptron NN model with one hidden layer and one output neuron was chosen as a basis for all the different network configurations examined.
As described by , the general approach used by a CAD system consists in describing the skin lesion by means of a set of textural and geometrical shape features known as the ABCD rule (asymmetry, border, color and diameter). Software WEKA was used to apply 13 different techniques and a statistical test K-folds to obtain the classification accuracy.
A different approach proposed by  named Modified Texture Distinctiveness Lesion Segmentation algorithm (M-TDLS) to segment the skin lesion. Two steps are involved: TD metric calculation and region classification. The RGB image is converted into XYZ color space and the TD metric is calculated to find dissimilarity between two texture distributions.
In , they addressed two different systems for the detection of melanomas in dermoscopy images. The first system used global methods to classify skin lesions, whereas the second system used local features and the bag-of-features classifier.
3. Research methodology
This section describes four main stages: preprocessing, segmentation, features extraction, and classification. We have started by reading an RGB image, as for example, as shown in Figure 1.
3.1 Preprocessing stage
The preprocessing stage consists of four sequential steps described as follows:
3.1.1 Step 1
For each channel in the RGB image, a 2-D median filtering for noise reduction with mask of size 5 × 5 is implemented and their associated results are depicted in Figure 2.
3.1.2 Step 2
For hair removal, two morphological operations are applied on grayscale image f, dilation followed by an erosion with a small shape or template called a structuring element s denoted by (f ⊕ s and f Θ s, respectively). The results are depicted in Figure 3.
3.1.3 Step 3
Brightness enhancement operation is applied separately on R, G, and B images. Figure 4 shows the result of the brightness enhancement operation.
3.1.4 Step 4
Based on our experimental studies, the channel B is chosen because it provides better segmentation results compared to others. Therefore, the third channel (B)-image is converted into a binarized form using Otsu’s method, and then converted white pixels into black pixels and vice versa to present the pigment skin lesion. The results of this step are depicted in Figure 5.
3.2 Segmentation stage
For each candidate region, the statistical properties such as center of mass (xc, yc) and area A are calculated. Based on the size of the region and the overlapping with the center of mass of the image, the region-of-interest (ROI) is identified as depicted in Figure 6.
3.3 Features extraction stage
This section presents and discusses in detail the methods used to extract the four features asymmetry (A), border irregularity (B), color (C), and diameter (D) from the segmented lesion. According to characteristics of the ABCD rule, each extracted feature plays a distinctive role with its associative weight to calculate the total dermoscopy score (TDS).
To calculate asymmetry, firstly, the skin lesion is converted into grayscale values. Secondly, it is rotated to vertically and horizontally partitioned into two equal halves. Finally, two methods called Entropy and Bi-fold are implemented, and their calculated average value is assigned as an asymmetry score of the segmented lesion.
Compared with Figure 6, the ROI is rotated by θ° to align the (x, y) coordinated with centroid principal axes as shown in Figure 7. The orientation angle θ° is defined as the angle between the x-axis and axis around which the object can be rotated with minimum inertia:
where m1,1, m2,0, and m0,2 are the second order moments or moment of inertia defined as:
where (x0, y0) is the centroid.
Figure 8 shows the result of the partition operation of the ROI over its closest line of symmetry (i.e., centroid) into two equal parts vertically and horizontally.
The asymmetry feature plays an important role in melanoma diagnosis and for this reason we have suggested two methods for implementation:
22.214.171.124 Entropy function
The entropy is a statistical measure of randomness that can be used to characterize the texture of grayscale image as described by the following:
where p contains the histogram counts of intensity values.
To find the similarity between the two parts (left vs. right and upper vs. lower) of the segmented lesion, their entropies are calculated as follows:
The same process is repeated to find the E(U, L). Therefore, the entropy asymmetry is calculated as follows:
where TE is the entropy threshold value.
126.96.36.199 Bi-fold method
The symmetry obtained by overlapping the two vertical (left vs. right) and horizontal (upper vs. lower) parts along the principal axes of the inertia. The non-overlapped is then compared with the total area of the lesion as follows:
where ∆A is the non-overlapping area between the original and reflected masks and A is the area of the original mask. The result of the non-overlapping operation between left and right halves is depicted in Figure 9a and the result of the non-overlapping operation between upper and right halves is depicted in Figure 9b as well. Hence, the overlapping asymmetry is calculated as follows:
where TO is an overlapping threshold value.
The overall asymmetry score (AsymScore) of the skin lesion is calculated as:
3.3.2 Border irregularity
From the binarized ROI (see Figure 7), the border irregularity index or compactness index is calculated as follows:
where P is the perimeter and A is the area of the lesion.
Among other edge detection methods, Sobel method is selected because it is relatively inexpensive in terms of computations. On the other hand, the gradient approximation that it produces is relatively crude, in particular for high-frequency variations in the image. As shown in Figure 10, the lesion’s boundary image is partitioned into eight equal segments and for each segment, we have computed its compact index.
The border irregularity index (BIScore) is calculated as follows:
3.3.3 Color feature
The existence of white, black, red, light-brown, dark-brown, and blue-gray colors in the true colored lesion are needed to be examined. Assume that Figure 11 presents the lesion that is needed to be examined for the six candidate colors appearance. The color score is incremented by 1, if the distance between the examined pixel’s value in the lesion and each color reference is below or equal to the pre-calculated threshold value.
Six RGB codes are chosen as reference points for each color used as shown in Table 1.
|(1, 1, 1)||(0, 0, 0)||(1, 0, 0)||(0.7843, 0.5882, 0.3922)||(0.5882, 0.3922, 0.3922)||(0.5882, 0.4902, 0.5882)|
|(0.9608, 0.9608, 0.9608)||(0.0392, 0.0392, 0.0392)||(1, 0.1961, 0.1961)||(0.7843, 0.3922, 0)||(0.4902, 0.2941, 0.2941)||(0.4902, 0.4902, 0.5882)|
|(0.9216, 0.9216, 0.9216)||(0.0784, 0.0784, 0.0784)||(0.7843, 0, 0)||(0.7843, 0.3922, 0.1961)||(0.3922, 0.1961, 0.1961)||(0.3922, 0.3922, 0.4902)|
|(0.8824, 0.8824, 0.8824)||(0.1176, 0.1176, 0.1176)||(0.7843, 0.1961, 0.1961)||(0.5882, 0.3922, 0.1961)||(0.3922, 0.1961, 0)||(0.3922, 0.4902, 0.5882)|
|(0.8431, 0.8431, 0.8431)||(0.1569, 0.1569, 0.1569)||(0.5882, 0 0)||(0.5882, 0.3922, 0)||(0.3922, 0, 0)||(0.1961, 0.3922, 0.5882)|
|(0.8039, 0.8039, 0.8039)||(0.1961, 0.1961, 0.1961)||(0.5882, 0.1961, 0.1961)||(0.5882, 0.1961, 0)||(0.1961, 0, 0)||(0, 0.3922, 0.5882)|
The distance of each pixel in the lesion and color reference is calculated by using the following Euclidean distance:
where k = 1, 2, …, 6 and (i, j) is the pixel’s position in the lesion.
The existence of colors in lesion depends on the comparison between Dk and threshold values. For each color, there is a threshold value Tk is calculated as a distance between the highest and smallest reference points. As for example, the threshold value for white color (i.e., T1) is calculated as follows:
The same process is repeated for other colors to calculate their threshold values. The color score (ColorScore) is incremented by 1 if the Dk ≤ Tk.
The number of pixels of the greatest diameter or major axis length of the segmented lesion is transferred into millimeter scale as follows:
where dpi is the dots-per-inch which equals to 96. Then, the diameter score (DMScore) is calculated as follows:
Finally, the calculated values of the four extracted features are multiplied by their weights to receive the total dermoscopic score (TDS). The TDS is calculated by the following equation:
3.4 Classification stage
Based on the result of the TDS, the lesion is classified based of the following criteria:
An example of the whole process yields to the printed results on bottom-left corner of the image is illustrated in Figure 12.
4. Experimental results and discussion
4.1 Hardware and software specifications
The experiments are executed on processor Intel, core i3-2330 M @ 2.20 GHz and RAM 4 GB. The system type is windows 7 ultimate of 64-bit operating and the software used for research implementation is MATLAB R2013a.
The performance of our research is tested on PH2 dataset . It consists of 200 8-bit RGB dermoscopic images of melanocytic lesions with a resolution of 768 × 560 pixels. This image database contains 80 common nevi, 80 atypical nevi, and 40 melanomas. The dermoscopic images were obtained at the Dermatology Service of Hospital Pedro Hispano (Matosinhos, Portugal) under the same conditions through tuebinger mole analyzer system using a magnification of 20 times.
4.3 Implementation of ABCD rule
The ABCD rule is implemented on the PH2 dataset and a random selection of segmentation and classification of successful results are presented in Figure 13. For each image, the segmented lesion is surrounded by a solid blue line, and the calculated value of the TDS and the classification result are presented in the bottom-left corner.
The results of this research are compared with the results of , in terms of accuracy, sensitivity, and specificity. The running time for the diagnosis process of 200 8-bit RGB images is 1670 s, or an average of 8.35 s per each examined lesion.
The performance of the proposed work is evaluated by one of the well-known metrics called the confusion matrix as described in Table 2. It presents the correct and wrong classification rates that resulted from the implementation of the ABCD rule on PH2 dataset. This image database contains 80 common nevi, 80 atypical nevi, and 40 melanomas.
|Benign (B)||Suspicious (S)||High suspicious (H)|
|Actual class||Benign (B)||TP(B) = 73||E(BS) = 5||E(BH) = 2|
|Suspicious (S)||E(SB) = 11||TP(S) = 63||E(SH) = 6|
|High suspicious (H)||E(HB) = 2||E(HS) = 4||TP(HS) = 34|
Table 3 summarizes the calculated values of true positive (TP), false negative (FN), false positive (FP), and true negative (TN) of the three classes, benign (B), suspicious (S), and high suspicious (H).
|Benign (B)||Suspicious (S)||High suspicious (H)|
The accuracy, sensitivity, and specificity formulas are described in the following equations, respectively. Table 4 summarizes the achieved performance of the three classes.
|Class||Average (our work)||Average |
5. Conclusion and future work
In this work, we have developed an automatic diagnostic system using image processing techniques for preliminary diagnosis of melanoma based on the well-proven commonly used ABCD medical procedure. The proposed work used different image processing capabilities to achieve fast, affordable, easily available and highly accurate melanoma diagnosis. The overall process includes multiple modules for handling various steps: noise removal, contrast enhancement, lesion segmentation, features extraction, and classification (diagnosis).
The achieved results show an acceptable performance rates, an accuracy 90%, sensitivity 85%, and specificity 92.22%.
The following opportunities are suggested for future work:
Increase the size of the dataset, and
Attach the proposed system to various mobile devices.
The author would like to thank Dr. Saeb Zaqout for his valuable comments and constructive criticism of this manuscript.