Deep Learning in Medical Imaging

Abstract

Medical image processing tools play an important role in clinical routine in helping doctors to establish whether a patient has or does not have a certain disease. To validate the diagnosis results, various clinical parameters must be defined. In this context, several algorithms and mathematical tools have been developed in the last two decades to extract accurate information from medical images or signals. Traditionally, the extraction of features using image processing from medical data are time-consuming which requires human interaction and expert validation. The segmentation of medical images, the classification of medical images, and the significance of deep learning-based algorithms in disease detection are all topics covered in this chapter.

1. Introduction

Recently, artificial intelligence (AI) is considered as a revolution across the medical field and one of the main factors of this AI revolution is deep learning (DL). The origin of DL and neural networks dates back to 1950. Yet, with the introduction of medically annotated big data, necessary for training, and the availability of high-performance computing, the recent years seem to mark a turning point for DL in medical imaging.

Accordingly, this branch of AI is recently applied to several healthcare problems such as computer-aided diagnosis, disease identification, image segmentation and classification, etc. Unlike classical tools, the powerful key of DL derives from the ability to automatically learn complex features without the need for human interaction. Nevertheless, many challenges still exist in medical health including privacy and heterogeneity of datasets. In this chapter, we will survey the application of DL in clinical imaging, and we will highlight the main challenges and future directions of this tool.

2. Deep learning-based segmentation in medical imaging

Deep learning algorithms were used in many medical applications to solve problems with segmentation, image classification, and pathology diagnosis. The manual segmentation process is time-consuming for radiologists because it is typically done slice by slice. Furthermore, segmentation results are susceptible to inter and interobserver variability. To address these limitations, several approaches based on active contour, level set, and statistical shape modeling [1, 2, 3] have been proposed to segment the extent of various pathologies or anatomical geometries. All of the methods mentioned above, however, are still semi-automated and require human interaction [4].

With the advent of DL, a fully automated segmentation of serial medical images is become possible in a few seconds. Several studies in the literature reported that segmentation algorithms based on AI outperformed the other classical models [5, 6]. Convolutional neural networks (CNNs) are the most used architecture to segment medical images. It consists of reducing the spatial dimensionality of the original image data through a series of the network layers by performing convolution and pooling operations. Other DL architectures were also proposed for this task such as deep neural network (DNN), artificial neural network (ANN), fully convolutional network (FCN), ResNet-50, and VGGNet-16 [7, 8, 9, 10]. Figure 1 describes the tasks involved in segmenting cardiac images for various imaging modalities.

Figure 1.

Overview of cardiac image segmentation tasks for different imaging modalities [11].

The success of DL-based medical image segmentation inspired other studies to reevaluate the traditional approaches to image segmentation and incorporate DL models into their work. Many factors have facilitated the increased use of DL. Among them, we can note the availability of medical data and the evolution of graphics processors’ performances.

Each year, large, annotated datasets were published online. These data were collected during many challenges such as medical segmentation decathlon and medical image computing and computer aided interventions (MICCAI). Table 1 summarizes the largest medical images datasets available online.

Dataset	Target	Modality	Source
Kaggle	Various diseases	X-rays, MRI and CT	Google LLC; https://www.kaggle.com/
NIH Image Gallery	Various diseases	X-rays, MRI, CT, PET.	National Institutes of Health (NIH); https://www.flickr.com/photos/nihgov/
ImageNet	Cancer, diabetes, and Alzheimer’s disease.	Genomic data	AI researchers; https://www.image-net.org/
Google Dataset Search	Various diseases	X-rays, MRI, CT, PET, echography	Google; https://datasetsearch.research.google.com/
UCI Machine Learning Repository	Various diseases	X-rays, MRI, CT, PET, echography	The National Science Foundation; https://archive.ics.uci.edu/ml/index.php
Stanford Medical ImageNet	Various diseases	X-rays, MRI scans, and CT scans.	Stanford University; https://aimi.stanford.edu/medical-imagenet
Open Images Dataset	Various diseases	All medical Imaging techniques	Google in collaboration with CMU and Cornell Universities; https://storage.googleapis.com/openimages/web/index.html
Cancer Imaging Archive	Cancer	Images from a variety of cancer types	National Cancer Institute (NCI); https://www.cancerimagingarchive.net/access-data/
Alzheimer’s Disease Neuroimaging Initiative	Alzheimer’s disease	brain scans and related data from MRI	Foundation for the National Institutes of Health; https://adni.loni.usc.edu/
The Microsoft COCO dataset	Various diseases	All medical Imaging techniques	Microsoft; https://cocodataset.org/#home
MIAS dataset	Various diseases	Mammographic images	Organization of UK research groups; https://www.kaggle.com/datasets/kmader/mias-mammography

Table 1.

Medical images datasets available online.

Segmentation based on DL were applied in different field of medical imaging [12, 13, 14]. In cardiac MRI, several DL models were used to delineate the contours of the myocardium which represent a crucial step to compute useful clinical parameters for the evaluation of cardiac function [15]. DL was also applied for the segmentation of different types and stage of cancer. For breast cancer, the data include mammography, ultrasound, and MRI images [16, 17, 18]. Other DL architectures were also proposed in the literature to segment cervical cancer based on Magnetic Resonance Imaging (MRI), computed tomography (CT), and positron emission tomography (PET) scan data [19]. Zhao et al. [20] proposed a new model of DL that combined U-net with progressive growing of U-net+ (PGU-net) for automated segmentation of cervical nuclei. In their study, they reported a segmentation accuracy of 92.5%. Similarly, Liu et al. [21] applied a modified U-net model on CT images for clinical target volume delineation in cervical cancer. In their proposed architecture, the encoder and decoder components were replaced with dual path network (DPN) components. The mean dice similarity coefficient (DSC) and the Hausdorff distance (HD) values of the model were 0.88 and 3.46 mm.

Although image segmentation based on DL facilitates the detection, characterization, and analysis of different lesions in medical images, it still suffers from several limitations. First, the problem of missing border regions in medical images should be considered [22]. Furthermore, the imbalanced data available online could significantly affect the segmentation performances. In medical imaging, the collection of balanced data is challenging since images related to controls are largely available compared to those associated with different pathologies. As a result, some models have been proposed to mitigate this problem. These models include convolutional autoencoders [23] and generative adversarial networks (GAN) [24]. The concept is based on the extraction of information from original images and to generate a similar dataset image based on linear transformation, e.g., reflection, rotation, translation.

3. Deep learning-based classification in medical imaging

Additionally, DL have demonstrated its superiority in the categorization of medical images, more notably in the distinction of various disorders. The extraction of key features is a step in the classification process that produces a model that can categorize a picture into multiple classes. To extract features using color or texture, several classical classifications have been dressed in the literature [25, 26, 27]. Support vector machines (SVMs), logistic regression, closest neighbors, etc. can be mentioned among them. These systems must, however, cope with other challenging problems related to medical imaging. First, the presence of artifacts in medical images may make it more difficult to categorize. Because of this, pre-processing is crucial to improving image quality. The second problem is the complexity of medical content captured by many modalities. The classification of medical images is extremely difficult because each modality has distinct characteristics.

Recently, several researchers used DL for the medical classification task and the results proved the accuracy of their models in comparison with the traditional machine learning approaches [28]. Deep learning’s key benefit is its ability to quickly distinguish between various structures in images without the need for manual feature extraction. Recent DL architectures also have the capacity to incorporate a variety of features gathered from many modalities to produce an effective classifier.

Yadav and Jadhav [29] used a DL algorithm based on the transfer learning of VGG16 to classify pneumonia from chest X rays’ images. In their study, they showed that the VGG16 outperformed the classical method based on SVM. The accuracy was 0.923 for VGG16 vs. 0.776 for SVM. Similarly, Xu et al. [30] tested a deep CNN in histopathology images to extract new features for the classification of colon cancer. Lai et al. [31] proposed a new architecture that combines coding network with multilayer perceptron (CNMP) with other features extracted from deep CNN.

In their study, they showed an accuracy of 90.2%. Although DL achieved high performance in the classification of medical images, it still suffers from numerous limitations. The major challenge is the reduced number of annotated data needed for the classification of medical images. Labeling data require the intervention of experienced radiologists. A few solutions have been proposed to resolve this issue. Pujitha and Sivaswamy [32] proposed a crowd-sourcing and synthetic image generation for training deep neural net-based lesion detection. In their study, they used color fundus retinal and they proved that crowd-sourcing improves the area under the curve (AUC) by 25%. The generative adversarial networks (GAN) is also another source of generating synthetic images with annotations. Aljohan and Alharbe [33] proposed a new GAN to generate synthetic medical images with the corresponding annotations from different medical modalities. The classification of medical images based on DL has shown good results. However, there are still several issues in medical image processing that need to be addressed with the different DL architectures.

4. Disease diagnosis based on deep learning

Early and precise diagnosis is crucial for the treatment of different diseases and for the estimation of a severity grade. The use of DL for the diagnosis of diseases is a dynamic research area that attracts several researchers worldwide. In fact, DL architectures have been applied to some specific pathologies such as cancer, heart disease, diabetes, and Alzheimer’s disease [34, 35]. The increasing number of medical imaging dataset led different researchers to use deep learning models for the diagnosis of different diseases.

DL algorithms have proven their performances in the prediction and diagnosis of cancer diseases. The availability of images derived from MRI, CT, mammography, and biopsy helped several researchers to use these data for early cancer detection. The analysis of cancer images includes the detection of tumor area, the classification of different cancer stages, and the extraction of different characteristics for tumors [36].

Recently, Shen et al. [37] used a modified version of CNNs for the screening of breast cancer using mammography data. The outcomes of their study showed an AUC of 0.95 and a specificity of 96.1%. A CNN was also applied for the classifications of different kinds of cancer and the detection of carcinoma. Figure 2 depicts the entire image categorization process for breast cancer screening using DL architecture.

Figure 2.

Deep learning for the screening of breast cancer [37].

Alanazi et al. [38] applied the transfer DL model to detect brain tumor in the early stage by using various types of tumor data. Furthermore, another study used a 3D deep CNN to assess the glioma grade (low or high-grade glioma). In their study, they reported an accuracy of 96.49% [39]. Compared to the classical algorithms, the different studies proved the efficiency of DL in the prediction and analysis of cancer. However, bigger medical data available online are needed for more adequate validation.

5. Conclusion

As has been shown, using medical image processing techniques in clinical practice is crucial for determining if a patient has a particular disease or not. The field of medical imaging has been transformed by AI and DL, which enable more precise and automatic feature extraction from medical data. DL has been used to address a variety of healthcare issues, including image segmentation and classification, disease detection, computer-aided diagnosis, and the learning of complex features without human interaction. Despite the advances made, many challenges still exist in medical health including privacy and heterogeneity of datasets.

Conflict of interest

The authors declare no conflict of interest.