Segmentation and Analysis of Diagnostic Images for Lesion Detection: Current Status and Future Potential

Lesion refers to any abnormal change in tissue or other organs due to disease or injury. All lesions can be broadly categorized as either benign or malignant. A statistics given by WHO [1] suggests that breast, cervical and colorectal cancers are predominantly claiming lives of most people worldwide. Traditionally the task of analyzing lesions is laid on the shoulders of the pathologists/ radiologists which is time-consuming. This necessitates the use of computer-assisted diagnosis (CAD) system for accurate lesion detection.

All lesion analysis and detection systems can be classified as either invasive, non-invasive and minimally invasive technique. An invasive technique is one which requires removal of a small amount of cell or tissue for microscopic examination. In contrary, non-invasive techniques do not involve the puncturing of the skin. Minimally invasive require a partial incision of a diagnostic device which is directed to the body cavity with minimal damage to a body tissue.

This study elaborates the recent state of the art techniques related to lesion segmentation. Prior to segmentation, preprocessing plays a vital role in any medical image analysis since these images are mostly of low quality and are prone to several artifacts. Amongst invasive techniques, microscopic images of the breast and cervical cancers were considered for the study. In non-invasive technique ultrasound images of breast cancer were considered and in minimally invasive technique colorectal cancer detection using colonoscopy has been considered.

The review is organized as follows: Various preprocessing techniques are discussed in Section 4, different segmentation techniques are discussed in Section 5, which is followed by existing challenges of different imaging modalities which are highlighted in Section 6. Different available datasets are summarized in Section 7. Finally, the conclusion and future potential of this work are summarized in Section 8.

Preprocessing technique is an important prerequisite for lesion segmentation since lesion images are often corrupted with multiple artifacts. In microscopic imaging preprocessing is required for removing artifacts arising due to poor staining and contrast enhancement. Illumination and color normalization [2] is employed for contrast enhancement. Thresholding, morphological operations, adaptive filters are used for noise reduction and image smoothening.

Ultrasound images are often corrupted with three predominant artifacts [3] namely speckle noise, acoustic shadowing, and reverberation. Conventional filters like median, adaptive weighted mean, Weiner, Gaussian, Kuan[4], Lee [5] filters have been used widely used by researchers to eliminate these artifacts. However, these filters have failed to give satisfactory results since they blur details and lesion edges.

The colonoscopy frames often suffer from non-uniform appearance, prone to noise, artifacts due to scene illumination (over-exposed region and specular highlights) [6]. Histogram modification, noise filtering, edge detection, specular highlights correction are evaluated as preprocessing step to mitigate such effects with minimal loss of data [7].

In microscopic imaging, segmentation mainly aims at individual nucleus detection and overlapping nuclei separation. This is a challenging task since each nucleus in the image does look different due to its type, malignancy of the disease and nuclei life cycle. The segmentation techniques for nucleus detection can be broadly classified as either thresholding based, clustering based, graph cuts, watershed-based, active contour-based, shape-based prior, entropy-based, deep learning based etc.

Figure 1:(a) Microscopic image of Pap smear cervical cells
(b) Segmented nuclei
(c) Segmented Cytoplasm

Figure 2: (a) Microscopic image of H&E stained biopsy slide (b) Segmented cells

Lee et al. [8] have used super pixel partitioning followed by triangle based thresholding and graph cut for cervical cell segmentation. They have worked on ISBI 2014 and 2015 challenge datasets and have achieved dice coefficient of 0.897 and 0.879 respectively on the two datasets. Zhang et al. [9] have used graph cut based approach for cervical cell segmentation and have achieved 93% accuracy for cytoplasm segmentation. Clustering based approach followed by radiating gradient vector flow snake has been used by Li et al. [10] for cervical cell segmentation. Genctav et al. [11] have used binary classifier and watershed segmentation for segmenting the cervical cells and have obtained Zijdenbos similarity index (ZSI) of 0.89. Veta et al. [12] have used a marker- controlled watershed algorithm and fast radial symmetry transform to segment the microscopic images of H&E stained breast cancer slides and obtained accuracy 81.2%. Jain et al. [13] have used Active Contour Model with General Classifier Neural Network (GCNN) for segmenting breast cancer histopathology images and have attained accuracy 83.47%. Basavanhally et al. [14] have used Active Contour model based on a color gradient with a hierarchical normalized cut for segmenting breast cancer images and obtained segmentation accuracy 89%. Bergmeir et al. [15] have used voting scheme and prior knowledge for nuclei detection and elliptical shape prior to cytoplasm segmentation in Pap smear images of cervical cells and have attained TPR of 95.63%. Nosrati et al. [16] have used starshaped prior using directional directives for segmenting cervical cells in ISBI 2014 challenge dataset and DC obtained was them was 0.88. Entropy-based thresholding have been used by Paul et al. [17] for mitosis detection in Mitos-Atypia-14 challenge dataset. Song et al. [18] have used deep learning multi-scale convolutional network (MSCN), a graph partitioning based approach for cervical cell segmentation and obtained 90% accuracy. Some of the segmented lesions in microscopic images are shown in (Figure 1 & 2).

Figure 3: (a) Ultrasound image of breast lesion (b) Segmented lesion

The field of ultrasound imaging for lesion detection has been very less explored and very few state-of-arts is present for segmenting lesions from ultrasound breast cancer images. Tan et al. [19] have used voxel features characterizing coronal speculation patterns, blobs, contrast, depth for breast cancer segmentation in ultrasound images and obtained 64% accuracy. Sadek et al. [20] have used median filtering for noise removal followed by the normalized cut approach and k-means clustering for ultrasound breast cancer image segmentation and obtained DC 0.8133. Yap et al. [21] have used deep learning based approach for segmentation purpose and obtained TPR of 0.91 and 0.77 respectively on two different datasets. An ultrasound image of a breast lesion and its segmented result is shown in Figure 3.

Detection and localization of polyp [22] from colonoscopy frames aims to accurately segment the area of the image where the polyp is in the frame. Though initially, polyps are benign, they might become malignant over time being ultimately responsible for complications. So segmentation of polyp in its early stage plays a crucial role in decreasing the mortality rate of patients. Nima Tajbakhsh et al. [23] presented a hybrid approach for polyp detection. Authors have combined the shape and context information around the polyp boundaries to detect the polyp. But the method fails to detect some complex shapes. Two datasets have been evaluated in their study namely CVC-Colon DB and ASU-Mayo. The sensitivity for CVC-Colon DB is 88% whereas for ASU-Mayo it is 48%. In [7] authors have made an attempt to segment a polyp from the preprocessed gradient image using the watershed algorithm. The segmentation process will yield a large number of regions so region merging is done to get relevant regions. The performance of their method is justified by Annotated Area Covered (AAC) and Dice Similarity Coefficient (DICE). AAC for their method is 70.29% and DICE is 44.6%. The cons of their method are the reliability of the method on color and texture and the detection rate declines in presence of lumen, specular highlights, and wrinkles. In another work Ruikai Zhang et al. [24] have introduced CNN architecture with transfer learning strategy from two nonmedical databases Places205 and ILSVRC2012. The trained set is tested on PWH database of colorectal frames. The accuracy, recall, and precision of their transfer learning strategy are 85.9%, 87.4%, and 87.3% respectively. The major drawback of this learning strategy is lack of a number of samples in the dataset which declines the performance measure. A polyp and its segmented result are depicted in Figure 4.

Figure 4 : (a) Polyp Image of colon
(b) Segmented polyp

This section describes some of the major challenges in invasive, non-invasive, and minimally invasive techniques for lesion detection. Amongst invasive technique, microscopic imaging requires exact extraction of nucleus and cytoplasm boundaries. However, most of the works in literature fails to give promising results for nuclei and cytoplasm segmentation with extensive overlapping. In addition, very less work has been reported for mitotic nuclei detection since mitotic nuclei segmentation is very challenging owing to its huge variation in shape, size. The main challenges in non-invasive techniques (ultrasound imaging) is conventional filters used widely by researchers for eliminating speckle noise fail to give satisfactory results since they blur lesion edges. In addition, no work in literature has been done for acoustic shadowing and reverberation removal. Some of the notable challenges in minimally-invasive technique (colonoscopy) are due to poor image quality/artifacts captured by the colonoscopy camera due to the complex structure of the gastrointestinal tract. Furthermore, due to frequent movement of the camera same polyp appears in different shape and size. Homogeneity in color intensity between polyp and the non-polyp region is also a major issue.

This section highlights the different available datasets for lesion detection using different imaging modalities. The available datasets for invasive microscopic imaging are Break His dataset [25] (9109 microscopic biopsy images of breast cancer), Mitos-Atypia-14 challenge dataset [26] (284 microscopic biopsy images of breast cancer), Mitos dataset [27] (50 microscopic biopsy images of breast cancer), ISBI 2014 [28] and ISBI 2015 [29] challenge datasets for cervical segmentation having 16 and 17 extended depth of field (EDF) images respectively. Only one dataset [30] of Sirindhorn International Institute is available for non-invasive ultrasound breast cancer imaging having 226 images. For minimally invasive colonoscopy imaging there are three datasets namely [31] CVC- Colon DB, ASU-Mayo Clinic Colonoscopy Video Database and PWH database having 300, 5200, 1104 frames respectively.

The review conducted in this study demonstrates different methodologies proposed by different researchers for lesion detection using different imaging modalities. Though an extensive research work has been done for lesion detection, there are certain fields which are yet to be explored. In microscopic imaging, no state of the art presented above could handle overlapping of nuclei and cytoplasm especially in the cases where the degree of overlap is very high. In addition, most of the works in literature segments normal or lymphocytic nuclei and very less work have been done on segmentation of mitotic nuclei which is crucial for malignant lesion detection. In ultrasound imaging, filters designed for removing the speckle noise, blur, lesion edges and other minute details which is undesirable so proper filters need to be designed. In addition, no work has been reported on acoustic shadowing and reverberation removal. Many handcrafted methodologies have been explored to localize and segment a polyp in colonoscopy frames but the methods cannot cope with all the aforementioned challenges. Current trends inhigh-level complex image processing techniques can be able to detect a polyp in light of such challenges with an acceptable degree of accuracy. Computer-aided accurate detection of lesions from different imaging modalities canlimit the technical efforts of medical experts. Hence, it will help them in the complex decision- making process in the management of the disease.

The first author is grateful to Department of Science and Technology (DST), Government of India for providing her Junior Research Fellowship (JRF) under DST INSPIRE fellowship program (IF170366).

1. (2018) World Health Organization (WHO) report on "Cancer”.
2. Gurcan MN, Boucheron LE, Can A, Madabhushi A, Rajpoot NM, et al. (2009) Histopathological image analysis: A review. IEEE reviews in biomedical engineering 2: 147-171.?
3. Kremkau FW, Taylor KJW (1986) Artifacts in ultrasound imaging. Journal of ultrasound in medicine 5(4): 227-237.
4. Kuan DT, Sawchuck AA, Strand TC, Chavel P (1985) Adaptive noise smoothing filter for images with signal-dependent noise. IEEE Transactions on Pattern Analysis and Machine Intelligence 7(2): 165177.
5. Lee J (1980) Digital image enhancement and noise filtering by use of local statistics. IEEE Transactions on Pattern analysis and machine intelligence 2(2): 165-168.
6. Bernal J, Sanchez FJ, Fernandez-Esparrach G, Gil D, Rodriguez C, et al. (2015) WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians. Comput Med Imag Graph 43: 99-111.
7. Bernal J, Vilarino SF (2012) Towards automatic polyp detection with a polyp appearance model. Pattern Recognition 45 (9): 3166-3182.
8. Hansang L, Junmo K (2016) Segmentation of overlapping cervical cells in microscopic images with super pixel partitioning and cell-wise contour refinement. IEEE International Conference on Computer Vision and Pattern Recognition Workshops, pp. 63-69.
9. Zhang L, Kong H, Chin CT, Shaoxiong L, Zhi C, et al. (2014) Segmentation of cytoplasm and nuclei of abnormal cells in cervical cytology using global and local graph cuts. Comput Med Imaging Graph 38(5): 369-380.
10. Kuan L, Zhi L, Wenyin L, Jianping Y (2012) Cytoplasm and nucleus segmentation in cervical smear images using Radiating GVF Snake. Pattern Recognition 45(4): 1255-1264.
11. Genftav A, Aksoy S, Onder S (2012) Unsupervised segmentation and classification of cervical cell images. Pattern Recognition 45(12): 41514168.
12. Veta M, Diest PJV, Kornegoor R, Huisman A, Viergever MA, et al. (2013) Automatic nuclei segmentation in H&E stained breast cancer histopathology images. PloS one 8(7): e70221.
13. Aashna J, Shwetal A, Satender V, Srivastava V (2014) Cancerous cell detection using histopathological image analysis. International Journal of Innovative Research in Computer and Communication Engineering 2(12): 7419-7426.
14. Basavanhally A, Ganesan S, Feldman M, Shih N, Carolyn M, et al. (2013) Multi-field-of-view framework for distinguishing tumor grade in ER+ breast cancer from entire histopathology slides. IEEE Trans Biomed Eng 60(8): 2089-2099.
15. Bergmeir C, Silvente MG, Benitez JM (2012) Segmentation of cervical cell nuclei in high-resolution microscopic images: A new algorithm and a web-based software framework. Computer methods and programs in biomedicine 107(3): 497-512.
16. Nosrati MS, Hamarneh G (2015) Segmentation of overlapping cervical cells: A variational method with star-shape prior. International Symposium on Biomedical Imaging (ISBI), pp. 186-189.
17. Angshuman P, Mukherjee DP (2015) Mitosis detection for invasive breast cancer grading in histopathological images. IEEE Transactions on Image Processing 24(11): 4041-4054.
18. Youyi S, Ling Z, Siping C, Dong N, Baiying L, et al. (2015) Accurate segmentation of cervical cytoplasm and nuclei based on multiscale convolutional network and graph partitioning. IEEE Transactions on Biomedical Engineering, IEEE 62(10): 2421-2433.
19. Tan T, Platel B, Roel M, Lazlo T, Mann RM, et al. (2013) Computer-aided detection of cancer in automated 3-D breast ultrasound. IEEE Trans Med Imaging 32(9): 1698-1706.
20. Ibrahim S, Elawady M, Stefanovski V (2016) Automated breast lesion segmentation in ultrasound images. arXiv preprint arXiv: 1609.08364.
21. MoiHoon Y, Gerard P, Joan M, Sergi G, Melcior S, et al. (2017) Automated breast ultrasound lesions detection using convolutional neural networks. IEEE J Biomed Health Inform.
22. Silva J, Histace A, Romain O, Dray X, Granado B, et al. (2014) Toward embedded detection of polyps in WCE images for early diagnosis of colorectal cancer. Int J Comput Assist Radiol Surg 9(2): 283-293.
23. Tajbakhsh N, Gurudu SR, Liang J (2016) Automated polyp detection in colonoscopy videos using shape and context information, IEEE Trans Med Imag 35(2): 630-644.
24. Zhang R, Zheng Y, Mak TW, Yu R, Wong SH, et al. (2017) Automatic detection and classification of colorectal polyps by transferring low- level CNN features from the nonmedical domain. IEEE J Biomed Health Inform 21(1): 41-47.
25. (2018) Breast Cancer Histopathological Database (BreakHis).
26. (2018) Mitos-Atypia-14.
27. (2018) MITOS Dataset.
28. (2014) Overlapping Cervical Cytology Image Segmentation Challenge -ISBI 2014.
29. (2015) The second overlapping cervical cytology image segmentation challenge - ISBI 2015.
30. Medical image database provided by Biomedical Engineering Unit of Sirindhorn International Institute of Technology.
31. Bernal J (2017) Comparative validation of polyp detection methods in video colonoscopy: Results from the MICCAI 2015 endoscopic vision challenge. IEEE Trans Med Imaging 36(6): 1231-1249.