Promising Deep Semantic Nuclei Segmentation Models for Multi-Institutional Histopathology Images of Different Organs.

Loay Hassan; Adel Saleh; Mohamed Abdel Nasser; Osama A. Omer; Domenec Puig

doi:10.9781/ijimai.2020.10.004

Authors

Loay Hassan Aswan University
Adel Saleh Gaist Solutions Ltd (United Kingdom).
Mohamed Abdel Nasser Aswan University
Osama A. Omer Aswan University
Domenec Puig Universitat Rovira i Virgili

DOI:

https://doi.org/10.9781/ijimai.2020.10.004

Keywords:

Digital Pathology, Nuclei Segmentation, Whole Slide Imaging, Deep Learning

Supporting Agencies

M. Abdel-Nasser and D. Puig are partially supported by the Spanish Government through Project under Grant PID2019-105789RB-I00.

Abstract

Nuclei segmentation in whole-slide imaging (WSI) plays a crucial role in the field of computational pathology. It is a fundamental task for different applications, such as cancer cell type classification, cancer grading, and cancer subtype classification. However, existing nuclei segmentation methods face many challenges, such as color variation in histopathological images, the overlapping and clumped nuclei, and the ambiguous boundary between different cell nuclei, that limit their performance. In this paper, we present promising deep semantic nuclei segmentation models for multi-institutional WSI images (i.e., collected from different scanners) of different organs. Specifically, we study the performance of pertinent deep learning-based models with nuclei segmentation in WSI images of different stains and various organs. We also propose a feasible deep learning nuclei segmentation model formed by combining robust deep learning architectures. A comprehensive comparative study with existing software and related methods in terms of different evaluation metrics and the number of parameters of each model, emphasizes the efficacy of the proposed nuclei segmentation models.

Downloads

Download data is not yet available.

References

Moen, E., Bannon, D., Kudo, T. et al. “Deep learning for cellular image analysis,” in Nat Methods, vol. 16, no. 12, pages. 1233–1246, 2019, doi: 10.1038/s41592-019-0403-1.

Carpenter, A.E., Jones, T.R., Lamprecht, M.R. et al. “CellProfiler: image analysis software for identifying and quantifying cell phenotypes,” in Genome Biol, vol 7, R100, 2006, doi: 10.1186/gb-2006-7-10-r100.

Schindelin, J., Arganda-Carreras, I., Frise, E. et al., “Fiji: an open-source platform for biological-image analysis,” in Nat Methods, vol. 9, no. 7, pages. 676-682, 2012.

Niazi, Muhammad Khalid Khan et al. “Digital pathology and artificial intelligence,” in The Lancet. Oncology, vol. 20, no. 5, pages e253-e261, 2019, doi: 10.1016/S1470-2045(19)30154-8.

P. Naylor, M. Laé, F. Reyal and T. Walter, “Nuclei segmentation in histopathology images using deep neural networks,” in 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), Melbourne, VIC, 2017, pp. 933-936, doi: 10.1109/ISBI.2017.7950669.

P. Naylor, M. Laé, F. Reyal and T. Walter, “Segmentation of Nuclei in Histopathology Images by Deep Regression of the Distance Map,” in 2019 IEEE Transactions on Medical Imaging, vol. 38, no. 2, pp. 448-459, Feb. 2019, doi: 10.1109/TMI.2018.2865709.

H. Wang, M. Xian and A. Vakanski, “Bending Loss Regularized Network for Nuclei Segmentation in Histopathology Images,” in 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI), Iowa City, IA, USA, 2020, pp. 1-5, doi: 10.1109/ISBI45749.2020.9098611.

Graham, Simon Matthew, Quoc Dang Vu, Shan-e-Ahmed Raza, Jin Tae Kwak and Nasir M. Rajpoot. “XY Network for Nuclear Segmentation in Multi-Tissue Histology Images,” in ArXiv abs/1812.06499 2018.

Al-Kofahi, Y., Zaltsman, A., Graves, R. et al. “A deep learning-based algorithm for 2-D cell segmentation in microscopy images,” in BMC Bioinformatics, vol. 19, no. 365, 2018, doi: 10.1186/s12859-018-2375-z.

Cui, Y., Zhang, G., Liu, Z. et al. “A deep learning algorithm for one-step contour aware nuclei segmentation of histopathology images,” in Med Biol Eng Comput, vol. 57, pages 2027–2043, 2019, doi: 10.1007/s11517-019-02008-8.

H. Qu et al., “Weakly Supervised Deep Nuclei Segmentation Using Partial Points Annotation in Histopathology Images,” in 2020 IEEE Transactions on Medical Imaging, doi: 10.1109/TMI.2020.3002244.

F. Mahmood et al., “Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images,” in 2019 IEEE Transactions on Medical Imaging, 2019, doi: 10.1109/TMI.2019.2927182.

Zhou, Y., Onder, O.F., Dou, Q., Tsougenis, E., Chen, H., Heng, P.A., “CIA-Net: Robust Nuclei Instance Segmentation with Contour-Aware Information Aggregation,” International Conference on Information Processing in Medical Imaging, 2019, pp. 682-693.

Wang, E.K., Zhang, X., Pan, L., Cheng, C., Dimitrakopoulou-Strauss, A., Li, Y., Zhe, N., “Multi-Path Dilated Residual Network for Nuclei Segmentation and Detection,” Cells Journal, Multidisciplinary Digital Publishing Institute, 2019, vol. 8, no. 5, page 499.

D. S. Mercadier, B. Besbinar and P. Frossard, “Automatic Segmentation of Nuclei in Histopathology Images Using Encoding-decoding Convolutional Neural Networks,” in 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, United Kingdom, 2019, pp. 1020-1024, doi: 10.1109/ICASSP.2019.8682502.

Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh and J. Liang, “UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation,” in 2020 IEEE Transactions on Medical Imaging, vol. 39, no. 6, pp. 1856-1867, June 2020, doi: 10.1109/TMI.2019.2959609.

Jung, H., Lodhi, B., Kang, J., “An automatic nuclei segmentation method based on deep convolutional neural networks for histopathology images,” BMC Biomedical Engineering Journal, vol.1, no. 24, 2019, doi: 10.1186/s42490-019-0026-8.

K. He, G. Gkioxari, P. Dollár and R. Girshick, “Mask R-CNN,” in 2017 IEEE International Conference on Computer Vision (ICCV), Venice, 2017, pp. 2980-2988, doi: 10.1109/ICCV.2017.322.

Lateef, Fahad, and Yassine Ruichek. “Survey on semantic segmentation using deep learning techniques,” in Neurocomputing, vol. 338, pp. 321-348, 2019.

A. Vahadane, T. Peng, A. Sethi, S. Albarqouni, L. Wang, M. Baust, K. Steiger, A. M. Schlitter, I. Esposito, and N. Navab, “Structurepreserving color normalization and sparse stain separation for histological images,” in 2016 IEEE transactions on medical imaging, vol. 35, no. 8, pp. 1962–1971, 2016.

J. Long, E. Shelhamer and T. Darrell, “Fully convolutional networks for semantic segmentation,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, 2015, pp. 3431-3440, doi: 10.1109/CVPR.2015.7298965.

Krizhevsky, Alex & Sutskever, Ilya & Hinton, Geoffrey. “ImageNet Classification with Deep Convolutional Neural Networks,” Neural Information Processing Systems, vol. 25, 2012.

Simonyan, Karen and Zisserman, Andrew. “Very Deep Convolutional Networks for Large-Scale Image Recognition,” in Computing Research Repository (CoRR). [Online]. Available: http://arxiv.org/abs/1409.1556

C. Szegedy et al., “Going deeper with convolutions,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, 2015, pp. 1-9, doi: 10.1109/CVPR.2015.7298594.

S. Jégou, M. Drozdzal, D. Vazquez, A. Romero and Y. Bengio, “The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, HI, 2017, pp. 1175-1183, doi: 10.1109/CVPRW.2017.156.

G. Huang, Z. Liu, L. Van Der Maaten and K. Q. Weinberger, “Densely Connected Convolutional Networks,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, 2017, pp. 2261-2269, doi: 10.1109/CVPR.2017.243.

N. Kumar, R. Verma, S. Sharma, S. Bhargava, A. Vahadane and A. Sethi, “A Dataset and a Technique for Generalized Nuclear Segmentation for Computational Pathology,” in 2017 IEEE Transactions on Medical Imaging, vol. 36, no. 7, pp. 1550-1560, July 2017, doi: 10.1109/TMI.2017.2677499.

O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation”, International Conference on Medical image computing and computer-assisted intervention. Springer, vol. 9351, pp. 234–241, 2015, doi: 10.1007/978-3-319-24574-4_28.

V. Badrinarayanan, A. Kendall and R. Cipolla, “SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation,” in 2017 IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 12, pp. 2481-2495, 1 Dec. 2017, doi: 10.1109/TPAMI.2016.2644615.

H. Zhao, J. Shi, X. Qi, X. Wang and J. Jia, “Pyramid Scene Parsing Network,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, 2017, pp. 6230-6239, doi: 10.1109/CVPR.2017.660.

Li, P., Xu, Y., Wei, Y., and Yang, Y. “Self-correction for human parsing,” 2019, arXiv:1910.09777.

Ruan, T., Liu, T., Huang, Z., Wei, Y., Wei, S., and Zhao, Y., “Devil in the Details: Towards Accurate Single and Multiple Human Parsing”. In 2019 Proceedings of the AAAI Conference on Artificial Intelligence, 2019 volume 33, pages 4814–4821.

Abdel-Nasser, M., Saleh, A., and Puig, D. “Channel-wise Aggregation with Self-correction Mechanism for Multi-center Multi-Organ Nuclei Segmentation in Whole Slide Imaging”. In 2020 Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, VISIGRAPP 2020, Vol. 4: VISAPP, Valletta, Malta, February 27-29, 2020 (pp. 466–473).

K. He, X. Zhang, S. Ren and J. Sun, “Deep Residual Learning for Image Recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, 2016, pp. 770-778, doi: 10.1109/CVPR.2016.90.

J. Deng, W. Dong, R. Socher, L. Li, Kai Li and Li Fei-Fei, “ImageNet: A large-scale hierarchical image database,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, 2009, pp. 248-255.

Abdel-Nasser, M., Saleh, A., Moreno, A., and Puig, D. “Automatic nipple detection in breast thermograms,” Expert Systems with Applications, vol. 64, pp. 365-374, 2016, doi: 10.1016/j.eswa.2016.08.026.

Abdel-Nasser, M., Moreno, A., and Puig, D. “Temporal mammogram image registration using optimized curvilinear coordinates,” Computer methods and programs in biomedicine, vol. 127, pp. 1-14, 2016. doi: 10.1016/j.cmpb.2016.01.019.

Wang, Xiaohong; Jiang, Xudong; Ren, Jianfeng. “Blood vessel segmentation from fundus image by a cascade classification framework,” in Pattern Recognition, 2019, vol. 88, p. 331-341.

Roberts, N., Magee, D., Song, Y., Brabazon, K., Shires, M., et. al. “Toward routine use of 3D histopathology as a research tool,” The American journal of pathology, vol. 180, no. 5, pp. 1835-1842, 2012.