Journal Article2022

TD-CNN-LSTM: A Hybrid approach combining CNN and LSTM to classify brain tumor on 3D MRI scans performing ablation study

Sidratul Montaha, Sami Azam, AKM Rakibul Haque Rafid, Md Zahid Hasan, Asif Karim, Ashraful Islam

IEEE Access

IEEE, Vol. 10, pp. 60039–60059, ISBN: 2169-3536

DOI: 10.1109/access.2022.3179577URL

CCDS Authors

Ashraful Islam

Ashraful Islam, PhD

Center DirectorDirector, Human-Computer Interaction Wing

References

  1. 1.David N. Louis, Arie Perry, Guido Reifenberger, Andreas von Deimling, Dominique Figarella-Branger, Webster K. Cavenee, Hiroko Ohgaki, Otmar D. Wiestler, Paul Kleihues, David W. Ellison. (2016). The 2016 World Health Organization Classification of Tumors of the Central Nervous System: a summary. Acta Neuropathologica, 131(6), 803–820[10.1007/s00401-016-1545-1]
  2. 2.Konstantinos Kamnitsas, Christian Ledig, Virginia Newcombe, Joanna Simpson, Andrew D. Kane, David Menon, Daniel Rueckert, Ben Glocker. (2016). Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Medical Image Analysis, 36, 61–78[10.1016/j.media.2016.10.004]
  3. 3.Marios Anthimopoulos, Stergios Christodoulidis, Lukas Ebner, Andreas Christe, Stavroula Mougiakakou. (2016). Lung Pattern Classification for Interstitial Lung Diseases Using a Deep Convolutional Neural Network. IEEE Transactions on Medical Imaging, 35(5), 1207–1216[10.1109/tmi.2016.2535865]
  4. 4.Xuan-Hien Le, Hung Viet Ho, Giha Lee, Sungho Jung. (2019). Application of Long Short-Term Memory (LSTM) Neural Network for Flood Forecasting. Water, 11(7), 1387[10.3390/w11071387]
  5. 5.Evangelia I. Zacharaki, Sumei Wang, Sanjeev Chawla, Dong Soo Yoo, Ronald L. Wolf, Elias R. Melhem, Christos Davatzikos. (2009). Classification of brain tumor type and grade using MRI texture and shape in a machine learning scheme. Magnetic Resonance in Medicine, 62(6), 1609–1618[10.1002/mrm.22147]
  6. 6.Milica Badža Atanasijević, Marko Barjaktarović. (2020). Classification of Brain Tumors from MRI Images Using a Convolutional Neural Network. Applied Sciences, 10(6), 1999[10.3390/app10061999]
  7. 7.Wenjie Lu, Jiazheng Li, Yifan Li, Sun Aijun, Jingyang Wang. (2020). A CNN-LSTM-Based Model to Forecast Stock Prices. Complexity, 2020, 1–10[10.1155/2020/6622927]
  8. 8.Ibrahem Kandel, Mauro Castelli. (2020). The effect of batch size on the generalizability of the convolutional neural networks on a histopathology dataset. ICT Express, 6(4), 312–315[10.1016/j.icte.2020.04.010]
  9. 9.Emrah Irmak. (2021). Multi-Classification of Brain Tumor MRI Images Using Deep Convolutional Neural Network with Fully Optimized Framework. Iranian Journal of Science and Technology Transactions of Electrical Engineering, 45(3), 1015–1036[10.1007/s40998-021-00426-9]
  10. 10.Jin Liu, Min Li, Jianxin Wang, Fang‐Xiang Wu, Tianming Liu, Yi Pan. (2014). A survey of MRI-based brain tumor segmentation methods. Tsinghua Science & Technology, 19(6), 578–595[10.1109/tst.2014.6961028]
  11. 11.Mohamed A. Naser, M. Jamal Deen. (2020). Brain tumor segmentation and grading of lower-grade glioma using deep learning in MRI images. Computers in Biology and Medicine, 121, 103758[10.1016/j.compbiomed.2020.103758]
  12. 12.Richard Bitar, General Leung, Richard Perng, Sameh Tadros, Alan R. Moody, Josée Sarrazin, Caitlin McGregor, Monique Christakis, Sean Symons, Andrew Nelson, Timothy P. L. Roberts. (2006). MR Pulse Sequences: What Every Radiologist Wants to Know but Is Afraid to Ask. Radiographics, 26(2), 513–537[10.1148/rg.262055063]
  13. 13.Hiba Mzoughi, Ines Njeh, Ali Wali, Mohamed Ben Slima, Ahmed BenHamida, Chokri Mhiri, Kharedine Ben Mahfoudhe. (2020). Deep Multi-Scale 3D Convolutional Neural Network (CNN) for MRI Gliomas Brain Tumor Classification. Journal of Digital Imaging, 33(4), 903–915[10.1007/s10278-020-00347-9]
  14. 14.Chiyu Feng, Ahmed Elazab, Peng Yang, Tianfu Wang, Feng Zhou, Huoyou Hu, Xiaohua Xiao, Baiying Lei. (2019). Deep Learning Framework for Alzheimer’s Disease Diagnosis via 3D-CNN and FSBi-LSTM. IEEE Access, 7, 63605–63618[10.1109/access.2019.2913847]
  15. 15.Gopal S. Tandel, Antonella Balestrieri, Tanay Jujaray, Narendra N. Khanna, Luca Saba, Jasjit S. Suri. (2020). Multiclass magnetic resonance imaging brain tumor classification using artificial intelligence paradigm. Computers in Biology and Medicine, 122, 103804[10.1016/j.compbiomed.2020.103804]
  16. 16.Vincent K.Y. Ho, Jaap C. Reijneveld, Roelien H. Enting, Henri P. Bienfait, Pierre A. Robe, Brigitta G. Baumert, Otto Visser. (2014). Changing incidence and improved survival of gliomas. European Journal of Cancer, 50(13), 2309–2318[10.1016/j.ejca.2014.05.019]
  17. 17.Ramy A. Zeineldin, Mohamed E. Karar, Jan Coburger, Christian R. Wirtz, Oliver Burgert. (2020). DeepSeg: deep neural network framework for automatic brain tumor segmentation using magnetic resonance FLAIR images. International Journal of Computer Assisted Radiology and Surgery, 15(6), 909–920[10.1007/s11548-020-02186-z]
  18. 18.Ying Zhuge, Holly Ning, Peter Mathen, Jason Cheng, Andra Krauze, Kevin Camphausen, Robert W. Miller. (2020). Automated glioma grading on conventional MRI images using deep convolutional neural networks. Medical Physics, 47(7), 3044–3053[10.1002/mp.14168]
  19. 19.D. Jude Hemanth, J. Anitha, Antoanela Naaji, Oana Geman, Daniela Elena Popescu, Le Hoang Son. (2018). A Modified Deep Convolutional Neural Network for Abnormal Brain Image Classification. IEEE Access, 7, 4275–4283[10.1109/access.2018.2885639]
  20. 20.Dinthisrang Daimary, Mayur Bhargab Bora, Khwairakpam Amitab, Debdatta Kandar. (2020). Brain Tumor Segmentation from MRI Images using Hybrid Convolutional Neural Networks. Procedia Computer Science, 167, 2419–2428[10.1016/j.procs.2020.03.295]
  21. 21.Huihui Qiao, Taiyong Wang, Peng Wang, Shi-bin Qiao, Lan Zhang. (2018). A Time-Distributed Spatiotemporal Feature Learning Method for Machine Health Monitoring with Multi-Sensor Time Series. Sensors, 18(9), 2932[10.3390/s18092932]
  22. 22.Khizir Mahmud, Sami Azam, Asif Karim, SM Zobaed, Bharanidharan Shanmugam, Deepika Mathur. (2021). Machine Learning Based PV Power Generation Forecasting in Alice Springs. IEEE Access, 9, 46117–46128[10.1109/access.2021.3066494]
  23. 23.Chaity Banerjee, Tathagata Mukherjee, Eduardo L. Pasiliao. (2019). An Empirical Study on Generalizations of the ReLU Activation Function. , 164–167[10.1145/3299815.3314450]
  24. 24.Linmin Pei, Lasitha Vidyaratne, Md Monibor Rahman, Khan M. Iftekharuddin. (2020). Context aware deep learning for brain tumor segmentation, subtype classification, and survival prediction using radiology images. Scientific Reports, 10(1), 19726[10.1038/s41598-020-74419-9]
  25. 25.Saima Rathore, Hamed Akbari, Jimit Doshi. (2018). Radiomic signature of infiltration in peritumoral edema predicts subsequent recurrence in glioblastoma: implications for personalized radiotherapy planning. Journal of Medical Imaging, 5(02), 1[10.1117/1.jmi.5.2.021219]
  26. 26.BrainLes 2017 Québec, Alessandro Crimi, Spyridon Bakas, Hugo J. Kuijf, Bjoern Menze, Mauricio Reyes, MICCAI 2017 Québec. (2018). Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Lecture notes in computer science[10.1007/978-3-319-75238-9]
  27. 27.Iram Shahzadi, Tong Boon Tang, Fabrice Meriadeau, Abdul Quyyum. (2018). CNN-LSTM: Cascaded Framework For Brain Tumour Classification. , 633–637[10.1109/iecbes.2018.8626704]
  28. 28.Thao Thi Ho, Taewoo Kim, Woo Jin Kim, Chang Hyun Lee, Kum Ju Chae, So Hyeon Bak, Sung Ok Kwon, Gong Yong Jin, Eun-Kee Park, Sanghun Choi. (2021). A 3D-CNN model with CT-based parametric response mapping for classifying COPD subjects. Scientific Reports, 11(1), 34[10.1038/s41598-020-79336-5]
  29. 29.Eali Stephen Neal Joshua, Debnath Bhattacharyya, Midhun Chakkravarthy, Yung-Cheol Byun. (2021). 3D CNN with Visual Insights for Early Detection of Lung Cancer Using Gradient-Weighted Class Activation. Journal of Healthcare Engineering, 2021, 1–11[10.1155/2021/6695518]
  30. 30.Fangyuan Lei, Xun Liu, Qingyun Dai, Bingo Wing‐Kuen Ling. (2019). Shallow convolutional neural network for image classification. SN Applied Sciences, 2(1)[10.1007/s42452-019-1903-4]
  31. 31.Hira Saleem, Ahmad Raza Shahid, Basit Raza. (2021). Visual interpretability in 3D brain tumor segmentation network. Computers in Biology and Medicine, 133, 104410[10.1016/j.compbiomed.2021.104410]
  32. 32.Hassan A. Khalil, Saad M. Darwish, Yasmine M. Ibrahim, Osama Farouk Hassan. (2020). 3D-MRI Brain Tumor Detection Model Using Modified Version of Level Set Segmentation Based on Dragonfly Algorithm. Symmetry, 12(8), 1256[10.3390/sym12081256]
  33. 33.Alexey A. Novikov, David Major, Maria Wimmer, Dimitrios Lenis, Katja Buhler. (2018). Deep Sequential Segmentation of Organs in Volumetric Medical Scans. IEEE Transactions on Medical Imaging, 38(5), 1207–1215[10.1109/tmi.2018.2881678]
  34. 34.Tommaso Banzato, Marco Bernardini, Giunio Bruto Cherubini, Alessandro Zotti. (2018). A methodological approach for deep learning to distinguish between meningiomas and gliomas on canine MR-images. BMC Veterinary Research, 14(1), 317[10.1186/s12917-018-1638-2]
  35. 35.Sheeraz Arif, Jing Wang, Tehseen Ul Hassan, Zesong Fei. (2019). 3D-CNN-Based Fused Feature Maps with LSTM Applied to Action Recognition. Future Internet, 11(2), 42[10.3390/fi11020042]
  36. 36.Shaode Yu, Shibin Wu, Lei Wang, Fan Jiang, Yaoqin Xie, Leida Li. (2017). A shallow convolutional neural network for blind image sharpness assessment. PLoS ONE, 12(5), e0176632[10.1371/journal.pone.0176632]
  37. 37.Augustin Lecler, Loïc Duron, Daniel Balvay, Julien Savatovsky, Olivier Bergès, Mathieu Zmuda, E. Farah, O. Galatoire, Afef Bouchouicha, Laure Fournier. (2019). Combining Multiple Magnetic Resonance Imaging Sequences Provides Independent Reproducible Radiomics Features. Scientific Reports, 9(1), 2068[10.1038/s41598-018-37984-8]
  38. 38.Caroline Chung, Ur Metser, Cynthia Ménard. (2015). Advances in Magnetic Resonance Imaging and Positron Emission Tomography Imaging for Grading and Molecular Characterization of Glioma. Seminars in Radiation Oncology, 25(3), 164–171[10.1016/j.semradonc.2015.02.002]
  39. 39.Giovanni Ercolano, Silvia Rossi. (2021). Combining CNN and LSTM for activity of daily living recognition with a 3D matrix skeleton representation. Intelligent Service Robotics, 14(2), 175–185[10.1007/s11370-021-00358-7]
  40. 40.Pengpeng Ding, Jinguo Li, Liangliang Wang, Mi Wen, Yuyao Guan. (2020). HYBRID-CNN: An Efficient Scheme for Abnormal Flow Detection in the SDN-Based Smart Grid. Security and Communication Networks, 2020, 1–20[10.1155/2020/8850550]
  41. 41.T. Kalaiselvi, S. Padmapriya, P. Sriramakrishnan, V. Priyadharshini. (2020). Development of automatic glioma brain tumor detection system using deep convolutional neural networks. International Journal of Imaging Systems and Technology, 30(4), 926–938[10.1002/ima.22433]
  42. 42.Ahmed Ibrahim Tawfik, Wael Hamza Kamr. (2020). Diagnostic value of 3D-FLAIR magnetic resonance sequence in detection of white matter brain lesions in multiple sclerosis. The Egyptian Journal of Radiology and Nuclear Medicine, 51(1)[10.1186/s43055-020-00247-6]
  43. 43.Lu Lu, Yuhong Yang, Yuzhi Jiang, Haojun Ai, Weiping Tu. (2018). Shallow Convolutional Neural Networks for Acoustic Scene Classification. Wuhan University Journal of Natural Sciences, 23(2), 178–184[10.1007/s11859-018-1308-z]
  44. 44.Siyoun Sung, Hyun Su Kim, Jong Won Kwon. (2017). MRI assessment of sacroiliitis for the diagnosis of axial spondyloarthropathy: comparison of fat-saturated T2, STIR and contrast-enhanced sequences. British Journal of Radiology, 90(1078), 20170090[10.1259/bjr.20170090]
  45. 45.Sheng He, Randy L. Gollub, Shawn N. Murphy, Juan David Perez, Sanjay P. Prabhu, Rudolph Pienaar, Richard L. Robertson, P. Ellen Grant, Yangming Ou. (2020). Brain Age Estimation Using LSTM on Children's Brain MRI. , 2020, 1–4[10.1109/isbi45749.2020.9098356]
  46. 46.Mohammad Faranoush, Mohammad Torabi-Nami, Azim Mehrvar, Amir Abbas Hedayati-Asl, Maryam Tashvighi, Reza Ravan parsa, Mohammad Ali Fazeli, Behdad Sobuti, Narjes Mehrvar, Ali Jafarpour, Rokhsareh Zangooei, Mardawij Alebouyeh, Mohammadreza Abolghasemi, Abdol‐Hossein Vahabie, Parvaneh Vossough. (2013). Classifying Pediatric Central Nervous System Tumors through near Optimal Feature Selection and Mutual Information: A Single Center Cohort. DOAJ (DOAJ: Directory of Open Access Journals)[link]
  47. 47.Richard Meyes, Melanie Lu, Constantin Waubert de Puiseau, Tobias Meisen. (2019). Ablation Studies in Artificial Neural Networks. arXiv (Cornell University)[10.48550/arxiv.1901.08644]
  48. 48.Subhashis Banerjee, Sushmita Mitra, Francesco Masulli, Stefano Rovetta. (2019). Deep Radiomics for Brain Tumor Detection and Classification from Multi-Sequence MRI. arXiv (Cornell University)[10.48550/arxiv.1903.09240]
  49. 49.Coen de Vente, Luuk Boulogne, Kiran Vaidhya Venkadesh, Cheryl Sital, Nikolas Leßmann, Colin Jacobs, Clara I. Sá‎nchez, Bram van Ginneken. (2020). Improving Automated COVID-19 Grading with Convolutional Neural Networks in Computed Tomography Scans: An Ablation Study. arXiv (Cornell University)[10.48550/arxiv.2009.09725]
  50. 50.Ramin Ranjbarzadeh, Abbas Bagherian Kasgari, Saeid Jafarzadeh Ghoushchi, Shokofeh Anari, Maryam Naseri, Malika Bendechache. (2021). Brain tumor segmentation based on deep learning and an attention mechanism using MRI multi-modalities brain images. Scientific Reports, 11(1), 10930[10.1038/s41598-021-90428-8]
  51. 51.Sidratul Montaha, Sami Azam, A. K. M. Rakibul Haque Rafid, Pronab Ghosh, Md. Zahid Hasan, Mirjam Jonkman, Friso De Boer. (2021). BreastNet18: A High Accuracy Fine-Tuned VGG16 Model Evaluated Using Ablation Study for Diagnosing Breast Cancer from Enhanced Mammography Images. Biology, 10(12), 1347[10.3390/biology10121347]
  52. 52.Institut Teknologi Sepuluh Nopember, Mauridhi Purnomo. (2021). Brain Tumor Classification in MRI Images Using En-CNN. International journal of intelligent engineering and systems, 14(4), 437–451[10.22266/ijies2021.0831.38]
← All publications