Conference Paper2021Cited by 2

Feature subset selection based on redundancy maximized clusters

Md Hasan Tarek, Md Eusha Kadir, Sadia Sharmin, Abu Ashfaqur Sajib, Amin Ahsan Ali, Mohammad Shoyaib

2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA)

IEEE, pp. 521–526

DOI: 10.1109/icmla52953.2021.00087URL

Abstract

Feature selection plays a vital role in the field of data mining and machine learning for analyzing high-dimensional data. A popular criteria for feature selection is Mutual Information (MI) as it can capture both the linear and non-linear relationship among different features and class variable. Existing MI based feature selection methods use different approximation techniques to capture the joint performance of features, their relationship with the classes and eliminate the redundant features. However, these approximations may fail to select the optimal set of features, especially when the feature dimension is high. Besides, due to the absence of an appropriate searching strategy, these MI based approximations may select unnecessary features. To address these issues, we propose a method namely Feature Selection based on Redundancy maximized Clusters (FSRC) that creates the clusters of redundant features and then selects a subset of representative features from each cluster. We also propose to use bias corrected normalized MI in this regard. Rigorous experiments performed on thirty benchmark datasets demonstrate that FSRC outperforms the existing state-of-the-art methods in most of the cases. Moreover, FSRC is applied to three gene expression datasets which are high-dimensional but small sample datasets. The result shows that FSRC can select the features (genes) that are not only discriminating but also biologically relevant.

Keywords

  • Feature selection
  • high-dimensional data
  • gene expression data
  • Mutual information
  • small sample size

CCDS Authors

Amin Ahsan Ali, PhD
Amin Ahsan Ali, PhD

Amin Ahsan Ali, PhD

  • Director, AI/ML Wing

Professor

Department of Computer Science and Engineering

Independent University, Bangladesh

Published at

ACL, CHI, UbiComp, ECCV, IJCAI, ECAI, WACV, IPSN, PAKDD, LREC, IJCNN, ICPR, ICIP, BEA, BLP, ICMLA, Scientific Reports, Scientific Data, Pattern Recognition, Information Sciences, Ad Hoc Networks, Drug and Alcohol Dependence, PLOS ONE

Research Interests

Artificial Intelligence, Machine Learning

References

  1. 1.Arthur Asuncion. (2007). UCI Machine Learning Repository. Medical Entomology and Zoology
  2. 2.Hanchuan Peng, Fuhui Long, C. Ding. (2005). Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(8), 1226–1238[10.1109/tpami.2005.159]
  3. 3.Jesús Alcalá‐Fdez, Alberto Fernández, Julián Luengo, J. Derrac, Salvador García, Luciano Sánchez, Francisco Herrera. (2011). KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework. , 17, 255–287
  4. 4.Lei Yu, Huan Liu. (2003). Feature selection for high-dimensional data: a fast correlation-based filter solution. , 856–863
  5. 5.Qinbao Song, Jingjie Ni, Guangtao Wang. (2011). A Fast Clustering-Based Feature Subset Selection Algorithm for High-Dimensional Data. IEEE Transactions on Knowledge and Data Engineering, 25(1), 1–14[10.1109/tkde.2011.181]
  6. 6.David D. Lewis. (1992). Feature selection and feature extraction for text categorization. , 212[10.3115/1075527.1075574]
  7. 7.Norma Masson, Peter J Ratcliffe. (2014). Hypoxia signaling pathways in cancer metabolism: the importance of co-selecting interconnected physiological pathways. Cancer & Metabolism, 2(1), 3[10.1186/2049-3002-2-3]
  8. 8.K.Z. Mao. (2004). Orthogonal Forward Selection and Backward Elimination Algorithms for Feature Subset Selection. IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics), 34(1), 629–634[10.1109/tsmcb.2002.804363]
  9. 9.Albert Gutierrez, Renee C. Tschumper, Xiaosheng Wu, Tait D. Shanafelt, Jeanette Eckel-Passow, Paul M. Huddleston, Susan L. Slager, Neil E. Kay, Diane F. Jelinek. (2010). LEF-1 is a prosurvival factor in chronic lymphocytic leukemia and is expressed in the preleukemic state of monoclonal B-cell lymphocytosis. Blood, 116(16), 2975–2983[10.1182/blood-2010-02-269878]
  10. 10.Ryan P McMullin, Ben S Wittner, Chuanwei Yang, Benjamin R Denton-Schneider, Daniel Hicks, Raj Singavarapu, Sharon Moulis, Jeongeun Lee, Mohammad R Akbari, Steven A Narod, Kenneth D Aldape, Patricia S Steeg, Sridhar Ramaswamy, Dennis C Sgroi. (2014). A BRCA1deficient-like signature is enriched in breast cancer brain metastases and predicts DNA damage-induced poly (ADP-ribose) polymerase inhibitor sensitivity. Breast Cancer Research, 16(2), R25[10.1186/bcr3625]
  11. 11.Antonietta Rosella Farina, Lucia Cappabianca, Michela Sebastiano, Veronica Zelli, Stefano Guadagni, Andrew Reay Mackay. (2020). Hypoxia-induced alternative splicing: the 11th Hallmark of Cancer. Journal of Experimental & Clinical Cancer Research, 39(1), 110[10.1186/s13046-020-01616-9]
  12. 12.Phungern Khongthong, Antonia K Roseweir, Joanne Edwards. (2019). The NF-KB pathway and endocrine therapy resistance in breast cancer. Endocrine Related Cancer, 26(6), R369–R380[10.1530/erc-19-0087]
  13. 13.Sadia Sharmin, Mohammad Shoyaib, Amin Ahsan Ali, Muhammad Asif Hossain Khan, Oksam Chae. (2019). Simultaneous feature selection and discretization based on mutual information. Pattern Recognition, 91, 162–174[10.1016/j.patcog.2019.02.016]
  14. 14.Maria Carla Bosco, Daniele Pierobon, Fabiola Blengio, Federica Raggi, Cristina Vanni, Marco Gattorno, Alessandra Eva, Francesco Novelli, Paola Cappello, Mirella Giovarelli, Luigi Varesio. (2010). Hypoxia modulates the gene expression profile of immunoregulatory receptors in human mature dendritic cells: identification of TREM-1 as a novel hypoxic marker in vitro and in vivo. Blood, 117(9), 2625–2639[10.1182/blood-2010-06-292136]
  15. 15.Brian Ortmann, Jimena Druker, Sonia Rocha. (2014). Cell cycle progression in response to oxygen levels. Cellular and Molecular Life Sciences, 71(18), 3569–3582[10.1007/s00018-014-1645-9]
  16. 16.Nancy T. Chee, Ines Lohse, Shaun P. Brothers. (2019). mRNA-to-protein translation in hypoxia. Molecular Cancer, 18(1), 49[10.1186/s12943-019-0968-4]
  17. 17.Nguyen Xuan Vinh, Shuo Zhou, Jeffrey Chan, James Bailey. (2015). Can high-order dependencies improve mutual information based feature selection?. Pattern Recognition, 53, 46–58[10.1016/j.patcog.2015.11.007]
  18. 18.Wanfu Gao, Liang Hu, Ping Zhang. (2020). Feature redundancy term variation for mutual information-based feature selection. Applied Intelligence, 50(4), 1272–1288[10.1007/s10489-019-01597-z]
  19. 19.Tofigh Naghibi, Sarah Hoffmann, Beat Pfister. (2014). A Semidefinite Programming Based Search Strategy for Feature Selection with Mutual Information Measure. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(8), 1529–1541[10.1109/tpami.2014.2372791]
  20. 20.Songyot Nakariyakul. (2019). A hybrid gene selection algorithm based on interaction information for microarray-based cancer classification. PLoS ONE, 14(2), e0212333[10.1371/journal.pone.0212333]
  21. 21.Andrea Brumwell, Leslie Fell, Lindsay Obress, James Uniacke. (2020). Hypoxia influences polysome distribution of human ribosomal protein S12 and alternative splicing of ribosomal protein mRNAs. RNA, 26(3), 361–371[10.1261/rna.070318.119]
  22. 22.Ting-Ting Han, Lei Fan, Jian-Yong Li, Wei Xu. (2013). Role of chemokines and their receptors in chronic lymphocytic leukemia. Cancer Biology & Therapy, 15(1), 3–9[10.4161/cbt.26607]
  23. 23.Mohammad Kamalabadi-Farahani, Mohammad Reza H Najafabadi, Zahra Jabbarpour. (2019). Apoptotic Resistance of Metastatic Tumor Cells in Triple Negative Breast Cancer: Roles of Death Receptor-5. Asian Pacific Journal of Cancer Prevention, 20(6), 1743–1748[10.31557/apjcp.2019.20.6.1743]
  24. 24.Pravin B. Sehgal, Jason E. Lee. (2011). Protein Trafficking Dysfunctions: Role in the Pathogenesis of Pulmonary Arterial Hypertension. Pulmonary Circulation, 1(1), 17–32[10.4103/2045-8932.78097]
  25. 25.Sadia Sharmin, Farzana Aktar, Amin Ahsan Ali, Muhammad Asif Hossain Khan, Mohammad Shoyaib. (2017). BFSp: A feature selection method for bug severity classification. , 750–754[10.1109/r10-htc.2017.8289066]
  26. 26.Puloma Roy, Sadia Sharmin, Amin Ahsan Ali, Mohammad Shoyaib. (2020). Discretization and Feature Selection Based on Bias Corrected Mutual Information Considering High-Order Dependencies. Lecture notes in computer science, 830–842[10.1007/978-3-030-47426-3_64]
  27. 27.Paul J. Camarata, Robert E. McGeachie, Stephen J. Haines. (1990). Dorsal midbrain encephalitis caused by Propionibacterium acnes. Journal of neurosurgery, 72(4), 654–659[10.3171/jns.1990.72.4.0654]
  28. 28.Suravi Akhter, Sadia Sharmin, Sumon Ahmed, Abu Ashfaqur Sajib, Mohammad Shoyaib. (2021). mRelief: A Reward Penalty Based Feature Subset Selection Considering Data Overlapping Problem. Lecture notes in computer science, 278–292[10.1007/978-3-030-77961-0_24]
  29. 29.Patricia Cruz Castellanos, Laura Quintana, Javier de Castro. (2018). Small Cell Breast Cancer with Lung Metastases. Archivos de Bronconeumología, 54(11), 586–587[10.1016/j.arbr.2018.09.001]

Cited By

  1. 1.Md Hasan Tarek, Suravi Akhter, Sumon Ahmed, Md Shariful Islam, Mohammad Shoyaib, Zerina Begum. (2023). A Clustering based Feature Selection Approach using Maximum Spanning Tree. Dhaka University Journal of Applied Science and Engineering, 7(2), 47–55[10.3329/dujase.v7i2.65094]
  2. 2.Md. Al-Amin, Prosanto Deb, Ishrat Jahan Rintu, Mohammed Maruf Islam, Dipok Chandra Das, Sabbir Ahmed Khan. (2024). Genuine or Spammer? Enhanced Fake Profile Detection using Feature Synthesis. , 3499–3504[10.1109/iccit64611.2024.11022341]
← All publications