Conference Paper2021

Comparing recent Swarm Algorithms with Information Theoretic Filter criterion for Feature Selection

Hasnain Hossain, Tahmid Bin Mahmud, AKM Mahbubur Rahman, M Ashraful Amin, Amin Ahsan Ali

2021 International Conference on Electrical, Computer and Energy Technologies (ICECET)

IEEE, pp. 1–6

DOI: 10.1109/icecet52533.2021.9698430URL

Abstract

Feature selection is getting significant importance in the fields of machine learning specially for dealing with high dimensional datasets. Feature selection techniques such as swarm algorithms have proven to be very effective in extracting meaningful information from large amounts of high dimensional data, thus improving prediction performance and reducing computation time. In this research, we evaluate the effectiveness of introducing mRMR (minimum Redundancy-Maximum Rel-evance) as an information theoretic filtering criterion on top of swarm optimization algorithms to further improve on existing techniques. We also present a comparison of the performances of two recent swarm algorithms against the well-known Artificial Bee Colony algorithm after incorporating mRMR-based filtering to demonstrate the applicability of this technique.

Keywords

  • Feature selection
  • minimum redundancy maximum relevance
  • high-dimensional data
  • artificial bee colony algorithm

CCDS Authors

Hasnain Hossain
Hasnain Hossain

Hasnain Hossain

System Analyst

+1 more affiliation

CholPori

Research Interests

Machine Learning

AKM Mahbubur Rahman, PhD
AKM Mahbubur Rhman, PhD

AKM Mahbubur Rahman, PhD

  • Director, DS & IPT Wings

Associate Professor

Department of Computer Science and Engineering

Independent University, Bangladesh

Published at

CHI, ECCV, WACV, PAKDD, LREC, IJCNN, ICPR, ICIP, HCII, BEA, Scientific Reports, Information Sciences

Prof. M Ashraful Amin, PhD
M Ashraful Amin, PhD

Prof. M Ashraful Amin, PhD

  • Founder Director, CCDS
  • Director, AI/ML, HCI & DS Wings

Professor

+1 more affiliation

Department of Information Sciences and Technology

George Mason University, USA

Published at

ACL, CHI, UbiComp, ECAI, WACV, PAKDD, IJCNN, ICPR, ICIP, HCII, CLEF, BLP, Scientific Reports, JMIR, IEEE Access, Smart Agricultural Technology, Frontiers in Computational Neuroscience, Diabetes Research and Clinical Practice, PLOS ONE

Research Interests

Machine Learning, Cognitive & Vision Science, Cybernetics, Surveillance & Security, ICT in Education, Health, & Agriculture, Human-Computer Interaction, Internet of Things, Robotics

Amin Ahsan Ali, PhD
Amin Ahsan Ali, PhD

Amin Ahsan Ali, PhD

  • Director, AI/ML Wing

Professor

Department of Computer Science and Engineering

Independent University, Bangladesh

Published at

ACL, CHI, UbiComp, ECCV, IJCAI, ECAI, WACV, IPSN, PAKDD, LREC, IJCNN, ICPR, ICIP, BEA, BLP, ICMLA, Scientific Reports, Scientific Data, Pattern Recognition, Information Sciences, Ad Hoc Networks, Drug and Alcohol Dependence, PLOS ONE

Research Interests

Artificial Intelligence, Machine Learning

References

  1. 1.R.C. Eberhart, James Kennedy. (2002). A new optimizer using particle swarm theory. , 39–43[10.1109/mhs.1995.494215]
  2. 2.Seyedali Mirjalili, Andrew Lewis. (2016). The Whale Optimization Algorithm. Advances in Engineering Software, 95, 51–67[10.1016/j.advengsoft.2016.01.008]
  3. 3.GuyonIsabelle, ElisseeffAndré. (2003). An introduction to variable and feature selection. Journal of Machine Learning Research[10.5555/944919.944968]
  4. 4.Derviş Karaboğa, Bahriye Akay. (2007). A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm. Journal of Global Optimization, 39(3), 459–471[10.1007/s10898-007-9149-x]
  5. 5.Max Kühn, Kjell Johnson. (2013). Applied Predictive Modeling. [10.1007/978-1-4614-6849-3]
  6. 6.Shahrzad Saremi, Seyedali Mirjalili, Andrew Lewis. (2017). Grasshopper Optimisation Algorithm: Theory and application. Advances in Engineering Software, 105, 30–47[10.1016/j.advengsoft.2017.01.004]
  7. 7.Huan Liu, Lei Yu. (2005). Toward integrating feature selection algorithms for classification and clustering. IEEE Transactions on Knowledge and Data Engineering, 17(4), 491–502[10.1109/tkde.2005.66]
  8. 8.El‐Ghazali Talbi. (2009). Metaheuristics. [10.1002/9780470496916]
  9. 9.E. Emary, Hossam M. Zawbaa, Aboul Ella Hassanien. (2015). Binary grey wolf optimization approaches for feature selection. Neurocomputing, 172, 371–381[10.1016/j.neucom.2015.06.083]
  10. 10.(2007). Computational Methods of Feature Selection. [10.1201/9781584888796]
  11. 11.Gavin Brown, Adam Pocock, Mingjie Zhao, Mikel Luján. (2012). Conditional likelihood maximisation: a unifying framework for information theoretic feature selection. Research Explorer (The University of Manchester), 13(1), 27–66[link]
  12. 12.Huan Liu, Rudy Setiono. (2002). Chi2: feature selection and discretization of numeric attributes. , 388–391[10.1109/tai.1995.479783]
  13. 13.C. Ding, Hujin Peng. (2004). Minimum redundancy feature selection from microarray gene expression data. , 523–528[10.1109/csb.2003.1227396]
  14. 14.James Kennedy. (2006). Swarm Intelligence. Kluwer Academic Publishers eBooks, 187–219[10.1007/0-387-27705-6_6]
  15. 15.Majdi Mafarja, Ibrahim Aljarah, Ali Asghar Heidari, Abdelaziz I. Hammouri, Hossam Faris, Ala’ M. Al-Zoubi, Seyedali Mirjalili. (2017). Evolutionary Population Dynamics and Grasshopper Optimization approaches for feature selection problems. Knowledge-Based Systems, 145, 25–45[10.1016/j.knosys.2017.12.037]
  16. 16.Hala Alshamlan, Ghada Badr, Yousef Al-Ohali. (2015). mRMR-ABC: A Hybrid Gene Selection Algorithm for Cancer Classification Using Microarray Gene Expression Profiling. BioMed Research International, 2015, 1–15[10.1155/2015/604910]
  17. 17.Fernando Fausto, Adolfo Reyna-Orta, Erik Cuevas, Ángel G. Andrade, Marco Pérez‐Cisneros. (2019). From ants to whales: metaheuristics for all tastes. Artificial Intelligence Review, 53(1), 753–810[10.1007/s10462-018-09676-2]
  18. 18.Celal Öztürk, Emrah Hançer, Derviş Karaboğa. (2014). A novel binary artificial bee colony algorithm based on genetic operators. Information Sciences, 297, 154–170[10.1016/j.ins.2014.10.060]
  19. 19.Sadia Sharmin, Mohammad Shoyaib, Amin Ahsan Ali, Muhammad Asif Hossain Khan, Oksam Chae. (2019). Simultaneous feature selection and discretization based on mutual information. Pattern Recognition, 91, 162–174[10.1016/j.patcog.2019.02.016]
  20. 20.Nguyễn Xuân Vinh, Shuo Zhou, Jeffrey Chan, James Bailey. (2015). Can high-order dependencies improve mutual information based feature selection?. Pattern Recognition, 53, 46–58[10.1016/j.patcog.2015.11.007]
  21. 21.Ivan Zelinka. (2015). A survey on evolutionary algorithms dynamics and its complexity – Mutual relations, past, present and future. Swarm and Evolutionary Computation, 25, 2–14[10.1016/j.swevo.2015.06.002]
  22. 22.Urszula Stańczyk. (2014). Feature Evaluation by Filter, Wrapper, and Embedded Approaches. Studies in computational intelligence, 29–44[10.1007/978-3-662-45620-0_3]
  23. 23.Susana M. Vieira, João M. C. Sousa, Uzay Kaymak. (2011). Fuzzy criteria for feature selection. Fuzzy Sets and Systems, 189(1), 1–18[10.1016/j.fss.2011.09.009]
  24. 24.Dimitrios Ververidis, Constantine Kotropoulos. (2005). Sequential Forward Feature Selection With Low Computational Cost. , 1–4[10.5281/zenodo.39057]
  25. 25.Heba F. Eid. (2018). Binary whale optimisation: an effective swarm algorithm for feature selection. International Journal of Metaheuristics, 7(1), 67[10.1504/ijmheur.2018.091880]
  26. 26.Bin Chen, Jiarong Hong, Yadong Wang. (1997). The minimum feature subset selection problem. Journal of Computer Science and Technology, 12(2), 145–153[10.1007/bf02951333]
  27. 27.Suleiman Mustafa. (2017). Feature selection using sequential backward method in melanoma recognition. , 1–4[10.1109/icecco.2017.8333341]
  28. 28.Sherry Chalotra, Sumeet Kaur Sehra, Sukhjit Singh Sehra. (2016). A systematic review of applications of Bee Colony Optimization. , 257–260[10.1109/iciccs.2016.7542297]
  29. 29.Quanquan Gu, Zhenhui Li, Jiawei Han. (2012). Generalized Fisher Score for Feature Selection. arXiv (Cornell University)[10.48550/arxiv.1202.3725]
← All publications