Conference Paper2024Cited by 3

Empowering Multiclass Classification and Data Augmentation of Arabic News Articles Through Transformer Model

Jahanggir Hossain Setu, Nabarun Halder, Sankar Sikder, Ashraful Islam, Md Zahangir Alam

2024 International Joint Conference on Neural Networks (IJCNN)

IEEE, pp. 1-7

DOI: 10.1109/ijcnn60899.2024.10650716URL

Abstract

Arabic Natural Language Processing (NLP) presents unique challenges due to the complexity of the language, including variations in dialects, rich morphology, and context-dependent semantics. These linguistic intricacies make accurate classification a challenging task. This study presents the development and evaluation of an Arabic Bidirectional Encoder Representations from Transformers (AraBERT) model trained on a publicly available dataset named UltimateArabic dataset, which comprises 10 diverse classes. The aim is to perform multiclass classification and to address the imbalance class problem, using AraVec, of Arabic text data in the field of NLP. The model achieved an overall accuracy of 98% on augmented data, showcasing its proficiency in categorizing a wide range of Arabic texts including medical, technology, art, culture, diversity, society, religion, sports, politics, and economy. The AraBERT model exhibits strong classification performance compared to non-augmented data across the majority of categories, with an encouraging raise on macro-average precision of 4%, recall of 4%, and F1-score of 4%. This study underscores the potential of AraBERT along with AraVec for diverse applications in imbalanced Arabic text classification tasks and highlights the need for continued research to address the complexities of multiclass classification in Arabic NLP.

Keywords

  • AraBERT
  • multi-class classification
  • Arabic text classification
  • data augmentation
  • Arabic language processing
  • class imbalance
  • transformer models

CCDS Authors

Nabarun Halder
Nabarun Halder

Nabarun Halder

  • Research Assistant, CCDS

Published at

UbiComp, IJCNN, HCII, IEEE Access

Research Interests

Machine Learning, Deep Learning, Natural Language Processing, Health Informatics, Human-Computer Interaction

Ashraful Islam, PhD
Ashraful Islam

Ashraful Islam, PhD

  • Center Director, CCDS
  • Director, HCI Wing

Assistant Professor

Department of Computer Science and Engineering

Independent University, Bangladesh (IUB)

Published at

CHI, UIST, UbiComp, IJCNN, ICIP, HCII, ICDMW, BLP, IEEE Internet of Things Journal, JMIR, JMIR Human Factors, JMIR Formative Research, IEEE Access, Smart Agricultural Technology, Diabetes Research and Clinical Practice, PLOS ONE

Research Interests

Human-Computer Interaction, AI for Social Good, AI for Public Health, AI for Impact

References

  1. 1.Jakubův, Jan, Chvalovský, Karel, Goertzel, Zarathustra, Kaliszyk, Cezary, Olšák, Mirek, Piotrowski, Bartosz, Schulz, Stephan, Suda, Martin, Urban, Josef. (2023). MizAR 60 for Mizar 50. DROPS (Schloss Dagstuhl – Leibniz Center for Informatics)[10.4230/lipics.itp.2023.19]
  2. 2.Holter, Ole Magnus, Ell, Basil. (2018). AI-Assisted Pipeline for Dynamic Generation of Trustworthy Health Supplement Content at Scale. DROPS (Schloss Dagstuhl – Leibniz Center for Informatics)[10.4230/lipics.cosit.2022.18]
  3. 3.T. B. Brown, Gao, Song. (2023). Aion Framework: Dimensional Emergence of AI Consciousness, Observer-Induced Collapse, and Cosmological Portal Dynamics. DROPS (Schloss Dagstuhl – Leibniz Center for Informatics)[10.4230/lipics.giscience.2023.43]
  4. 4.Željko Vujović. (2021). Classification Model Evaluation Metrics. International Journal of Advanced Computer Science and Applications, 12(6)[10.14569/ijacsa.2021.0120670]
  5. 5.Abu Bakr Soliman, Kareem Eissa, Samhaa R. El-Beltagy. (2017). AraVec: A set of Arabic Word Embedding Models for use in Arabic NLP. Procedia Computer Science, 117, 256–265[10.1016/j.procs.2017.10.117]
  6. 6.Qian Li, Hao Peng, Jianxin Li, Congying Xia, Renyu Yang, Lichao Sun, Philip S. Yu, Lifang He. (2022). A Survey on Text Classification: From Traditional to Deep Learning. ACM Transactions on Intelligent Systems and Technology, 13(2), 1–41[10.1145/3495162]
  7. 7.Muhammad Abdul-Mageed, AbdelRahim Elmadany, El Moatez Billah Nagoudi. (2021). ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic. , 7088–7105[10.18653/v1/2021.acl-long.551]
  8. 8.Tira Nur Fitria. (2021). QuillBot as an online tool: Students’ alternative in paraphrasing and rewriting of English writing. Englisia Journal of language education and humanities, 9(1), 183[10.22373/ej.v9i1.10233]
  9. 9.Imad Zeroual, Dirk Goldhahn, Thomas Eckart, Abdelhak Lakhouaja. (2019). OSIAN: Open Source International Arabic News Corpus - Preparation and Integration into the CLARIN-infrastructure. , 175–182[10.18653/v1/w19-4619]
  10. 10.Samah M. Alzanin, Aqil M. Azmi, Hatim Aboalsamh. (2022). Short text classification for Arabic social media tweets. Journal of King Saud University - Computer and Information Sciences, 34(9), 6595–6604[10.1016/j.jksuci.2022.03.020]
  11. 11.Ibrahim Abu Farha, Walid Magdy. (2021). Benchmarking Transformer-based Language Models for Arabic Sentiment and Sarcasm Detection. Edinburgh Research Explorer, 21–31[link]
  12. 12.Kosisochukwu Judith Madukwe, Xiaoying Gao, Bing Xue. (2022). Token replacement-based data augmentation methods for hate speech detection. World Wide Web, 25(3), 1129–1150[10.1007/s11280-022-01025-2]
  13. 13.Meshrif Alruily, Abdul Manaf Fazal, Ayman Mohamed Mostafa, Mohamed Ezz. (2023). Automated Arabic Long-Tweet Classification Using Transfer Learning with BERT. Applied Sciences, 13(6), 3482[10.3390/app13063482]
  14. 14.Najla Z. Alhazzani, Isra Al-Turaiki, Sarah A. Alkhodair. (2023). Text Classification of Patient Experience Comments in Saudi Dialect Using Deep Learning Techniques. Applied Sciences, 13(18), 10305[10.3390/app131810305]
  15. 15.Wissam Antoun, Fady Baly, Hazem Hajj. (2020). AraBERT: Transformer-based Model for Arabic Language Understanding. arXiv (Cornell University)[10.48550/arxiv.2003.00104]
  16. 16.Ibrahim Abu El-khair. (2016). 1.5 billion words Arabic Corpus. arXiv (Cornell University)[10.48550/arxiv.1611.04033]

Cited By

  1. 1.Abdulaziz M. Alayba, Mohammed Altamimi. (2025). Optimization of Arabic text classification using SVM integrated with word embedding models on a novel dataset. International Journal of ADVANCED AND APPLIED SCIENCES, 12(9), 140–151[10.21833/ijaas.2025.09.013]
  2. 2.Benamar Hamzaoui, Djelloul Bouchiha, Abdelghani Bouziane. (2025). Hierarchical Arabic Text Classification: BERT-ANN vs. BERT-BiLSTM. International Conference on Pattern Analysis and Intelligent Systems (PAIS), 1–5[10.1109/pais66004.2025.11126481]
  3. 3.A. Moustapha, B. Sharada, Wadeea R. Naji, Channabasava Chola. (2025). Exploring the Efficacy of Text Data Augmentation in Arabic Text Emotion Intensity Classification. Lecture notes in networks and systems, 333–347[10.1007/978-981-96-6715-4_24]
← All publications