Conference Paper2025

Effectiveness of Transfer Learning Approach for Low-Resource Bangla Dialect Translation

Deawan Rakin Ahamed Remal, Sinthia Chowdhury, Ashraful Islam, SM Rakibul Islam, Nabarun Halder, Faiza Omar Arpita, Jahanggir Hossain Setu, Asif Mahmud, M Ashraful Amin

2025 IEEE 23rd Student Conference on Research and Development (SCOReD)

IEEE, pp. 1-6

DOI: 10.1109/scored68498.2025.11399036URL

Abstract

The Chittagonian dialect is largely used in the Chittagong region of Bangladesh. The lack of data makes translating Chittagonian dialects extremely challenging. Even so, the problem of data scarcity is lessened when transfer learning is incorporated into pretrained models. The linguistic richness of both standard Bangla and the Chittagonian dialect makes machine translation more difficult. Without a large amount of data, machine translation models often perform badly. The purpose of this study is to compare the performance results of the suggested models and apply transfer learning for efficient machine translation. With an emphasis on the effects of fine-tuning in a dataset with minimal data, this research investigates the efficiency of encoder-decoder-based models in translating the low-resource Chittagonian dialect. For dialect translation in this research, we used fine-tuned mBART, NLLB, and mT5 models along with transfer learning, and even though we kept the tuning parameter constant, we witnessed distinct variations in the models’ performances. This was evident when the Bilingual Evaluation Understudy (BLEU) score, Word Error Rate (WER), and Character Error Rate (CER) differed significantly. The accuracy of the NLLB model was the highest of the three models, with a BLEU score of 82.67, 8.25% WER, and 4.77% CER.

Keywords

  • character error rate
  • mT5
  • transfer learning
  • low-resource translation
  • fine-tuning
  • encoder-decoder architecture
  • Word error rate
  • NLLB
  • mBART
  • data scarcity

CCDS Authors

Ashraful Islam, PhD
Ashraful Islam

Ashraful Islam, PhD

  • Center Director, CCDS
  • Director, HCI Wing

Published at

CHI, UIST, UbiComp, IJCNN, ICIP, HCII, ICDMW, BLP, Pattern Recognition, IEEE Internet of Things Journal, JMIR, JMIR Human Factors, JMIR Formative Research, IEEE Access, Smart Agricultural Technology, Diabetes Research and Clinical Practice, PLOS ONE

Nabarun Halder
Nabarun Halder

Nabarun Halder

  • Research Assistant, CCDS

Published at

UbiComp, IJCNN, HCII, Pattern Recognition, IEEE Access

Asif Mahmud
Asif Mahmud

Asif Mahmud

  • Co-Director, IPT Wing

  • Lecturer

    Department of Computer Science and Engineering

    Independent University, Bangladesh (IUB)

Prof. M Ashraful Amin, PhD
M Ashraful Amin, PhD

Prof. M Ashraful Amin, PhD

  • Founder Director, CCDS
  • Director, AI/ML, HCI & DS Wings

Published at

ACL, CHI, UbiComp, ECAI, WACV, PAKDD, IJCNN, ICPR, ICIP, HCII, CLEF, BLP, Scientific Reports, Pattern Recognition, JMIR, IEEE Access, Smart Agricultural Technology, Frontiers in Computational Neuroscience, Diabetes Research and Clinical Practice, PLOS ONE

References

  1. 1.Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, Colin Raffel. (2021). mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer. , 483–498[10.18653/v1/2021.naacl-main.41]
  2. 2.Lisa Torrey, Jude Shavlik. (2010). Transfer Learning. IGI Global eBooks, 242–264[10.4018/978-1-60566-766-9.ch011]
  3. 3.(2016). Transfer learning for low-resource neural machine translation. Digital Collections portal (Koç University)[10.18653/v1/d16-1163]
  4. 4.Surangika Ranathunga, En-Shiun Annie Lee, Marjana Prifti Skenduli, Ravi Shekhar, Mehreen Alam, Rishemjit Kaur. (2022). Neural Machine Translation for Low-resource Languages: A Survey. ACM Computing Surveys, 55(11), 1–37[10.1145/3567592]
  5. 5.P. Brown, J. Cocke, S. Della Pietra, V. Della Pietra, F. Jelinek, R. Mercer, P. Roossin. (1988). A statistical approach to language translation. , 1, 71–76[10.3115/991635.991651]
  6. 6.Orhan Firat, Baskaran Sankaran, Yaser Al-Onaizan, Fatos T. Yarman Vural, Kyunghyun Cho. (2016). Zero-Resource Translation with Multi-Lingual Neural Machine Translation. , 268–277[10.18653/v1/d16-1026]
  7. 7.Long Zhou, Wenpeng Hu, Jiajun Zhang, Chengqing Zong. (2017). Neural System Combination for Machine Translation. , 378–384[10.18653/v1/p17-2060]
  8. 8.Saméh Kchaou, Rahma Boujelbane, Lamia Hadrich. (2022). Hybrid Pipeline for Building Arabic Tunisian Dialect-standard Arabic Neural Machine Translation Model from Scratch. ACM Transactions on Asian and Low-Resource Language Information Processing, 22(3), 1–21[10.1145/3568674]
  9. 9.Shaykh Siddique, Tahmid Ahmed, Md. Rifayet Azam Talukder, Md. Mohsin Uddin. (2020). English to Bangla Machine Translation Using Recurrent Neural Network. International Journal of Future Computer and Communication, 46–51[10.18178/ijfcc.2020.9.2.564]
  10. 10.Nouhaila Bensalah, Habib Ayad, Abdellah Adib, Abdelhamid Ibn El Farouk. (2021). LSTM vs. GRU for Arabic Machine Translation. Advances in intelligent systems and computing, 156–165[10.1007/978-3-030-73689-7_16]
  11. 11.Roqayah Al-Ibrahim, Rehab M. Duwairi. (2020). Neural Machine Translation from Jordanian Dialect to Modern Standard Arabic. , 173–178[10.1109/icics49469.2020.239505]
  12. 12.Hafizur Rahman Milon, Sheikh Nasir Uddin Sabbir, Azfar Inan, Nahid Hossain. (2020). A Comprehensive Dialect Conversion Approach from Chittagonian to Standard Bangla. 2020 IEEE Region 10 Symposium (TENSYMP), 214–217[10.1109/tensymp50017.2020.9230714]
  13. 13.Goutam Datta, Nisheeth Joshi, Kusum Gupta. (2023). Hyper-parameter optimization in neural-based translation systems: A case study. International Journal on Smart Sensing and Intelligent Systems, 16(1)[10.2478/ijssis-2023-0010]
  14. 14.Khoula Al-Kharusi, Abdurahman AAlAbdulsalam. (2023). Machine Translation of Omani Arabic Dialect from Social Media. , 302–309[10.18653/v1/2023.arabicnlp-1.24]
  15. 15.Philipp Koehn, Franz J. Och, Daniel Marcu. (2003). Statistical Phrase-Based Translation. [10.21236/ada461156]
  16. 16.Sinthia Chowdhury, Deawan Rakin Ahamed Remal, Syed Tangim Pasha, Ashraful Islam, Sheak Rashed Haider Noori. (2025). ChatgaiyyaAlap: A dataset for conversion from Chittagonian dialect to standard Bangla. Data in Brief, 59, 111413[10.1016/j.dib.2025.111413]
  17. 17.Andreas Maletti. (2012). Statistical Machine Translation. SpringerReference[10.1007/springerreference_179495]
← All publications