Effectiveness of Transfer Learning Approach for Low-Resource Bangla Dialect Translation

Deawan Rakin Ahamed Remal; Sinthia Chowdhury; Ashraful Islam; SM Rakibul Islam; Nabarun Halder; Faiza Omar Arpita; Jahanggir Hossain Setu; Asif Mahmud; M Ashraful Amin

doi:10.1109/scored68498.2025.11399036

Conference Paper2025

Effectiveness of Transfer Learning Approach for Low-Resource Bangla Dialect Translation

Deawan Rakin Ahamed Remal, Sinthia Chowdhury, Ashraful Islam, SM Rakibul Islam, Nabarun Halder, Faiza Omar Arpita, Jahanggir Hossain Setu, Asif Mahmud, M Ashraful Amin

2025 IEEE 23rd Student Conference on Research and Development (SCOReD)

In: 2025 IEEE 23rd Student Conference on Research and Development (SCOReD)

IEEE, pp. 1-6

DOI: 10.1109/scored68498.2025.11399036 URL

Abstract

The Chittagonian dialect is largely used in the Chittagong region of Bangladesh. The lack of data makes translating Chittagonian dialects extremely challenging. Even so, the problem of data scarcity is lessened when transfer learning is incorporated into pretrained models. The linguistic richness of both standard Bangla and the Chittagonian dialect makes machine translation more difficult. Without a large amount of data, machine translation models often perform badly. The purpose of this study is to compare the performance results of the suggested models and apply transfer learning for efficient machine translation. With an emphasis on the effects of fine-tuning in a dataset with minimal data, this research investigates the efficiency of encoder-decoder-based models in translating the low-resource Chittagonian dialect. For dialect translation in this research, we used fine-tuned mBART, NLLB, and mT5 models along with transfer learning, and even though we kept the tuning parameter constant, we witnessed distinct variations in the models’ performances. This was evident when the Bilingual Evaluation Understudy (BLEU) score, Word Error Rate (WER), and Character Error Rate (CER) differed significantly. The accuracy of the NLLB model was the highest of the three models, with a BLEU score of 82.67, 8.25% WER, and 4.77% CER.

Keywords

BengaliMachine translationTransfer of learningWord error rateTranslation (biology)Word (group theory)Character (mathematics)

CCDS Authors

Ashraful Islam, PhD

Center Director, CCDS
Director, HCI Wing

Assistant Professor

Department of Computer Science and Engineering

Independent University, Bangladesh (IUB)

Research Interests

Human-Computer Interaction, AI for Social Good, AI for Public Health, AI for Impact

References

1.Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, Colin Raffel. (2021). mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer. , 483–498[10.18653/v1/2021.naacl-main.41]
2.Lisa Torrey, Jude Shavlik. (2010). Transfer Learning. IGI Global eBooks, 242–264[10.4018/978-1-60566-766-9.ch011]
3.(2016). Transfer learning for low-resource neural machine translation. Digital Collections portal (Koç University)[10.18653/v1/d16-1163]
4.Surangika Ranathunga, En-Shiun Annie Lee, Marjana Prifti Skenduli, Ravi Shekhar, Mehreen Alam, Rishemjit Kaur. (2022). Neural Machine Translation for Low-resource Languages: A Survey. ACM Computing Surveys, 55(11), 1–37[10.1145/3567592]
5.P. Brown, J. Cocke, S. Della Pietra, V. Della Pietra, F. Jelinek, R. Mercer, P. Roossin. (1988). A statistical approach to language translation. , 1, 71–76[10.3115/991635.991651]
6.Orhan Firat, Baskaran Sankaran, Yaser Al-Onaizan, Fatos T. Yarman Vural, Kyunghyun Cho. (2016). Zero-Resource Translation with Multi-Lingual Neural Machine Translation. , 268–277[10.18653/v1/d16-1026]
7.Long Zhou, Wenpeng Hu, Jiajun Zhang, Chengqing Zong. (2017). Neural System Combination for Machine Translation. , 378–384[10.18653/v1/p17-2060]
8.Saméh Kchaou, Rahma Boujelbane, Lamia Hadrich. (2022). Hybrid Pipeline for Building Arabic Tunisian Dialect-standard Arabic Neural Machine Translation Model from Scratch. ACM Transactions on Asian and Low-Resource Language Information Processing, 22(3), 1–21[10.1145/3568674]
9.Shaykh Siddique, Tahmid Ahmed, Md. Rifayet Azam Talukder, Md. Mohsin Uddin. (2020). English to Bangla Machine Translation Using Recurrent Neural Network. International Journal of Future Computer and Communication, 46–51[10.18178/ijfcc.2020.9.2.564]
10.Nouhaila Bensalah, Habib Ayad, Abdellah Adib, Abdelhamid Ibn El Farouk. (2021). LSTM vs. GRU for Arabic Machine Translation. Advances in intelligent systems and computing, 156–165[10.1007/978-3-030-73689-7_16]
11.Roqayah Al-Ibrahim, Rehab M. Duwairi. (2020). Neural Machine Translation from Jordanian Dialect to Modern Standard Arabic. , 173–178[10.1109/icics49469.2020.239505]
12.Hafizur Rahman Milon, Sheikh Nasir Uddin Sabbir, Azfar Inan, Nahid Hossain. (2020). A Comprehensive Dialect Conversion Approach from Chittagonian to Standard Bangla. 2020 IEEE Region 10 Symposium (TENSYMP), 214–217[10.1109/tensymp50017.2020.9230714]
13.Goutam Datta, Nisheeth Joshi, Kusum Gupta. (2023). Hyper-parameter optimization in neural-based translation systems: A case study. International Journal on Smart Sensing and Intelligent Systems, 16(1)[10.2478/ijssis-2023-0010]
14.Khoula Al-Kharusi, Abdurahman AAlAbdulsalam. (2023). Machine Translation of Omani Arabic Dialect from Social Media. , 302–309[10.18653/v1/2023.arabicnlp-1.24]
15.Philipp Koehn, Franz J. Och, Daniel Marcu. (2003). Statistical Phrase-Based Translation. [10.21236/ada461156]
16.Sinthia Chowdhury, Deawan Rakin Ahamed Remal, Syed Tangim Pasha, Ashraful Islam, Sheak Rashed Haider Noori. (2025). ChatgaiyyaAlap: A dataset for conversion from Chittagonian dialect to standard Bangla. Data in Brief, 59, 111413[10.1016/j.dib.2025.111413]
17.Andreas Maletti. (2012). Statistical Machine Translation. SpringerReference[10.1007/springerreference_179495]

← All publications

Effectiveness of Transfer Learning Approach for Low-Resource Bangla Dialect Translation

Deawan Rakin Ahamed Remal, Sinthia Chowdhury, Ashraful Islam, SM Rakibul Islam, Nabarun Halder, Faiza Omar Arpita, Jahanggir Hossain Setu, Asif Mahmud, M Ashraful Amin

2025 IEEE 23rd Student Conference on Research and Development (SCOReD)

In: 2025 IEEE 23rd Student Conference on Research and Development (SCOReD)

IEEE, pp. 1-6

Abstract

References

1.Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, Colin Raffel. (2021). mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer. , 483–498[10.18653/v1/2021.naacl-main.41]

2.Lisa Torrey, Jude Shavlik. (2010). Transfer Learning. IGI Global eBooks, 242–264[10.4018/978-1-60566-766-9.ch011]

3.(2016). Transfer learning for low-resource neural machine translation. Digital Collections portal (Koç University)[10.18653/v1/d16-1163]

4.Surangika Ranathunga, En-Shiun Annie Lee, Marjana Prifti Skenduli, Ravi Shekhar, Mehreen Alam, Rishemjit Kaur. (2022). Neural Machine Translation for Low-resource Languages: A Survey. ACM Computing Surveys, 55(11), 1–37[10.1145/3567592]

5.P. Brown, J. Cocke, S. Della Pietra, V. Della Pietra, F. Jelinek, R. Mercer, P. Roossin. (1988). A statistical approach to language translation. , 1, 71–76[10.3115/991635.991651]

6.Orhan Firat, Baskaran Sankaran, Yaser Al-Onaizan, Fatos T. Yarman Vural, Kyunghyun Cho. (2016). Zero-Resource Translation with Multi-Lingual Neural Machine Translation. , 268–277[10.18653/v1/d16-1026]

7.Long Zhou, Wenpeng Hu, Jiajun Zhang, Chengqing Zong. (2017). Neural System Combination for Machine Translation. , 378–384[10.18653/v1/p17-2060]

8.Saméh Kchaou, Rahma Boujelbane, Lamia Hadrich. (2022). Hybrid Pipeline for Building Arabic Tunisian Dialect-standard Arabic Neural Machine Translation Model from Scratch. ACM Transactions on Asian and Low-Resource Language Information Processing, 22(3), 1–21[10.1145/3568674]

9.Shaykh Siddique, Tahmid Ahmed, Md. Rifayet Azam Talukder, Md. Mohsin Uddin. (2020). English to Bangla Machine Translation Using Recurrent Neural Network. International Journal of Future Computer and Communication, 46–51[10.18178/ijfcc.2020.9.2.564]

10.Nouhaila Bensalah, Habib Ayad, Abdellah Adib, Abdelhamid Ibn El Farouk. (2021). LSTM vs. GRU for Arabic Machine Translation. Advances in intelligent systems and computing, 156–165[10.1007/978-3-030-73689-7_16]

11.Roqayah Al-Ibrahim, Rehab M. Duwairi. (2020). Neural Machine Translation from Jordanian Dialect to Modern Standard Arabic. , 173–178[10.1109/icics49469.2020.239505]

12.Hafizur Rahman Milon, Sheikh Nasir Uddin Sabbir, Azfar Inan, Nahid Hossain. (2020). A Comprehensive Dialect Conversion Approach from Chittagonian to Standard Bangla. 2020 IEEE Region 10 Symposium (TENSYMP), 214–217[10.1109/tensymp50017.2020.9230714]

13.Goutam Datta, Nisheeth Joshi, Kusum Gupta. (2023). Hyper-parameter optimization in neural-based translation systems: A case study. International Journal on Smart Sensing and Intelligent Systems, 16(1)[10.2478/ijssis-2023-0010]

14.Khoula Al-Kharusi, Abdurahman AAlAbdulsalam. (2023). Machine Translation of Omani Arabic Dialect from Social Media. , 302–309[10.18653/v1/2023.arabicnlp-1.24]

15.Philipp Koehn, Franz J. Och, Daniel Marcu. (2003). Statistical Phrase-Based Translation. [10.21236/ada461156]

16.Sinthia Chowdhury, Deawan Rakin Ahamed Remal, Syed Tangim Pasha, Ashraful Islam, Sheak Rashed Haider Noori. (2025). ChatgaiyyaAlap: A dataset for conversion from Chittagonian dialect to standard Bangla. Data in Brief, 59, 111413[10.1016/j.dib.2025.111413]

17.Andreas Maletti. (2012). Statistical Machine Translation. SpringerReference[10.1007/springerreference_179495]