Can Small LLMs Replace Fine-Tuned BERT for Bangla Natural Language Inference?

Keywords

• small language models
• Bengali language
• natural language inference
• fine-tuned BERT
• multilingual language models

CCDS Authors

Istiaq Ahmed

Istiaq Ahmed

• Research Assistant, HCI Wing

Published at

CHI, HCII

Research Interests

mHealth, Human-Computer Interaction, Cross-Platform App/Website Development

MD Asif Bin Khaled

MD Asif Bin Khaled

• Co-Director, MIRA Wing

Published at

IEEE Internet of Things Journal

Ashraful Islam, PhD

Ashraful Islam, PhD

• Center Director, CCDS
• Director, HCI Wing

Assistant Professor

Department of Computer Science and Engineering

Independent University, Bangladesh (IUB)

Published at

CHI, UIST, UbiComp, IJCNN, ICIP, HCII, ICDMW, BLP, IEEE Internet of Things Journal, JMIR, JMIR Human Factors, JMIR Formative Research, IEEE Access, Smart Agricultural Technology, Diabetes Research and Clinical Practice, PLOS ONE

Research Interests

Human-Computer Interaction, AI for Social Good, AI for Public Health, AI for Impact

Prof. M Ashraful Amin, PhD

Prof. M Ashraful Amin, PhD

• Founder Director, CCDS
• Director, AI/ML, HCI & DS Wings

Professor

+1 more affiliation

Department of Information Sciences and Technology

George Mason University, USA

Published at

ACL, CHI, UbiComp, ECAI, WACV, PAKDD, IJCNN, ICPR, ICIP, HCII, CLEF, BLP, Scientific Reports, JMIR, IEEE Access, Smart Agricultural Technology, Frontiers in Computational Neuroscience, Diabetes Research and Clinical Practice, PLOS ONE

Research Interests

Machine Learning, Cognitive & Vision Science, Cybernetics, Surveillance & Security, ICT in Education, Health, & Agriculture, Human-Computer Interaction, Internet of Things, Robotics

References

1. 1.Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, Veselin Stoyanov. (2018). XNLI: Evaluating Cross-lingual Sentence Representations. , 2475–2485[10.18653/v1/d18-1269]
2. 2.Abhik Bhattacharjee, Tahmid Hasan, Wasi Ahmad, Kazi Samin Mubasshir, Md Saiful Islam, Anindya Iqbal, M. Sohel Rahman, Rifat Shahriyar. (2022). BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Bangla. Findings of the Association for Computational Linguistics: NAACL 2022, 1318–1327[10.18653/v1/2022.findings-naacl.98]
3. 3.Haoyang Huang, Tianyi Tang, Dongdong Zhang, Xin Zhao, Ting Song, Yan Xia, Furu Wei. (2023). Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting. , 12365–12394[10.18653/v1/2023.findings-emnlp.826]
4. 4.Samuel Cahyawijaya, Holy Lovenia, Pascale Fung. (2024). LLMs Are Few-Shot In-Context Low-Resource Language Learners. , 405–433[10.18653/v1/2024.naacl-long.24]
5. 5.Nishat Raihan, Marcos Zampieri. (2025). TigerLLM - A Family of Bangla Large Language Models. , 887–896[10.18653/v1/2025.acl-short.69]
6. 6.Md. Arid Hasan, Shudipta Das, Afiyat Anjum, Firoj Alam, Anika Anjum, Avijit Sarker, Sheak Rashed Haider Noori. (2024). Zero- and Few-Shot Prompting with LLMs: A Comparative Study with Fine-tuned Models for Bangla Sentiment Analysis. , 17808–17818[10.63317/3edi4825svh3]
7. 7.Shahriar Kabir Nahin, Rabindra Nath Nandi, Sagor Sarker, Quazi Sarwar Muhtaseem, Md Kowsher, Apu Chandraw Shill, Md Ibrahim, Mehadi Hasan Menon, Tareq Al Muntasir, Firoj Alam. (2025). TituLLMs: A Family of Bangla LLMs with Comprehensive Benchmarking. , 24922–24940[10.18653/v1/2025.findings-acl.1279]
8. 8.Md. Sajjad Hossain, Kawsar Ahmed, SUNY MD ASHRAF KHAN, Mohammed Moshiul Hoque. (2026). Exploring Large Language Models for Multitask Learning in Bengali Text Classification. , 92–104[10.63317/54y27enmr5e9]