Aambela at BLP-2023 Task 2: Enhancing BanglaBERT Performance for Bangla Sentiment Analysis Task with In Task Pretraining and Adversarial Weight Perturbation

Md Fahim

doi:10.18653/v1/2023.banglalp-1.42

Conference Paper2023Cited by 3

Aambela at BLP-2023 Task 2: Enhancing BanglaBERT Performance for Bangla Sentiment Analysis Task with In Task Pretraining and Adversarial Weight Perturbation

Md Fahim

Proceedings of the First Workshop on Bangla Language Processing (BLP-2023)

Association for Computational Linguistics, pp. 317-323

DOI: 10.18653/v1/2023.banglalp-1.42 Open Access URL

Abstract

This paper introduces the top-performing approachof “Aambela” for the BLP-2023 Task2: “Sentiment Analysis of Bangla Social MediaPosts”. The objective of the task was tocreate systems capable of automatically detectingsentiment in Bangla text from diverse socialmedia posts. My approach comprised finetuninga Bangla Language Model with threedistinct classification heads. To enhance performance,we employed two robust text classificationtechniques. To arrive at a final prediction,we employed a mode-based ensemble approachof various predictions from different models,which ultimately resulted in the 1st place in thecompetition.

Keywords

BengaliComputer scienceTask (project management)Artificial intelligenceNatural language processingSentiment analysisSpeech recognitionAdversarial systemTask analysis

References

1.Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova. (2019). , 4171–4186[10.18653/v1/n19-1423]
2.Chi Sun, Xipeng Qiu, Yige Xu, Xuanjing Huang. (2019). How to Fine-Tune BERT for Text Classification?. Lecture notes in computer science, 194–206[10.1007/978-3-030-32381-3_16]
3.Abhik Bhattacharjee, Tahmid Hasan, Wasi Ahmad, Kazi Samin Mubasshir, Md Saiful Islam, Anindya Iqbal, M. Sohel Rahman, Rifat Shahriyar. (2022). BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Bangla. Findings of the Association for Computational Linguistics: NAACL 2022, 1318–1327[10.18653/v1/2022.findings-naacl.98]
4.Khondoker Ittehadul Islam, Sudipta Kar, Md Saiful Islam, Mohammad Ruhul Amin. (2021). SentNoB: A Dataset for Analysing Sentiment on Noisy Bangla Texts. , 3265–3271[10.18653/v1/2021.findings-emnlp.278]
5.Minaee, Shervin, Kalchbrenner, Nal, Cambria, Erik, Nikzad, Narjes, Chenaghlu, Meysam, Gao, Jianfeng. (2020). Deep Learning Based Text Classification: A Comprehensive Review. arXiv (Cornell University)[10.48550/arxiv.2004.03705]
6.Wu, Dongxian, Xia, Shu-tao, Wang, Yisen. (2020). Adversarial Weight Perturbation Helps Robust Generalization. arXiv (Cornell University)[10.48550/arxiv.2004.05884]
7.Tianyi Zhang, Felix Wu, Arzoo Katiyar, Kilian Q. Weinberger, Yoav Artzi. (2021). Revisiting Few-sample BERT Fine-tuning. arXiv (Cornell University)[link]
8.Md. Arid Hasan, Firoj Alam, Anika Anjum, Shudipta Das, Afiyat Anjum. (2023). BLP-2023 Task 2: Sentiment Analysis. , 354–364[10.18653/v1/2023.banglalp-1.48]

Cited By

1.Md. Arid Hasan, Firoj Alam, Anika Anjum, Shudipta Das, Afiyat Anjum. (2023). BLP-2023 Task 2: Sentiment Analysis. , 354–364[10.18653/v1/2023.banglalp-1.48]
2.Sovon Chakraborty, Protiva Das, Shakib Mahmud Dipto, Md. Aktaruzzaman Pramanik, Jannatun Noor. (2025). An Analytical Review of Preprocessing Techniques in Bengali Natural Language Processing. IEEE Access, 13, 112428–112445[10.1109/access.2025.3574234]
3.Farhan Noor Dehan, Md Fahim, A. K. M. Mahabubur Rahman, M. Ashraful Amin, Amin Ahsan Ali. (2024). TinyLLM Efficacy in Low-Resource Language: An Experiment on Bangla Text Classification Task. Lecture notes in computer science, 472–487[10.1007/978-3-031-78495-8_30]

← All publications

Abstract

References

1.Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova. (2019). , 4171–4186[10.18653/v1/n19-1423]

2.Chi Sun, Xipeng Qiu, Yige Xu, Xuanjing Huang. (2019). How to Fine-Tune BERT for Text Classification?. Lecture notes in computer science, 194–206[10.1007/978-3-030-32381-3_16]

3.Abhik Bhattacharjee, Tahmid Hasan, Wasi Ahmad, Kazi Samin Mubasshir, Md Saiful Islam, Anindya Iqbal, M. Sohel Rahman, Rifat Shahriyar. (2022). BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Bangla. Findings of the Association for Computational Linguistics: NAACL 2022, 1318–1327[10.18653/v1/2022.findings-naacl.98]

4.Khondoker Ittehadul Islam, Sudipta Kar, Md Saiful Islam, Mohammad Ruhul Amin. (2021). SentNoB: A Dataset for Analysing Sentiment on Noisy Bangla Texts. , 3265–3271[10.18653/v1/2021.findings-emnlp.278]

5.Minaee, Shervin, Kalchbrenner, Nal, Cambria, Erik, Nikzad, Narjes, Chenaghlu, Meysam, Gao, Jianfeng. (2020). Deep Learning Based Text Classification: A Comprehensive Review. arXiv (Cornell University)[10.48550/arxiv.2004.03705]

6.Wu, Dongxian, Xia, Shu-tao, Wang, Yisen. (2020). Adversarial Weight Perturbation Helps Robust Generalization. arXiv (Cornell University)[10.48550/arxiv.2004.05884]

7.Tianyi Zhang, Felix Wu, Arzoo Katiyar, Kilian Q. Weinberger, Yoav Artzi. (2021). Revisiting Few-sample BERT Fine-tuning. arXiv (Cornell University)[link]

8.Md. Arid Hasan, Firoj Alam, Anika Anjum, Shudipta Das, Afiyat Anjum. (2023). BLP-2023 Task 2: Sentiment Analysis. , 354–364[10.18653/v1/2023.banglalp-1.48]

Cited By

1.Md. Arid Hasan, Firoj Alam, Anika Anjum, Shudipta Das, Afiyat Anjum. (2023). BLP-2023 Task 2: Sentiment Analysis. , 354–364[10.18653/v1/2023.banglalp-1.48]

2.Sovon Chakraborty, Protiva Das, Shakib Mahmud Dipto, Md. Aktaruzzaman Pramanik, Jannatun Noor. (2025). An Analytical Review of Preprocessing Techniques in Bengali Natural Language Processing. IEEE Access, 13, 112428–112445[10.1109/access.2025.3574234]

3.Farhan Noor Dehan, Md Fahim, A. K. M. Mahabubur Rahman, M. Ashraful Amin, Amin Ahsan Ali. (2024). TinyLLM Efficacy in Low-Resource Language: An Experiment on Bangla Text Classification Task. Lecture notes in computer science, 472–487[10.1007/978-3-031-78495-8_30]