Book Chapter2025Cited by 2

TinyLLM Efficacy in Low-Resource Language: An Experiment on Bangla Text Classification Task

Farhan Noor Dehan, Md Fahim, A. K. M. Mahabubur Rahman, M. Ashraful Amin, Amin Ahsan Ali

Pattern Recognition

Springer Nature Switzerland, pp. 472-487, ISBN: 9783031784941

DOI: 10.1007/978-3-031-78495-8_30URL

Keywords

  • low-resource languages
  • BanglaBERT
  • TinyLlama
  • Sentiment analysis
  • sarcasm detection
  • Emotion recognition
  • hate speech detection
  • fake news detection
  • zero-shot prompting
  • ChatGPT
  • model comparison

CCDS Authors

Farhan Noor Dehan
Farhan Noor Dehan

Farhan Noor Dehan

  • Alumni, CCDS

  • Master's Student

    Georgia Southern University, USA

  • BSc

    Independent University, Bangladesh (IUB)

Published at

ICPR, Pattern Recognition

MD Md Fahim
Md Fahim

MD Md Fahim

  • Alumni, CCDS

  • PhD Student

    University of Texas at Dallas, USA

  • BSc

    University of Dhaka, Bangladesh

Published at

ACL, EMNLP, ECAI, WACV, LREC, ICPR, BEA, BLP, Pattern Recognition, Information Sciences, Smart Agricultural Technology

Prof. M Ashraful Amin, PhD
M Ashraful Amin, PhD

Prof. M Ashraful Amin, PhD

  • Founder Director, CCDS
  • Director, AI/ML, HCI & DS Wings

Published at

ACL, CHI, UbiComp, ECAI, WACV, PAKDD, IJCNN, ICPR, ICIP, HCII, CLEF, BLP, Scientific Reports, Pattern Recognition, JMIR, IEEE Access, Smart Agricultural Technology, Frontiers in Computational Neuroscience, Diabetes Research and Clinical Practice, PLOS ONE

Prof. Amin Ahsan Ali, PhD
Amin Ahsan Ali, PhD

Prof. Amin Ahsan Ali, PhD

  • Director, AI/ML Wing
  • Supervisor, ComBio Wing

  • Professor

    Department of Computer Science and Engineering

    Independent University, Bangladesh (IUB)

Published at

ACL, CHI, UbiComp, ECCV, IJCAI, ECAI, WACV, IPSN, PAKDD, LREC, IJCNN, ICPR, ICIP, BEA, BLP, ICMLA, Scientific Reports, Scientific Data, Pattern Recognition, Information Sciences, Ad Hoc Networks, Drug and Alcohol Dependence, PLOS ONE

References

  1. 1.Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 4171–4186[10.18653/v1/n19-1423]
  2. 2.Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, Veselin Stoyanov. (2020). Unsupervised Cross-lingual Representation Learning at Scale. Proceedings of the conference - Association for Computational Linguistics. Meeting, 8440–8451[10.18653/v1/2020.acl-main.747]
  3. 3.Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego De Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Thomas Hennigan, Eric Noland, Katherine Millican, George Van Den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karén Simonyan, Erich Elsen, Oriol Vinyals, Jack Rae, Laurent Sifre. (2022). Training Compute-Optimal Large Language Models. neural information processing systems, 30016–30030[10.52202/068431-2176]
  4. 4.Abhik Bhattacharjee, Tahmid Hasan, Wasi Ahmad, Kazi Samin Mubasshir, Md Saiful Islam, Anindya Iqbal, M. Sohel Rahman, Rifat Shahriyar. (2022). BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Bangla. Findings of the Association for Computational Linguistics: NAACL, 1318–1327[10.18653/v1/2022.findings-naacl.98]
  5. 5.Nafis Irtiza Tripto, Mohammed Eunus Ali. (2018). Detecting Multilabel Sentiment and Emotions from Bangla YouTube Comments. International Conference on Bangla Speech and Language Processing (ICBSLP), 1–6[10.1109/icbslp.2018.8554875]
  6. 6.Md. Rezaul Karim, Bharathi Raja Chakravarthi, John P. McCrae, Michael Cochez. (2020). Classification Benchmarks for Under-resourced Bengali Language based on Multichannel Convolutional-LSTM Network. IEEE International Conference on Data Science and Advanced Analytics (DSAA), 390–399[10.1109/dsaa49011.2020.00053]
  7. 7.Khondoker Ittehadul Islam, Sudipta Kar, Md Saiful Islam, Mohammad Ruhul Amin. (2021). SentNoB: A Dataset for Analysing Sentiment on Noisy Bangla Texts. Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP), 3265–3271[10.18653/v1/2021.findings-emnlp.278]
  8. 8.Nicholas Kluge Corrêa, Sophia Falk, Shiza Fatimah, Aniket Sen, Nythamar De Oliveira. (2024). TeenyTinyLlama: Open-source tiny language models trained in Brazilian Portuguese. Machine Learning with Applications, 16, 100558[10.1016/j.mlwa.2024.100558]
  9. 9.Xiaoqian Li, Ercong Nie, Sheng Liang. (2023). Crosslingual Retrieval Augmented In-context Learning for Bangla. Workshop on Bangla Language Processing (BLP), 136–151[10.18653/v1/2023.banglalp-1.15]
  10. 10.Farhan Dehan, Md Fahim, Amin Ahsan Ali, M Ashraful Amin, Akmmahbubur Rahman. (2023). Investigating the Effectiveness of Graph-based Algorithm for Bangla Text Classification. Workshop on Bangla Language Processing (BLP), 104–116[10.18653/v1/2023.banglalp-1.12]
  11. 11.Tasnim Sakib Apon, Ramisa Anan, Elizabeth Antora Modhu, Arjun Suter, Ifrit Jamal Sneha, MD. Golam Rabiul Alam. (2022). BanglaSarc: A Dataset for Sarcasm Detection. IEEE Asia-Pacific Conference on Computer Science and Data Engineering (CSDE), 1–5[10.1109/csde56538.2022.10089322]
  12. 12.Md Fahim. (2023). Aambela at BLP-2023 Task 2: Enhancing BanglaBERT Performance for Bangla Sentiment Analysis Task with In Task Pretraining and Adversarial Weight Perturbation. Workshop on Bangla Language Processing (BLP), 317–323[10.18653/v1/2023.banglalp-1.42]
  13. 13.Md Fahim, Amin Ahsan Ali, M Ashraful Amin, Akmmahbubur Rahman. (2023). Contextual Bangla Neural Stemmer: Finding Contextualized Root Word Representations for Bangla Words. Workshop on Bangla Language Processing (BLP), 94–103[10.18653/v1/2023.banglalp-1.11]

Cited By

  1. 1.Megha Hegde, Jean-Christophe Nebel, Farzana Rahman. (2025). Language Modelling Techniques for Analysing the Impact of Human Genetic Variation. Bioinformatics and Biology Insights, 19, 11779322251358314[10.1177/11779322251358314]
  2. 2.Ashiqur Rahman Saron, Hasin Almas Sifat, Afsah Sharmin. (2026). Bengali Political Narrative Understanding via Transformer Models for Web Applications. IEEE International Conference on Quantum Photonics, Artificial Intelligence & Networking (QPAIN), 1–6[10.1109/qpain69676.2026.11546235]
← All publications