BaFCo: A Document Understanding Benchmark for Complex Bangla Form Comprehension

Keywords

• key information extraction
• multimodal large language models
• document layout analysis
• government forms
• chain-of-thought prompting
• low-resource languages
• zero-shot prompting
• document understanding
• Bengali language
• annotated dataset

CCDS Authors

Ishita Sur Apan

Ishita Sur Apan

• Intern, CCDS

Published at

ECCV, BEA

Research Interests

Natural Language Processing, Multimodal Learning, Software Engineering

Fahim Ahmed

Fahim Ahmed

• Intern, CCDS

Published at

ECCV, ECAI

Armun Alam

Armun Alam

• Research Assistant, CCDS

Published at

ECCV, Scientific Reports

Research Interests

Machine Learning

Amin Ahsan Ali, PhD

Amin Ahsan Ali, PhD

• Director, AI/ML Wing

Professor

Department of Computer Science and Engineering

Independent University, Bangladesh

Published at

ACL, CHI, UbiComp, ECCV, IJCAI, ECAI, WACV, IPSN, PAKDD, LREC, IJCNN, ICPR, ICIP, BEA, BLP, ICMLA, Scientific Reports, Scientific Data, Pattern Recognition, Information Sciences, Ad Hoc Networks, Drug and Alcohol Dependence, PLOS ONE

Research Interests

Artificial Intelligence, Machine Learning

Aman Chadha

Aman Chadha

• Collaborator, AI/ML Wing

Generative AI Leadership

Amazon Web Services

Published at

ECCV

Research Interests

Artificial Intelligence System Design

MD Mofijul Islam

MD Mofijul Islam

• Collaborator, AI/ML Wing

Applied Scientist II

AWS GenAI Innovation Center

Published at

ECCV

Research Interests

Multimodal Machine Learning (Vision + NLP), Large Language Model, Knowledge Graph, Multitask Learning, Generative AI

AKM Mahbubur Rahman, PhD

AKM Mahbubur Rahman, PhD

• Director, DS & IPT Wings

Associate Professor

Department of Computer Science and Engineering

Independent University, Bangladesh

Published at

CHI, ECCV, WACV, PAKDD, LREC, IJCNN, ICPR, ICIP, HCII, BEA, Scientific Reports, Information Sciences

References

1. 1.Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C. Lawrence Zitnick. (2014). Microsoft COCO: Common Objects in Context. Lecture notes in computer science, 740–755[10.1007/978-3-319-10602-1_48]
2. 2.Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou. (2022). Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models. , 24824–24837[10.52202/068431-1800]
3. 3.Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, Furu Wei. (2022). LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking. Proceedings of the 30th ACM International Conference on Multimedia, 4083–4091[10.1145/3503161.3548112]
4. 4.Xu Zhong, Jianbin Tang, Antonio Jimeno Yepes. (2019). PubLayNet: Largest Dataset Ever for Document Layout Analysis. , 1015–1022[10.1109/icdar.2019.00166]
5. 5.Minghao Li, Yiheng Xu, Lei Cui, Shaohan Huang, Furu Wei, Zhoujun Li, Ming Zhou. (2020). DocBank: A Benchmark Dataset for Document Layout Analysis. , 949–960[10.18653/v1/2020.coling-main.82]
6. 6.Galal M. Binmakhashen, Sabri A. Mahmoud. (2019). Document Layout Analysis. ACM Computing Surveys, 52(6), 1–36[10.1145/3355610]
7. 7.Jaekyu Ha, R.M. Haralick, I.T. Phillips. (2002). Recursive X-Y cut using bounding boxes of connected components. , 2, 952–955[10.1109/icdar.1995.602059]
8. 8.Jiapeng Wang, Lianwen Jin, Kai Ding. (2022). LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 7747–7757[10.18653/v1/2022.acl-long.534]
9. 9.Birgit Pfitzmann, Christoph Auer, Michele Dolfi, Ahmed S. Nassar, Peter Staar. (2022). DocLayNet: A Large Human-Annotated Dataset for Document-Layout Segmentation. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 3743–3751[10.1145/3534678.3539043]
10. 10.Teakgyu Hong, DongHyun Kim, Mingi Ji, Wonseok Hwang, Daehyun Nam, Sungrae Park. (2022). BROS: A Pre-trained Language Model Focusing on Text and Layout for Better Key Information Extraction from Documents. Proceedings of the AAAI Conference on Artificial Intelligence, 36(10), 10767–10775[10.1609/aaai.v36i10.21322]
11. 11.Yiheng Xu, Tengchao Lv, Lei Cui, Guoxin Wang, Yijuan Lu, Dinei Florencio, Cha Zhang, Furu Wei. (2022). XFUND: A Benchmark Dataset for Multilingual Visually Rich Form Understanding. Findings of the Association for Computational Linguistics: ACL 2022, 3214–3224[10.18653/v1/2022.findings-acl.253]
12. 12.Dongsheng Wang, Natraj Raman, Mathieu Sibue, Zhiqiang Ma, Petr Babkin, Simerjot Kaur, Yulong Pei, Armineh Nourbakhsh, Xiaomo Liu. (2024). DocLLM: A Layout-Aware Generative Language Model for Multimodal Document Understanding. , 8529–8548[10.18653/v1/2024.acl-long.463]
13. 13.Yuchen Xia, Jiho Kim, Yuhan Chen, Haojie Ye, Souvik Kundu, Cong Callie Hao, Nishil Talati. (2024). Understanding the Performance and Estimating the Cost of LLM Fine-Tuning. , 210–223[10.1109/iiswc63097.2024.00027]
14. 14.Linke Ouyang, Yuan Qu, Hongbin Zhou, Jiawei Zhu, Rui Zhang, Qunshu Lin, Bin Wang, Zhiyuan Zhao, Man Jiang, Xiaomeng Zhao, Jin Shi, Fan Wu, Pei Chu, Minghao Liu, Zhenxiang Li, Chao Xu, Bo Zhang, Botian Shi, Zhongying Tu, Conghui He. (2025). OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations. , 24838–24848[10.1109/cvpr52734.2025.02313]
15. 15.Yuchen Duan, Zhe Chen, Yusong Hu, Weiyun Wang, Shenglong Ye, Botian Shi, Lewei Lu, Qibin Hou, Tong Lu, Hongsheng Li, Jifeng Dai, Wenhai Wang. (2025). Docopilot: Improving Multimodal Models for Document-Level Understanding. , 4026–4037[10.1109/cvpr52734.2025.00381]
16. 16.Syed Saqib Bukhari, Mayce Ibrahim Ali Al Azawi, Faisal Shafait, Thomas M. Breuel. (2010). Document image segmentation using discriminative learning over connected components. , 183–190[10.1145/1815330.1815354]
17. 17.Štěpán Šimsa, Milan Šulc, Michal Uřičář, Yash Patel, Ahmed Hamdi, Matěj Kocián, Matyáš Skalický, Jiří Matas, Antoine Doucet, Mickaël Coustaty, Dimosthenis Karatzas. (2023). DocILE Benchmark for Document Information Localization and Extraction. Lecture notes in computer science, 147–166[10.1007/978-3-031-41679-8_9]
18. 18.Yihao Ding, Siqu Long, Jiabin Huang, Kaixuan Ren, Xingxiang Luo, Hyunsuk Chung, Soyeon Caren Han. (2023). Form-NLU: Dataset for the Form Natural Language Understanding. , 2807–2816[10.1145/3539618.3591886]
19. 19.Wenhui Liao, Jiapeng Wang, Hongliang Li, Chengyu Wang, Jun Huang, Lianwen Jin. (2025). DocLayLLM: An Efficient Multi-modal Extension of Large Language Models for Text-rich Document Understanding. , 4038–4049[10.1109/cvpr52734.2025.00382]
20. 20.Abdellatif Sassioui, Rachid Benouini, Yasser El Ouargui, Mohamed El Kamili, Meriyem Chergui, Mohammed Ouzzif. (2023). Visually-Rich Document Understanding: Concepts, Taxonomy and Challenges. , 1–7[10.1109/wincom59760.2023.10322990]
21. 21.Jiaxin Zhang, Wentao Yang, Songxuan Lai, Zecheng Xie, Lianwen Jin. (2025). DocKylin: A Large Multimodal Model for Visual Document Understanding with Efficient Visual Slimming. Proceedings of the AAAI Conference on Artificial Intelligence, 39(9), 9923–9932[10.1609/aaai.v39i9.33076]
22. 22.Masato Fujitake. (2024). LayoutLLM: Large Language Model Instruction Tuning for Visually Rich Document Understanding. , 10219–10224[10.63317/428wjtmc3xvh]
23. 23.N. Journet, V. Eglin, J.Y. Ramel, R. Mullot. (2005). Text/graphic labelling of ancient printed documents. , 1010–1014 Vol. 2[10.1109/icdar.2005.235]
24. 24.Haoyu Cao, Xin Li, Jiefeng Ma, Deqiang Jiang, Antai Guo, Yiqing Hu, Hao Liu, Yinsong Liu, Bo Ren. (2022). Query-driven Generative Network for Document Information Extraction in the Wild. Proceedings of the 30th ACM International Conference on Multimedia, 4261–4271[10.1145/3503161.3547877]
25. 25.Md. Istiak Hossain Shihab, Md. Rakibul Hasan, Mahfuzur Rahman Emon, Syed Mobassir Hossen, Md. Nazmuddoha Ansary, Intesur Ahmed, Fazle Rabbi Rakib, Shahriar Elahi Dhruvo, Souhardya Saha Dip, Akib Hasan Pavel, Marsia Haque Meghla, Md. Rezwanul Haque, Sayma Sultana Chowdhury, Farig Sadeque, Tahsin Reasat, Ahmed Imtiaz Humayun, Asif Sushmit. (2023). BaDLAD: A Large Multi-Domain Bengali Document Layout Analysis Dataset. Lecture notes in computer science, 326–341[10.1007/978-3-031-41676-7_19]
26. 26.Panfeng Cao, Ye Wang, Qiang Zhang, Zaiqiao Meng. (2023). GenKIE: Robust Generative Multimodal Document Key Information Extraction. , 14702–14713[10.18653/v1/2023.findings-emnlp.979]
27. 27.Sampoorna Poria, Xiaolei Huang. (2025). Bhaasha, Bhāṣā, Zaban: A Survey for Low-Resourced Languages in South Asia – Current Stage and Challenges. , 1386–1406[10.18653/v1/2025.findings-emnlp.73]
28. 28.Chung-Chih Wu, Chien-Hsing Chou, Fu Chang. (2008). A machine-learning approach for analyzing document layout structures with two reading orders. Pattern Recognition, 41(10), 3200–3213[10.1016/j.patcog.2008.03.014]
29. 29.Haoyu Cao, Jiefeng Ma, Antai Guo, Yiqing Hu, Hao Liu, Deqiang Jiang, Yinsong Liu, Bo Ren. (2022). GMN: Generative Multi-modal Network for Practical Document Information Extraction. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 3768–3778[10.18653/v1/2022.naacl-main.276]
30. 30.Stanis{\l}awek, Tomasz, Grali\'nski, Filip, Wr\'oblewska, Anna, Lipi\'nski, Dawid, Kaliska, Agnieszka, Rosalska, Paulina, Topolski, Bartosz, Biecek, Przemys{\l}aw. (2021). Kleister: Key Information Extraction Datasets Involving Long Documents with Complex Layouts. arXiv (Cornell University)[10.1007/978-3-030-86549-8_36]