Journal Article2026Cited by 2

SuSastho.AI: A multimodal medical copilot for adolescents using evidence-based medicine and large language models

Shamim Ahamed, Moinul H. Chowdhury, Marzia Zaman, Abhishek Agarwala, Md Abdullah Al Mashud, Tariqul, Tareq Al Muntasir, Rifat Rahman, Rifat Shahriyar, Farhana Sarker, Khondaker A. Mamun

Informatics in Medicine Unlocked

Elsevier BV, Vol. 60, pp. 101720, ISBN: 2352-9148

DOI: 10.1016/j.imu.2025.101720Open AccessURL

Abstract

Adolescents in Bangladesh face serious sexual, reproductive, and mental health challenges due to cultural stigma, poverty, and limited healthcare infrastructure. Within the country, 63% of adolescents are deprived of essential sexual and reproductive health services, and only 13% receive mental health support. Adolescents living in urban slums and with disabilities face additional challenges in receiving reliable health information. This limited access exposes them to a high risk of STIs, unintended pregnancies, and serious mental health issues. Addressing these challenges our study introduces SuSastho.AI, a healthcare copilot to provide access to reliable health information to adolescents. We utilized large language models, along with evidence-based medicine, retrieval-augmented generation, and a clinically validated dataset to provide evidence-based responses. Our copilot supports both voice and text-based interactions. Clinical evaluation of a pilot study shows our method reduces incorrect responses by 26.9% and increases response correctness by 16.1% compared to other methods. It achieved an accuracy rate of 86.7% when specifically evaluated based on available knowledge. While the responses are mostly consistent with up-to-date medical practices, occasional less precise responses highlight the need for further refinement. Participants reported overall positive feedback, where 87% found their answers and 90.7% found responses relevant. Our results show that SuSastho.AI can provide reliable and evidence-based information while being an affordable way to support traditional healthcare systems with a high potential to transform digital health. The study sets an example as an evidence-based healthcare copilot to support adolescents and lays the foundation for future research where evidence-based tools overcome social barriers.

CCDS Authors

Farhana Sarker, PhD
Farhana Sarker, PhD

Farhana Sarker, PhD

  • Director, WiSE Wing

  • Associate Professor

    Department of Computer Science and Engineering

    Independent University, Bangladesh (IUB)

Published at

ICDMW, IEEE Access

References

  1. 1.Chin-Yew Lin. (2004). ROUGE: A Package for Automatic Evaluation of Summaries. , 74–81
  2. 2.Joseph Weizenbaum. (1966). ELIZA—A Computer Program For the Study of Natural Language Communication Between Man and Machine. Communications of the ACM, 9(1), 36–45[10.1145/365153.365168]
  3. 3.Gordon Guyatt. (1992). Evidence-Based Medicine. JAMA, 268(17), 2420[10.1001/jama.1992.03490170092032]
  4. 4.Satanjeev Banerjee, Alon Lavie. (2005). METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. , 65–72
  5. 5.Karan Singhal, Shekoofeh Azizi, Tao Tu, S. Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, Perry Payne, Martin Seneviratne, Paul Gamble, Chris Kelly, Abubakr Babiker, Nathanael Schärli, Aakanksha Chowdhery, Philip Mansfield, Dina Demner-Fushman, Blaise Agüera y Arcas, Dale Webster, Greg S. Corrado, Yossi Matias, Katherine Chou, Juraj Gottweis, Nenad Tomasev, Yun Liu, Alvin Rajkomar, Joelle Barral, Christopher Semturs, Alan Karthikesalingam, Vivek Natarajan. (2023). Large language models encode clinical knowledge. Nature, 620(7972), 172–180[10.1038/s41586-023-06291-2]
  6. 6.Kathleen Kara Fitzpatrick, Alison Darcy, Molly Vierhile. (2017). Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial. JMIR Mental Health, 4(2), e19[10.2196/mental.7785]
  7. 7.T. Greenhalgh, J. Howick, N. Maskrey, for the Evidence Based Medicine Renaissance Group. (2014). Evidence based medicine: a movement in crisis?. BMJ, 348(jun13 4), g3725[10.1136/bmj.g3725]
  8. 8.Lalit R. Bahl, Frederick Jelinek, Robert L. Mercer. (1983). A Maximum Likelihood Approach to Continuous Speech Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-5(2), 179–190[10.1109/tpami.1983.4767370]
  9. 9.Becky Inkster, Shubhankar Sarda, Vinod Subramanian. (2018). An Empathy-Driven, Conversational Artificial Intelligence Agent (Wysa) for Digital Mental Well-Being: Real-World Data Evaluation Mixed-Methods Study. JMIR mhealth and uhealth, 6(11), e12106[10.2196/12106]
  10. 10.Eliane M. Boucher, Nicole R. Harake, Haley E. Ward, Sarah Elizabeth Stoeckl, Junielly Vargas, Jared Minkel, Acacia C. Parks, Ran Zilca. (2021). Artificially intelligent chatbots in digital mental health interventions: a review. Expert Review of Medical Devices, 18(sup1), 37–49[10.1080/17434440.2021.2013200]
  11. 11.M D Romael Haque, Sabirat Rubya. (2023). An Overview of Chatbot-Based Mobile Mental Health Apps: Insights From App Description and User Reviews. JMIR mhealth and uhealth, 11, e44838[10.2196/44838]
  12. 12.Mohammad Didar Hossain, Helal Uddin Ahmed, Waziul Alam Chowdhury, Louis Wilhelmus Niessen, Dewan Shamsul Alam. (2014). Mental disorders in Bangladesh: a systematic review. BMC Psychiatry, 14(1), 216[10.1186/s12888-014-0216-9]
  13. 13.Mahyar Abbasian, Elahe Khatibi, Iman Azimi, David Oniani, Zahra Shakeri Hossein Abad, Alexander Thieme, Ram Sriram, Zhongqi Yang, Yanshan Wang, Bryant Lin, Olivier Gevaert, Li-Jia Li, Ramesh Jain, Amir M. Rahmani. (2024). Foundation metrics for evaluating effectiveness of healthcare conversations powered by generative AI. npj Digital Medicine, 7(1), 82[10.1038/s41746-024-01074-z]
  14. 14.Maria Carolina Klos, Milagros Escoredo, Angela Joerin, Viviana Noemí Lemos, Michiel Rauws, Eduardo L Bunge. (2021). Artificial Intelligence–Based Chatbot for Anxiety and Depression in University Students: Pilot Randomized Controlled Trial. JMIR Formative Research, 5(8), e20678[10.2196/20678]
  15. 15.Luke Balcombe. (2023). AI Chatbots in Digital Mental Health. Informatics, 10(4), 82[10.3390/informatics10040082]
  16. 16.Shi Min Lim, Chyi Wey Claudine Shiau, Ling Jie Cheng, Ying Lau. (2021). Chatbot-Delivered Psychotherapy for Adults With Depressive and Anxiety Symptoms: A Systematic Review and Meta-Regression. Behavior Therapy, 53(2), 334–347[10.1016/j.beth.2021.09.007]
  17. 17.Friederike Holderried, Christian Stegemann–Philipps, Lea Herschbach, Julia-Astrid Moldt, Andrew Nevins, Jan Griewatz, Martin Holderried, Anne Herrmann-Werner, Teresa Festl-Wietek, Moritz Mahling. (2024). A Generative Pretrained Transformer (GPT)–Powered Chatbot as a Simulated Patient to Practice History Taking: Prospective, Mixed Methods Study. JMIR Medical Education, 10, e53961[10.2196/53961]
  18. 18.Alaa Abd-Alrazaq, Zeineb Safi, Mohannad Alajlani, Jim Warren, Mowafa Househ, Kerstin Denecke. (2020). Technical Metrics Used to Evaluate Health Care Chatbots: Scoping Review. Journal of Medical Internet Research, 22(6), e18301[10.2196/18301]
  19. 19.Sooah Jang, Jae-Jin Kim, Soo-Jeong Kim, Jieun Hong, Suji Kim, Eunjoo Kim. (2021). Mobile app-based chatbot to deliver cognitive behavioral therapy and psychoeducation for adults with attention deficit: A development and feasibility/usability study. International Journal of Medical Informatics, 150, 104440[10.1016/j.ijmedinf.2021.104440]
  20. 20.Saeid Ashraf Vaghefi, Dominik Stammbach, Veruska Muccione, Julia Bingler, Jingwei Ni, Mathias Kraus, Simon Allen, Chiara Colesanti-Senni, Tobias Wekhof, Tobias Schimanski, Glen Gostlow, Tingyu Yu, Qian Wang, Nicolas Webersinke, Christian Huggel, Markus Leippold. (2023). ChatClimate: Grounding conversational AI in climate science. Communications Earth & Environment, 4(1)[10.1038/s43247-023-01084-x]
  21. 21.Hua Wang, Sneha Gupta, Arvind Singhal, Poonam Muttreja, Sanghamitra Singh, Poorva Sharma, Alice Piterova. (2022). An Artificial Intelligence Chatbot for Young People’s Sexual and Reproductive Health in India (SnehAI): Instrumental Case Study. Journal of Medical Internet Research, 24(1), e29969[10.2196/29969]
  22. 22.Kate Daley, Ines Hungerbuehler, Kate Cavanagh, Heloísa Garcia Claro, Paul Alan Swinton, Michael Kapps. (2020). Preliminary Evaluation of the Engagement and Effectiveness of a Mental Health Chatbot. Frontiers in Digital Health, 2, 576361[10.3389/fdgth.2020.576361]
  23. 23.Rifat Rahman, Md. Rishadur Rahman, Nafis Irtiza Tripto, Mohammed Eunus Ali, Sajid Hasan Apon, Rifat Shahriyar. (2021). AdolescentBot: Understanding Opportunities for Chatbots in Combating Adolescent Sexual and Reproductive Health Problems in Bangladesh. CHI Conference on Human Factors in Computing Systems, 1–15[10.1145/3411764.3445694]
  24. 24.Yifan Peng, Justin F. Rousseau, Edward H. Shortliffe, Chunhua Weng. (2023). AI-generated text may have a role in evidence-based medicine. Nature Medicine, 29(7), 1593–1594[10.1038/s41591-023-02366-9]
  25. 25.Rhiana Mills, Emily Rose Mangone, Neal Lesh, Diwakar Mohan, Paula Baraitser. (2023). Chatbots to Improve Sexual and Reproductive Health: Realist Synthesis. Journal of Medical Internet Research, 25, e46761[10.2196/46761]
  26. 26.Zubair Ahmed Ratan, Anne-Maree Parrish, Mohammad Saud Alotaibi, Hassan Hosseinzadeh. (2022). Prevalence of Smartphone Addiction and Its Association with Sociodemographic, Physical and Mental Well-Being: A Cross-Sectional Study among the Young Adults of Bangladesh. International Journal of Environmental Research and Public Health, 19(24), 16583[10.3390/ijerph192416583]
  27. 27.Giovannino Ciccone. (2014). Evidence-based medicine: un movimento in crisi?. Recenti Progressi in Medicina, 105(9), 315–6[10.1701/1606.17512]
  28. 28.Jona Kräenbring, Tika Monzon Penza, Joanna Gutmann, Susanne Muehlich, Oliver Zolk, Leszek Wojnowski, Renke Maas, Stefan Engelhardt, Antonio Sarikas. (2014). Accuracy and Completeness of Drug Information in Wikipedia: A Comparison with Standard Textbooks of Pharmacology. PLoS ONE, 9(9), e106930[10.1371/journal.pone.0106930]
  29. 29.Usaid K. Allahwala, Aniket Nadkarni, Deshan F. Sebaratnam. (2012). Wikipedia use amongst medical students – New insights into the digital revolution. Medical Teacher, 35(4), 337[10.3109/0142159x.2012.737064]
  30. 30.Samy A Azer, Nourah M AlSwaidan, Lama A Alshwairikh, Jumana M AlShammari. (2015). Accuracy and readability of cardiovascular entries on Wikipedia: are they reliable learning resources for medical students?. BMJ Open, 5(10), e008187[10.1136/bmjopen-2015-008187]
  31. 31.Vaghefi, Saeid Ashraf, Stammbach, Dominik, Muccione, Veruska, Bingler, Julia, Ni, Jingwei, Kraus, Mathias, Allen, Simon, Colesanti Senni, Chiara, Werkhof, Tobias, Schimanski, Tobias, Gostlow, Glen, Yu, Tingyu, Wang, Qian, Webersinke, Nicolas, Huggel, Christian, Leippold, Markus. (2023). ChatClimate: Grounding conversational AI in climate science. Zurich Open Repository and Archive (University of Zurich)[10.3929/ethz-b-000649578]
  32. 32.Mohammad Hammoud, Shahd Douglas, Mohamad Darmach, Sara Alawneh, Swapnendu Sanyal, Youssef Kanbour. (2024). Evaluating the Diagnostic Performance of Symptom Checkers: Clinical Vignette Study. JMIR AI, 3, e46875[10.2196/46875]
  33. 33.Nazmul M Huq, Abul Quasem Al-Amin, Sushil Ranjan Howlader, Mohammad Alamgir Kabir. (2015). Paying Out of Pocket for Healthcare in Bangladesh - A Burden on Poor?. Europe PMC (PubMed Central), 44(7), 1024–5[link]
  34. 34.Bavly Samy Helmy, Andrew Samy Helmy. (2021). Role of Artificial Intelligence in Mental Wellbeing: Opportunities and Challenges. Journal of Artificial Intelligence, 15(1), 1–8[10.3923/jai.2022.1.8]
  35. 35.Christiano, Paul, Leike, Jan, Brown, Tom B., Martic, Miljan, Legg, Shane, Amodei, Dario. (2017). Deep reinforcement learning from human preferences. arXiv (Cornell University), 30, 4299–4307[10.48550/arxiv.1706.03741]
  36. 36.Rafailov, Rafael, Sharma, Archit, Mitchell, Eric, Ermon, Stefano, Manning, Christopher D., Finn, Chelsea. (2023). Direct Preference Optimization: Your Language Model is Secretly a Reward Model. arXiv (Cornell University)[10.48550/arxiv.2305.18290]
  37. 37.Om P. Singh. (2023). Artificial intelligence in the era of ChatGPT - Opportunities and challenges in mental health care. Indian Journal of Psychiatry, 65(3), 297–298[10.4103/indianjpsychiatry.indianjpsychiatry_112_23]

Cited By

  1. 1.Zumrotul Ula, Awatiful Azza, Sitti Zakiyyah Putri, Risna Ayu Rahmadani, Indryani Indryani. (2026). Effectiveness of comprehensive reproductive health education on adolescent sexual risk behavior and health literacy: quasi-experimental study. JURNAL ILMIAH KESEHATAN SANDI HUSADA, 15(2), 512–521[10.35816/jiksh.v15i2.371]
  2. 2.Sinthia Chowdhury, Deawan Rakin Ahamed Remal, Sefatul Wasi, Ashraful Islam, Moinul Zaber, M. Ashraful Amin. (2026). Designing Conversational User Interface with Low-Resource Bi-directional Machine Translation for Chittagonian Dialect to Regular Bangla Health Dialogues. Communications in computer and information science, 193–203[10.1007/978-3-032-30833-7_21]
← All publications