Conference Paper2019Cited by 10

Medicine recognition from colors and text

Tanjina Piash Proma, Md Zakir Hossan, M Ashraful Amin

Proceedings of the 3rd International Conference on Graphics and Signal Processing

ACM, pp. 39–43

  • Good Health and Well-being — Relevance: 85%
  • Reduced Inequalities — Relevance: 64%
DOI: 10.1145/3338472.3338484URL

Abstract

People of hazy or blurred vision or the elderly people finds it way too challenging just to identify the pills if they are out of the box or packet. And various pills of various shapes, size, texture, color comes with a diverse set of medicinal components. It creates confusion among pills of same color and shape to identify based on a specific texture. For visually impaired people, even if they configure the shape of the pill, the color information and the texts imprinted on the pill remains unknown to them. In this paper, the splitting processes of a dataset according to the number of colors and the texts imprinted on the pills, will be described. Initially the color information were extracted by segmenting pill region from pill image and then some statistical measurements i.e. Kurtosis and skewness, are calculated for probability distributions generated from the image histograms. Thus figuring out the how many colors the pill surface consists of. For the text recognition, the probable text region is detected for an error free text detection. For high quality image data, the reference images from NLM RxIMAGE database has been utilized. The overall accuracy of the proposed system for number of color determination is 95.6% and text recognition accuracy is 81.32%.

Keywords

  • skewness and kurtosis
  • pill recognition
  • text recognition
  • image histogram
  • text region detection
  • drug identification
  • visually impaired people
  • color analysis

CCDS Authors

Md. Zakir Hossan
Md. Zakir Hossan

Md. Zakir Hossan

  • Research Manager, CCDS

Research Interests

Machine Learning, Computational Neuroscience, Mathematical Physics

Prof. M Ashraful Amin, PhD
M Ashraful Amin, PhD

Prof. M Ashraful Amin, PhD

  • Founder Director, CCDS
  • Director, AI/ML, HCI & DS Wings

Professor

+1 more affiliation

Department of Information Sciences and Technology

George Mason University, USA

Published at

ACL, CHI, UbiComp, ECAI, WACV, PAKDD, IJCNN, ICPR, ICIP, HCII, CLEF, BLP, Scientific Reports, JMIR, IEEE Access, Smart Agricultural Technology, Frontiers in Computational Neuroscience, Diabetes Research and Clinical Practice, PLOS ONE

Research Interests

Machine Learning, Cognitive & Vision Science, Cybernetics, Surveillance & Security, ICT in Education, Health, & Agriculture, Human-Computer Interaction, Internet of Things, Robotics

References

  1. 1.David Lowe. (2004). Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision, 60(2), 91–110[10.1023/b:visi.0000029664.99615.94]
  2. 2.Herbert Bay, Andreas Ess, Tinne Tuytelaars, Luc Van Gool. (2008). Speeded-Up Robust Features (SURF). Computer Vision and Image Understanding, 110(3), 346–359[10.1016/j.cviu.2007.09.014]
  3. 3.Lukáš Neumann, Jiřı́ Matas. (2012). Real-time scene text localization and recognition. , 3538–3545[10.1109/cvpr.2012.6248097]
  4. 4.Remco C. Veltkamp, Michiel Hagedoorn. (2001). State of the Art in Shape Matching. Advances in computer vision and pattern recognition, 87–119[10.1007/978-1-4471-3702-3_4]
  5. 5.Huizhong Chen, Sam S. Tsai, Georg Schroth, David M. Chen, Radek Grzeszczuk, Bernd Girod. (2011). Robust text detection in natural images with edge-enhanced Maximally Stable Extremal Regions. , 2609–2612[10.1109/icip.2011.6116200]
  6. 6.Li Yao, Huchuan Lu. (2016). Scene Text Detection via Stroke Width.
  7. 7.M. Ashraful Amin, Hong Yan. (2007). Sign Language Finger Alphabet Recognition from Gabor-PCA Representation of Hand Gestures. , 2218–2223[10.1109/icmlc.2007.4370514]
  8. 8.Young‐Beom Lee, Unsang Park, Anil K. Jain. (2010). PILL-ID: Matching and Retrieval of Drug Pill Imprint Images. , 2632–2635[10.1109/icpr.2010.645]
  9. 9.M. Ashraful Amin, Mahmood Kazi Mohammed. (2015). Overview of the ImageCLEF 2015 medical clustering task. CLEF (Working Notes)
  10. 10.Iqbal Nouyed, Bruce Poon, M. Ashraful Amin, Hong Yan. (2015). A study on the discriminating characteristics of Gabor phase-face and an improved method for face recognition. International Journal of Machine Learning and Cybernetics, 7(6), 1115–1130[10.1007/s13042-015-0440-8]
  11. 11.Andreas Hartl, Clemens Arth, Dieter Schmalstieg. (2011). Instant Medical Pill Recognition on Mobile Phones. Robotics and Applications[10.2316/p.2011.740-017]
  12. 12.K. Hasan, M. Ashraful Amin. (2012). Dual iris matching for biometric identification. Signal Image and Video Processing, 8(8), 1605–1611[10.1007/s11760-012-0399-9]
  13. 13.Iftakhar Hasan, Minnatul Fatema, M. Ashraful Amin. (2011). Dual iris based human identification. , 2, 79–84[10.1109/iccitechn.2011.6164878]
  14. 14.Michiel Hagedoorn. (2000). Pattern matching using similarity measures. [link]
  15. 15.Andreas Hartl. (2022). Computer-vision based pharmaceutical pill recognition on mobile phones. [10.3217/casr3-4ew24]

Cited By

  1. 1.Khalil Al-Hussaeni, Ioannis Karamitsos, Ezekiel Adewumi, Rema M. Amawi. (2023). CNN-Based Pill Image Recognition for Retrieval Systems. Applied Sciences, 13(8), 5050[10.3390/app13085050]
  2. 2.Anh Duy Nguyen, Huy Hieu Pham, Huynh Thanh Trung, Quoc Viet Hung Nguyen, Thao Nguyen Truong, Phi Le Nguyen. (2023). High accurate and explainable multi-pill detection framework with graph neural network-assisted multimodal data fusion. PLoS ONE, 18(9), e0291865[10.1371/journal.pone.0291865]
  3. 3.Joe Llerena-Izquierdo, Fernando Procel-Jupiter, Alison Cunalema-Arana. (2020). Mobile Application with Cloud-Based Computer Vision Capability for University Students’ Library Services. Advances in intelligent systems and computing, 3–15[10.1007/978-3-030-60467-7_1]
  4. 4.Ruth Alvarado-Salazar, Joe Llerena-Izquierdo. (2022). Revisión de la literatura sobre el uso de Inteligencia Artificial enfocada a la atención de la discapacidad visual. Revista InGenio, 5(1), 10–21[10.18779/ingenio.v5i1.472]
  5. 5.Anh Duy Nguyen, Thuy Dung Nguyen, Huy Hieu Pham, Thanh Hung Nguyen, Phi Le Nguyen. (2022). Image-Based Contextual Pill Recognition with Medical Knowledge Graph Assistance. Communications in computer and information science, 354–369[10.1007/978-981-19-8234-7_28]
  6. 6.Zulfadhlina Amira Nor Hisham, Masyura Ahmad Faudzi, Azimah Abdul Ghapar, Fiza Abdul Rahim. (2020). A Systematic Literature Review of the Mobile Application for Object Recognition for Visually Impaired People. , 37, 316–322[10.1109/icimu49871.2020.9243523]
  7. 7.R Shashidhar, V Sahana, Sudeshna Chakraborty, S B Puneeth, M Roopa. (2021). Recognition of Tablet using Blister Strip for Visually Impaired using SIFT Algorithm. Indian Journal of Science and Technology, 14(23), 1953–1960[10.17485/ijst/v14i23.750]
  8. 8.Wenjun Liu. (2021). Collaborative Innovation of Online Ideological Education Platform with Data Mining and Text Recognition Algorithms. , 2, 1372–1375[10.1109/iccmc51019.2021.9418306]
  9. 9.João Barata, Mateus Mendes, Ana Paula Melo Santos, Maria João Melo Neves, Kamil Wozniak, Anna Drozdz, José Sousa. (2022). Artificial Intelligence-Enabled Medicine Dispensing: From Concepts to Practice in the Community Pharmacy (Preprint). [10.2196/preprints.38426]
  10. 10.João Barata, Mateus Mendes, Ana Paula Melo Santos, Maria João Melo Neves, Kamil Woźniak, Anna Drozdz, José Sousa. (2026). IT-enabled medicine dispensing in the community pharmacy: an action design research project. Health Systems, 15(2), 174–187[10.1080/20476965.2025.2591042]
← All publications