Conference Paper2007Cited by 51

Sign language finger alphabet recognition from Gabor-PCA representation of hand gestures

M Ashraful Amin, Hong Yan

2007 International Conference on Machine Learning and Cybernetics

IEEE, Vol. 4, pp. 2218–2223

  • Reduced Inequalities — Relevance: 81%
DOI: 10.1109/icmlc.2007.4370514URL

Abstract

During recent years a large number of computer aided applications have been developed to help the disabled people. This has improved the communication between the able and the hearing impaired community. An intelligent signed alphabet recognizer can work as an aiding agent to translate the signs to words (and also sentences) and vice versa. To achieve this goal few steps to be followed, among which the first complicated task is to recognize the sign-language alphabets from hand gesture images. In this paper, we propose a system that is able to recognize American Sign Language (ASL) alphabets from hand gesture with average 93.23% accuracy. The classification is performed with fuzzy-c-mean clustering on a lower dimensional data which is acquired from the Principle Component Analysis (PCA) of Gabor representation of hand gesture images. Out of the top 20 Principle Components (PCs) the best combination of PCs is determined by finding the best fuzzy cluster for the corresponding PCs of the training data. The best result is obtained from the combination of the fourth to seventh principle components.

Keywords

  • fuzzy c-means clustering
  • Principal component analysis
  • American Sign Language
  • hand gesture recognition
  • sign language recognition
  • Gabor features
  • Feature selection
  • hearing-impaired communication

CCDS Authors

Prof. M Ashraful Amin, PhD
M Ashraful Amin, PhD

Prof. M Ashraful Amin, PhD

  • Founder Director, CCDS
  • Director, AI/ML, HCI & DS Wings

Professor

+1 more affiliation

Department of Information Sciences and Technology

George Mason University, USA

Published at

ACL, CHI, UbiComp, ECAI, WACV, PAKDD, IJCNN, ICPR, ICIP, HCII, CLEF, BLP, Scientific Reports, JMIR, IEEE Access, Smart Agricultural Technology, Frontiers in Computational Neuroscience, Diabetes Research and Clinical Practice, PLOS ONE

Research Interests

Machine Learning, Cognitive & Vision Science, Cybernetics, Surveillance & Security, ICT in Education, Health, & Agriculture, Human-Computer Interaction, Internet of Things, Robotics

References

  1. 1.James C. Bezdek. (1981). Pattern Recognition with Fuzzy Objective Function Algorithms. [10.1007/978-1-4757-0450-1]
  2. 2.John Daugman. (1988). Complete discrete 2-D Gabor transforms by neural networks for image analysis and compression. IEEE Transactions on Acoustics Speech and Signal Processing, 36(7), 1169–1179[10.1109/29.1644]
  3. 3.Stanley R. Deans. (1983). The Radon Transform and Some of Its Applications.
  4. 4.Vladimir Pavlović, Rajeev Sharma, Thomas S. Huang. (1997). Visual interpretation of hand gestures for human-computer interaction: a review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), 677–695[10.1109/34.598226]
  5. 5.Tai Sing Lee. (1996). Image representation using 2D Gabor wavelets. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(10), 959–971[10.1109/34.541406]
  6. 6.Thad Starner, Joshua Weaver, Alex Pentland. (1998). Real-time American sign language recognition using desk and wearable computer based video. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(12), 1371–1375[10.1109/34.735811]
  7. 7.Bruce A. Draper, Kyungim Baek, Marian Stewart Bartlett, J. Ross Beveridge. (2003). Recognizing faces with PCA and ICA. Computer Vision and Image Understanding, 91(1-2), 115–137[10.1016/s1077-3142(03)00077-8]
  8. 8.Ying Wu, Thomas S. Huang. (1999). Vision-Based Gesture Recognition: A Review. Lecture notes in computer science, 103–115[10.1007/3-540-46616-9_10]
  9. 9.Clayton Valli, Ceil Lucas, Kristin J. Mulrooney. (1995). Linguistics of American Sign Language: An Introduction. Internet Archive (Internet Archive)
  10. 10.Rung‐Huei Liang, Ming Ouhyoung. (2002). A real-time continuous gesture recognition system for sign language. , 558–567[10.1109/afgr.1998.671007]
  11. 11.K. Murakami, Hitomi Taguchi. (1991). Gesture recognition using recurrent neural networks. , 237–242[10.1145/108844.108900]
  12. 12.Christian Vogler, Dimitris Metaxas. (2001). A Framework for Recognizing the Simultaneous Aspects of American Sign Language. Computer Vision and Image Understanding, 81(3), 358–384[10.1006/cviu.2000.0895]
  13. 13.Christian Vogler, D. Metaxas. (2002). ASL recognition based on a coupling between HMMs and 3D motion analysis. , 363–369[10.1109/iccv.1998.710744]
  14. 14.Aleix M. Martı́nez, Ronnie Β. Wilbur, Robin A. Shay, A.C. Kak. (2003). Purdue RVL-SLLL ASL database for automatic recognition of American Sign Language. , 167–172[10.1109/icmi.2002.1166987]
  15. 15.Shinichi Tamura, Shingo Kawasaki. (1988). Recognition of sign language motion images. Pattern Recognition, 21(4), 343–353[10.1016/0031-3203(88)90048-9]
  16. 16.Sidney Fels, Geoffrey E. Hinton. (1998). Glove-TalkII-a neural-network interface which maps gestures to parallel formant speech synthesizer controls. IEEE Transactions on Neural Networks, 9(1), 205–212[10.1109/72.655042]
  17. 17.Tomoichi Takahashi, Fumio Kishino. (1991). Hand gesture coding based on experiments using a hand gesture interface device. ACM SIGCHI Bulletin, 23(2), 67–74[10.1145/122488.122499]
  18. 18.Christian Vogler, Dimitris Metaxas. (1999). Toward Scalability in ASL Recognition: Breaking Down Signs into Phonemes. Lecture notes in computer science, 211–224[10.1007/3-540-46616-9_19]
  19. 19.C. Charayaphan, A. E. Marble. (1992). Image processing system for interpreting motion in American Sign Language. Journal of Biomedical Engineering, 14(5), 419–425[10.1016/0141-5425(92)90088-3]

Cited By

  1. 1.Cao Dong, Ming C. Leu, Zhaozheng Yin. (2015). American Sign Language alphabet recognition using Microsoft Kinect. , 44–52[10.1109/cvprw.2015.7301347]
  2. 2.Mao Ye, Qing Zhang, Liang Wang, Jiejie Zhu, Ruigang Yang, Juergen Gall. (2013). A Survey on Human Motion Analysis from Depth Data. Lecture notes in computer science, 149–187[10.1007/978-3-642-44964-2_8]
  3. 3.Shikha Gupta, Jafreezal Jaafar, Wan Fatimah wan Ahmad, Arpit Bansal. (2013). Feature Extraction Using Mfcc. Signal & Image Processing An International Journal, 4(4), 101–108[10.5121/sipij.2013.4408]
  4. 4.S. Padam Priyal, Prabin Kumar Bora. (2013). A robust static hand gesture recognition system using geometry based normalizations and Krawtchouk moments. Pattern Recognition, 46(8), 2202–2219[10.1016/j.patcog.2013.01.033]
  5. 5.Eva Kollorz, Jochen Penne, Joachim Hornegger, Alexander Barke. (2008). Gesture recognition with a Time-Of-Flight camera. International Journal of Intelligent Systems Technologies and Applications, 5(3/4), 334[10.1504/ijista.2008.021296]
  6. 6.Ahmad Sami Al-Shamayleh, Rodina Ahmad, Mohammad A. M. Abushariah, Khubaib Amjad Alam, Nazean Jomhari. (2018). A systematic literature review on vision based gesture recognition techniques. Multimedia Tools and Applications, 77(21), 28121–28184[10.1007/s11042-018-5971-z]
  7. 7.Deng-Yuan Huang, Wu-Chih Hu, Sung-Hsiang Chang. (2010). Gabor filter-based hand-pose angle estimation for hand gesture recognition under varying illumination. Expert Systems with Applications, 38(5), 6031–6042[10.1016/j.eswa.2010.11.016]
  8. 8.Dominique Uebersax, Juergen Gall, Michael Van den Bergh, Luc Van Gool. (2011). Real-time sign language letter and word recognition from depth data. [10.1109/iccvw.2011.6130267]
  9. 9.Deng-Yuan Huang, Wu-Chih Hu, Sung-Hsiang Chang. (2009). Vision-Based Hand Gesture Recognition Using PCA+Gabor Filters and SVM. , 1–4[10.1109/iih-msp.2009.96]
  10. 10.Shengli Zhou, Fei Fei, Guanglie Zhang, John D. Mai, Yunhui Liu, Jay Y. J. Liou, Wen J. Li. (2013). 2D Human Gesture Tracking and Recognition by the Fusion of MEMS Inertial and Vision Sensors. IEEE Sensors Journal, 14(4), 1160–1170[10.1109/jsen.2013.2288094]
  11. 11.C. Jose L. Flores, A. E. Gladys Cutipa, R. Lauro Enciso. (2017). Application of convolutional neural networks for static hand gestures recognition under different invariant features. , 1–4[10.1109/intercon.2017.8079727]
  12. 12.Shikha Gupta, Jafreezal Jaafar, Wan Fatimah Wan Ahmad. (2012). Static Hand Gesture Recognition Using Local Gabor Filter. Procedia Engineering, 41, 827–832[10.1016/j.proeng.2012.07.250]
  13. 13.Ravinder Ahuja, Daksh Jain, Deepanshu Sachdeva, Archit Garg, Chirag Rajput. (2019). Convolutional Neural Network Based American Sign Language Static Hand Gesture Recognition. International Journal of Ambient Computing and Intelligence, 10(3), 60–73[10.4018/ijaci.2019070104]
  14. 14.Haitham Badi. (2016). Recent methods in vision-based hand gesture recognition. International Journal of Data Science and Analytics, 1(2), 77–87[10.1007/s41060-016-0008-z]
  15. 15.Javier Jimenez, Anabel Martin, Victor Uc, Arturo Espinosa. (2017). Mexican Sign Language Alphanumerical Gestures Recognition using 3D Haar-like Features. IEEE Latin America Transactions, 15(10), 2000–2005[10.1109/tla.2017.8071247]
  16. 16.Fang-Ting Liu, Yong-Ting Wang, Hsi-Pin Ma. (2017). Gesture recognition with wearable 9-axis sensors. [10.1109/icc.2017.7997414]
  17. 17.Watcharin Tangsuksant, Suchin Adhan, Chuchart Pintavirooj. (2014). American Sign Language recognition by using 3D geometric invariant feature and ANN classification. , 1–5[10.1109/bmeicon.2014.7017372]
  18. 18.Youdong Ding, Haibo Pang, Xuechun Wu, Jianliang Lan. (2011). Recognition of hand-gestures using improved local binary pattern. , 3171–3174[10.1109/icmt.2011.6001919]
  19. 19.Kanjana Pattanaworapan, Kosin Chamnongthai, Jing-Ming Guo. (2016). Signer-independence finger alphabet recognition using discrete wavelet transform and area level run lengths. Journal of Visual Communication and Image Representation, 38, 658–677[10.1016/j.jvcir.2016.04.015]
  20. 20.Rajat Agarwal, Balasubramanian Raman, Ankush Mittal. (2015). Hand gesture recognition using discrete wavelet transform and support vector machine. , 489–493[10.1109/spin.2015.7095326]
  21. 21.Bhumika Nandwana, Satyanarayan Tazi, Sheifalee Trivedi, Dinesh Kumar, Santosh Kumar Vipparthi. (2017). A survey paper on hand gesture recognition. , 15, 147–152[10.1109/csnt.2017.8418527]
  22. 22.Fahad Ullah. (2011). American Sign Language recognition system for hearing impaired people using Cartesian Genetic Programming. , 1802, 96–99[10.1109/icara.2011.6144863]
  23. 23.R. Sathya, M. Kalaiselvi Geetha. (2015). Framework for Traffic Personnel Gesture Recognition. Procedia Computer Science, 46, 1700–1707[10.1016/j.procs.2015.02.113]
  24. 24.Suchin Adhan, Chuchart Pintavirooj. (2016). Thai sign language recognition by using geometric invariant feature and ANN classification. , 2016, 4[10.1109/bmeicon.2016.7859627]
  25. 25.Hoshang Kolivand, Saba Joudaki, Mohd Shahrizal Sunar, David Tully. (2021). An implementation of sign language alphabet hand posture recognition using geometrical features through artificial neural network (part 2). Neural Computing and Applications, 33(20), 13885–13907[10.1007/s00521-021-06025-3]
  26. 26.Tanjina Piash Proma, Md. Zakir Hossan, M. Ashraful Amin. (2019). Medicine Recognition from Colors and Text. , 39–43[10.1145/3338472.3338484]
  27. 27.Yong-Ting Wang, Hsi-Pin Ma. (2018). Real-Time Continuous Gesture Recognition with Wireless Wearable IMU Sensors. , 1–6[10.1109/healthcom.2018.8531095]
  28. 28.Lokesh Kumar Viswavarapu. (2018). Real-Time Finger Spelling American Sign Language Recognition Using Deep Convolutional Neural Networks. [10.12794/metadc1404616]
  29. 29.Priya Prajapati, Gorrepati Sai Surya, M. Nithya. (2020). An Interpreter for the Differently Abled using Haptic Feedback and Machine Learning. 2020 Third International Conference on Smart Systems and Inventive Technology (ICSSIT), 1–7[10.1109/icssit48917.2020.9214398]
  30. 30.Ying Zheng, Su Xiao. (2016). Performance Analysis of a Moving Target Tracking Method Based on Computer Vision. , 1, 467–470[10.1109/icmtma.2016.117]
  31. 31.Zhou Lu, Zhang Li-Shuang, Sun Lei, Zhang Xue-Bo. (2016). Dynamic hand gesture recognition using HMM-BPNN model. , 422–426[10.1109/rcar.2016.7784066]
  32. 32.Zhan Song, Hanxuan Yang, Yanguo Zhao, Feng Zheng. (2010). Hand Detection and Gesture Recognition Exploit Motion Times Image in Complicate Scenarios. Lecture notes in computer science, 628–636[10.1007/978-3-642-17274-8_61]
  33. 33.Padmaja Vivek Kulkarni, Boris Illing, Bastian Gaspers, Bernd Brüggemann, Dirk Schulz. (2019). Mobile manipulator control through gesture recognition using IMUs and Online Lazy Neighborhood Graph search. ACTA IMEKO, 8(4), 3[10.21014/acta_imeko.v8i4.677]
  34. 34.Arif-Ul-Islam, Shamim Akhter. (2018). Orientation Hashcode and Articial Neural Network Based Combined Approach to Recognize Sign Language. , 1–5[10.1109/iccitechn.2018.8631967]
  35. 35.I. C. Triviño-López, C. H. Rodríguez-Garavito, J. S. Martinez-Caldas. (2020). Hand Gesture Recognition Using Computer Vision Applied to Colombian Sign Language. Lecture notes in computer science, 207–214[10.1007/978-3-030-45096-0_26]
  36. 36.Nabil Boughnim, Julien Marot, Caroline Fossati, Salah Bourennane. (2012). Hand Posture Classification by Means of a New Contour Signature. Lecture notes in computer science, 384–394[10.1007/978-3-642-33140-4_34]
  37. 37.Haitham S. Hasan. (2014). STATIC HAND GESTURE RECOGNITION USING ARTIFICIAL NEURAL NETWORK.
  38. 38.You Dong Ding, Hai Bo Pang. (2012). An Improved Algorithm of Hand-Gesture Recognition Based on Haar-Like Features and Adaboost. Advanced materials research, 588-589, 1238–1241[10.4028/www.scientific.net/amr.588-589.1238]
  39. 39.Ferhat Roumiassa, Salah Eddine Agab, Fatma Zohra Chelali. (2022). Hand Gesture Recognition System Based on Textural Features. 2022 2nd International Conference on Advanced Electrical Engineering (ICAEE), 10 1, 1–6[10.1109/icaee53772.2022.9962080]
  40. 40.Santosh Kumar, Arvind Kaurav. (2018). Hand gesture through geometric moments (HCI based). 2018 2nd International Conference on Inventive Systems and Control (ICISC), 10, 561–565[10.1109/icisc.2018.8398862]
  41. 41.Liang SHA, Guijin WANG, Xinggang LIN, Kongqiao WANG. (2011). A Framework of Real Time Hand Gesture Vision Based Human-Computer Interaction. IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences, E94-A(3), 979–989[10.1587/transfun.e94.a.979]
  42. 42.Priyanka Muruganandham, Sangeetha Jayaraman, Ramesh Raman. (2025). Continuous speech recognition for Tamil language using a novel semantic verification integrated with the transformer model. EURASIP Journal on Audio Speech and Music Processing, 2025(1)[10.1186/s13636-025-00424-3]
  43. 43.Shikha Gupta, Jafreezal Jaafar, Wan Fatimah Wan Ahmad. (2012). Hybrid algorithm for hand gesture recognition. , 3, 538–542[10.1109/iccisci.2012.6297304]
  44. 44.Pang Haibo, Chengming Liu, Zhe Zhao, Shuyan Zhang. (2015). Gesture Recognition Based on Hexagonal Structure Histograms of Oriented Gradients. International Journal of Signal Processing Image Processing and Pattern Recognition, 8(8), 239–250[10.14257/ijsip.2015.8.8.26]
  45. 45.Itishree Mandal, Samiksha Ray. (2014). Hand gesture based digit recognition.
  46. 46.Ashutosh Kumar Singh .. (2015). USING SYSTEMATIC DATA FLOW WITH HELP OF HAND GESTURE BASED DIGIT RECOGNITION. International Journal of Research in Engineering and Technology, 04(22), 14–22[10.15623/ijret.2015.0422005]
  47. 47.Shamim Akhter, Shah Jafor Sadeek Quaderi, Saleh Ud-Din Ahmed. (2023). Deep Learning with OBH for Real-Time Rotation-Invariant Signs Detection. , 290–296[10.1145/3587828.3587884]
  48. 48.Zong-chang Yang. (2016). Compressive representation based pattern analysis for correlation image. International Journal of Machine Learning and Cybernetics, 9(2), 359–370[10.1007/s13042-016-0511-5]
  49. 49.Ali Asgarov, Ali Parsayan. (2023). 3D-CNNs-Based Touchless Human-Machine Interface. Lecture notes in computer science, 70–80[10.1007/978-3-031-43111-1_7]
  50. 50.Subhamoy Chatterjee. (2014). Hand gesture recognition based on fusion of moments.
  51. 51.Parul Chauhan, Chirag Sharma, Harleen Kaur Khehra. (2013). Gurumukhi Sign Language Translation Using FCM.
← All publications