Conference Paper2008Cited by 8

Expression intensity measurement from facial images by self organizing maps

Md Ashraful Amin, Hong Yan

2008 International Conference on Machine Learning and Cybernetics

IEEE, Vol. 6, pp. 3490–3496

DOI: 10.1109/icmlc.2008.4621008URL

Abstract

Facial expression recognition and inferring emotion from an expression is a challenging task. Many methods have been proposed to recognize facial expressions, but the more challenging task “facial expression intensity classification” remains less focused. Here we propose a system that is able to provide an estimation of facial expression intensity from facial images. At first each image of these sequences are normalized and cropped based on a fixed template. Then, features are captured from Gabor wavelet transformation of these images followed by Principle Component Analysis (PCA). Finally, Self Organizing Maps (SOM) are applied to determine the intensity of emotion from these principle components. In this work we propose a heuristic; MDC (minimum distance criterion) that is able to provide a quantitative measurement about the goodness of a combination of PCs from the intensity measurement point of view. Moreover, we propose a method to represent the results of SOM in the form of membership functions to visualize the qualitative performance.

Keywords

  • minimum distance criterion
  • Gabor wavelet features
  • Principal component analysis
  • facial expression intensity
  • membership functions
  • self-organizing maps
  • facial expression recognition
  • Emotion recognition
  • image normalization

CCDS Authors

Prof. M Ashraful Amin, PhD
M Ashraful Amin, PhD

Prof. M Ashraful Amin, PhD

  • Founder Director, CCDS
  • Director, AI/ML, HCI & DS Wings

Professor

+1 more affiliation

Department of Information Sciences and Technology

George Mason University, USA

Published at

ACL, CHI, UbiComp, ECAI, WACV, PAKDD, IJCNN, ICPR, ICIP, HCII, CLEF, BLP, Scientific Reports, JMIR, IEEE Access, Smart Agricultural Technology, Frontiers in Computational Neuroscience, Diabetes Research and Clinical Practice, PLOS ONE

Research Interests

Machine Learning, Cognitive & Vision Science, Cybernetics, Surveillance & Security, ICT in Education, Health, & Agriculture, Human-Computer Interaction, Internet of Things, Robotics

References

  1. 1.James C. Bezdek. (1981). Pattern Recognition with Fuzzy Objective Function Algorithms. [10.1007/978-1-4757-0450-1]
  2. 2.Matthew Turk, Alex Pentland. (1991). Eigenfaces for Recognition. Journal of Cognitive Neuroscience, 3(1), 71–86[10.1162/jocn.1991.3.1.71]
  3. 3.Teuvo Kohonen. (1995). Self-Organizing Maps. Springer series in information sciences[10.1007/978-3-642-56927-2]
  4. 4.B. Fasel, Juergen Luettin. (2002). Automatic facial expression analysis: a survey. Pattern Recognition, 36(1), 259–275[10.1016/s0031-3203(02)00052-3]
  5. 5.Y.-I. Tian, T. Kanade, J.F. Cohn. (2001). Recognizing action units for facial expression analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(2), 97–115[10.1109/34.908962]
  6. 6.Tai Sing Lee. (1996). Image representation using 2D Gabor wavelets. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(10), 959–971[10.1109/34.541406]
  7. 7.I.A. Essa, A.P. Pentland. (1997). Coding, analysis, interpretation, and recognition of facial expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), 757–763[10.1109/34.598232]
  8. 8.Michael J. Black, Yaser Yacoob. (1997). Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion. International Journal of Computer Vision, 25(1), 23–48[10.1023/a:1007977618277]
  9. 9.M Pantic, L.J.M Rothkrantz. (2000). Expert system for automatic analysis of facial expressions. Image and Vision Computing, 18(11), 881–905[10.1016/s0262-8856(00)00034-2]
  10. 10.Matthew N. Dailey, Garrison W. Cottrell, Curtis Padgett, Ralph Adolphs. (2002). EMPATH: A Neural Network that Categorizes Facial Expressions. Journal of Cognitive Neuroscience, 14(8), 1158–1173[10.1162/089892902760807177]
  11. 11.S. Kimura, M. Yachida. (2002). Facial expression recognition and its degree estimation. , 295–300[10.1109/cvpr.1997.609338]
  12. 12.Paul Ekman, Wallace V. Friesen. (1978). Facial Action Coding System. PsycTESTS Dataset[10.1037/t27734-000]

Cited By

  1. 1.Meng-Ju Han, Chia-How Lin, Kai-Tai Song. (2012). Robotic Emotional Expression Generation Based on Mood Transition and Personality Model. IEEE Transactions on Cybernetics, 43(4), 1290–1303[10.1109/tsmcb.2012.2228851]
  2. 2.Bhakti Sonawane, Priyanka Sharma. (2020). Review of automated emotion-based quantification of facial expression in Parkinson’s patients. The Visual Computer, 37(5), 1151–1167[10.1007/s00371-020-01859-9]
  3. 3.Bhakti Sonawane, Priyanka Sharma. (2020). Deep Learning Based Approach of Emotion Detection and Grading System. Pattern Recognition and Image Analysis, 30(4), 726–740[10.1134/s1054661820040239]
  4. 4.Kenny Hong, Stephan K. Chalup, Robert A.R. King. (2010). A component based approach improves classification of discrete facial expressions over a holistic approach. , 3, 1–8[10.1109/ijcnn.2010.5596836]
  5. 5.Chandrani Saha, Kuntal Ghosh. (2011). Estimation of facial expression intensity from a sequence of binary face images. , 3212, 1–6[10.1109/iciip.2011.6108935]
  6. 6.H. Madokoro, K. Sato. (2012). Facial Expression Spacial Charts for Describing Dynamic Diversity of Facial Expressions. Journal of Multimedia, 7(4)[10.4304/jmm.7.4.314-324]
  7. 7.Xia Mao, Chongping Wang, Yuli Xue. (2012). Expression Intensity Recognition Based on Multilayer Hybrid Classifier. Advances in intelligent systems and computing, 739–748[10.1007/978-3-642-33932-5_69]
  8. 8.Liang Shao. (2013). Robotic Emotional Expression Generation Based on Mood Transition and Personality Model.
← All publications