Conference Paper2016Cited by 45

Fish activity tracking and species identification in underwater video

Ekram Hossain, SM Shaiful Alam, Amin Ahsan Ali, M Ashraful Amin

2016 5th International conference on informatics, electronics and vision (ICIEV)

IEEE, pp. 62–66

  • Life Below Water — Relevance: 88%
DOI: 10.1109/iciev.2016.7760189URL

Abstract

In this paper we propose an automatic marine life monitoring system. First task in the monitoring process is to detect underwater moving objects as fishes. Second Task is to identify the species of the detected fish. Third task is to track the detected fish to avoid multiple counting and record their activities. Detection is performed using GMM based background subtraction method, classification is performed using Pyramid Histogram Of visual Words (PHOW) features with SVM classifier and finally identified fishes are tracked using “Kalman Filter”. This experiment is performed using data-set from the CLEF 2015. The proposed system can detect and track fishes with 48.94 percent accuracy in videos, and it can identify fishes in high resolution still image with 91.7 percent accuracy where as in the low quality video fishes are detected with 40.1 percent accuracy.

Keywords

  • underwater video
  • Kalman filter tracking
  • fish detection
  • support vector machine classification
  • fish identification
  • fish tracking

CCDS Authors

Ekram Hossain
Ekram Hossain

Ekram Hossain

  • Collaborator, HCI Wing

  • Research Data Engineer II

    University of Rochester Medical Center

Prof. Amin Ahsan Ali, PhD
Amin Ahsan Ali, PhD

Prof. Amin Ahsan Ali, PhD

  • Director, AI/ML Wing
  • Supervisor, ComBio Wing

  • Professor

    Department of Computer Science and Engineering

    Independent University, Bangladesh (IUB)

Published at

ACL, CHI, UbiComp, ECCV, IJCAI, ECAI, WACV, IPSN, PAKDD, LREC, IJCNN, ICPR, ICIP, BEA, BLP, ICMLA, Scientific Reports, Scientific Data, Pattern Recognition, Information Sciences, Ad Hoc Networks, Drug and Alcohol Dependence, PLOS ONE

Prof. M Ashraful Amin, PhD
M Ashraful Amin, PhD

Prof. M Ashraful Amin, PhD

  • Founder Director, CCDS
  • Director, AI/ML, HCI & DS Wings

Published at

ACL, CHI, UbiComp, ECAI, WACV, PAKDD, IJCNN, ICPR, ICIP, HCII, CLEF, BLP, Scientific Reports, Pattern Recognition, JMIR, IEEE Access, Smart Agricultural Technology, Frontiers in Computational Neuroscience, Diabetes Research and Clinical Practice, PLOS ONE

References

  1. 1.David Lowe. (1999). Object recognition from local scale-invariant features. , 1150–1157 vol.2[10.1109/iccv.1999.790410]
  2. 2.Svetlana Lazebnik, C. Schmid, Jean Ponce. (2006). Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories. , 2, 2169–2178[10.1109/cvpr.2006.68]
  3. 3.Anna Bosch, Andrew Zisserman, Xavier Muñoz. (2007). Image Classification using Random Forests and Ferns. , 1–8[10.1109/iccv.2007.4409066]
  4. 4.Oncel Tuzel, Fatih Porikli, Peter Meer. (2006). Region Covariance: A Fast Descriptor for Detection and Classification. Lecture notes in computer science, 589–600[10.1007/11744047_45]
  5. 5.John G. Allen, Richard Yi Da Xu, Jesse S. Jin. (2004). Object tracking using CamShift algorithm and multiple quantized feature spaces. , 3–7
  6. 6.Mohinder S. Grewal, Angus P. Andrews. (2014). Kalman Filtering. [10.1002/9781118984987]
  7. 7.Concetto Spampinato, Yun-Heh Chen-Burger, Gayathri Devi Nadarajan, Robert B. Fisher. (2008). DETECTING, TRACKING AND COUNTING FISH IN LOW QUALITY UNCONSTRAINED UNDERWATER VIDEOS. , 514–519[10.5220/0001077705140519]
  8. 8.Concetto Spampinato, Daniela Giordano, Roberto Di Salvo, Yun-Heh Chen-Burger, Robert B. Fisher, Gayathri Devi Nadarajan. (2010). Automatic fish classification for underwater species behavior understanding. , 45–50[10.1145/1877868.1877881]
  9. 9.Mauricio Villegas, Henning Müller, Andrew Gilbert, Luca Piras, Josiah Wang, Krystian Mikolajczyk, Alba García Seco de Herrera, Stefano Bromuri, M. Ashraful Amin, Mahmood Kazi Mohammed, Burak Acar, Suzan Üsküdarlı, Neda Barzegar Marvasti, José F. Aldana, María del Mar Roldán‐García. (2015). General Overview of ImageCLEF at the CLEF 2015 Labs. Lecture notes in computer science, 444–461[10.1007/978-3-319-24027-5_45]
  10. 10.Fatih Porikli. (2006). Achieving real-time object detection and tracking under extreme conditions. Journal of Real-Time Image Processing, 1(1), 33–40[10.1007/s11554-006-0011-z]
  11. 11.Phoenix X. Huang, Bastiaan J. Boom, Robert B. Fisher. (2013). Underwater Live Fish Recognition Using a Balance-Guaranteed Optimized Tree. Lecture notes in computer science, 422–433[10.1007/978-3-642-37331-2_32]
  12. 12.Concetto Spampinato, Simone Palazzo, Bastian J. Boom, Jacco van Ossenbruggen, Isaak Kavasidis, Roberto Di Salvo, Fang‐Pang Lin, Daniela Giordano, Lynda Hardman, Robert B. Fisher. (2012). Understanding fish behavior during typhoon events in real-life underwater environments. Multimedia Tools and Applications, 70(1), 199–236[10.1007/s11042-012-1101-5]
  13. 13.(2012). COVARIANCE BASED FISH TRACKING IN REAL-LIFE UNDERWATER ENVIRONMENT. , 409–414[10.5220/0003866604090414]
  14. 14.Bastiaan J. Boom, Phoenix X. Huang, Cigdem Beyan, Concetto Spampinato, Simone Palazzo, Jiyin He, Emma Beauxis-Aussalet, Sun-In Lin, Hsiu‐Mei Chou, Gayathri Devi Nadarajan, Jessica Chen-Burger, Jacco van Ossenbruggen, Daniela Giordano, Lynda Hardman, Fang‐Pang Lin, Bob Fisher. (2012). Long-term underwater camera surveillance for monitoring and analysis of fish populations. Centrum Wiskunde & Informatica (CWI), the national research institute for mathematics and computer science in the Netherlands[link]
  15. 15.M. Ashraful Amin, Mahmood Kazi Mohammed. (2015). Overview of the ImageCLEF 2015 medical clustering task. CLEF (Working Notes)
  16. 16.Katy Blanc, Diane Lingrand, Fŕed́eric Precioso. (2014). Fish Species Recognition from Video using SVM Classifier. , 1–6[10.1145/2661821.2661827]
  17. 17.Pierre-Hugues Joalland, Sébastien Paris, Hervé Glotin. (2014). Efficient instance-based fish species visual identification by global representation. CLEF (Working Notes), 785–789

Cited By

  1. 1.Ling Yang, Yeqi Liu, Huihui Yu, Xiaomin Fang, Lihua Song, Daoliang Li, Yingyi Chen. (2020). Computer Vision Models in Intelligent Aquaculture with Emphasis on Fish Detection and Behavior Analysis: A Review. Archives of Computational Methods in Engineering, 28(4), 2785–2816[10.1007/s11831-020-09486-2]
  2. 2.Daoliang Li, Yinfeng Hao, Yanqing Duan. (2019). Nonintrusive methods for biomass estimation in aquaculture with emphasis on fish: a review. Reviews in Aquaculture, 12(3), 1390–1411[10.1111/raq.12388]
  3. 3.Alzayat Saleh, Marcus Sheaves, Mostafa Rahimi Azghadi. (2022). Computer vision and deep learning for fish classification in underwater habitats: A survey. Fish and Fisheries, 23(4), 977–999[10.1111/faf.12666]
  4. 4.Alzayat Saleh, Marcus Sheaves, Dean Jerry, Mostafa Rahimi Azghadi. (2023). Applications of deep learning in fish habitat monitoring: A tutorial and survey. Expert Systems with Applications, 238, 121841[10.1016/j.eswa.2023.121841]
  5. 5.Mutasem K. Alsmadi, Ibrahim Almarashdeh. (2020). A survey on fish classification techniques. Journal of King Saud University - Computer and Information Sciences, 34(5), 1625–1638[10.1016/j.jksuci.2020.07.005]
  6. 6.Alexa Sugpatan Abangan, Dorothée Kopp, Robin Faillettaz. (2023). Artificial intelligence for fish behavior recognition may unlock fishing gear selectivity. Frontiers in Marine Science, 10, 1010761[10.3389/fmars.2023.1010761]
  7. 7.Yupeng Mei, Boyang Sun, Daoliang Li, Huihui Yu, Hanxiang Qin, Huihui Liu, Ni Yan, Yingyi Chen. (2022). Recent advances of target tracking applications in aquaculture with emphasis on fish. Computers and Electronics in Agriculture, 201, 107335[10.1016/j.compag.2022.107335]
  8. 8.Vanesa Lopez-Vazquez, Jose Manuel Lopez-Guede, Damianos Chatzievangelou, Jacopo Aguzzi. (2023). Deep learning based deep-sea automatic image enhancement and animal species classification. Journal Of Big Data, 10(1)[10.1186/s40537-023-00711-w]
  9. 9.Adamu Ali-Gombe, Eyad Elyan, Chrisina Jayne. (2017). Fish Classification in Context of Noisy Images. Communications in computer and information science, 216–226[10.1007/978-3-319-65172-9_19]
  10. 10.Ramil Lumauag, Marevic Nava. (2018). Fish Tracking and Counting using Image Processing. IEEE International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management (HNICEM), 1–4[10.1109/hnicem.2018.8666369]
  11. 11.Cyril Jay L. Chan, Ethan James A. Reyes, Noel B. Linsangan, Roben A. Juanatas. (2022). Real-time Detection of Aquarium Fish Species Using YOLOv4-tiny on Raspberry Pi 4. IEEE International Conference on Artificial Intelligence in Engineering and Technology (IICAIET), 1–6[10.1109/iicaiet55139.2022.9936790]
  12. 12.Krithika M. Pai, K. B. Ajitha Shenoy, M. M. Manohara Pai. (2022). A Computer Vision Based Behavioral Study and Fish Counting in a Controlled Environment. IEEE Access, 10, 87778–87786[10.1109/access.2022.3197887]
  13. 13.Sudhakara Malla, M. Janaki Meena, Obulakonda Reddy. R, V. Mahalakshmi, Awatef Balobaid. (2022). A Study on Fish Classification Techniques using Convolutional Neural Networks on Highly Challenged Underwater Images. International Journal on Recent and Innovation Trends in Computing and Communication, 10(4), 01–09[10.17762/ijritcc.v10i4.5524]
  14. 14.Jianhao Yin, Junfeng Wu, Chunqi Gao, Zhongai Jiang. (2022). LIFRNet: A Novel Lightweight Individual Fish Recognition Method Based on Deformable Convolution and Edge Feature Learning. Agriculture, 12(12), 1972[10.3390/agriculture12121972]
  15. 15.Universitas Dian Nuswantoro, Ricardus Pramunendar, Dwi Prabowo, Universitas Dian Nuswantoro, Dewi Pergiwati, Universitas Dian Nuswantoro, Yuslena Sari, Universitas Lambung Mangkurat, Pulung Andono, Universitas Dian Nuswantoro, Moch Soeleman, Universitas Dian Nuswantoro. (2020). New Workflow for Marine Fish Classification Based on Combination Features and CLAHE Enhancement Technique. International journal of intelligent engineering and systems, 13(4), 293–304[10.22266/ijies2020.0831.26]
  16. 16.S. Shreesha, M. M. Manohara Pai, Ujjwal Verma, Radhika M. Pai. (2023). Fish Tracking and Continual Behavioral Pattern Clustering Using Novel Sillago Sihama Vid (SSVid). IEEE Access, 11, 29400–29416[10.1109/access.2023.3247143]
  17. 17.K Banno, H Kaland, AM Crescitelli, SA Tuene, GH Aas, LC Gansel. (2022). A novel approach for wild fish monitoring at aquaculture sites: wild fish presence analysis using computer vision. Aquaculture Environment Interactions, 14, 97–112[10.3354/aei00432]
  18. 18.Junhui Wang, Meng Zhao, Li Zou, Yi Hu, Xuezhen Cheng, Xiaofeng Liu. (2019). Fish Tracking Based on Improved TLD Algorithm in Real-World Underwater Environment. Marine Technology Society Journal, 53(3), 80–89[10.4031/mtsj.53.3.8]
  19. 19.Universitas Gadjah Mada, Ricardus Pramunendar, Sunu Wibirama, Universitas Gadjah Mada, Paulus Santosa, Universitas Gadjah Mada, Pulung Andono, Universitas Dian Nuswantoro, Moch Soeleman, Universitas Dian Nuswantoro. (2019). A Robust Image Enhancement Techniques for Underwater Fish Classification in Marine Environment. International journal of intelligent engineering and systems, 12(5), 116–129[10.22266/ijies2019.1031.12]
  20. 20.Alberto Maximiliano Crescitelli, Lars Christian Gansel, Houxiang Zhang. (2021). NorFisk: fish image dataset from Norwegian fish farms for species recognition using deep neural networks. Modeling Identification and Control A Norwegian Research Bulletin, 42(1), 1–16[10.4173/mic.2021.1.1]
  21. 21.Girish S. Gaude, Samarth Borkar. (2019). Fish Detection And Tracking For Turbid Underwater Video. International Conference on Intelligent Computing and Control Systems (ICICCS), 326–331[10.1109/iccs45141.2019.9065425]
  22. 22.P Varalakshmi, J Julanta Leela Rachel. (2019). Recognition Of Fish Categories Using Deep Learning Technique. International Conference on Computing and Communications Technologies (ICCCT), 168–172[10.1109/iccct2.2019.8824916]
  23. 23.Zhou Shen, Chuong Nguyen. (2020). Temporal 3D RetinaNet for fish detection. International Conference on Digital Image Computing: Techniques and Applications (DICTA), 1–5[10.1109/dicta51227.2020.9363372]
  24. 24.Airi Kitasato, Tomo Miyazaki, Yoshihiro Sugaya, Shinichiro Omachi. (2018). Automatic Discrimination between Scomber japonicus and Scomber australasicus by Geometric and Texture Features. Fishes, 3(3), 26[10.3390/fishes3030026]
  25. 25.Devi Fitrianah, Kristien Margi Suryaningrum, Noviyanti Tri Maretta Sagala, Vina Ayumi, Siew Mooi Lim. (2022). Fine-Tuned MobileNetV2 and VGG16 Algorithm for Fish Image Classification. International Conference on Informatics, Multimedia, Cyber and Information System (ICIMCIS), 384–389[10.1109/icimcis56303.2022.10017901]
  26. 26.Li Zou, Meng Zhao, Fangfang Cao, Shiliang Zan, Xuezhen Cheng, Xiaofeng Liu. (2021). Fish Tracking Based on Feature Fusion and Scale Adaptation in a Real-World Underwater Environment. Marine Technology Society Journal, 55(2), 45–53[10.4031/mtsj.55.2.12]
  27. 27.Thitinun Pengying, Marius Pedersen, Jon Yngve Hardeberg, Jon Museth. (2019). Underwater Fish Classification of Trout and Grayling. International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), 2019, 268–273[10.1109/sitis.2019.00052]
  28. 28.Gehad Ismail Sayed, Aboul Ella Hassanien, Ahmed Gamal, Hassan Aboul Ella. (2018). An Automated Fish Species Identification System Based on Crow Search Algorithm. Advances in intelligent systems and computing, 112–123[10.1007/978-3-319-74690-6_12]
  29. 29.Jonathan Gregory, Scott M. Miehls, Jesse L. Eickholt, Daniel P. Zielinski. (2025). A Real-Time Fish Detection System for Partially Dewatered Fish to Support Selective Fish Passage. Sensors, 25(4), 1022[10.3390/s25041022]
  30. 30.J Julanta Leela Rachel, P Varalakshmi. (2019). Classification of Breeding Fish using Deep Learning from the Captured Video. International Conference on Advanced Computing, 48–55[10.1109/icoac48765.2019.246815]
  31. 31.Leandro Magno C. Silva, Franklin Cesar Flores, Rosilene Luciana Delariva, Gustavo Z. Felipe, Yandre M. G. Da Costa. (2024). Classification of Highly Similar Fish Species Using Machine Learning. International Conference on Systems, Signals and Image Processing, 1–8[10.1109/iwssip62407.2024.10634024]
  32. 32.Jorge E. Pezoa, Diego A. Ramírez, Cristofher A. Godoy, María F. Saavedra, Silvia E. Restrepo, Pablo A. Coelho-Caro, Christopher A. Flores, Francisco G. Pérez, Sergio N. Torres, Mauricio A. Urbina. (2023). A Spatial-Spectral Classification Method Based on Deep Learning for Controlling Pelagic Fish Landings in Chile. Sensors, 23(21), 8909[10.3390/s23218909]
  33. 33.Alberto Maximiliano Crescitelli, Lars Christian Gansel, Houxiang Zhang. (2020). Semi-automatic Approach to Create Fish Image Datasets for Aquaculture Applications. IEEE Conference on Industrial Electronics and Applications (ICIEA), 1450–1455[10.1109/iciea48937.2020.9248107]
  34. 34.Diego A. Ramirez, Jorge E. E. Pezoa. (2018). Spectral vision system for discriminating small pelagic species caught by small-scale fishing. Infrared Sensors, Devices, and Applications, 33[10.1117/12.2325457]
  35. 35.Maria Gemel Palconit, Ronnie S. Concepcion, Jonnel D. Alejandrino, Edwin Sybingco, Ryan Rhay P. Vicerra, Argel A. Bandala, Elmer P. Dadios. (2021). Fish Stereo Matching using Modified k-Dimensional Tree Nearest Neighbor Search. IEEE International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management (HNICEM), 25, 1–6[10.1109/hnicem54116.2021.9731879]
  36. 36.Chad Whitney, Mahid Ahmed, Lei Huang, Shuiling He, Zhaoxian Zhou, Chaoyang Zhang. (2026). A Comparative Analysis of ResNet Models on Fish Classification. Mathematics, 14(6), 1055[10.3390/math14061055]
  37. 37.Ashok P, Bhoomika V, Janani A, Lavanya S. (2025). A CNN Based Underwater Aquatic Exploring System for Real Time Identification of Fish Species. , 1–6[10.1109/iccct63501.2025.11019036]
  38. 38.Shreesha S, Manohara Pai M. M, Ujjwal Verma, Radhika M. Pai, Girisha S. (2021). Behavioural Pattern Analysis of Fishes for Smart Aquaculture: An Object Centric Approach. IEEE Region 10 Conference (TENCON), 917–922[10.1109/tencon54134.2021.9707293]
  39. 39.Daxiong Ji, Jialong Zhou, Minghui Xu, Zhangying Ye, Songming Zhu, Shuo Liu. (2021). Automatic Counting Algorithm of Fry Based on Machine Vision System. WRC Symposium on Advanced Robotics and Automation (WRC SARA), 36, 104–109[10.1109/wrcsara53879.2021.9612698]
  40. 40.Susmita Panda, Pradipta Kumar Nanda. (2020). MRF Model-Based Estimation of Camera Parameters and Detection of Underwater Moving Objects. International Journal of Cognitive Informatics and Natural Intelligence, 14(4), 1–29[10.4018/ijcini.2020100101]
  41. 41.Santosh Kumar, Sanjay Kumar Singh, Rishav Singh, Amit Kumar Singh. (2017). Analytical Study of Animal Biometrics: A Technical Survey. Springer eBooks, 21–78[10.1007/978-981-10-7956-6_2]
  42. 42.Pooja Prasenan, M H Supriya. (2023). An Automated Fish Species Classification Using Modified Neural Network and Firefly Algorithm and its Empirical Analysis. International Symposium on Ocean Technology (SYMPOL), 1–9[10.1109/sympol59195.2023.10455081]
  43. 43.Divya Velayudhan, Adarsh Ghimire, Jorge Dias, Naoufel Werghi, Sajid Javed. (2023). Underwater Fish Tracking-by-Detection: An Adaptive Tracking Approach. Lecture notes in computer science, 504–515[10.1007/978-3-031-37731-0_37]
  44. 44.Israa Mohammed Hassoon, Shaymaa Akram Hantoosh. (2023). EDIBLE FISH IDENTIFICATION BASED ON MACHINE LEARNING. Iraqi Journal for Computers and Informatics, 49(2), 62–72[10.25195/ijci.v49i2.455]
  45. 45.Ramon G. Garcia, Jonen Lery L. Ibanez, Flordeliza L. Valiente, Jeffrey T. Macadangdang, John Edward G. Macario, Charmaine C. Paglinawan, Paola Jamie V. Uy. (2019). Identification of Various Osteichthyes Class of Fishes Through Gaussian Mixture Model, Kalman Filter, Pyramid Histogram of Visual Words and Support Vector Machine. International Conference on Biomedical Engineering and Technology (ICBET), 210–216[10.1145/3326172.3326223]
← All publications