Monitoring and analyzing the growth and behavior of fish at different stages of growth can help fish farmers estimate changes in fish abundance, length, and behavior, and optimize fish farm management.
Unfortunately, traditional methods for analyzing fish growth and behavior primarily rely on manual observations and measurements. This not only demands time and labor but also affects the welfare and health of the fish.
With advances in computer vision techniques and deep learning (DL), object detection techniques based on DL have been widely applied in aquaculture to simultaneously classify and locate the fish of interest in images.
A team of researchers from Shandong University, Shandong Marine Group Ltd., and Shandong Deep Sea Green Farming Ltd. published a scientific review highlighting the research status of object detection techniques based on deep learning in fish counting, body length measurement, and individual behavior analysis.
Although scientific reviews on the application of deep learning in aquaculture have already been published, the present study focuses on object detection techniques for fish counting, body length measurement, and behavior analysis.
Image databases and pre-processing
Because deep neural networks (DNN) generally rely on large-scale and high-quality datasets for training to achieve satisfactory performance.
Public datasets related to the application of object detection techniques in aquaculture mainly include Fish4Knowledge, LifeCLEF2014, LifeCLEF2015, NOAA, NCFM, and ImageNet.
- Fish4Knowledge: The database is maintained by the European Commission and is jointly developed by a team from the University of Edinburgh, Academia Sinica, and other groups, with the primary goal of assisting in marine ecosystem research. This dataset contains around 700,000 video clips, which can be used for fish identification, detection, and tracking in images and videos.
- LifeCLEF2014: It was built based on the Fish4-Knowledge database by a project team from the University of Catania and Edinburgh, which contains about 1,000 videos with 10 fish species. Approximately 20,000 fish in this database were linked to species in the videos.
- NOAA: The database was developed by the National Oceanic and Atmospheric Administration (NOAA) during studies of sea bass in the Southern California Bight. This dataset contains 929 images and 1,005 annotations.
- NCFM: This database is supported by the Kaggle-organized global competition “The Nature Conservancy Fisheries Monitoring,” which contains about 3,777 fish images.
- ImageNet: It was initiated by Fei-Fei Li’s team at Stanford University and contains over 14 million images. ImageNet is an image dataset organized by the WordNet hierarchy, in which each node is linked to hundreds or thousands of images.
Application of DL in fish farming
The study reviews the application of object detection techniques based on deep learning in aquaculture: fish counting, body length measurement, and individual behavior analysis.
Fish counting
Numerous deep learning approaches have been developed for counting fish in images, which are mainly divided into detection-based and density estimation-based approaches.
Density estimation-based fish counting methods can obtain the total number of fish in the image by integrating the predicted density map. However, the main limitation of these methods is the inability to distinguish fish by density.
Object detection techniques based on deep learning can automatically extract discriminatory features of the object of interest in images and simultaneously classify and locate objects.
Fish body length measurement
Due to the fact that the ideal bounding box length predicted by DL-based object detection algorithms corresponds to the distal ends of the morphological features that represent the length of the fish body, this has become the preferred method for measuring fish body length, especially, Mask R-CNN can perform fish detection and body area segmentation simultaneously using a network structure, which has been widely used in fish body length measurement.
Regarding the analysis of fish population behavior, deep learning-based image classification methods have been widely used to classify the dispersion and aggregation states of fish in images to identify different behaviors of fish populations.
In recent years, deep learning-based object detection algorithms have been increasingly applied to the analysis of individual fish behavior. Image and video processing are the two main means for analyzing individual fish behavior.
Conclusion
“The review shows that generic deep learning-based object detection architectures have played an important role in aquaculture,” the researchers conclude.
However, fish detection in commercial aquaculture still faces several challenges, such as the complex environment and fish characteristics, real-time detection requirements, lack of large-scale database, among others.
The study was funded by the Key Research and Development Program of Shandong Province and the Marine Science and Technology Innovation Project of Qingdao City.
Reference (open access)
Liu, Hanchi, Xin Ma, Yining Yu, Liang Wang, and Lin Hao. 2023. “Application of Deep Learning-Based Object Detection Techniques in Fish Aquaculture: A Review” Journal of Marine Science and Engineering 11, no. 4: 867. https://doi.org/10.3390/jmse11040867
