FIN-PRINT a fully-automated multi-stage deep-learning-based framework for the individual recognition of killer whales

Bergler C, Gebhard A, Towers JR, Butyrev L, Sutton GJ, Shaw TJH, Maier A, Nöth E (2021)


Publication Language: English

Publication Type: Journal article

Publication year: 2021

Journal

Book Volume: 11

Pages Range: 1-16

Article Number: 23480

Journal Issue: 1

DOI: 10.1038/s41598-021-02506-6

Abstract

Biometric identification techniques such as photo-identification require an array of unique natural markings to identify individuals. From 1975 to present, Bigg’s killer whales have been photo-identified along the west coast of North America, resulting in one of the largest and longest-running cetacean photo-identification datasets. However, data maintenance and analysis are extremely time and resource consuming. This study transfers the procedure of killer whale image identification into a fully automated, multi-stage, deep learning framework, entitled FIN-PRINT. It is composed of multiple sequentially ordered sub-components. FIN-PRINT is trained and evaluated on a dataset collected over an 8-year period (2011–2018) in the coastal waters off western North America, including 121,000 human-annotated identification images of Bigg’s killer whales. At first, object detection is performed to identify unique killer whale markings, resulting in 94.4% recall, 94.1% precision, and 93.4% mean-average-precision (mAP). Second, all previously identified natural killer whale markings are extracted. The third step introduces a data enhancement mechanism by filtering between valid and invalid markings from previous processing levels, achieving 92.8% recall, 97.5%, precision, and 95.2% accuracy. The fourth and final step involves multi-class individual recognition. When evaluated on the network test set, it achieved an accuracy of 92.5% with 97.2% top-3 unweighted accuracy (TUA) for the 100 most commonly photo-identified killer whales. Additionally, the method achieved an accuracy of 84.5% and a TUA of 92.9% when applied to the entire 2018 image collection of the 100 most common killer whales. The source code of FIN-PRINT can be adapted to other species and will be publicly available.

Authors with CRIS profile

How to cite

APA:

Bergler, C., Gebhard, A., Towers, J.R., Butyrev, L., Sutton, G.J., Shaw, T.J.H.,... Nöth, E. (2021). FIN-PRINT a fully-automated multi-stage deep-learning-based framework for the individual recognition of killer whales. Scientific Reports, 11(1), 1-16. https://dx.doi.org/10.1038/s41598-021-02506-6

MLA:

Bergler, Christian, et al. "FIN-PRINT a fully-automated multi-stage deep-learning-based framework for the individual recognition of killer whales." Scientific Reports 11.1 (2021): 1-16.

BibTeX: Download