RT Journal Article
SR Electronic
T1 PoseR - A deep learning toolbox for decoding animal behavior
JF bioRxiv
FD Cold Spring Harbor Laboratory
SP 2023.04.07.535991
DO 10.1101/2023.04.07.535991
A1 Mullen, Pierce N
A1 Bowlby, Beatrice
A1 Armstrong, Holly C
A1 Zwart, Maarten F
YR 2023
UL http://biorxiv.org/content/early/2023/04/08/2023.04.07.535991.abstract
AB The actions of animals provide a window into how their minds work. Recent advances in deep learning are providing powerful approaches to recognize patterns of animal movement from video recordings, including markerless pose estimation models. However, tools to efficiently parse coordinates of animal position and pose into meaningful semantic behavioral labels are lacking. Here, we present PoseRecognition (PoseR), a behavioral decoder leveraging state- of-the-art action recognition models using spatio-temporal graph convolutional networks. We show that it can be used to decode animal behavior quickly and accurately from pose estimations, using zebrafish larvae and mice as model organisms. PoseR can be accessed using a Napari plugin, which facilitates efficient behavioral extraction, annotation, model training and deployment. We have simplified the workflow of behavioral analysis after pose estimation, transforming coordinates of animal position and pose into meaningful semantic behavioral labels, using methods designed for fast and accurate behavioral extraction, annotation, model training and deployment. Furthermore, we contribute a novel method for unsupervised clustering of behaviors and provide open-source access to our zebrafish datasets and models. The design of our tool ensures scalability and versatility for use across multiple species and contexts, improving the efficiency of behavioral analysis across fields.Competing Interest StatementThe authors have declared no competing interest.