AlphaTracker: a multi-animal tracking and behavioral analysis tool

Zexin Chen; Ruihan Zhang; Hao-Shu Fang; Yu E Zhang; Aneesh Bal; Haowen Zhou; Rachel R Rock; Nancy Padilla-Coreano; Laurel R Keyes; Haoyi Zhu; Yong-Lu Li; Takaki Komiyama; Kay M Tye; Cewu Lu

doi:10.3389/fnbeh.2023.1111908

AlphaTracker: a multi-animal tracking and behavioral analysis tool

Front Behav Neurosci. 2023 May 30:17:1111908. doi: 10.3389/fnbeh.2023.1111908. eCollection 2023.

Authors

Zexin Chen^#¹, Ruihan Zhang^#^{2

3}, Hao-Shu Fang^#¹, Yu E Zhang^{4

5}, Aneesh Bal^{6

7}, Haowen Zhou², Rachel R Rock⁷, Nancy Padilla-Coreano^{7

8}, Laurel R Keyes^{7

9}, Haoyi Zhu¹, Yong-Lu Li¹, Takaki Komiyama⁵, Kay M Tye^{7

9}, Cewu Lu^{1

10}

Affiliations

¹ Department of Computer Science, Shanghai Jiao Tong University, Shanghai, China.
² Zhiyuan College, Shanghai Jiao Tong University, Shanghai, China.
³ Media Arts and Sciences, Massachusetts Institute of Technology, Cambridge, MA, United States.
⁴ Department of Neurobiology, Center for Neural Circuits and Behavior, University of California, San Diego, La Jolla, CA, United States.
⁵ Department of Neurosciences, University of California, San Diego, La Jolla, CA, United States.
⁶ Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD, United States.
⁷ Salk Institute for Biological Studies, La Jolla, CA, United States.
⁸ Department of Neuroscience, University of Florida, Gainesville, FL, United States.
⁹ Howard Hughes Medical Institute, The Salk Institute, La Jolla, CA, United States.
¹⁰ Shanghai Artificial Intelligence Laboratory, Shanghai, China.

^# Contributed equally.

Abstract

Computer vision has emerged as a powerful tool to elevate behavioral research. This protocol describes a computer vision machine learning pipeline called AlphaTracker, which has minimal hardware requirements and produces reliable tracking of multiple unmarked animals, as well as behavioral clustering. AlphaTracker pairs a top-down pose-estimation software combined with unsupervised clustering to facilitate behavioral motif discovery that will accelerate behavioral research. All steps of the protocol are provided as open-source software with graphic user interfaces or implementable with command-line prompts. Users with a graphical processing unit (GPU) can model and analyze animal behaviors of interest in less than a day. AlphaTracker greatly facilitates the analysis of the mechanism of individual/social behavior and group dynamics.

Keywords: animal behavior; animal tracking; behavioral clustering; computer vision; neuroscience.

Grants and funding

KT is an HHMI Investigator and the Wylie Vale Professor at the Salk Institute for Biological Studies and this work was supported by finance from the JPB Foundation, the Dolby Family Fund, R01-MH115920 (NIMH), and Pioneer Award DP1-AT009925 (NCCIH). NP-C was supported by the Simons Center for the Social Brain, the Ford Foundation, L'Oreal For Women in Science, the Burroughs Wellcome Fund, and K99 MH124435-01. CL was supported by the AI Institute, SJTU, the Shanghai Qi Zhi Institute, and the Meta Technology Group. This work was also supported by the National Key R&D Program of China (No. 2021ZD0110704), Shanghai Municipal Science and Technology Major Project (2021SHZDZX0102), Shanghai Qi Zhi Institute, and Shanghai Science and Technology Commission (21511101200).