Pixying Behavior: A Versatile Real-Time and Post Hoc Automated Optical Tracking Method for Freely Moving and Head Fixed Animals

Abstract Here, we describe an automated optical method for tracking animal behavior in both head-fixed and freely moving animals, in real time and offline. It takes advantage of an off-the-shelf camera system, the Pixy camera, designed as a fast vision sensor for robotics that uses a color-based filtering algorithm at 50 Hz to track objects. Using customized software, we demonstrate the versatility of our approach by first tracking the rostro-caudal motion of individual adjacent row (D1, D2) or arc whiskers (β, γ), or a single whisker and points on the whisker pad, in head-fixed mice performing a tactile task. Next, we acquired high-speed video and Pixy data simultaneously and applied the pixy-based real-time tracking to high-speed video data. With this approach, we expand the temporal resolution of the Pixy camera and track motion ( post hoc ) at the limit of high-speed video frame rates. Finally, we show that this system is flexible: it can be used to track individual whisker or limb position without any sophisticated object tracking algorithm, it can be used in many lighting conditions including infrared (IR); it can be used to track head rotation and location of multiple animals simultaneously. Our system makes behavioral monitoring possible in virtually any biological setting.