How AI and Fluorescent Dye Markers Can Track Animal Behavior with High Resolution and Versatility

Animal behavior is a fascinating subject that can reveal a lot about how the brain operates and controls the body. However, tracking and quantifying animal movement is not an easy task. It requires sophisticated methods and tools that can capture the subtle and complex motions of different body parts in various environments.

One of the most popular methods to track animal movement is to use artificial intelligence (AI) models that can automatically recognize and label body parts in video recordings. These models are trained with large amounts of data, where human annotators manually mark each body part hundreds to thousands of times. This process is time-consuming, labor-intensive, and prone to human error and bias. Moreover, these models are limited by the quality and diversity of the training data they receive. If the lighting, orientation, camera angle, or any other factor changes, the model may fail to recognize the body part correctly.

To overcome these limitations, a team of scientists from the Salk Institute has developed a new method called GlowTrack, which uses fluorescent dye markers to train AI models to track animal movement. GlowTrack is non-invasive, time-efficient, and high definition. It can track a single digit on a mouse’s paw or hundreds of landmarks on a human hand.

GlowTrack works by applying fluorescent dye markers to the body parts of interest. These markers are invisible to the naked eye, but they glow under ultraviolet light. The researchers then record videos of the animal or human moving under both normal and ultraviolet light. The videos are then fed into an AI model that learns to associate the glowing markers with the corresponding body parts. Once trained, the model can track the movement of the body parts under normal light without the need for fluorescent markers.

The advantage of GlowTrack is that it can generate a large amount of visually diverse data quickly and easily, without requiring human annotation. This makes the AI model more robust and versatile, as it can handle different situations and environments. GlowTrack can also capture movement at a high resolution, which is important for studying fine motor control and coordination.

The researchers demonstrated the effectiveness of GlowTrack by applying it to various scenarios, such as tracking mouse paw movements during skilled reaching tasks, tracking human hand movements during sign language gestures, and tracking human facial expressions during speech. They showed that GlowTrack outperformed existing methods in terms of accuracy, speed, and versatility.

GlowTrack has many potential applications in biology, robotics, medicine, and beyond. It can help researchers study how the brain controls movement and how movement disorders such as ALS and Parkinson’s disease affect motor function. It can also help engineers design better robots and prosthetics that can mimic natural movements. Furthermore, it can help artists and entertainers create realistic and immersive visual effects and experiences with augmented reality and virtual production.

GlowTrack is a novel and powerful method that unleashes the power of AI to track animal behavior with high resolution and versatility. It opens new possibilities for understanding and enhancing movement in humans and animals alike.

Source: Salk Institute (2023). Unleashing the power of AI to track animal behavior: Scientists create GlowTrack to track human and animal behavior with better resolution and more versatility [press release]. Retrieved from here.