Our team explore representation capacities of deep learning in the context of neuroscience and behavior.
Behavior is the primary outcome of the central nervous system. To understand how brains work in health and disease, we need to measure behavior precisely — and describe it richly in natural settings. Yet accurately quantifying it has been challenging. Animal behavior is routinely quantified with a few 1D measurements, such as trial count or position, and limited to comparing pre- and post-stimulus responses. This lack of methods limits the behaviors we have access to, limiting the neuronal processes we can understand.
Today, state-of-the-art cameras capture freely moving animals in 2D and 3D, while miniature microscope and neuroprobes simulataneously record hundreds of neurons. These advancements create an immense opportunity for robust and general analysis methods that can accurately quantify both unconstrained behavior and associated high-dimensional neuronal activity.
Two major challenges remain. First, behavior unfolds across time scales — from subtle shifts that never show up as discrete action classes, to long-timescale planning and strategy. Second, we still lack a systematic way to link the determinants of behavior to the behavior we observe.
The revolution in language processing has introduced techniques like transformers and self-supervised learning (SSL), but these methods are rarely applied to studying animal behavior. We aim to incorporate these innovations into our research on time series data, additionally drawing insights from fields like computer vision and image analysis.
⇨ Using powerful training paradigms and heterogeneous time series datasets, we build general models usable across labs and behavior paradigms.
⇨ Integrating interpretability features such as confidence estimation and illustrative examples, we aim at enhancing the adoption of these techniques in both animal studies and human clinical practice.
Check out our recent Nature methods paper!
Rose, F. et al. (2025). Deep Imputation for Skeleton Data (DISK) in Behavioral Science.
