Our research group has been developing a unique trajectory in the field of automated behavior analysis of rodents, combining academic foundations with collaborative projects across institutions and industry.
2014–2015 (NAIST, Komai Lab)
The foundation was laid by Jin Watanabe’s master’s thesis and conference presentations, where we introduced high-precision tracking methods and applied non-parametric Bayesian clustering (Infinite Gaussian Mixture Model, IGMM). This approach allowed us to detect undefined behaviors without fixing the number of clusters in advance. We further extended IGMM to a semi-supervised framework (Ss-IGMM), enabling the effective use of limited labeled data alongside large amounts of unlabeled data.
2018 (Kyoto University, in collaboration with Komai Lab)
Building on this foundation, Atsushi Hashimoto (then at Kyoto University) and Yushi Seigenji presented the results of our joint research at the IEICE General Conference. In this work, we explored self-supervised learning approaches for behavior analysis. This marked a significant step forward: moving beyond semi-supervised methods to frameworks that can learn robust representations directly from unlabeled data, anticipating later international trends in self-supervised representation learning.
Present (Komai, Morimoto, Hashimoto Collaboration)
Our current joint research integrates diverse features such as CHLAC, optical flow, locomotion activity, head orientation, proximity measures, and mutual information. These features are combined with hierarchical and grammar-based models (HMM, HDP-HMM, Transformer architectures) to estimate higher-order “behavioral grammar.” We are extending this framework to capture multi-animal social interactions, aiming to uncover the relational structures and causal dynamics underlying collective behavior.
This trajectory illustrates a clear progression:
Undefined behavior detection (IGMM) → Semi-supervised learning (Ss-IGMM) → Self-supervised learning → Grammar-based modeling of social interactions.
Our work represents a domestic lineage of innovation that connects to and anticipates international developments such as MoSeq, B-SOID, MotionMapper, and modern self-supervised learning frameworks. By bridging academic research and industrial collaboration, we are contributing a distinctive perspective to the global field of behavioral neuroscience and computational ethology.
Origins of Mathematical Understanding of Behavior
My exploration into the mathematical understanding of animal behavior began in 2009, during my participation in the JST PRESTO program “Understanding and Application of Brain Information.”
The foundation was laid earlier, between 2003 and August 2005, when I studied at the Max Planck Institute of Medical Research in Heidelberg, Germany. There, using the Two-Photon Laser Microscope targeted patch clamp method (TPTP), I performed in vivo measurements from genetically modified neurons. This work revealed the critical importance of back-propagating action potentials in the process of neural network formation.
Building on these findings, I sought to collect information from single-cell recordings and use it to design Brain-Machine Interfaces (BMI). However, I realized that to identify behaviors induced by single-cell activity, precise measurement of behavior itself was indispensable. This necessity became the starting point for my subsequent research into the mathematical modeling and analysis of behavior.
An experimental learning ecosystem where students design their own university
In Keihanna, I have been developing a new model of learning in which students themselves design the spaces, structures, and cultures of their education. Drawing on perspectives from behavioral science, institutional design, and educational culture, I support students in creating learning environments that emerge from their own curiosity and initiative.
Student-led design of learning spaces
Collaborative projects with local communities and industry partners
Exploration of educational systems that prioritize autonomy and intrinsic motivation
The Keihanna initiative is an ongoing experiment in rethinking what a university can be. It aims to build a future-oriented educational culture where students are not passive recipients but active creators of their learning.
Urban co-creation where students and communities shape the future together
In Shinjuku North, I lead urban co-creation projects that bring together students, residents, local businesses, and government. Through data analysis, institutional design, and dialogue-based fieldwork, we work collaboratively to identify local challenges and prototype solutions.
Data-driven revitalization of local shopping districts
Community workshops for discovering shared challenges and aspirations
New models of community design that connect young people with urban life
These projects explore how learning and society intersect in the heart of the city. They offer students a chance to grow through real-world engagement while contributing to the future of the community.
Across both Keihanna and Shinjuku North, my work is guided by a single theme:
connecting systems, culture, and technology to create environments where people can act autonomously.
By moving back and forth between research, education, and social implementation, I aim to design new forms of learning and new possibilities for society.
In our current joint research, we not only employ Mutual Information to quantify the strength of relationships between multiple animals, but also Transfer Entropy to capture the directionality of causal influence. This allows us to move beyond correlation and toward a grammar of social interaction that reflects who drives whom in collective behavior. By integrating TE into grammar-based models, we aim to uncover the causal dynamics of multi-animal interactions and extend these insights to human social systems.
Application to Human Behavior
Translating methods originally developed for mice to human contexts, enabling quantitative analysis of everyday actions and social interactions.
Integration with Neuroscience
Linking behavioral grammar with neural activity patterns, aiming to uncover how cognitive processes and brain dynamics shape observable behavior.
Understanding “Society” as Integrated Human Behavior
Expanding from individual actions to collective dynamics, we seek to model society as a complex system of interacting behavioral grammars.
Visualization of Invisible Cognitive Activities
Making latent processes such as cognitive biases visible through computational modeling, thereby providing new tools to study decision-making and perception.
Our long-term vision is to establish a framework that connects behavioral science, neuroscience, and social systems research. By integrating advanced machine learning techniques with experimental and observational data, we aim to contribute to a deeper understanding of human cognition and society, and to provide methodologies that can be applied across disciplines—from psychology and neuroscience to education, healthcare, and social innovation.