*My office is in Room G18 in SportExR Building (Y14 in the campus map). Please drop me an email to arrange a meeting (s.yeo bham ac UK)
Motion capture is a technique used to accurately measure human movement, and it finds applications across various fields such as biomechanics, physiotherapy, computer animation, and ergonomics. In biomechanics, motion capture is often combined with physiological signals, such as electromyography (EMG) and muscle ultrasound, to provide a multimodal assessment of how human movement is generated and controlled. In this project, you will utilise our newly acquired motion capture system (VICON), along with EMG and ultrasound sonography, to measure muscle activities in conjunction with body kinematics. The results will be compared with the predictions from OpenSIM, a state-of-the-art musculoskeletal simulator in biomechanics.
In this project, you will be involved in the following sub-projects based on your background:
CS students: You will be involved in the motion capture and musculoskeletal model development using OpenSIM or other simulators.
SportExR students: You will run an experiment examining human motor behaviour, e.g. locomotion or bicyling.
Related papers:
De Groote, F., & Falisse, A. (2021). Perspective on musculoskeletal modelling and predictive simulations of human movement to assess the neuromechanics of gait. Proceedings of the Royal Society B, 288(1946), 20202432.
Yeo, S. H., Verheul, J., Herzog, W., & Sueda, S. (2023). Numerical instability of Hill-type muscle models. Journal of the Royal Society Interface, 20(199), 20220430.
When intercepting a moving object, such as catching a thrown ball, your eyes and hand need to collaborate very closely to gather information on the thrown ball, predict its future position, and plan and execute hand movement to catch it. Previous studies have revealed that humans incorporate two different eye movement strategies: saccade (fast, reactive, and intermittent) and smooth pursuit (slow, predictive, and continuous). The hand movement is controlled in synchrony with these eye movement patterns. This special pattern of eye-hand coordination provides a new way to examine the brain's strategy for sensorimotor control and can be applied to many related areas such as stroke rehabilitation and humanoid robot control.
In this project, you will be involved in the following sub-projects based on your background:
CS students: You will be involved in the development of a software system that combines motion tracking devices, a VR headset, and eye tracking.
SportExR students: You will run an experiment examining eye-hand coordination behaviour during interception and perform data analysis.
Related papers:
Yeo, S. H., Lesmana, M., Neog, D. R., & Pai, D. K. (2012). Eyecatch: Simulating visuomotor coordination for object interception. ACM Transactions on Graphics (TOG), 31(4), 1-10. https://dl.acm.org/doi/abs/10.1145/2185520.2185538
Fooken, J., Yeo, S. H., Pai, D. K., & Spering, M. (2016). Eye movement accuracy determines natural interception strategies. Journal of Vision, 16(14), 1-1. https://jov.arvojournals.org/article.aspx?articleid=2579407
Learning sensorimotor mapping is a critical component of human behaviour and has significant implications for skill acquisition and human-computer interaction. Imagine you are playing a first-person shooter game – you will need to learn how your motor commands (e.g. button press, and joystick/mouse gesture) are related to the actions of your avatar. In addition, when you perceive a sensory event (e.g. an enemy attacking your avatar), your brain needs to generate proper motor commands to control your avatar in a desirable way (e.g. move away while aiming your gun at the enemy). All these skills require you to learn a new mapping that connects your sensation and your movement in a novel way. Some mappings are intuitive and easy to learn, while others are incredibly hard to master (imagine playing a game with the monitor rotated by 90 degrees!). Despite this obvious difference, we still have a very limited understanding of the neural representations underlying this mapping procedure.
In this project, you will be involved in the following sub-projects based on your background:
CS students: You will be involved in the development of a VR-based software environment to study sensorimotor mapping, combining haptic devices and a VR headset.
SportExR students: You will run an experiment examining mechanisms involved in sensorimotor mapping and perform a data analysis.
When performing spatial navigation—whether walking, running, or driving a vehicle—our eyes constantly monitor the dynamic visual environment to extract critical information required to plan our movements. To deal with the many different objects approaching us, the brain must dynamically coordinate our gaze depending on the distances and velocities of these objects. For example, assume there are two cars approaching you from the front: one car is closer but moving slowly, and the other is further away but moving fast. Which one will you look at first and more often, and how will this relate to your movement to avoid a collision? The primary goal of this study is to obtain a statistical relationship between the distance/velocity of objects and eye movement. We will combine state-of-the-art deep-learning image localisation and classification methods (Mask R-CNN) with binocular eye-tracking glasses (Pupil Lab) to acquire extensive data on gaze behaviour combined with environmental information, i.e., where and what object the subject is gazing upon at each moment during free navigation. Once we establish this relationship, the secondary goal is to implement it to develop a practical control logic for collision avoidance algorithms in automated vehicles, such as mobile robots, self-driving cars, and powered wheelchairs.
In this project, you will be involved either 1) in developing an integrated system that combines eye tracking, deep-learning-based computational vision, and other novel motion tracking and image processing techniques, or 2) in data collection and analysis.
Related paper:
Deane, O., Toth, E., & Yeo, S. H. (2022). Deep-SAGA: a deep-learning-based system for automatic gaze annotation from eye-tracking data. Behavior Research Methods, 1-20. https://link.springer.com/article/10.3758/s13428-022-01833-4
We ride bicycles every day, yet it is surprising that scientists still do not fully understand how we ride bicycles in complex environments, such as urban areas, with stability and flexibility. By combining cutting-edge technologies in control engineering, robotics, computer vision, and biomechanics, this project aims to push the boundaries of our understanding of bicycle riding and how the human brain controls bicycles in complex environments.
The project involves building a tracking system that enables accurate tracking of the bicycle's motion and the human rider's eye and body movements, along with the surrounding outdoor environment. We combine various techniques including SLAM, mobile eye-tracking, IMU-based motion tracking, and deep-learning-based object detection. Our goal is to understand how the cyclist's manoeuvring behaviour changes in various scenarios.
As part of this project, you will have the opportunity to participate in various sub-projects based on your background:
CS students: You will be involved in the development of a bicycle system that integrates multiple sensors, including video (with object detection/tracking), mobile eye-tracker, LIDAR sensor, and IMU-based motion trackers. This system will provide a comprehensive dataset to help us understand the complex processes involved in bicycle riding. Those with prior experience in deep-learning-based computer vision, sensor fusion, and hardware such as motion capture/tracking or embedded sensor systems are preferred.
SportExR students: You will run experiments with human riders in different scenarios. These experiments will involve investigating various manoeuvring behaviours of riders and collecting data to better understand the complex processes involved in bicycle riding.
Using deep neural networks to understand biomedical images, such as MRI, ultrasound, CT, and X-ray images, sheds new insight on biomedical image processing, which previously depended mainly on qualitative/unstructured analyses based on human (doctor's) expertise. In this project, we will test whether state-of-the-art deep learning image processing techniques can be successfully applied to regression problems in biomedical image processing. We will focus on estimating the force generated by a muscle using an ultrasound image, a core technique in biomechanics. We will use a dynamometer (figure - top) to measure force output from the tibialis anterior muscle (the ankle dorsiflexor) and pair that with the corresponding ultrasound image of the muscle (figure - bottom). This set of ultrasound-force pairs will be used as training data for a deep neural network, which will learn to regress force from an ultrasound image. After training, the regressor will be used to estimate muscle force output during natural movement, such as locomotion.
CS students: You will be involved in developing and training a deep neural network to solve the regression problem.
SportExR students: You will run human experiments using dynamometry, ultrasound sonography, and motion capture to measure the kinematics and kinetics of muscles during movement.
Our lab is based in the SportExR building and is equipped with state-of-the-art VR and motion capture devices listed below:
Meta ARIA eye-tracking glasses
HTC Vive VR headset and motion controllers
Oculus Rift VR headset with built-in eye tracker
FOVE VR headset with built-in eye tracker
Polhemus magnetic motion tracker
Hydra magnetic motion tracker
Eyelink 1000 eye tracker (1 kHz)
Pupil Lab eye tracking glasses
Phantom haptic robot arm
Other gaming devices: Kinect, Leap Motion, Eye Tribe, Myo armband, Wii balance board, etc.
CS students: In terms of the skill sets required, students who are fluent in programming languages (C++, C#, or Python) or have experience in game development using game engines (Unity3D or Unreal) are preferred. Additionally, those who have experience with sensors (e.g. IMU, gyroscope, or depth camera) or embedded systems are strongly encouraged to discuss with Dr. Yeo. Working in our lab will give you a unique opportunity to learn and develop your expertise in VR development, a highly sought-after skill in today's IT industry, as well as your knowledge of human movement neuroscience. You don't need to have a specific background or knowledge about sport science.
SportExR students: Students who have taken A-level math and have strong data analysis skills are preferred. Working in our lab will provide a good opportunity for you to learn to use state-of-the-art technologies currently revolutionising the healthcare industry, to acquire valuable research experience in human motor control, and to improve your data analysis skills. You don't necessarily need to have, although it is preferred, experience or skills in computer programming.
Contact Dr. Yeo (s.yeo at bham.ac.uk) if you want further discussion
