During postdoc, I worked on the Visual Experience Data Base project, focusing on a wide range of topics. First developing an in-house end-to-end hardware and software system that allowed us to collect > 50 hours of human daily activity under a wide range of environments, and groups of participants. Second, developing modern algorithms for eye-head-body tracking under these uncontrolled indoor-outdoor environments in terms of data quality and user's ease of calibration and validation. I also co-organized a workshop at ETRA 2021 on these topics in which industry-academia experts presented their solutions to issues involving active participant sensing. We are hosting the database at UNR and part of my focus was on designing and maintaining the data analysis pipeline. Since we are collecting data over a much wider range of environments and tasks, privacy is a huge issue that we are trying to tackle both at the algorithmic level and also from a broader standards/ethical perspective. 

During my internship at NVIDIA Research, we worked on studying the effect of latency vs. refresh rate in the context of e-sports. Our team developed an end-to-end hardware-software system that allowed us to manipulate the click to photon latency independent of the system refresh rate (uncorrelated). We tested this over four ranges of refresh rates (as high as 360) and three ranges of latency representing local, cloud, and e-sport scenarios. We published our results at Siggraph Asia and a patent on the late latch and warp algorithms for latency mitigation. 

This study utilized a VR system to investigate the predictive strategies that we use when visual information is not available.  Our visual system (to be precise our brain) is capable of precisely predicting the consequence of self-action and also outside events such as the motion of targets in our field of view.  Here we systematically occlude the trajectory of the ball in order to first measure predictive eye movements and second how does this affect the hand movement to catch the moving (red) ball. We also compare the performance under the no-occlusion condition. What we showed in this study is that not only the gaze successfully predicts the movements of the ball under different speeds and types of occlusion, but also the hand predictive movement is correlated with the gaze, suggesting a common predictive strategy driving both eye and hand movements.  The yellow and orange lines are the left and right gaze, the red disc is the VR badminton paddle.

In 2014 when we started working on Eye Tracking in VR the only commercially available headset with a built-in eye tracker was SMI. At the time, the gaze tracking algorithms were not robust. Therefore, I had to create a calibration and validation technique for robust gaze tracking in depth which allowed us to also compensate for possible drift of the eye tracker on the user's face. We used intermediate calibration and validations to check if the quality if the estimated gaze remains high and if not using a temporal metric we were able to correct for it. Another addition was to calibrate at different depths and add more robustness  to the 3D eye surface model. 

In this study, we used a 3D projector in a motion-captured lab and along with 3D shutter glasses, we projected objects on the floor that appeared convincing and close to real 3D objects. After projector calibration and implementing the stimulus presentation code we design a walking task and asked 14 participants to walk between start and end points while stepping over an obstacle of a certain height. We first examined the fidelity of the system by comparing biomechanical properties of the participants' walking compared to when avoiding real obstacles and then added eye-tracking to the experiment in order to investigate the gaze pattern during walking.

During my masters degree I focused on medical image enhancement techniques using adaptive filters. My focus was on computational modeling of multiplicative noise which best explains the degradations in the ultrasound imaging pipeline. After image enhancement and verifying that the image quality is improved, noisy areas are reduced and important information such as edges are preserved, I also developed lesion segmentation algorithms robust to noise. Since the boundaries of the lesion are most affected by the noise and it has direct influence of the segmentation accuracy, our two-fold approach improved the segmentation results significantly.