Daniel Farkash

Hello! My name is Daniel Farkash.

My research has mainly focused on using deep learning methods for perception in robotics applications.

You can check some of it out down below!

General Robotics, Automation, Sensing, and Perception Lab

- University of Pennsylvania

This project aims to improve the metric depth estimates produced by Meta's vision foundation model DINOv2 without fine-tuining or reducing generalizability. This approach involves using a YOLOv7-based instance segmentation model to identify objects of known types within a given image, and uses the apparent heights of each object as indications of their depth. For each image, we combine priors on the height of known object types with the depth estimates provided by DINOv2 to determine a scaling factor which when applied to the depth map.

The diagram below describes this method.

Autonomous Mobile Robotics Laboratory

- The University of Texas at Austin

This project uses deep learning for robot navigation and aims to create an unsupervised method of learning costs for different terrains based on visual and inertial data from unlabeled human navigation demonstrations.

The resulting paper was accepted for presentation at the Conference on Robot Learning (CoRL) and will be published in Proceedings of Machine Learning Research (PMLR).

The video to the right contains an example deployment of a learned terrain cost function on the lab's Spot robot.

The video in the bottom right is a deployment over a 3 mile stretch.

The bar graph below shows the performance of unsupervised classification on the model's learned representations when compared to state of the art models.

2022-10-03-demo.mp4

The figure to the left shows a visual representation of the visual-inertial invariance and viewpoint invariance that were used to learn effective representations.

Laboratory for Intelligent Systems and Controls

- Cornell University

This project uses Computer Vision methods for identification and prediction in Hockey Sports Analytics, including using a conditional generative adversarial model for homography estimation and using action recognition and keypoint detection for hockey puck position estimation.

The video on the right shows the homography estimation and player detection in action. The field of view of the camera from the video above is estimated and projected on the 2D representation of the field below.

output_vid.mp4

output_rink.mp4

video1.mp4

Both of the videos of the left are of the same game and start at the same time. They show the results of using the trained models for detecting the players, which team they are on (and the referees), what actions they are taking (ex. shot, pass, dribble), and the location of the puck.

The model to the left shows representations of the players (change color with action) and puck (in pink) projected on a 2D representation of the field.

Page updated

Google Sites

Report abuse