Projects
Comcast
In collaboration with Comcast, we explore the idea of using small, safe autonomous aerial vehicles to gather information about indoor and built spaces. Problem of building maps showing how WiFi signal strength varies over a single floor of a building.In these kinds of indoor environments GPS signals are not available which necessitates the development of systems that can autonomously localize and navigate in complex and possibly cluttered environments without the need for external signals or infrastructure. We are developing technologies to measure RF propagation characteristics over one floor of a multi dwelling unit (MDU) using flying robots.
The ultimate goal is to have a
team of two or more robots automatically navigate to prescribed locations within a known floor plan
acquire signal strength readings from each others WiFi transceivers.
the resulting measurements would be used to verify or refine an RF propagation model of the space.
Mohamed bin Zayed International Robotics Challenge (MBZIRC) 2020
The MBZIRC 2020 competition took place in Abu Dhabi during 23-25 February 2020 preceded by three days of rehearsals. With collaboration with New York University, Czech Technical University we successfully competed in all disciplines. In Challenge 1, three drones were supposed to pop balloons and catch a ball carried by foreign UAV following a 3D trajectory. We placed 2nd in the Challenge 1. In Challenge 2, three UAVs and one UGV had to cooperatively build multiple walls. We won the wall building challenge and got the US$250,000 price money. Challenge 3 consisted of outdoor and indoor fires that the team of three UAVs and one UGV had to extinguish. We finished the fire fighting challenge at 4th place. Finally, due to our great results in all three challenges, we qualified to the Grand Challenge where all three challenges are combined together and solved simultaneously. We won the Grand Challenge among all teams from prestigious universities.
Nuclear Environments Inspection with Micro Aerial Vehicles
In collaboration with Southwest Research Institute (SwRI) we are developing unmanned aerial system (UAS) technology to fly into the containment vessels of the damaged units at Japan’s Fukushima Daiichi nuclear power station and assess conditions. Tokyo Electric Power Company Holdings, Incorporated (TEPCO Holdings) contracted SwRI-UPenn to explore the use of UAS, or drones, within the containment. We are helping adapt small drones to autonomously operate within the containment.
When a 9.0 magnitude earthquake and a tsunami with estimated wave heights of 13 meters struck the power station in 2011, this one-two punch initiated a series of events that ultimately caused three reactors to fail. Since then, a number of ground- and underwater-based robotic systems have been sent inside the containment. However, damage and high radiation levels have limited access to information vital to decontamination and decommissioning efforts.
The team successfully demonstrated the core feasibility of their approach in a test fixture at SwRI’s San Antonio campus late last year. During Phase 1 of the project, the team also verified that the UAS components could survive the harsh radiation conditions within the containment.
References:
Experimental Evaluation and Characterization of Radioactive Source Effects on Robot Visual Localization and Mapping
E. S. Lee, G. Loianno, D. Thakur and V. Kumar
IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 3259-3266, April 2020.
[DOI]
Autonomous Inspection of a Containment Vessel using a Micro Aerial Vehicle
D. Thakur, G. Loianno, L. Jarin-Lipschitz, A. Zhou and V. Kumar
2019 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), Würzburg, Germany, 2019, pp. 1-7,
[DOI]
Nuclear Environments Inspection with Micro Aerial Vehicles: Algorithms and Experiments
Dinesh Thakur, Giuseppe Loianno, Wenxin Liu, and Vijay Kumar
Proceedings of the 2018 International Symposium on Experimental Robotics, 2020, Springer International Publishing, pp. 191-200.
KUKA Innovation Award
Team UPEnD was selected as one of the five finalists for the 2018 KUKA Innovation Award on the topic “Real-World Interaction Challenge”.
The competition focused on robots that can interact outside as well as inside the industrial environment, with emphasis on direct support for humans.
References:
[Timelapse1] [Timelapse2] [Setup]
Precise Dispensing of Liquids Using Visual Feedback
Monroe Kennedy, Kendall Queen, Dinesh Thakur, Kostas Daniilidis, and Vijay Kumar.
IEEE/RSJ International Conference on Intelligent Robots and Systems, (IROS), 2017.
Multi Vehicle Stereo Event Camera Dataset
The Multi Vehicle Stereo Event Camera dataset is a collection of data designed for the development of novel 3D perception algorithms for event based cameras. Stereo event data is collected from car, motorbike, hexacopter and handheld data, and fused with lidar, IMU, motion capture and GPS to provide ground truth pose and depth images. In addition, we provide images from a standard stereo frame based camera pair for comparison with traditional techniques.
References:
Dataset Available at https://daniilidis-group.github.io/mvsec/
The Multivehicle Stereo Event Camera Dataset: An Event Camera Dataset for 3D Perception
A. Z. Zhu, D. Thakur, T. Özaslan, B. Pfrommer, V. Kumar and K. Daniilidis,
in IEEE Robotics and Automation Letters, vol. 3, no. 3, pp. 2032-2039, July 2018.
NRI-LARGE
References:
Optimal Paths for Polygonal Robots in SE(2)
Monroe Kennedy, III, Dinesh Thakur, M. Ani Hsieh, Subhrajit Bhattacharya and Vijay Kumar
ASME 2017 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Cleveland, Ohio, USA, August 6–9, 2017
UAV Autonomy Project:
References:
Anytime Planning for Decentralized Multirobot Active Information Gathering
B. Schlotfeldt, D. Thakur, N. Atanasov, V. Kumar and G. J. Pappas,
in IEEE Robotics and Automation Letters, vol. 3, no. 2, pp. 1025-1032, April 2018.
[DOI]
A column generation approach for optimized routing and coordination of a UAV fleet
Julia Zillies Stephan Westphal, Dinesh Thakur, Vijay Kumar, George Pappas and David Scheidt
IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), 2016.
Mohamed bin Zayed International Robotics Challenge (MBZIRC) 2017
Our team consisting of Czech Technical University , University of Pennsylvania and University of Lincoln competed and won in the Mohamed Bin Zayed International Robotics Challenge (MBZIRC) held during 16-18 March 2017 at the Yas Marina Circuit in Abu Dhabi, UAE. There, teams attempted up to three different tasks, each carrying a $350,000 prize, and the Grand Challenge, where the team with the highest cumulative score wins $1 million. We competed with the worldwide best universities in the field of Micro Aerial Vehicles (MAV) in two Challenges. The aim of Challenge 1 is to autonomously localize a moving vehicle in the arena by a single MAV and then land on a landing platform carried by the vehicle. In the second challenge, a group of three MAVs had to search the arena for various static and moving color objects, then pick them and move them to a dedicated area. We won first place ($350,000) in the Challenge 3 and took silver position in the Challenge 1. This success earned the team the possibility to participate in the Grand Challenge competition, in which we won the bronze medal together with colleagues from the Italian University of Padua.
References:
Experimental Evaluation and Characterization of Radioactive Source Effects on Robot Visual Localization and Mapping
E. S. Lee, G. Loianno, D. Thakur and V. Kumar
IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 3259-3266, April 2020.
[DOI]
Localization, Grasping, and Transportation of Magnetic Objects by a team of MAVs in Challenging Desert like Environments
G. Loianno and V. Spurny and J. Thomas and T. Baca and D. Thakur and D. Hert and R. Penicka and T. Krajnik and A. Zhou and A. Cho and M. Saska and V. Kumar
in IEEE Robotics and Automation Letters, vol. 3, no. 3, pp. 1576-1583, July 2018.
[DOI]
Press: [Grasp] [Heise] [GulfNews] [Heise] [Emirates24]
DARPA FLA
https://www.darpa.mil/program/fast-lightweight-autonomy
References:
Fast, autonomous flight in GPS‐denied and cluttered environments
Kartik Mohta, Michael Watterson, Yash Mulgaonkar, Sikang Liu, Chao Qu, Anurag Makineni, Kelsey Saulnier, Ke Sun, Alex Zhu, Jeffrey Delmerico, Dinesh Thakur, Konstantinos Karydis, Nikolay Atanasov, Giuseppe Loianno, Davide Scaramuzza, Kostas Daniilidis, Camillo Jose Taylor, Vijay Kumar
Journal of Field Robotics,vol. 35, no. 1, pp. 101-120, 2018
Semantic Mapping of Retail Environment:
Walgreens was interested in automatic creation of an drawing that shows the following store features, along with their locations and dimensions; interior walls, merchandise holding fixtures, structural features such as columns, and identities of the departments within the merchandise holding fixtures using autonomous robots. What followed was a two year project leading to a GRASP Spin-off COSY
References:
GRASP Arche
Automated System for Semantic Object Labeling with Soft Object Recognition and Dynamic Programming Segmentation.
Jonas Cleveland, Dinesh Thakur, Philip Dames, Cody Phillips, Terry Kientz, Kostas Daniilidis, John Bergstrom, and Vijay Kumar.
IEEE Transactions on Automation Science and Engineering, vol. 14, no. 2, pp. 820-833, April 2017.
An Automated System for Semantic Object Labeling with Soft Object Recognition and Dynamic Programming Segmentation.
Jonas Cleveland, Dinesh Thakur, Philip Dames, Cody Phillips, Terry Kientz, Kostas Daniilidis, John Bergstrom, and Vijay Kumar.
IEEE International Conference on Automation Science and Engineering (CASE), 2015.
Best Application Paper Finalist
DARPA Robotics Challenge, June 2013 - June 2015
The DARPA Robotics Competition was a three phase challenge. The first phase, a Virtual Challenge in June 2013, had teams from around the world complete a virtual obstacle course with a virtual robot. Team TROOPER which was collaboration of UPenn, Lockheed Martin Advanced Technology Laboratories (ATL) and Rensselaer Polytechnic Institute was successful in getting its virtual robot to drive a vehicle, walk across difficult terrain, and attach a hose connector to a spigot and turn a valve. We were one of seven teams selected to receive an Atlas robot for use in the next phases of the DRC. The Phase 2 was the two-day Trials on December 20 and 21, 2013, in Homestead, Florida. For this challenge, 16 teams guided their robots through eight individual physical tasks that tested mobility, manipulation, dexterity, perception, and operator control mechanisms. The TROOPER team completed tasks with their Atlas robot that included removing debris blocking an entryway, climbing an industrial ladder, and opening a door to enter a building, finishing in the top 8 to secure a spot in the Finals. For the DRC’s third and final phase, Team TROOPER used an upgraded Atlas robot that’s 6’2” tall and weighs 400 pounds, which is two inches taller and 70 pounds heavier than the robot used in the Trials. The tasks performed in the DRC Finals were similar to those of the Trials and performed at one time in a single series. Also, robot was no longer connected to power chords or fall arrestors.
DARPA Robotics Challenge Finals (Jun 2015):
DARPA Robotics Challenge Trials (Dec 2013):
DARPA Virtual Challenge (Jun 2013):
Search and Persistent Surveillance by Multiple Heterogeneous UAVs:
References:
Coordinated Path Planning for Fixed-wing UAS Conducting Persistent Surveillance Missions
James Keller, Dinesh Thakur, Maxim Likhachev, Jean Gallier and Vijay Kumar
IEEE Transactions on Automation Science and Engineering, vol. 14, no. 1, pp. 17-24, Jan. 2017.
Obstacle avoidance and path intersection validation for UAS: A B-spline approach
James Keller, Dinesh Thakur, Maxim Likhachev, Jean Gallier and Vijay Kumar
International Conference on Unmanned Aircraft Systems (ICUAS), 2016.
Coordinated Path Planning for Fixed-wing UAS Conducting Persistent Surveillance Missions
James Keller, Dinesh Thakur, Maxim Likhachev, Jean Gallier and Vijay Kumar
IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), 2015.
Coordinated commencement of pre-planned routes for fixed-wing UAS starting from arbitrary locations-a near real-time solution.
James Keller, Dinesh Thakur, Vladimir Dobrokhodov, Kevin Jones, Maxim Likhachev, Jean Gallier, Isaac Kaminer and Vijay Kumar.
International Conference on Unmanned Aircraft Systems (ICUAS), 2014.
A Computationally Efficient Approach to Trajectory Management for Coordinated Aerial Surveillance.
James Keller, Dinesh Thakur, Vladimir Dobrokhodov, Kevin Jones, Mihail Pivtoraiko, Jean Gallier, Isaac Kaminer and Vijay Kumar.
Unmanned Systems, 1. 2013
[DOI]
Planning for Opportunistic Surveillance with Multiple Robots.
Dinesh Thakur, Maxim Likhachev, James Keller, Vijay Kumar, Vladimir Dobrokhodov, Kevin Jones, Jeff Wurz and Isaac Kaminer,
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2013.
Collaborative Autonomous Navigation in a Networked Environment (CANINE), Jun 2012:
The CANINE program was a collaborative effort conducted by TARDEC and the Robotics Technology Consortium (RTC) in which industry and academia demonstrated their unique technology approaches to replicate a military dog locating threats, providing protection and clearing patrol routes. UPenn's team in collaboration with SRI Sarnoff was one of the six teams selected to participate out of 28 proposals. The CANINE was structured as a competition involving six teams involving both industry and academic which was held at Ft. Benning, Georgia as part of the 2012 Robotics Rodeo. The challenge required autonomous robotic “fetch” operation, where a robot is shown a novel object and then asked to locate it in the field, retrieve it and bring it back to the human operator.
References:
Playing Fetch with Your Robot: The Ability of Robots to Locate and Interact with Objects.
Philip Dames, Dinesh Thakur, Mac Schwager, and Vijay Kumar.
IEEE Robotics and Automation Magazine 21.2 (2014): 46-52. June 2014.
R-MASTIF: Robotic mobile autonomous system for threat interrogation and object fetch.
Aveek Das, Dinesh Thakur, James Keller, Sujit Kuthirummal, Zsolt Kira and Mihail Pivtoraiko.
SPIE Conference, 2013.
Adaptive Information Gathering Using Visual Sensors.
Philip Dames, Dinesh Thakur, Mac Schwager and Vijay Kumar.
In Workshop on Stochastic Geometry in SLAM at the International Conference on Robotics and Automation. Saint Paul, MN, USA, May. 2012
[pdf]
