Projects
PhD Research
Abstract - The paper presents an end-to-end approach that leverages images for estimating an ordered list of 3D key-points. Most of the existing methods either use point clouds or multiple RGB/depth images to estimate 3D key-points, whereas the proposed approach requires only a single-view RGB image. It is based on three steps: extracting latent codes, computing pixel-wise features, and estimating 3D key-points. It also computes a confidence score of every key-point that enables it to predict a different number of key-points based on an object’s shape. Therefore, unlike existing approaches, the network can be trained to address several categories at once. For evaluation, we first estimate 3D key-points for two views of an object and then use them for finding a relative pose between the views. The results show that the average angular distance error of our approach (6.39◦) is 8.01◦ lower than that of KP-Net (14.40◦).
Comparison with the other paradigms. Some existing methods use point clouds (top) or multiple images representing different views of an object (middle) as inputs and compute 2D/3D features for key-points estimation. In comparison, the proposed approach considers a single-view RGB image, extracts object 2D features, and use them for estimating 3D key-points (bottom).
Towards Reconstruction of 3D Shapes in a Realistic Environment [Paper]
Abstract - This paper presents an end-to-end approach for single-view 3D object reconstruction in a realistic environment. Most of the existing reconstruction approaches are trained for synthetic data and they fail when evaluated on real images. On the other hand, some of the methods require pre-processing in order to separate an object from the background. In contrast, the proposed approach learns to compute stable features for an object by reducing the influence of the image background. This is achieved by feeding two images simultaneously to the model; synthetic with white background and its realistic variant with a natural background. The encoder extracts the common features from both images and hence separates features of the object from features of the background. The extracted features allow the model to predict an accurate 3D object surface from a real image. The approach is evaluated for both real images of the Pix3D dataset and realistic images rendered from the ShapeNet dataset. The results are compared with state-of-the-art approaches in order to highlight the significance of the proposed approach. Our approach achieves an increase in reconstruction accuracy of approximately 6.1 percentage points in the F1 score with respect to Mesh R-CNN on the Pix3D dataset.
Overview of the presented approach. Reconstruction systems with high accuracy do not perform well when applied to natural images directly (top). An instance segmentation algorithm can be considered as a simple solution for removing the background (middle). In comparison, the proposed approach reconstructs accurate 3D shapes by estimating common features for realistic and white background images (bottom).
Supervised Undergraduate Projects
6. A Collaborative Approach For Map Creation Using Swarm Robots: Feb 2019 to Feb 2020
In the project, we are introducing an algorithm for map generation using a robot swarm. The robots localize themself in an unknown environment and generate separate maps. However, after some time when the robots reach a previously observed area (by some other robots), their maps will be merged into a single joint map. This reduces the time required to generate a map of a wide area and increases the efficiency such that a small environmental change can be observed. Meanwhile, the robots generate the possible routes and propose an optimal path between the given positions. We will design simulations in V-Rep using ROS in a Linux environment. Python or C++ will be used for implementations.
Funding: Funded by National Grassroots ICT Research Initiative (NGIRI) 2019 - PKR 80,000
Group members: Asad Wali, M. Arslan Pervaiz Natt, and Osama Azhar
5. Autonomous Room Cleaner Robot: Feb 2019 to Feb 2020
We are aimed to design a wheeled mobile robot "Autonomous Room Cleaner" in a Simulator as well as a hardware prototype. It will navigate autonomously in a room and create a grid map. It will clean the entire room while localizing itself. It will learn the changes made in the room and update the generated map accordingly. It will clean the target area in minimum time by utilizing the integrated vacuum cleaner. Tools that will be used in the project are; Sweep 2D lidar sensor, GPS, Raspberry Pi, Robot Operating System (ROS), V-Rep, Python/C++, etc.
Funding: Funded by National Grassroots ICT Research Initiative (NGIRI) 2019 - PKR 71,109
Group members: Arif Mustafa, Muhammad Usman Sifarish, and Asfand Yar Khan Lodhi
4. Simultaneous Localization And Mapping (SLAM) In Dynamic Environment: Feb 2019 to Feb 2020
Simultaneous Localization and Mapping (SLAM) in dynamic environments is an under considered problem in the field of robotics. It is due to the fact that without environmental information, the robot may not complete any task with accuracy. The existing mapping techniques are widely used for a static environment such that G-mapping or Hector SLAM. However, they may fail in the case of dynamic objects. Therefore we are aimed to design an algorithm that helps the robot in generating the environmental map that presents only static objects. Meanwhile, it also highlights the dynamic object separately.
The algorithm takes environmental scans (using 2D/3D laser sensor) as input and after pre-processing, it generates the clusters of objects in the environment. These clusters are examined for in detail dimension extractions. The extracted dimensions will be utilized for the separation and classification of static and dynamic objects. The extracted information is mapped on the 2D/3D grid. We will use V-Rep as a simulation environment. The Python or C++ language will be used with ROS in the Linux environment.
Group members: Waqas Ahmed, Shoaib Razzaq, Naeem Ashraf
3. An approach for object recognition with estimated dimensions: Feb 2019 to Feb 2020
In this project, we will present an algorithm that utilizes the vision sensor (stereo camera / 3D laser) in order to identify the objects by considering the estimated dimensions. The algorithm takes environmental scans as input and after pre-processing, it generates the clusters of objects in the environment. These clusters are examined for in detail dimension extractions. The extracted dimensions will be utilized from object identifications and/or classification. Furthermore, the equilibrium position of the identified object is also suggested so that an articulated robot can pick the object from an appropriate position. We will use V-Rep as s simulation environment. The Python or C++ language will be used with ROS in the Linux environment.
Group members: Muhammad Junaid, Muhammad Sheharyar Khan, Muhammad Haseeb Sheikh
2. Cotton Detection & Identification: Oct 2018 to Sep 2019
In this project, we are creating an agricultural system that detects cotton and identifies its height, size, and color. The system helps the farmer in monitoring their fields. The system provides a map of fields including cotton growth information. We are aimed to create a system that can be integrated with an autonomous copter. The V-Rep simulator with C++ or Python language is used in Robot Operating System (ROS).
Group members: Usman Dilbahar, Haider Ali Ishtiaq and Syed Mohammad Salman Tahir
1. V-FIXX: Oct 2018 to Sep 2019
We are creating a web and mobile-based system that helps people to find and contact fixer (technician) for solving their technical/nontechnical problems at their doorstep. They use the applications, mention their problem and find the nearest fixer. The fixer will reach on time and solve the problem with minimum time and high accuracy.
Group members: Muhammad Shaharyar and Ahmad Subhan Asif
Self BS Projects
Mobile Robotics:
Dynamic SLAM with Object Detection and Tracking [MS thesis]
Modeling and Autonomous Control of Four-Wheeled Robot with Obstacle Avoidance [BS Thesis]
Multi-legged Pneumatic Control [Term project]
Line and Wall Tracking Robots [Term project]
Implementations of AI algorithms including filtering and prediction, smoothing, most likely sequence etc. [Assignment work]
Medical Soft-robots
Assessment of Muscle Stiffness using CSLDV
Modeling and Analysis of damages (crack, edge slot, delamination) in a beam using ANSYS