Recent Projects

1) Transfer learning from synthetic to real images using VAEs for robotic applications
Tadanobu InoueSubhajit Chaudhury, Giovanni De Magistris and Sakyasingha Dasgupta, IBM Research - Tokyo
We propose a method that transfers learned capability of detecting object position from simulation environment to the real world using variational auto-encoders (VAE). Firstly the VAE is trained on numerous synthetic images, which are easy to synthesize, followed by fine-tuning the encoder on real images with fixed decoder. The proposed method detects object positions 6 to 7 times more precisely than the baseline of directly learning from the dataset of the real images. We show that the proposed method performs robustly in different lighting conditions or in the presence of other distractors.
    [Paper]       [Video]    

2) Conditional generation of multi-modal data using constrained embedding space mapping
Subhajit Chaudhury, Sakyasingha Dasgupta, Asim Munawar, Md. A. Salam Khan, Ryuki Tachibana, IBM Research - Tokyo
In this work, we developed a multi-modal generative method that maps multiple data modalities to a common latent space enabling simple cross modal inference by simple encoder decoder rearrangement. The proposed method can synthesize images from text and audio input and produces better PSNR values for the reconstructed images than directly regressing images from text and audio embeddings
    [Paper]       [Poster]    

3) Spatial-temporal motion analysis for invisible crack detection
Subhajit Chaudhury, Nakano Gaku, Jun Takada, Akihiko Iketani, NEC Central Reseach Labs
In this project, a crack detection system is proposed, that can identify internal and early-stage cracks by analyzing motion patterns to find local strain discontinuities for dynamic loading of bridges. Crack localization is formulated as an energy minimization problem on Conditional Random Fields (CRF) framework for robust structured prediction which was experimentally shown to be superior (by 0.14 to 0.22 F1 Score) to state-of-the-art image-based crack detection methods. 

4) Can fully convolutional networks perform well for general image restoration problems?
Subhajit Chaudhury, NEC Central Research Labs ; Hiya Roy, University of Tokyo
This work proposes an FCN model, that learns a direct end-to-end mapping between the corrupted images as input and the desired clean images as output. Our FCN model outperforms traditional sparse coding based methods and demonstrates competitive performance compared to the state-of-the-art methods for image denoising. Our model also performs well (less than state-of-the-art performance) for blind image inpainting task as well.
   [Paper]       [Project Page]    

5) Convolutional Neural Network Layer Re-ordering for acceleration
Vijay Daultani, Subhajit Chaudhury, Kazuhisa Ishizaka, NEC Central Reseach Labs
In this project, we improve the time performance of Convolutional Neural Networks (CNN) by interchanging the order of activation and pooling layers with no change to the output of the network. We performed experimental verification of our hypothesis with VGG16, VGG19, and AlexNet CNN architectures and showed that our proposed method can obtain up to 4x speed-up in activation unit(close to the theoretical values) and 5% overall time improvements on both CPU and GPU tests. 

M.Tech Projects

1) Volume Preserving Haptic Pottery
Subhajit Chaudhury, Subhasis Chaudhuri, IIT Bombay

This project proposes a realistic deformation model for pottery in which the user can interact with a rotating clay volume using a haptic tool. The deformation algorithm preserves the volume of clay to model the incompressible nature of semi-solid clay used in pottery. The interactive clay volume consists of an array of cylindrical elements stacked up vertically, providing simple and efficient collision detection and response. The simple design of the proposed deformable virtual clay model allows real-time force computation at 1Khz and graphical rendering at 25fps, while producing very similar deformation results to real life pottery. 
    [Paper]       [Video]       [doi]

2Vision-Based Human Pose Estimation for Virtual Cloth Fitting
Sourav Saha, Pritha Ganguly, Subhajit ChaudhuryIIT Bombay
This project proposes a real-time solution to setting up a virtual trial room for on-line portals selling apparels using a generic web camera interface to the portal. The user selects an image of an apparel from the on-line display and captures his/her own videos. The proposed method detects the pose of the user as well as various anthropomorphic features such as length and thickness of upper limbs and the dimensions of the torso. Using the detected human body points from image frames, we use a cloth fitting algorithm to fit the garment to the user's body. The entire process is performed at 30 fps, providing a realistic rendering of virtual clothing for any user. 
  [Paper]       [Project Page]       [doi]

3) Experiencing Physical Interactions through Virtualization in a Shared 3D Virtual Environment
Subhajit Chaudhury, Vineet Gokhale, Subhasis Chaudhuri, IIT Bombay
This project proposes a 3D virtual reality based conferencing system where multiple users can share 3D space in the same virtual environment while interacting with each other through audio, video and tactile feedback with the objects in the environment. Users are represented in the virtual environment with virtual avatars, which are character animated, according to the corresponding user movements, by detecting the 3D location of joints in the user's body in each frame with the aid of a marker-less pose detection algorithm. The key contribution of this work is to incorporate tactile feedback along with visual and audio feedback for a more immersive virtual reality experience. 
     [Report]       [Video(single-client)]

Bachelor's Project

1) Door detection for mobile robot navigation using fuzzy door classifier
Subhajit Chaudhury, Hiya Roy, Jadavpur University
(under the guidance of Prof. Amitava Chatterjee)
This project proposed an automatic door detection algorithm for visually guided autonomous mobile robot navigation. Initial candidate door detection is achieved leveraging intensity variation in the captured image(s) and consequent edge detection methodology. Then, a fuzzy decision is made on the detected candidate framework by defining the likeliness of various features for a general door. Based on the percentage of trust generated by fuzzy likeliness kernel, the final decision is made about the likeliness of the detected candidate to be a genuine door or not.
    [Report]         [Code]

2) Peak Voltmeter error minimization by Lagrangian Polynomial Regression
Hiya Roy, Subhajit Chaudhury, Jadavpur University
This project proposed a polynomial regression model for learning the relationship between errors and uncompensated voltage readings from high voltage Peak Voltmeter reading. The proposed technique reduced error by about 7% compared to an uncompensated setting.

Other Relevant Projects
1) Classification of Hyperspectral Satellite Image Using Deep Convolutional Neural Networks
Subhajit Chaudhury, Hiya Roy
Implemented land-cover classification from hyperspectral satellite images using convolutional neural networks. Results demonstrate that CNN based classification performs considerably well compared to traditional feature classifiers like support vector machines.

2) Determination of scene depth using disparity map obtained from stereo images of the scene
Subhajit Chaudhury, Avik Hati
Course: Computer Vision (EE 702)
Obtained the pixel-wise disparity map from pixel correspondence using block-wise correlation matching.
     [Report]          [Code]

3) Shape from Shading Using Surface Normals
 Avik Hati, Subhajit Chaudhury
Course: Computer Vision (EE 702)
Estimated the shape of an object using the intensity value from the image. We used the classical "shape from shading" using p, q values, and global minimization technique. Then we used a simplified approach of treating p as the x-derivative and q as the y-derivative of the depth map.
      [Report]          [Code]

4) Implementation of Vanilla ray-tracing algorithm
Satwik Kottur, Subhajit Chaudhury
Course: Advanced Computer Graphics (CS 775)
Implemented vanilla ray-tracing from scratch given a set of objects in the scene, light source, and surface reflection parameters.

5) The Music Box : Creating a Computer Graphics animation in OpenGL from scratch.
Subhajit Chaudhury, Urmi Sadhu
Course: Computer Graphics (CS 675)
Modeled an articulated human figure using basic geometry transformations, prepared a virtual environment, provided appropriate lighting and texture maps to finally create a full animation in OpenGL from scratch.
    [Video]           [Code]

6) Face recognition using Gabor Filters with Minimum Euclidean Distance based classifier
Subhajit Chaudhury, Bitan Bhar, Naireeta Kansabanik
Course: Digital Signal Processing (EE 603)
Extracted feature vectors from query image at different scale and orientation using Gabor filters and performed classification using multi-class maximum margin SVM classifier.

7) Image Compression using Discrete Haar Wavelet transform
Subhajit Chaudhury, Bitan Bhar
Course: Wavelets (EE 678)
Obtained the various levels of the image(LL,LH,HL,HH) and reconstructed the image using these intermediate levels according to the required level of detail and PSNR.

8) Facial feature based authentication using Zernike Moment features and Nearest neighbor + Fuzzy Clustering classification
Abhyuday Das, Subhajit Chaudhury
Generated an n-dimensional feature vector from the query image using the first n ordered Zernike moments of the image matrix and classified it using nearest neighbor and fuzzy clustering algorithm