Embedded Files
2021 - DepTherm, Software Tool to Extrinsically Calibrate Thermal, RGB and Depth Cameras
DepTherm allows merging 3 types of images of different kind: thermal images, thermal images, visible spectrum images and depth images. The system presented is composed of 2 calibration patterns, an image acquisition system and a software application that includes the following functionalities: the camera calibration procedure, image fusion, point cloud registration, 3D point cloud visualization with information from temperature and generate a thermographic inspection report.
The acquisition system consists of a thermal camera and a Kinect sensor, which are placed on a fixed base with a small separation between them to increase the overlap of fields of view. The calibration patterns consist of a chessboard with copper squares and a chessboard with rectangular holes respectively.
Extrinsic calibration uses the patterns in such that they are visible in the fields of view of a pair of cameras, capturing pairs of images at different positions, and creating ordered pairs of points that correspond to the calibrate pattern corners in both images. These points are used to calculate the homography matrix, which allows applying a projective transformation these cameras. Since there are 2 pairs of cameras: thermal-rgb and rgb-depth, there will be two homography matrices, allowing an image from any camera to be projected onto the image from any other camera.
Actually, DepTherm is a registered software legally recognized by the Interior Ministry of Colombia. This document can be downloaded here.
At the very beginning, DepTherm requires the following login information:
Email.
Password.
In case the user is not registered, an additional dialog is shown requesting the following information:
Name.
Last name.
Email.
Phone.
Password.
and, user type.
Then, DepTherm will show to the user the main image acquisition and processing tools:
Intrinsic calibration image acquisition and processing.
Extrinsic calibration image acquisition and processing.
and, all cameras image acquisition and processing.
At the left of the GUI, DepTherm will show direct links to the following tools:
Only image acquisition module.
Observing the images captured by the cameras.
Intrinsic calibration image acquisition and processing.
Extrinsic calibration image acquisition and processing.
Thermographic inspection and data fusion.
Clean and reset the session.
In case of selecting the Intrinsic calibration module, DepTherm will start this procedure where the user must give the following information:
Dimension of the calibration pattern (number of squares, h x w).
Time delay between calibration patterns show to the camera.
Number of calibration images
Selecting what camera will be calibrated (RGB, Depth or Thermal)
Once these data are selected into the GUI, the calibration process can be started or stopped using two buttons:
Executing the calibration process
and stopping the calibration process.
Another option is selecting the Extrinsic calibration module. Here, DepTherm will start this procedure, and the user must give the following information:
Dimension of the calibration pattern (number of squares, h x w).
Time delay between calibration patterns show to the camera.
Number of calibration images.
Selecting what camera set will be calibrated (RGB-Depth or Depth-Thermal)
Once these data are selected into the GUI, the calibration process can be started or stopped using two buttons:
Executing the calibration process
and stopping the calibration process.
Once the Intrinsic and Extrinsic calibrations are performed, the image acquisition process for the thermographic inspection can be started. To do so, the GUI shows the following data and tools:
The RGB image at the left.
The Depth image at the center.
The Thermal image at the right.
Button to capture one set of images.
Text field to define the amount of set of images to capture.
Text field to define the time delay between images.
Button to start the acquisition process
Button to stop the acquisition process.
Afterwards, the image inspection tool can be started. In this GUI, DepTherm allows:
Visualizing one set of captures vertically.
Visualizing the image data fusion applied.
Changing color properties: color spaces, pseudo-color palettes, histogram equalization and binarization.
Defining the data fusion constants using sliders.
Image registration considering a defined number of images, time interval between images, and type of image to register.
Cloud point filters.
and a status bar in the bottom of the GUI.
The thermographic inspection generates the following results:
A 3D cloud point with temperature information on it.
Using many images, the ICP registering algorithm can fill the gaps which depend on the image acquisition point of view.
The 3D cloud point can be analyzed with tools that can:
Change the color of the palette.
Applying cloud point filters
Change the constants to perform the data fusion.
Finally, DepTherm allows generating a report that includes information such as:
Inspection date.
Inspection title.
Address of the inspection.
Inspector name.
Description of the inspection.
Institution name.
RGB, Depth and Thermal images captured.
Fused images processed
Weather conditions.
Comments and conclusions.
This software was developed for industrial service and academic use. If you like to perform a field test, please contact me:
Prof. Bladimir Bacca Cortes Ph.D.
Address: Cra. 100, Street 13, Universidad del Valle, Melendez, Building 354, Office 2006.
Tel: +5723212100 Ext. 7656
Page updated
Report abuse