Vision Based Wire Detection Dataset

1 Overview

The Near Earth Autonomy Vision Based Wire Detection (NE-VBWD) dataset provides a public benchmark for evaluating wire detection algorithm for full scale helicopter safety systems. This dataset is designed for computer vision researchers who want to advance the state of the art in object recognition and structure from motion techniques for wires. It allows them to test new algorithms against one of the most challenging problems in helicopter safety, wire hazards, and compare against our published work "Detection and Reconstruction of Wires using Cameras for Aircraft Safety Systems".

The dataset contains 41 approaches to wire hazards at 5 distinct locations around the Pittsburgh, PA metropolitan area. Each approach consists of approximately 3 minutes of flight data, GPS/INS position data and video from a calibrated forward looking 2hz, 6576 x 4384 pixel RGB camera. Ground truth for the dataset is provided in the form of LIDAR surveys of the obstacle wires as well as 125 manually annotated image frames.

The dataset was collected using Near Earth Autonomy’s m4 perception suite which was developed as a part of the Office of Naval Research’s Autonomous Aerial Cargo Utility System (AACUS) program. m4’s sensor head contains a forward looking high resolution camera as well as a scanning lidar. During each dataset flight, the m4 equipped helicopter approached the wires from a distance of over 2km at an altitude slightly above the highest wire to maintain safety. By 300m from the crossing point, the helicopter began to climb to at least 40m above the wire to safely clear the obstacles.

If you use this dataset in your work, please cite:

Stambler, A., Sherwin, G., Rowe, P. (2019). Detection and Reconstruction of Wires Using Cameras for Aircraft Safety Systems. In International Conference on Robotics and Automation. Montreal, Cananda: IEEE.

2 Data set files and formats

The raw data for each run towards wire hazards is recorded as ROS bags. All of the standard ROS utilities can be used to view and manipulate the data.

2.1 Raw Data

The raw data is organized by test flight

• forward_high_def_camera*.bag contains the forward high resolution camera data. The forward camera is recorded as a calibrated camera with a sensor_msgs::Image and accompanying sensor_msgs::CameraInfo message specifying the calibrated camera intrinsic data. The Image sensor is in BayerGRBG8 format and must be decoded into RGB or monochrome format for proper viewing.
• tf*.bag file contains the GPS/INS measured global position of the camera. The pose is encoded in the ROS /tf along with the extrinsic calibration of the camera to the INS. The position is expressed as a transform relative to the ECEF frame.
• forward_camera.webm, is a non-ROS preview of the forward camera imagery. It allows you to view the video with any standard movie player like VLC.

2.2 Ground Truth Labels

• Lidar Survey: All wire hazards have been surveyed by m4 perception suite's GPS INS and lidar. The point cloud was registered and hand labeled as wire points. This point cloud is a 3d ground truth for the location of the wire hazard. Note, this point cloud cannot be used to exactly label wire pixels in the image because the survey and camera position accuracy is not high enough for pixel level reprojection at long idstances.
• Per-Pixel Image Labels: Select camera image frames have been labeled with pixel level annotation of the wire's location. The original image is in the "labeled_imagerg/imgs" while the labeled mask are in the "labeled_imagery/labels" folder. The labels are a binary image where pixels greater than 0 are wires while all others are not wires. Wires projections that overlap in image space have at least a one pixel gap between their binary masks so that they pixels can be clustered into wire segements. RGB images and label images share a common name which indicate their time stamp. Names with the format img_XXXXXXXXXX.XXXXXX where Xs represent seconds from the UTC epoch. This time stamp can be matched with the raw data rosbag image timestamps.

3 Download Instructions

1. Download the download script.
2. Use python to execute the script and create a local copy of the dataset:

python download_dataset.py  vbwd-dataset