EDFLOW21 Benchmarking

MVSEC sequences and ground truth flow NPZ files are from https://daniilidis-group.github.io/mvsec/download/
The slider_hdr_far sequence is from the Event Camera Dataset for SLAM hosted by RPG http://rpg.ifi.uzh.ch/davis_data.html . Please note: for this sequence we generated the ground truth flow from the known depth and camera motion. See our slider_hdr_far folder.
The spinning dot sequence is available in our spinningDot folder.
The transboxes sequence is from DVSFLOW16. Its GT flow is from IMU with lens focal length of 4.5mm. Remember to load the calibration AEDAT file before using IMU for GT flow.

Measuring flow accuracy

The EDFLOW methods can be easily benchmarked in jAER on either AEDAT-2.0 or rosbag recordings, as long as ground truth flow is available. They can be compared with LP, DS, and other methods implemeented in jAER.

EDFLOW is called PatchMatchFlow in jAER

To measure accuracy, activate the measureAccuracy option in any flow filter that subclasses AbstractMotionFlowIMU, e.g. PatchMatchFlow, LocalPlanesFlow, DirectionSelectiveFlow, LukasKanadeFlow, etc.

IMU flow (camera rotation)

Ground truth flow is computed automatially from the Davis IMU gyro rotation when the IMU samples are present. Remember to correctly set the lens focal length.

Remember also to load a file or play part of a file where the camera is not moving to capture IMU zero-motion calibration offset samples

You can then monitor the accuracy at the bottom of the AEViewer display. (You might need o zoom out a bit to see it.)

You can also print the statistics accumulated (up to statisticsWindowSize) by clicking PrintStatistics

External GT flow (as in MVSEC or slider sequences)

jAER can also load the numpy flow vector frames from MVSEC.

To use these, download the NPZ file, then extract the numpy .npy files to a folder. You can use 7-zip or other extractor; the .npz file is just a zip archive.

In the jAER flow filter, click the ImportGTFromNPZ. Navigate to the folder with the npy files and click Open. It may take some time to load these huge files that contain thousands of frames of vx,vy data.

Increasing JVM memory

You may need to increase your VM memory with -Xmx=20g, for example, for 20GB of VM.

Validating the ground truth flow with IMUFlow

To validate the GT flow, use IMUFlow. Load the GT if using NPZ or Matlab ground truth. Then each DVS event should be drawn with a barb showing the speed. Make sure displayMotionVectors is enabled. You can use ppsScale to set the scale of the drawn flow barbs.

This sample from MVSEC's outdoor_night1 shows obvious incorrect flow on the car, which should be pointing left. Otherwise, the flow appears reasonable.

Recording motion vectors

To record motion vectors, load a DVS data file, activate a flow filter, and click the StartLoggingMotionVectorEvents button. Select a folder and filename base (sans extension). jAER will now start writing a CSV file with the output flow vectors. This file can be processed by the Matlab script in our Code folder to compute the final flow accuracy statistics reported in the paper.

Algorithm parameters

EDFLOW

We used the parameters reported in the paper. These can be set by hitting the Defaults button.

LocalPlanes

We used these parameters

EV-Flownet

We used the pretrained EV-Flownet_2R network.

As in the EV-Flownet paper, we generated event volumes for EV-Flownet with the same frame rate as the recording environment (30Hz indoors, 45Hz outdoors day, 10Hz outdoors night).

Page updated

Google Sites

Report abuse