Results (full area)


Tile server (all versions): www.mat.uc.pt/autoprobadtm


v5 - Jul 2014 (current)
  • Work in progress  
    v4 - Jul 2013 (latest)
    • New extraction algorithm with variable peak width, which is more robust than the previous version with fixed width
    • Total run time 3:30 including ground filtering and automatic estimation of peak width (slower than with fixed width)
    • Typical run time for a 5 GB file swath (40 M waveforms): 35 min (single thread); breakdown see table below
    • New layer: peak widening, shows the ratio between expected width (FWHM=4ns) and received ground peak width
    • New proxy functions help compute the uncertainty as a function of peak width, amplitude, estimator type and noise properties 
    • Noise correlation (rho=0.75) taken into account in the peak detection algorithm, and in uncertainty proxy functions
    • Updated php interface, new layer
    Procedure description   % run time
    full Gaussian peak estimation + uncertainty  43
     truncated Gaussian peak estimation + uncertainty   27
    partial deconvolution  16 
     outlier filtering and faint ground return recovery  7
     reading, decoding, conversion, overhead  4
      emitted waveform fitting (fixed width Gaussian), pulse sorting  3

    Type of extracted information   % total waveforms

    processed

    95.0

    significant last returns detected

    94.4

    ground points created

    89.7

    deconvolved waveforms

    75.0

    reprocessed after outlier detection

    7.5

    no return

    5.2

    positive outliers rejected

    4.6

    recovered points (significant faint returns)

    2.9

    negative outliers rejected

    <0.1


    v3 - Mar 2013 
    • New ground filtering algorithm (1D, applied on scanlines) successfully removed the high vegetation
    • New layer: ground fraction, or transparency - shows the ratio between the ground peak energy and the total wave energy
    • Updated extraction algorithm, faster and more robust to peak mixtures - 90 min total processing time (80 min without ground filtering)
    • Fixed known bug with DEM errors 
    • Updated format decoding interface, with wave block concatenation and blunder rejection, extracts one row at a time
    • Updated php interface, now showing tile location on general area map (tile page), and new layer
    v2 - Jan 2013 
    • Extraction: Bayesian waveform processing with truncated Gaussian peak detection for range bias correction (new), efficient false alarm filtering (new), range/amplitude robust error estimation using proxies (new), intensity correction based on range and planar shading (new): 14 files of 6 GB each, processed 40 M waves per strip, total 530 M pts; run time 100 min (parallel runs, 4 threads, Core i7 2.67 MHz)
    • Georeferencing & tiling of the point clouds from each strip: produced 214 tiles, 3-4 M pts each except on boundary, safety margin and timestamp preserving capabilities added for future attitude correction (new); run time 12 min (sequential), 
    • Gridding of elevation, intensity, inter-strip discrepancy (new) and corrected range bias (new): 1000x1000 pixels at 1 m GSD, produced 7 tiff files for each tile (Z I A N D B S) - DEM, intensity image, DEM accuracy map, intensity accuracy map, point density, range bias, geometric discrepancy; run time 7 min (sequential)
    • Conversion for visualization using gdaldem and convert (imagemagick), to produce composite color hill-shaded jpegs for elevation, accuracy and bias maps; discrepancy and density maps were color-coded without relief shading, and intensity images were gamma-corrected
    • Conversion to geotiff (new) using gdal_translate to ensure the final products can be used easily (full georeferencing info. included except vertical units); file size 10 MB per tile, or 2.14 GB total
    • Known issues: bug in z error computation due to amplitude loss, will be fixed soon
      
    PHP web interface allowing users to explore and download the tiles for the 6 products of version 2


    20 m DEM of the entire area 
    (hillshaded, elevation-colorized downsampled jpeg)
    The results were initially gridded at 1 m GSD, projection UTM29 with WGS84 ellipsoidal heights, ITRF2005 reference frame
    Click here to download a 5 m GSD version (4 MB)

    20 m accuracy map of the entire area 
    (hillshaded, predictive error-colorized downsampled jpeg)
    Standard deviation (cm): purple 0.5, green 1.0, yellow 2.0, orange 3.0, red 5.0, yellow 10.0

    20 m density map of the entire area 
    (point density-colorized downsampled jpeg)
    Approx density (pts/m2): light pink 1, pink 2, purple 3, blue 4, dark blue 5+ 

    20 m bias map of the entire area 
    (hillshaded, range bias-colorized downsampled jpeg)
    Corrected bias (cm): orange 6, red 12, purple 30 - approx 0.5 1 and 2 waveform time sampling units (ns) 
    20 m intensity image of the entire area 
    (range and planar shading corrected, gamma 1.5, downsampled jpeg)
    Range and simple shading calibration only, no DEM shading correction

    20 m geometric discrepancy map of the entire area 
    (inter-strip discrepancy-colorized downsampled jpeg)
    Discrepancy = standard deviation computed between gridded DEM and point cloud. Scale(cm): black 0, blue 3, purple 6, red 9, orange 15, yellow 20, white 25+ 


    v1 - Nov 2012 
    • Extraction (Bayesian waveform processing): 14 files, 6 GB each, processed 40 M waves per strip, total 560 M pts; run time approx. 200 min (parallel runs, 4 threads, Core i7 2.67 MHz)
    • Georeferencing & tiling of the point clouds from each strip: produced 216 tiles, 3-4 M pts each except on boundary; run time 12 min (sequential)
    • Gridding of both elevation and range-corrected intensity: 1000x1000 pixels at 1 m GSD, produced 5 tiff files for each tile (Z I A N D) - DEM, intensity image, DEM accuracy map, intensity accuracy map, point density; run time 5 min (sequential)
    • Conversion for visualization using gdaldem and convert (imagemagick), to produce composite color hill-shaded jpegs, using elevation information and combining accuracy maps and intensity images
    20 m DEM of the entire area 
    (hillshaded colorized downsampled jpeg)
    The results were initially gridded at 1 m GSD, projection UTM29 with WGS84 ellipsoidal heights, ITRF2005 reference frame
    Click here to download a 5 m GSD version (4 MB)
    ą
    André Jalobeanu,
    Jan 31, 2013, 11:25 AM
    ą
    André Jalobeanu,
    Nov 30, 2012, 4:16 PM
    Comments