R-ASIAS

  • High helicopter accident rate

  • General Aviation (GA) and small helicopter have low participation rate in the flight data monitoring programs (e.g. FDR)

=> video data analysis based approach is proposed

  • Objective: to develop an intelligent video data analysis tool for flight data monitoring

  • Flight state information inference for basic flight parameters that most FDRs are required to record

Attitude Estimation from Outside View

Using DBSCAN, core points are extracted that can

  • reduce computational cost

  • deal with significant camera’s movement

  • deal with noises (e.g. light)

Analogue Gauge Reading

Using DBSCAN & threshold-based filter

  • Can reduce computational cost using only core points

  • Can deal with various layouts of flight instrument panels by automatically detect the positions of analogue gauges

Digital Display Reading

Digital Number Indicators

Original images of digital numbers

  • Feature Selection: HOG (Histogram of Oriented Gradients)

  • Classification: Support Vector Machines (SVM) => Precision: 92.5%

Primary Flight Display (PFD)

  • Hough transform-based line detection

  • Electronic Attitude Display System (EADS)

    • Red line in the right video: detected horizon

  • Precision: 91.2%

Detected roll angle

Detected pitch angle

Alarms Sound Detection

  • Single alarm detection:

    • Short Time Fourier Transform (STFT) => type and occurrence time detection

  • Multiple alarm detection: integrate STFT with

    • Statistical evaluation to find relationships between two data:

    • Change detection algorithm (CUSUM) to detect small changes in values of signal

Five alarm mixed sound

  • Autopilot disco: 2.12 – 11.68 sec

  • Dash outer: 7.49 – 9.83 sec

  • Dot inner: 9.24 – 13.74 sec

  • Dash middle: 8.03 – 17.36 sec

  • Stall: 11.86 – 13.75 sec

Engine Parameter Estimation

Using statistical model and DBSCAN

  • Can deal with various noises by efficiently identifying dominant frequency correlated with flight parameter

  • Can estimate the engine parameters using a statistical model without sound profiles

Correlation between Flight Parameters and Noise

Statistical model created using audio from different flights

Flight parameters can be estimated by frequency analysis and the statistical model

Related Publications

  • S. Shin and I. Hwang, “Data Mining Based Computer Vision Analytics for Automated Helicopter Flight State Inference,” AIAA Journal of Aerospace Information Systems, (submitted on November 19, 2016; 1st revision submitted on May 15, 2017; Accepted on September 6, 2017)

  • S. Shin and I. Hwang, “Helicopter Cockpit Audio Data Analysis to Infer Flight State Information,” Journal of the American Helicopter Society, (submitted on November 19, 2016; under review)