Title: From Classical Universal Approximation to Deep Regression in Machine Learning

Abstract:

Recently there arise plenty of algorithms supporting artificial intelligence (AI) based applications. However, most of them are simply reporting experimental results without theoretical analyses. In this talk, we attempt to interpret deep regression, a new approach to solving classical signal processing problems leveraging upon machine learning and big data paradigms. Based on Komogorov’s Representation Theorem (1957), a multivariate scalar function can be expressed exactly as a superposition of a finite number of outer functions with another linear combination of inner functions embedded within. Cybenko (1989) developed a universal approximation theorem showing such a scalar function can be approximated by a superposition of sigmoid functions, inspiring a new wave of neural network algorithms. Barron (1993) later proved that the error in approximation can be tightly bounded and related to the representation power in learning theory. In order to make the mapping learnable and computable for some practical applications, we cast the classical function approximation problems into a nonlinear regression setting using deep neural networks (DNNs) as mapping functions, such that the DNN parameters can be estimated with deep learning and big data configurations for machine learning. In this talk, we first develop four new theorems to generalize the universal approximation theorems from sigmoid to DNNs and from vector-to-scalar to vector-to-vector regression. We also show that the generalization loss or regression error in machine learning can be decomposed into three terms, approximation, estimation and optimization errors, such that each of them can be tightly bounded, separately.

Many classical speech processing problems, such as enhancement, source separation and dereverberation, can be formulated as finding mapping functions to transform input to output spectra. Our developed theorems also provide some guidelines for parameter and architecture selections in DNN designs. In a series of experiments for high-dimensional nonlinear regression, we validate our theory in terms of representation and generalization powers in machine learning for speech spectrum mapping. As a result, DNN-transformed speech usually exhibits a good quality and a clear intelligibility under adverse acoustic conditions. Finally, our proposed deep regression framework was also tested on recent challenging tasks in CHiME-2, CHiME-4, CHiME-5, CHiME-6, REVERB and DIHARD III. Based on the top quality achieved in microphone-array based enhancement, separation and dereverberation, our teams scored the lowest error rates in almost all the above-mentioned open evaluation scenarios.

Speaker: Chin-Hui Lee, School of ECE, Georgia Tech

Chin-Hui Lee is a professor at School of Electrical and Computer Engineering, Georgia Institute of Technology. Before joining academia in 2001, he had accumulated 20 years of industrial experience ending in Bell Laboratories, Murray Hill, as the Director of the Dialogue Systems Research Department. Dr. Lee is a Fellow of the IEEE and a Fellow of ISCA. He has published over 550 papers and 30 patents, with more than 55,000 citations and an h-index of 80 on Google Scholar. He received numerous awards, including the Bell Labs President's Gold Award in 1998. He won the SPS's 2006 Technical Achievement Award for “Exceptional Contributions to the Field of Automatic Speech Recognition''. In 2012 he gave an ICASSP plenary talk on the future of automatic speech recognition. In the same year he was awarded the ISCA Medal in Scientific Achievement for “pioneering and seminal contributions to the principles and practice of automatic speech and speaker recognition''. His two pioneering papers on deep regression accumulated over 2000 citations and won a Best Paper Award from IEEE Signal Processing Society in 2019.

Gallery

主辦／協辦單位:

聯絡人:

• 智能系統研究所 所長 廖元甫 yfliao@nycu.edu.tw

地理位置:

交通資訊:

 自行開行開車：

•  南下 : 中山高速公路新竹交流道(94.5K)下，新竹光復路方向第一個紅 綠燈左轉進入大學路，即可抵達本校北大門。
  （台北⟶中壢⟶新竹交流道（靠右）⟶（左轉入）大學路（加油站旁）⟶交大光復區

•  北上 :

  • 中山高速公路新竹交流道(94.5K)下，往新竹方向，行駛光復路，左轉大學路，即可抵達本校北大門。

  • 中山高速公路新竹交流道(97K)下，左轉園區二路直駛約三分鐘，在園區二路盡頭左轉新安路，即可抵達本校南大門。
    （台中⟶中山高北上（往台北方向）⟶新竹交流道（左轉往新竹市區方向）⟶經光復路（高速公路下）⟶（左轉入）大學路（加  油站旁）⟶交大光復校區 

  搭乘客運：  搭乘統聯「 板橋–北二高–新竹」線，即可直達本校。

  搭乘高鐵：   高鐵（接駁專車）。高鐵新竹站–交通大學站下車，轉市區2路公車或步行至本校。

  搭乘公車：

1. • 新竹客運2號公車（火車站⟶交通大學）約每小時一班

     • 火車站⟶東門市場⟶東門國小⟶公園⟶學園商場⟶交大博愛校區⟶學府路口⟶光復中學⟶清華大學⟶神學院⟶清華山莊⟶交大光復校區

   • 新竹客運1號公車（火車站⟶過溝）約每 10~15 分鐘一班

備註：

• 於大學路的北大門進入交大校園後，通過大門警衛室後需「立即」左轉，沿著竹湖行約三百公尺，於左手邊的荷花池旁向左轉至電子與資訊研究大樓。

• 因會場附近停車位有限，自行開車者可能需停於較遠處，再步行至會場。

• 於科學園區新安路的南大門近交大校園後，到Ｔ型路口右轉，沿著主要道路行駛，過了奈米研究大樓（對面是活動中心），右轉向前沿著荷花池即可到達電子與資訊研究大樓。