Machine Learning for Signal Processing
General Information 6 CFUCredits: ING-IND/31Scientific sector: EnglishCourse language: Laurea Magistrale in Ingegneria Elettronica (LMIE) and Master Degree in Electronics Engineering (MDEE). The class can be also freely chosen and attended by interested students from other degree programs of Sapienza University.Offered programs: February 28 - May 31, 2019Calendar: Thursday 8:30-10:00, Room 6 - Friday 8:30-10:45, Room 32Class timing: by appointmentOffice hours: Interested students are advised to send an email to subscribe to the MLSP course mailing list (or directly join the list by Google Groups) and receive any kind of communication related to the course.News and updates:Course DescriptionInformation conveyed in real-world signals may be often affected by noise, partially corrupted or even unavailble. Thus, extracting desired information from real-world signals can be even very complicated.
Machine learning for signal processing (MLSP) is the science that deals with the development of efficient algorithms and models that are able to detect and unveil a possible hidden structure in signals, thus recovering a desired information. This process is autonomously and automatically performed by MLSP algorithms, by simply learning from the available data, which is the basis of any science related to artificial intelligence.This course aims at presenting the main machine learning paradigms and applying them for the processing of a variety of signals, including audio and speech, images, movies, music, biological, electrical and mechanical, among many others. The course is based on regular classroom lessons, which also include regular exercises in Python on practical problems.
PrerequisitesMost of the prerequisites will be briefly recalled in classes. However, basic knowledge of linear algebra, signal theory and stochastic processes are warmly recommended, as well as basic programming skills.
Exam assessment and grade evaluationFinal projects will be assigned to small teams of students. The theoretical skills acquired by the student will be evaluated as well as the ability to apply and implement a specific methodology in a practical problem. Exam grades (in thirties) will be based on homework assignments (30%) and final project (70%).
Program**Introduction to MLSP.**Using machine learning methods signal processing problems. Introduction to signals, time series and possible representations. Introduction to regression and classification problems. MLSP taxonomy. Typical applicative examples in MLSP.**MLSP basics.**Brief recall of the main concepts of linear algebra, probability and random variables, distributions, stochastic processes, information theory, related to MLSP. Practical examples using typical MLSP data and scenarios.**Learning in parametric modeling.**Parametric estimation. Bias-variance dilemma. Deterministic and stochastic estimation. Cost functions and optimization for machine learning. Normal equations. Least-square optimal estimation. Linear and logistic regression. Applicative examples.**Nonlinear learning algorithms.**Nonlinear modeling and estimation. Nonlinear online learning algorithms. Learning with kernels. Regression and classification using kernels. Kernel adaptive filters. Applicative examples.**Neural networks and deep learning for signal processing.**Introduction to neural networks, universal approximation, backpropagation, feed-forward neural networks. Recurrent neural networks for signal processing. Long short-term memory models. Deep learning basics. Convolutional neural networks. Advanced deep neural networks. Examples with application to audio/image/music/biomedical signals.
Textbooks and materialMain textbook:
- Sergios Theodoridis, "Machine Learning: A Bayesian and Optimization Perspective". Elsevier, 2015.
Further references:
- Aurelio Uncini, "Introduction to Adaptive Algorithms and Machine Learning", 2018 (printed copies available at AZ Centro Stampa).
- Kevin P. Murphy, "Machine Learning: A Probabilistic Perspective". The MIT Press, 2012.
- Ian Goodfellow, Yoshua Bengio, Aaron Courville, “Deep Learning”. The MIT Press, 2016 (free pdf available online).
- Christopher M. Bishop, "Pattern Recognition and Machine Learning". Springer, 2006 (free pdf available online)
- Danilo Comminiello and José C. Príncipe (Eds.), "Adaptive Learning Methods for Nonlinear System Modeling". Elsevier, 2018.
Lectures 2018/2019[L01/L02] Machine Learning for Signal Processing: A Course Introduction. [February 28/March 1, 2019] (pdf)Lab Sessions 2018/2019[EX01] Lab Session 1: Introduction to Python. [March 8, 2019][EX02] Lab Session 2: Linear Algebra and its Application to Signals. [March 15, 2019][EX03] Lab Session 3: Empirical Estimation of Statistic Averages. [March 22, 2019][EX04] Lab Session 4: Least-Square Parametric Modeling. [March 28, 2019][EX05] Lab Session 5: Noise Cancellation by Optimal MSE Linear Filtering. [April 5, 2019][EX06] Lab Session 6: Adaptive System Identification. [April 12, 2019][EX07] Lab Session 7: Blind Source Separation. [May 2, 2019][EX08] Lab Session 8: Classification by Logistic Regression. [May 10, 2019][EX09] Lab Session 9: Introduction to TensorFlow. (Guest Lecturer: Simone Scardapane) [May 17, 2019][EX10] Lab Session 10: Neural Networks with TensorFlow. (Guest Lecturer: Simone Scardapane) [May 23, 2019][EX11] Lab Session 11: Convolutional Neural Networks. [May 24, 2019]*Lab notebooks can be required by emal.Exam datesStudents are reminded that exams are booked electronically via the INFOSTUD portal. Booking for extraordinary exam sessions is allowed only to part-time students, students enrolled in supplementary years or non-conventional paths, and graduating students.
Scheduled exam sessions for the year 2018/2019: **Session I (2018/2019):**June 14, 2019**Session II (2018/2019):**July 19, 2019**Session III (2018/2019):**September 10, 2019**Extraordinary Session I (2018/2019):**October 22, 2019**Session IV (2018/2019):**January 2020 TBD**Session V (2018/2019):**February 2020 TBD**Extraordinary Session II (2018/2019):**April 2020 TBD
About Machine Learning for Signal ProcessingHere is some video by the IEEE Signal Processing Society about typical MLSP applications.
In the next video Prof. Bernard Widrow introduces ADALINE and talks about a "machine that learns from its own experience".
...and some fun |