TICS854 - Neural Networks

Course Overview

01 – Introduction (motivations, brief history)

02 – The Multilayer Perceptron

03 – Kernels and Radial-Basis Function Networks

04 – Regularization and Support Vector Machines

05 – Self-Organizing Maps

06 – Deep Feed-Forward Networks

07 – Convolutional Neural Networks

08 – Generative Adversarial Networks

09 – Potential topics (if time allows) Other Architectures and Learning Frameworks (Filtering, Recursion. Auto-encoders. Granular Neural Nets. Online Learning and Evolving Nets. Spiking Neural Nets)

Objectives of the Course

01 – Introduce foundations and concepts of neural nets

02 – Become familiar with the most used neural architectures, their learning algorithms, and key hyper-parameters

03 – Discuss issues on design; optimization, regularization, generalization; and validation –“best practices”

04 – Discuss capabilities, challenges, and consequences of neural networks

05 – Effectively develop neural models of systems

06 – Solve problems in the context of data-driven modeling for classification and prediction

Main Textbooks

[1] Simon Haykin. Neural Networks and Learning Machines, 3rd edition. Person, Prentice Hall, 2008

[2] Charu Aggarwal. Neural Networks and Deep Learning: A Textbook. Springer, 2018

[3] Ian Goodfellow, Yoshua Bengio, Aaron Courville. Deep Learning. The MIT Press, 2016

[4] Jason Brownlee. Generative Adversarial Networks. Machine Learning Mystery, v.1.5, 2019

Other Textbooks

[5] Richard Duda, Peter Hart, David Stork. Pattern classification, 2nd edition. Wiley-Interscience, 2000

[6] Cristopher Bishop, Pattern Recognition and Machine Learning. Springer, 2006

[7] Trevor Hastie, R. Tibshirani, J. H. Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. 2nd edition, Springer, 2009

Additional Supporting Material

Class Notes (available below)

Conference Papers

– NIPS conference (https://papers.nips.cc/)

– IJCNN conf. (https://ieeexplore.ieee.org/xpl/conhome/1000500/all-proceedings)

Journals

– IEEE Transactions on Neural Networks and Learning Systems (IEEE)

– Neural Networks (Pergamon Press)

– Neural Computation (MIT Press)

– Neurocomputing (Elsevier)

– International Journal of Neural Systems (World Scientific)

– IEEE Transaction on Cybernetics (IEEE)

– Neural Processing Letters (Springer)

– Information Sciences (Elsevier)

– Machine Learning (Springer)

– Neural Computing & Applications (Springer)

Datasets

– Google Dataset Search (Link) - UCI Machine Learning Repository (Link) - Kaggle (Link)