Teaching
This course covers fundamentals of machine learning over networks (MLoNs). It starts from a conventional single-agent setting where one server runs a convex/non-convex optimization problem to learn an unknown function. We introduce several approaches to address this seemingly, simple yet fundamental, problem. We introduce an abstract form of MLoNs, present centralized and distributed solution approaches to address this problem, and exemplify via training a deep neural network over a network. The course covers various important aspects of MLoNs, including optimality, computational complexity, communication complexity, security, large-scale learning, online learning, MLoN with partial information, and several application areas. As most of these topics are under heavy researches nowadays, the course is not based on a single textbook but builds on a series of key publications in the field. The course also includes a two-days workshop on recent advancements on fundamentals of MLoNs.
- Institution : Royal Institute of Technology (KTH)
- Dates: Jan 2019 - March 2019
- Course Responsible: H. Shokri (KTH), H. Ghauch, C. Fischione (KTH)
This course offers an in-depth study on the mathematical foundations of deep neural networks (DNNs). The first part of the course covers the fundamental aspects of statistical learning and large-scale optimization methods for modern machine learning tasks.. We then focus on learning by a DNNs, pose the resulting learning problem as an empirical risk minimization, discuss in-detail popular training methods (e.g., gradient descent and batch optimization using back propagation) and regularization. We then focus on common DNN architectures/types such as deep convolutional/recurrent neural networks, as well as factor models, and discuss challenges and practical issues for training DNNs.. Finally, we close the course by reviewing current and future research trends on DNNs and (potentially) interesting applications in wireless communications.
- Institution : University of Adger (UiA)
- Dates: April 2019 - May 2019
- Course Responsible: H. Ghauch, H. Shokri (KTH)
This course offers an in-depth study on the mathematical foundations of deep neural networks (DNNs), which are at the heart of the AI revolution. The first part of the course covers the fundamental aspects of learning and large-scale convex methods for modern machine learning tasks. We then focus on learning by a DNNs, pose the resulting learning problem as an empirical risk minimization, discuss in-detail state-of-the-art methods of large-scale training, e.g., back propagation, stochastic gradient descent, AdaGrad, RMSProp, and ADAM . We then focus on common DNN architectures/types such as deep convolutional neural networks, as well as recurrent neural networks and LSTM networks.
- Institution : CWC, University of Oulu, Finland
- Dates: Dec 2019
- Course Responsible: H. Ghauch, H. Shokri (KTH)
The course is an introduction to wireless communication, for undergraduate students. The course requires minimum background and ramps up slowly. We start with reviewing the basics in communication theory, signal processing and estimation theory. We then introduce the different functions at the transmitter, receiver and the different channel models. Then, we present in detail the inter-symbol interference (ISI) channel, the optimal receiver for ISI mitigation (matched filter, maximum likelihood), and suboptimal receivers (sphere decoding, Viterbi). We then move to the multi-antenna (MIMO) system, where we discuss optimal (in the maximal likelihood-sense) and suboptimal/linear (zero forcing, MMSE, DFE ) receivers. After that, we derive the achievable rate for a MIMO system, with linear precoding and combining; we also discuss capacity-achieving precoding and combining methods (water-filling ), and suboptimal methods (zero forcing, MMSE ). We then discuss the problem of channel estimation for the MIMO case (with a known pilot sequence), and blind estimation methods for receiver estimation (CMA). Finally, we close the course by discussing an important technique, OFDM, in single and multiple antenna setting.
- Institution : Jiaotong University, China
- Dates: March - April 2020
- Course Responsible: H. Ghauch,