Part 1: Signal Processing and Sparsity

• • Definition of signals, signal properties, discrete representations, Fourier transforms, filtering, sampling theory, applications to audio signals and images.

  • Basics of convex optimization: Convex sets, convex functions, convex optimization problems.

  • Sparse representations, compressive sensing, application to image recovery.

  • Matrix completion, application to recommendation systems.

  • Sparse plus low-rank models, Application to traffic prediction over networks.

  • Tensor factorization and completion.

Part 2: Graph Signal Processing

• • Algebraic graph theory, graph properties, connectivity, degree centrality, eigenvector centrality, PageRank, betweeness, modularity

  • Graph partitioning

  • Graph filtering, sampling, prediction of graph processes

  • Independence graphs: Markov networks, Bayes networks, Gaussian Markov Random Fields, inference of graph topology from data. Application to brain functional connectivity inference.

  • Graph convolutional neural networks. Application to Geometric Deep Learning.

Part 3: Distributed Optimization and Machine Learning

• Duality theory: Lagrange dual problem, Slater's constraint qualifications, KKT conditions.

• Optimization algorithms: Primal methods (steepest descent, gradient projection, Newton method, proximal gradient), primal-dual methods (dual ascent, method of multipliers, ADMM).

• Distributed convex machine learning: Computing architectures (parallel, federated, etc.). Splitting across examples and/or features; Application to distributed regression and classification.

• Distributed non-convex machine learning: Parallel SGD and the SCA framework. Application to Neural Network training and matrix/tensor completion.

Textbooks and resources:

[1] Slides and codes

[2] S. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, 2004;

[3] S. Foucart and R. Holger, A mathematical introduction to compressive sensing, Basel: Birkhäuser, 2013.

[4] S. Boyd et al., Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers, Foundations and Trends in Machine Learning, 3(1):1–122, 2011.

[5] CVX software for convex optimization.

[6] E.J. Candès et al., Exact matrix completion via convex optimization, Foundations of Computational mathematics, 9(6), 717-772, 2009.

[7] Cai, J. F., Candès, E. J., & Shen, Z., A singular value thresholding algorithm for matrix completion, SIAM Journal on Optimization, 20(4), 1956-1982.

[8] P. Di Lorenzo, S. Barbarossa, and P. Banelli, Sampling and Recovery of Graph Signals, Cooperative and Graph Signal Processing, P. Djuric and C. Richard Eds., Elsevier, 2018.

[9] Vetterli, Martin, Jelena Kovačević, and Vivek K. Goyal. Foundations of signal processing. Cambridge University Press, 2014.

[10] M.E.J. Newman, Networks: An Introduction, Oxford, UK: Oxford University Press.