TUTORIAL COURSE on Distributed Strategic Learning for Engineers
Place: Amphitheatre F.3.05, Plateau de Moulon, Supelec, 91192 Gif sur Yvette, France
Dates: February 2324, 2012 Thursday, February 23, 2012 (9h0012h00/14h0017h00) ,
Friday, February 24, 2012 (9h0012h00/14h0017h00)
Audience: Engineers, Master, Ph.D. Students, Researchers etc
This tutorial course will revisit the fundamental tools of distributed strategic learning in view of their applications to wireless networks. The course will be subdivided into a theoretical part where the classical methods and results for distributed learning are introduced, and an application part where practical considerations in engineering are visited. Each part will decline successively the basic tools and applications, the advanced methods and results known today, as well as current research activities.
A precise outline is given below:
First DAY: THEORETICAL NOTIONS
MORNING: (Basic notions)  Markov trees
 Dynamical systems
 Stochastic approximation
 Differential inclusion
 Basics of robust games
 Basics of dynamic robust games
 Dynamic solution concepts/stationary solution concepts
 Research today:
 fast convergence of iterative learning patterns,
 hitting time to a set,
 frequency of visits
 Introduction to strategy learning
AFTERNOON: (STRATEGY LEARNING and PERFORMANCE ESTIMATIONS)  Strategy learning
 Payoff learning,
 Combined learning (CODIPAS)
 Heterogeneous learning
 Hybrid learning
 Learning with random updates
 Risksensitive strategic learning in dynamic robust games
 Combined learning for continuous action space
 Mean field learning
 Research today:
 Best learning algorithms for stability/performance tradeoff,
 How to learn global optima in a fully distributed way?
Second DAY: APPLICATIONS TO WIRELESS NETWORKING, COMMUNICATIONS AND NETWORK ECONOMICS
MORNING: (WIRELESS NETWORKING and COMMUNICATIONS)  Distributed learning for parallel routing and frequency selection
 Strategic Learning in usercentric network selection
 Combined learning under noise in WLAN
 Cost of learning and Quality of Experience (QoE) in LTE
 Distributed Learning for Network Security
 Learning under uncertainty for Network MIMO
 Coalitional learning for cognitive radios
 Research today:
 Why should we care on distributed strategic learning?
 How to extract useful information from outdated and noisy measurements?
AFTERNOON: (ECONOMICS OF NETWORKS)  Mean field learning for the smart grid
 Hierarchical learning for network design
 Risksensitive learning for the economics of cloud computing
 Research today:
 Simultaneous, coalitional and mutual learning
 Learn how to flatten the peaks in largescale networks
Participants Ejder BASTUG   Manjesh Kumar HANAWAL   Essaid SABIR   Apostolos DESTOUNIS   Cedric AULIAC   Pascal BIANCHI   Matthieu DE MARI   Biruk Ashenafi MULUGETA   XuanThang VU   Arshad ALI   Chung Shue CHEN   Alexander PELOV   Giuseppe CAIRE   Abdoulaye BAGAYOKO   Dorin PANAITOPOL   Tijani CHAHED   Benoit LACROIX   ChienChun CHENG   Wissam CHAHIN   Habib SIDI   Heyfa AMMAR   Baher MAWLAWI   Julia VINOGRADOVA   Joanna TOMASIK   AbdulAziz MOHAMAD   Vivek PARASHAR   Alexis LAMIABLE   Jeremie JAKUBOWICZ   Ghassan ALNWAIMI   Philippe SEHIE   Mikael DAUTREY   Houda CHICHI   Mohammed ELTAYEB   Vincent GAUTHIER   Thi Mai Trang NGUYEN   Rong ZHENG   Vladimir FUX   Paul de KERRET   Chahe NERGUIZIAN   Bruno MODENA   Reuben George Stephen 
 Anup APREM   Manesh   Olivier BEAUDE   Hajer KHANFIR   Pol BLASCO   LI GUANGYU   Zhengguo SHENG   Romain COUILLET   Ivan STUPIA   Ahmed Farhan HANIF   David GESBERT   Marios KOUNTOURIS   Manoj Kumar PANDA   Yasir FAHEEM   Muhammad Shoaib SALEEM   Tao WANG   Mari KOBAYASHI   Abhik BANERJEE   Rachit AGARWAL   Patrick LOISEAU

 Sheng YANG
  Francesco PANTISANO
  Amadou Kountche DJIBRILLA
  Samson LASAULCE
  Raul DELACERDA
  Henrik LUNDGREN
  Nidhi HEGDE   Karen MIRANDA   RAISS EL FENNI MOHAMMED   Francois MERIAUX   GOUMBARK ALI   Wieslawa WAJDA   Meryem BENAMMAR    
Distributed Strategic Learning for Wireless Engineers
This course provides an introduction to distributed strategic learning for both static and dynamic models, with deterministic as well as stochastic descriptions. The coverage will encompass both theoretical and algorithmic developments, with applications in networking and communications.To follow the course, familiarity with optimization techniques, basics in algebra and analysis, and some background in probability theory are required.
Textbooks
 Sutton, R. S. and Barto, A. G. (1998) Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press
 D. Fudenberg & D. Levine, The Theory of Learning in Games, MIT Press, 1998
 Young, H. P. (2004). Strategic Learning and Its Limits. Oxford University Press, Oxford.
 Tamer Basar and Geert Jan Olsder, Dynamic Noncooperative Game Theory, 2nd edition, Classics in Applied Mathematics, SIAM, Philadelphia, 1999.
 V. Borkar, Stochastic Approximation: A Dynamical Systems Viewpoint, 2008.
 Jorgen Weibull, Evolutionary Game Theory. Cambridge, MA: The M.I.T. Press 1995
 W. H. Sandholm, Population Games and Evolutionary Dynamics, MIT Press, 2010.
 N. CesaroBianchi, G. Lugosi, Prediction, Learning and Games, Cambridge University Press, 2006
Course outline
Part 0: Introduction  Basics on stochastic approximations
 Recursive equations
 Fixed point iterative techniques and average shemes
 Basics on dynamical systems
 Basics on differential inclusions
Part I  Partially distributed algorithms in networked games  Cournot adjustement
 convergence results
 strict contraction and BanachPicard fixed point theorem
 nonconvergence results and logistic map
 Best response dynamics
 simultaneous updates
 sequential updates
 convergence of pseudopotential games
 nonconvergence results
 Betterreply dynamics
 difference between betterreply dynamics and bestreplydynamics
 convergence results
 nonconvergence results
 discrete time adjustment vs continuous time dynamics
 Loglinear dynamics
 logit map, smooth best response
 convergence time to approximated equilibria
 Fictitious play and its variants
 meaning of convergence in frequences
 nonconvergence in twobytwo games
 how to play equilibrium strategies when the frequencies are converging?
 learning Hannan set
Part II  Fully distributed learning algorithms in networked games  Trial and error learning
 Markov chain formulation
 Markov tree theorem
 application to finite games which have at least one pure equilibrium
 Reinforcement learning algorithms
 BushMosteller RL
 Cross learning
 Arthur model of RL
 Calibration and learning correlated equilibria
 Regret minimization
 convergence in frequencies
 BoltzmannGibbs based reinforcement learning algorithms
 Qlearning in Markov decision process
 Qlearning in stochastic games
 Hlearning
 Evolutionary game dynamics
 Evolutionary game dynamics with migration
 Delayed evolutionary game dynamics
 Learning Pareto optimal solutions
Part III: CODIPASRL (combined fully distributed payoff and strategy reinforcement learning)  Noregret based CODIPASRL
 Imitative BoltzmannGibbs based CODIPASRL
 BushMosteller based CODIPASRL
 BoltzmannGibbs based CODIPASRL
 Multiplicative weighted imitative CODIPASRL
 Heterogeneous CODIPASRL
 Hybrid CODIPASRL
 CODIPASRL for continuous action spaces
 CODIPASRL for Pareto Optimal solutions
Part IV: Distributed strategic learning for global optima in:  Coordination games
 Anticoordination games
 Aggregative games
 Stable games
 Robust games
 Dynamic robust games
Part V: Learning in risksensitive games:  Risksensitive finite games
 Risksensitive robust games
 Risksensitive payofflearning
 Risksensitive strategylearning
 Risksensitive CODIPASRL
 Connection to riskneutral learning
 Convergence in risksensitive potential games, risksensitive aggregative games, risksensitive monotone payoff games
 Selection of risksensitive evolutionary stable states
 Risksensitive Bayesian equilibria
 Risksensitive correlated equilibria
 Risksensitive satisfactory solution, risksensitive subgame perfect equilibria
 Learning in risksensitive stochastic games
Part VI: Projects  Cost of learning CODIPASRL
 CODIPASRL for evolutionary network formation games
 CODIPASRL for global optima in network selection games
 Coalitional learning CODIPASRL
 Risksensitive evolutionary coalitional games
 Mean field CODIPASRL
Part VII: Examples  Strategic learning in Twobytwocoordination games
 Strategic learning in Twobytwoanticoordination games
 Strategic learning in Frequency/Channel selection games
 Strategic learning in Network selection games
 Strategic learning in Routing games
 Strategic learning in Medium access control games
 Strategic learning in Multiple access channel in opportunistic networks
 Strategic learning in Resource allocation games
 Strategic learning in Wireless network security
 Strategic learning in UserCentric IPTV Services Selection in Heterogeneous Wireless Networks
 etc
