strategiclearning


TUTORIAL COURSE on Distributed Strategic Learning for Engineers


Place: Amphitheatre F.3.05,
Plateau de Moulon,
Supelec, 91192 Gif sur Yvette, France


Dates: February 23-24, 2012
Thursday, February 23, 2012 (9h00-12h00/14h00-17h00) ,
 Friday, February 24, 2012 (9h00-12h00/14h00-17h00)

Audience: Engineers, Master, Ph.D. Students, Researchers etc

This tutorial course will revisit the fundamental tools  of distributed strategic learning in view of their applications
 to wireless networks. The course will be subdivided into a theoretical part where the classical methods and results 
for distributed learning are introduced, and an application part where practical considerations in engineering are visited. 
Each part will decline successively the basic tools and applications, the advanced methods and results known today, 
as well as current research activities. 





A precise outline is given below:

First DAY: THEORETICAL NOTIONS

MORNING: (Basic notions)
  • Markov trees
  • Dynamical systems
  • Stochastic approximation
  • Differential inclusion
  • Basics of robust games
  • Basics of dynamic robust games
  • Dynamic solution concepts/stationary solution concepts
  • Research today: 
    • fast convergence of iterative learning patterns,
    •  hitting time to a set, 
    •  frequency of visits
  • Introduction to strategy learning

AFTERNOON: (STRATEGY LEARNING and PERFORMANCE ESTIMATIONS)
  • Strategy learning
  • Payoff learning,
  • Combined learning (CODIPAS)
  • Heterogeneous learning
  • Hybrid learning
  • Learning with random updates
  • Risk-sensitive strategic learning in dynamic robust games
  • Combined learning for continuous action space
  • Mean field learning
  • Research today:
    • Best learning algorithms for stability/performance tradeoff,
    • How to learn global optima in a fully distributed way?


Second DAY: APPLICATIONS TO WIRELESS NETWORKING, COMMUNICATIONS AND NETWORK ECONOMICS


MORNING: (WIRELESS NETWORKING and COMMUNICATIONS)
  • Distributed learning for parallel routing and frequency selection
  • Strategic Learning in user-centric network selection
  • Combined learning under noise in WLAN
  • Cost of learning and Quality of Experience (QoE) in LTE
  • Distributed Learning for Network Security
  • Learning under uncertainty for Network MIMO
  • Coalitional learning for cognitive radios
  • Research today: 
    •  Why should we care on distributed strategic learning?
    •  How to extract useful information from outdated and noisy measurements?

AFTERNOON: (ECONOMICS OF NETWORKS)
  • Mean field learning for the smart grid
  • Hierarchical learning for  network design
  • Risk-sensitive learning for the economics of cloud computing
  • Research today: 
    • Simultaneous, coalitional and mutual learning 
    • Learn how to flatten the peaks in large-scale networks

Participants
 Ejder BASTUG 
 Manjesh Kumar HANAWAL 
 Essaid SABIR 
 Apostolos DESTOUNIS 
 Cedric AULIAC 
 Pascal BIANCHI 
 Matthieu DE MARI 
 Biruk Ashenafi MULUGETA 
 Xuan-Thang VU 
 Arshad ALI 
 Chung Shue CHEN 
 Alexander PELOV 
 Giuseppe CAIRE 
 Abdoulaye BAGAYOKO 
  Dorin PANAITOPOL 
  Tijani CHAHED 
 Benoit LACROIX 
 Chien-Chun CHENG 
 Wissam CHAHIN 
 Habib SIDI 
 Heyfa AMMAR 
 Baher MAWLAWI 
 Julia VINOGRADOVA 
 Joanna TOMASIK 
 AbdulAziz MOHAMAD   
    
 Vivek PARASHAR         
 Alexis LAMIABLE  
 Jeremie JAKUBOWICZ 
 Ghassan ALNWAIMI 
 Philippe SEHIE 
 Mikael DAUTREY 
 Houda CHICHI 
 Mohammed ELTAYEB 
 Vincent GAUTHIER 
 Thi Mai Trang NGUYEN 
 Rong ZHENG 
  Vladimir FUX 
 Paul de KERRET  
 Chahe NERGUIZIAN 
 Bruno MODENA 
 Reuben George Stephen
 Anup APREM 
Manesh  
 Olivier BEAUDE 
 Hajer KHANFIR 
 Pol BLASCO 
 LI GUANGYU 
 Zhengguo SHENG 
 Romain COUILLET 
 Ivan STUPIA 
 Ahmed Farhan HANIF 
 David GESBERT 
 Marios KOUNTOURIS 
 Manoj Kumar PANDA 
 Yasir FAHEEM 
 Muhammad Shoaib SALEEM 
 Tao WANG 
 Mari KOBAYASHI 
 Abhik BANERJEE 
 Rachit AGARWAL 
 Patrick LOISEAU

 Sheng YANG
 
 Francesco PANTISANO
 
 Amadou Kountche DJIBRILLA
 
 Samson LASAULCE
 
 Raul DELACERDA
 
 Henrik LUNDGREN
 
 Nidhi HEGDE 
 Karen MIRANDA 
 RAISS EL FENNI MOHAMMED 
 Francois MERIAUX 
 GOUMBARK ALI 
  Wieslawa WAJDA 
 Meryem BENAMMAR 
  




Distributed  Strategic Learning for Wireless Engineers



This course provides an introduction to distributed strategic learning for both static and dynamic models, with deterministic as well as stochastic descriptions. The coverage will encompass both theoretical and algorithmic developments, with applications in networking and communications.

To follow the course, familiarity with optimization techniques, basics in algebra and analysis, and some background in probability theory are required.


Textbooks


  • Sutton, R. S. and Barto, A. G. (1998) Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press
  • D. Fudenberg & D. Levine, The Theory of Learning in Games, MIT Press, 1998
  • Young, H. P. (2004). Strategic Learning and Its Limits. Oxford University Press, Oxford.
  • Tamer Basar and Geert Jan Olsder, Dynamic Noncooperative Game Theory, 2nd edition, Classics in Applied Mathematics, SIAM, Philadelphia, 1999.
  • V. Borkar, Stochastic Approximation: A Dynamical Systems Viewpoint, 2008.
  • Jorgen Weibull, Evolutionary Game Theory. Cambridge, MA: The M.I.T. Press 1995
  • W. H. Sandholm, Population Games and Evolutionary Dynamics, MIT Press, 2010.
  • N. Cesaro-Bianchi, G. Lugosi, Prediction, Learning and Games, Cambridge University Press, 2006

Course outline 

Part 0: Introduction
  • Basics on stochastic approximations
  • Recursive equations
  • Fixed point iterative techniques and average shemes
  • Basics on dynamical systems
  • Basics on differential inclusions
Part I - Partially distributed algorithms in networked games
  • Cournot adjustement
    • convergence results
    • strict contraction and Banach-Picard fixed point theorem
    • non-convergence results and logistic map
  • Best response dynamics
    • simultaneous updates
    • sequential updates
    • convergence of pseudo-potential games
    • non-convergence results
  • Better-reply dynamics
    • difference between better-reply dynamics and best-reply-dynamics
    • convergence results
    • non-convergence results
    • discrete time adjustment vs continuous time dynamics
  • Log-linear dynamics
    • logit map, smooth best response
    • convergence time to approximated equilibria
  • Fictitious play and its variants
    • meaning of convergence in frequences
    • non-convergence in two-by-two games
    • how to play equilibrium strategies when the frequencies are converging?
    • learning Hannan set
Part II - Fully distributed learning algorithms in networked games
  • Trial and error learning
    • Markov chain formulation
    • Markov tree theorem
    • application to finite games which have at least one pure equilibrium
  •  Reinforcement learning algorithms
    • Bush-Mosteller RL
    • Cross learning
    • Arthur model of RL
  • Calibration and learning correlated equilibria
    • Regret minimization
    • convergence in frequencies
  • Boltzmann-Gibbs based reinforcement learning algorithms
  • Q-learning in Markov decision process
  • Q-learning in stochastic games 
  • H-learning 
  • Evolutionary game dynamics
  • Evolutionary game dynamics with migration
  • Delayed evolutionary game dynamics
  • Learning Pareto optimal solutions
Part III: CODIPAS-RL (combined fully distributed payoff and strategy reinforcement learning)
  • No-regret based CODIPAS-RL
  • Imitative Boltzmann-Gibbs based CODIPAS-RL
  • Bush-Mosteller based CODIPAS-RL
  • Boltzmann-Gibbs based CODIPAS-RL
  • Multiplicative weighted imitative CODIPAS-RL
  • Heterogeneous CODIPAS-RL
  • Hybrid CODIPAS-RL
  • CODIPAS-RL for continuous action spaces
  • CODIPAS-RL for Pareto Optimal solutions
Part IV: Distributed strategic learning for global optima in:
  • Coordination games
  • Anticoordination games
  • Aggregative games
  • Stable games
  • Robust games
  • Dynamic robust games
Part V: Learning in risk-sensitive games:
  • Risk-sensitive finite games
  • Risk-sensitive robust games
  • Risk-sensitive payoff-learning
  • Risk-sensitive strategy-learning
  • Risk-sensitive CODIPAS-RL
  • Connection to risk-neutral learning
  • Convergence in risk-sensitive potential games, risk-sensitive aggregative games, risk-sensitive monotone payoff games
  • Selection of risk-sensitive evolutionary stable states
  • Risk-sensitive Bayesian equilibria
  • Risk-sensitive correlated equilibria
  • Risk-sensitive satisfactory solution, risk-sensitive subgame perfect equilibria
  • Learning in risk-sensitive stochastic games
Part VI: Projects
  • Cost of learning CODIPAS-RL
  • CODIPAS-RL for evolutionary network formation games
  • CODIPAS-RL for global optima in network selection games
  • Coalitional learning CODIPAS-RL
  • Risk-sensitive evolutionary coalitional games
  • Mean field CODIPAS-RL

Part VII: Examples
  • Strategic learning in Two-by-two-coordination games
  • Strategic learning in Two-by-two-anti-coordination games
  • Strategic learning in Frequency/Channel selection games
  • Strategic learning in Network selection games
  • Strategic learning in Routing games
  • Strategic learning in Medium access control games
  • Strategic learning in Multiple access channel in opportunistic networks
  • Strategic learning in Resource allocation games
  • Strategic learning in Wireless network security
  • Strategic learning in User-Centric IPTV Services Selection in Heterogeneous Wireless Networks
  • etc



Comments