Zinal Summer School 2021

Data Science, Optimization and Operations Research

I am honored to be part of the EPFL Winter Summer School 2021 together with Yurii Nesterov in Zinal, Switzerland. Please find below the agenda along with the slides of the lecture series.







Lecture Series

  • Lecture 1: Introduction to Reinforcement Learning (RL)

    Date & time: 16.08.2021

    Slides: PDF

    This lecture will focus on fundamental concepts and challenges of RL.

    • Basics of RL
    • Exact solution methods for planning
    • Value-based vs policy-based methods
    • Modern challenges and open questions

  • Lecture 2: RL from Control Perspectives

    Date & time: 17.08.2021

    Slides: PDF

    This lecture will focus on the convergence analysis of value-based RL from unified dynamical systems perspectives.

    • TD learning
    • Q-learning
    • Double Q-learning
    • Variants without function approximation

  • Lecture 3: RL from Optimization Perspectives

    Date & time: 18.08.2021

    Slides: PDF

    This lecture will focus on the global convergence analysis of policy-based RL from nonconvex optimization perspectives.

    • Policy gradient method
    • Natural policy gradient method
    • Policy gradient methods with function approximation
    • Policy gradient methods with entropy regularization

  • Lecture 4: RL from Deep Learning Perspectives

    Date & time: 19.08.2021

    Slides: PDF

    This lecture will focus on the generalization and regularization of deep RL from deep learning perspectives.

    • A refresher of modern deep learning theory
    • TD learning with neural network approximation
    • Actor-critic with neural network approximation

  • Lecture 5: RL from Game Perspectives

    Date & time: 20.08.2021

    Slides: PDF

    This lecture will focus on game perspectives of RL.

    • Zero-sum game (from convex-concave to nonconvex-nonconcave)
    • Stackelberg game