Home

Assistant Professor (non-TT) at the Department of Computer Science, University of Toronto

Research Goal: Understanding the computational and statistical mechanisms required to design efficient RL agents that interact with their environment and adaptively improve their long-term performance.

Refer to Research and Publications for more information on my research, and to Bio and CV for more information about my academic background.

Links

News

  • (2022 Fall) Tyler Kastner joined my team and Murat Erdogdu’s as a PhD student. Welcome!
  • (2022 Fall) I taught the graduate level Introduction to Machine Learning course at DCS. This is a slightly updated version of the Fall 2021.
  • (2022 Summer) Farnam Mansouri did his MSc on risk-aware RL, and joined University of Waterloo afterwards. Stay tuned for some of his interesting results!
  • (2021 Fall) I taught the graduate level Introduction to Machine Learning course at DCS. I recorded the lectures, which you can find at this YouTube playlist.
  • (2021 Spring & Fall) Allen Bao, an MScAc student, joined my lab in Spring, and in collaboration with AMD worked on Gameplay Test Automation with Reinforcement Learning. He graduated in the Fall. Congratulations! He is currently Senior Software Development Engineer (ML) at AMD.
  • (2021 Summer) Dr. Yangchen Pan defended his PhD! He is my first graduated PhD student and I am very proud of him. He is currently a Departmental Lecturer at the University of Oxford. Congratulations on both achievements!
  • (2021-2023) I have not updated this place between 2021 Spring until 2023 Spring. I retroactively add a few important news above.
  • (2021 Spring) I developed and taught a new course on Reinforcement Learning. All videos can be accessed through this playlist. The course is accompanied by the Lecture Notes in Reinforcement Learning

Continue reading “News”

Publications

Textbook

I have been working on a textbook on reinforcement learning started from when I taught the new Introduction to Reinforcement Learning course in Spring 2021. The current version is from 2021. Expect major updates by the end of 2023.

  • A.M. Farahmand, Lecture Notes on Reinforcement Learning, 2021.

    The textbook is introductory in the sense that it does not assume prior exposure to reinforcement learning. It is not, however, focused on being a collection of algorithms or only providing high-level intuition. Instead, it tries to build the mathematical intuition behind many important ideas and concepts often encountered in RL. We prove many basic, or sometimes not so basic, results in RL. If the proof of some result is too complicated, we prove a simplified version of it.

    If you are a university instructor and wish to use slides for your own course, please contact me.

Papers

2024

2023

2022

2021

2020

2019

2018

2017

Continue reading “Publications”