• https://sites.ualberta.ca/~szepesva/papers/RLAlgsInMDPs.pdf

• There are things which are outside of my control

  • Just do as many as you can in the most sound way you can
• TODO: Email Bowling, checklist what to do from nanjiang, explain your
• Do the thing that is the baseline, so you can actually focus on doing things that you really care about
  • Like for school, dont spend too much time. Get it done. It's not worship or religion you know?
  • It's just a career move
  • And for other things as well. Set standards, meet them and improve them.
• At the end of the day, you should aim to spend your time doing the things that are important to you
  • and whereever you need to go

• Focus! Narrow it down

• CV

  STATUS

• ALBERTA: Bowling (50%), Schuurmans (Spinning to get more background, paper)

• Uni Urban Cham: Nan Jiang

  • https://nanjiang.cs.illinois.edu/prospective-student/
  • https://rltheorybook.github.io/rltheorybook_AJKS.pdf
    • I do research in RL theory, so please keep the description of your empirical and/or non-RL research experience short and concise. Showing that you have some exposure to ML/RL is enough.
    • [Most Important] Give me a short list of your grades in mathematical courses (in letter or points, whatever your institution’s native scale is), including (1) Calculus, Linear algebra, probability and statistics, etc. (2) Advanced courses that have a significant theoretical/mathematical component, including but not limited to: computational complexity, signal processing, information theory, control theory, optimization, etc. Include relevant ranking information as you like.
    • Describe your experience in theory research, and which of the research topics I work on attract you
    • https://rltheorybook.github.io/
    • https://wensun.github.io/CS6789fall2021.html
    • http://www.athenasc.com/index.html

• Check what kind of research the professors do, Email professors

• What I'm looking for. Not too strick guidelines/roals
• Have faith in God

• Less time presenting creating image and more time doing

• Good math and ai interdiscplinary program

• Deadline, requirements,
• CV/personal statement

Next steps.

• Narrow down your understanding of RL and focus down on the research paths you can go down on...
• Intertwined with mathematics

THIS IS A KEY REALIZATION

• YOU ALREADY HAVE A MASTERS IN MATH YECHANA
• Alot of what I feel like I wil be doing in Math PhD program is repeating what I have done in MAth MA
  • The only thing you really need is just pass prelim/quals/oral exam etc

• You just need to go to a decent school (in cs)

• Basic Background info

  • http://incompleteideas.net/forstudents.html

Good list

* https://simons.berkeley.edu/programs/rl20

Uni of Indiana

• https://cs.indiana.edu/research/index.html

• https://cs.indiana.edu/contact/profile/index.html?Roni_Khardon

  • https://cgi.luddy.indiana.edu/~rkhardon/contact.php

• Adversarially Trained Actor Critic for Offline Reinforcement Learning

  • Actor Critic
    • Policy Gradient Methods

• RL wiki

• Markov Decision Process

  • (S,A,Pa,Ra) States, Actions, Pa(sA,sB), Ra(sA,sB)
  • Goal in Markov decision, find a policy s.t.
    • \(E[\sum_{t=0}^{\infty} \gamma^t R_{a_t}(s_{t},s_{t+1}) ]\) where \(a_t=\pi(s_t)\) w/ discount factor gamma
    • Algorithms to finding optimal MDP
      • Dynamic Programming approach
      • background info
      • Book Sutton Bartol
      • https://en.wikipedia.org/wiki/Bellman_equation

        ASU

• https://degrees.apps.asu.edu/masters-phd/major/ASU00/ESCOMSCPHD/computer-science-phd

• 12/31: letterrec, cv, transcript, statement purpose
• Financial aid: https://scai.engineering.asu.edu/teaching-research-assistantships-fellowships-awards/

Uni Alberta

• Opens Oct 1, 2021
• Provides good financial aid
• 3 References, No Gre, No Lang

• Read paper little bit more in detail: ask about, opening for student, give your cv and background

• Department of Mathematical and Statistical Sciences

• Outline

• Candidacy Exam?

• Statistical Machine Learning (Ph.D.)

• statement of purpose (4500 character limit) and an abstract of their M.Sc Thesis in GARS.
• Your statement of purpose should outline your research goals in relation to the research being done by the Computing Science faculty member(s) listed on your application.
• research area

• 3 professors, contact supervisors discuss funding and supervision contact

• Marlos C. Machado

  • http://mcmachado.info/?page_id=416
• Adam White (Do not contact)
• RL homepage
• Michale Bowling (games and ml) Focus on first paper
  • Expert-level artificial intelligence in heads-up no-limit poker
  • Heads-up limit hold’em poker is solved
    • Extensive form game, classical representation of imperfect information setting
    • Normal form game
    • CFR
    • Seq form https://cw.fel.cvut.cz/b191/_media/courses/be4m36mas/6.pdf
• Dale Schuurmans
  • Very Math Heavy
  • FOCUS Characterizing the Gap Between Actor-Critic and Policy Gradient
  • On the Optimality of Batch Policy Optimization Algorithms
  • Leveraging Non-uniformity in First-order Non-convex Optimization
• Rich Sutton
  • These seem to be more classical approaches, not DL approaches
  • Average-Reward Off-Policy Policy Evaluation with Function Approximation
  • Learning and Planning in Average-Reward Markov Decision Processes
• Matthew Taylor (Industry)
  • Same with these I think I need to finish spinning up terminology. MDP?
  • Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey
  • Multi Type Mean Field Reinforcement Learning

---

• Rupam Mahmood

• Csaba Szepesvari

• http://rlai.ualberta.ca/research.html

CMU

Uni Florida

• https://www.cise.ufl.edu/ciseresearch/research-centers-labs/

  Uni Illinois

• https://nanjiang.cs.illinois.edu/prospective-student/

• https://math.illinois.edu/research-aiml-and-data-science-illinois
• grad requirement
  • Concentration in Computational Science and Engineering

• https://cse.illinois.edu/cse-educational-programs/graduate-concentration/

• No Gre

• 3 letters, personal statement

• What are they looking for in a graduate stduent

University of Washington

• https://math.washington.edu/how-apply
  • https://math.washington.edu/past-phd-preliminary-exams

GOOD RL

• McGill University
• Uni Sydeny
• University of Michigan
• Duke University
• University of Texas at Austin
• Stanford University
• UC Berkeley

FUTURE

• (Partial Differential Eq) https://people.clas.ufl.edu/yun/

Unviersity of Sydeney

• https://www.sydney.edu.au/science/study/study-areas/mathematics-and-statistics.html
• https://www.sydney.edu.au/science/schools/school-of-mathematics-and-statistics.html
• www.maths.usyd.edu.au/ut/printgrouplist.py?group=Applied%20Mathematics
• https://fabioramos.github.io/CVfiles/CVRamos2019Jan.pdf

---

GOOD TDA Georgia Tech University of Utah, Duke, Ohio State, University of Miam