ALL Logo

Autonomous Learning Laboratory

College of Information and Computer Sciences
University of Massachusetts Amherst

The Autonomous Learning Laboratory (ALL) conducts foundational artificial intelligence (AI) research, with emphases on AI safety and reinforcement learning (RL), and particularly the intersection of these two areas.

The long-term goals of the laboratory are to develop more capable artificial agents, ensure that systems that use artificial intelligence methods are safe and well-behaved, improve our understanding of biological learning and its neural basis, and to forge stronger links between studies of learning by computer scientists, engineers, neuroscientists, and psychologists.



Philip Thomas Philip S. Thomas Co-Director
Sridhar Mahadevan Sridhar Mahadevan Co-Director
Not accepting new students
Andrew Barto Andrew Barto Director Emeritus
Not accepting new students


Sarah Byrne Sarah Byrne Grants and Contracts Coordinator

Doctoral Students

Clemens Rosenbaum Clemens Rosenbaum PhD Candidate
Francisco Garcia Francisco Garcia PhD Candidate
Stephen Giguere Stephen Giguere PhD Candidate
Chris Nota Chris Nota PhD Student
James Kostas James Kostas PhD Student
James Kostas Yash Chandak PhD Student
Blossom Metevier Blossom Metevier PhD Student
Scott Jordan Scott Jordan PhD Student
Mengxue Zhang Mengxue Zhang PhD Student

Undergraduate Students

Sarah Brockman Sarah Brockman Honors Project

Alumni (Doctoral Students)

Name Adviser Year Current Website
Ian Gemp Sridhar Mahadevan 2019 link
Thomas Boucher Sridhar Mahadevan 2018 link
CJ Carey Sridhar Mahadevan 2017 link
Bo Liu Sridhar Mahadevan 2015 link
Chris Vigorito Andrew Barto 2015 link
Philip Thomas Andrew Barto 2015 link
Bruno Castro da Silva Andrew Barto 2015 link
William Dabney Andrew Barto 2014 link
Scott Niekum Andrew Barto 2013 link
Yariv Z. Levy Andrew Barto 2012 link
Scott Kuindersma Andrew Barto 2012 link
George Konidaris Andrew Barto 2011 link
Jeffrey Johns Sridhar Mahadevan 2010 link
Chang Wang Sridhar Mahadevan 2010 link
Alicia "Pippin" Peregrin Wolfe Andrew Barto 2010 link
Sarah Osentoski Sridhar Mahadevan 2009 link
Ashvin Shah Andrew Barto 2008 link
Özgür Şimşek Andrew Barto 2008 link
Khashayar Rohanimanesh Sridhar Mahadevan 2006
Mohammad Ghavamzadeh Sridhar Mahadevan 2005 link
Anders Jonsson Andrew Barto 2005 link
Thomas Kalt Andrew Barto 2005
Balaraman Ravindran Andrew Barto 2004 link
Michael Rosenstein Andrew Barto 2003
Michael Duff Andrew Barto 2002
Amy McGovern Andrew Barto 2002 link
Theodore Perkins Andrew Barto 2002 link
Doina Precup Andrew Barto 2000 link
Bob Crites Andrew Barto 1996
S. J. Bradtke Andrew Barto 1994
Satinder Singh Andrew Barto 1993 link
J. R. Backrach Andrew Barto 1992 link
Vijaykumar Gullapalli Andrew Barto 1992
Robert A. Jacobs Andrew Barto 1990 link
J. S. Judd Andrew Barto 1988
Charles W. Anderson Andrew Barto 1986 link
Richard S. Sutton Andrew Barto 1984 link

Past Postdocs

Name Adviser Year Current Website
Jay Buckingham Andrew Barto
Michael Kositsky Andrew Barto 1998 - 2001
Matthew Schlesinger Andrew Barto 1998 - 2000 link
Andrew H. Fagg Andrew Barto 1998 - 2004 link
Sascha E. Engelbrecht Andrew Barto 1996 - 2002
Vijaykumar Gullapalli Andrew Barto 1992 - 1994
Michael Jordan Andrew Barto link

Past Masters and Bachelors Students

Name Adviser Year Degree
Michael Amirault Sridhar Mahadevan 2018 BS
Stefan Dernbach Sridhar Mahadevan 2015 MS
Jonathan Leahey Sridhar Mahadevan 2013 MS
Jie Chen Sridhar Mahadevan 2013 MS
Andrew Stout Andrew Barto 2011 MS
Armita Kaboli Andrew Barto 2011 MS
Peter Krafft Andrew Barto 2010 BS
Colin Barringer Andrew Barto 2007 MS
Suchi Saria Sridhar Mahadevan 2002 - 2004 BS
Eric Sondhi Sridhar Mahadevan BS
Ilya Scheidwasser Sridhar Mahadevan BS



  • Y. Chandak, G. Theocharous, J. Kostas, S. Jordan, and P. S. Thomas
    Learning Action Representations for Reinforcement Learning
    In ICML, 2019.
    [arXiv] [pdf]
  • P. S. Thomas and E. Learned-Miller
    Concentration Inequalities for Conditional Value at Risk
    In ICML, 2019.
  • S. Tiwari and P. S. Thomas
    Natural Option Critic
    In AAAI, 2019.
    [arXiv] [pdf]
  • Y. Chandak, G. Theocharous, J. Kostas, S. Jordan, and P. S. Thomas
    Improving Generalization over Large Action Sets
    At RLDM, 2019.
  • C. Nota and P. S. Thomas
    Is the Policy Gradient a Gradient?
  • P. S. Thomas, S. Jordan, Y. Chandak, C. Nota, and J. Kostas
    Classical Policy Gradient: Preserving Bellman's Principle of Optimality
  • Y. Chandak, G. Theocharous, C. Nota, and P. S. Thomas
    Lifelong Learning with a Changing Action Set
  • Y. Chandak, G. Theocharous, B. Metevier, P. S. Thomas
    Reinforcement Learning When All Actions are Not Always Available
  • E. Learned-Miller and P.S. Thomas
    A New Confidence Interval for the Mean of a Bounded Random Variable
    [arXiv] [pdf]
  • J. Kostas, C. Nota, and P. S. Thomas
    Asynchronous Coagent Networks: Stochastic Networks for Reinforcement Learning without Backpropagation or a Clock
    [arXiv] [pdf]


  • P. S. Thomas, C. Dann, and E. Brunskill.
    Decoupling Gradient-Like Learning Rules from Representations
    In ICML, 2018.
    [ pdf ]
  • C. Rosenbaum, T. Klinger, and M. Riemer
    Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learning
    In ICLR, 2018.
    [ pdf ]
  • M. Machado, C. Rosenbaum, X. Guo, M. Liu, G. Tesauro, and M. Campbell
    Eigenoption Discovery through the Deep Successor Representation
    In ICLR, 2018.
    [ pdf ]
  • Y. Chandak, G. Theocharous, J. Kostas, and P. S. Thomas
    Reinforcement Learning with a Dynamic Action Set
    In Continual Learning workshop , NIPS 2018.
  • S. M. Jordan, D. Cohen, and P. S. Thomas
    Using Cumulative Distribution Based Performance Analysis to Benchmark Models
    In Critiquing and Correcting Trends in ML workshop, NIPS 2018.
    [ pdf ]
  • S. Giguere and P. S. Thomas.
    Classification with Probabilistic Fairness Guarantees
    Presented at FairWare, 2018.
  • A. Jagannatha, P. S. Thomas, and H. Yu.
    Towards High Confidence Off-Policy Reinforcement Learning for Clinical Applications
    Presented at CausalML, 2018.
    [ pdf ]


  • I. Durugkar, I. Gemp, and S. Mahadevan
    Generative Multi-Adversarial Networks
    In ICLR, 2017.
    [ pdf ]
  • X. Guo, T. Klinger, C. Rosenbaum, J. P. Bigus, M. Campbell, B. Kawas, K. Talamadupula, G. Tesauro, and S. Singh
    Learning to Query, Reason, and Answer Questions On Ambiguous Texts
    In ICLR, 2017.
    [ pdf ]
  • C. Rosenbaum, T. Gao, and T. Klinger
    e-QRAQ: A Multi-turn Reasoning Dataset and Simulator with Explanations
    In WHI@ICML, 2017.
    [ pdf ]

1978 – 2016

Click here for a listing of older publications.


Prospective Doctoral Students:

The Autonomous Learning Laboratory is not accepting applications for doctoral students at this time.

Prospective Interns:

The Autonomous Learning Laboratory is not accepting applications for interns at any level at this time.

Prospective Masters Students:

The Autonomous Learning Laboratory is not accepting applications for masters level positions at this time.

Prospective Postdoctoral Researchers:

The Autonomous Learning Laboratory is not accepting applications for postdoctoral researchers at this time.