Chapter 5: Monte Carlo Methods

10/14/99


Click here to start


Table of Contents

Chapter 5: Monte Carlo Methods

Monte Carlo Policy Evaluation

First-visit Monte Carlo policy evaluation

Blackjack example

Blackjack value functions

Backup diagram for Monte Carlo

The Power of Monte Carlo

Two Approaches

Monte Carlo Estimation of Action Values (Q)

Monte Carlo Control

Convergence of MC Control

Monte Carlo Exploring Starts

Blackjack example continued

On-policy Monte Carlo Control

On-policy MC Control

Off-policy Monte Carlo control

Learning about p while following

Off-policy MC control

Incremental Implementation

Racetrack Exercise

Summary

Author: Andy Barto

Email: barto@cs.umass.edu

Download presentation source