Chapter 5: Monte Carlo Methods


Click here to start

Table of Contents

Chapter 5: Monte Carlo Methods

Monte Carlo Policy Evaluation

First-visit Monte Carlo policy evaluation

Blackjack example

Blackjack value functions

Backup diagram for Monte Carlo

The Power of Monte Carlo

Two Approaches

Monte Carlo Estimation of Action Values (Q)

Monte Carlo Control

Convergence of MC Control

Monte Carlo Exploring Starts

Blackjack example continued

On-policy Monte Carlo Control

On-policy MC Control

Off-policy Monte Carlo control

Learning about p while following

Off-policy MC control

Incremental Implementation

Racetrack Exercise


Author: Andy Barto


Download presentation source