Model Minimization in Hierarchical Reinforcement Learning

Abstraction

Outline

Slide 4

Equivalence in MDPs

Modeling Equivalence

Modeling Equivalence (cont.)

Model Minimization

Symmetry

Symmetry example.

Symmetries in Minimization

Partial Equivalence

Abstraction in Hierarchical RL

Option specific minimization

"Task is to collect all..."

Relativized Options

"Especially useful when learning option..."

Experimental Setup

Reinforcement Learning
(Sutton and Barto ’98)

Results

Modified problem

Asymmetric Testbed

Results – Asymmetric Testbed

Results – Asymmetric Testbed

Approximate Equivalence

Summary

Summary (cont.)

Future Work

Issues