Publications on Partially Observable Problems

Mitchell, Matthew (
An Architecture for Situated-learning Agents
Ph.D. Thesis, Monash University, Australia 2004. (PDF - 1.9 MB) Abstract:
This thesis looks at the problem of situated learning agents operating in real-world environments ...

Bhulai, Sandjai (
Markov Decision Processes: the control of high-dimensional systems.
Ph.D. Thesis, Vrije Universiteit Amsterdam, 2002 (PDF - 1.1 MB) Abstract:
We develop algorithms for the computation of (nearly) optimal decision rules in high-dimensional sys...

Cassandra, Anthony , Michael L. Littman and Nevin L. Zhang(
Incremental pruning: A simple, fast, exact algorithm for partially observable Markov decision processes.
Proceedings of the Thirteenth Annual Conference on Uncertainty in Artificial Intelligence (UAI--97), 1997 (Postscript - 120 KB) Abstract:
Most exact algorithms for general partially observable Markov decision processes (POMDPs) use a for...

Kaelbling, Leslie Pack , Anthony R. Cassandra and Michael L. Littman(
Acting Optimally in Partially Observable Stochastic Domains
Proceedings of the Twelfth National Conference on Artificial Intelligence, 1994 (gzipped Postscript - 104 KB) Abstract:
In this paper, we describe the partially observable Markov decision process (POMDP) approach to fin...

Kalmar, Zsolt , Cs. Szepesvari and A. Lorincz
Module-Based Reinforcement Learning for a Real Robot
Proceedings of the 6th European Workshop on Learning Robots, Lecture Notes in AI, to appear. 1998 ( PDF - 845 KB) Abstract:
The behaviour of reinforcement learning (RL) algorithms is best understood in completely observable,...

Kalmar, Zsolt , Cs. Szepesvári and A. Lorincz
Module Based Reinforcement Learning for a Real Robot
Proceedings of the 6th European Workshop on Learning Robots, 22-32, 1997 ( PDF - 845 Kb) Abstract:
This is the shortest version of our Module-Based RL paper. The behaviour of reinforcement learnin...

Littman, Michael , Anthony Cassandra and Leslie Pack Kaelbling(
Efficient dynamic-programming updates in partially observable Markov decision processes
Brown University Technical Report CS-95-19 (Postscript - 1.2 MB) Abstract:
We examine the problem of performing exact dynamic-programming updates in partially observable Mark...

Littman, Michael , Anthony Cassandra and Leslie Kaelbling
Learning policies for partially observable environments: Scaling up
Proceedings of the Twelfth International Conference on Machine Learning (Postscript - 315K) Abstract:
Partially observable Markov decision processes (POMDPs) model decision problems in which an agent t...

Littman, Michael (
Memoryless policies: Theoretical limitations and practical results
From Animals to Animats 3: Proceedings of the Third International Conference on Simulation of Adaptive Behavior (Postscript - 416KB) Abstract:
One form of adaptive behavior is "goal-seeking" in which an agent acts so as to minimize the time i...

Littman, Michael (
An optimization-based categorization of reinforcement learning environments
This paper proposes a categorization of reinforcement learning environments based on the optimizati...

Mahadevan, Sridhar , Nikfar Khaleeli E-mail: (
Robot Navigation using Discrete-Event Markov Decision Process Models
unpublished ( gzipped Postscript - 200 kb) Abstract:
This paper describes a novel architecture for robot navigation based on semi-Markov decision proces...

Parr, Ronald, Stuart Russell
Approximating optimal policies for partially observable stochastic domains
Proceedings of the IJCAI, 1995 (Postscript - 157 KB) Abstract:
The problem of making optimal decisions in uncertain conditions is central to Artificial Intelligen...

Schmidhuber, Juergen (
REINFORCEMENT LEARNING AND POMDPs (dozens of papers on RL in partially observable environments since 1989)
Journal papers and conference papers (HTML - 100KB) Abstract:
Realistic environments are not fully observable. General learning agents need an internal state to m...

Singh, Satinder , Tommi Jaakkola, Michael Jordan(
Learning Without State-Estimation in Partially Observable Markovian Decision Processes
Proceedings of the Eleventh International Machine Learning Conference ( gzipped Postscript - ) Abstract: