|
One of the great scientific challenges of our time is that of
understanding the natural phenomenon
of human intelligence, and its twin challenge of engineering artificial
systems that display aspects
of intelligence. Since a key function of the brain is to enable an
animal to effectively exert control
over its environment, the question of how an intelligent agent modifies
its behaviour given experience
in the world is one of the foundational questions of Artificial
Intelligence. My research fits
broadly within the framework of reinforcement learning, a
machine
learning paradigm concerned
with the problem of learning how to act.
My primary interest is in building agents that must solve a range of
tasks over their operational
lifetimes, and can therefore use experience gained in solving some tasks
to build knowledge structures
that enable them to improve performance on other tasks.
My work is directed towards scaling
up to the kind of high-dimensional,
continuous problems faced by robots (and humans), with the goal of one
day creating
a mobile robot capable of autonomous operation, open-ended learning, and
broad competence.
My thesis will be on autonomous skill acquisition in continuous
domains.
One of the most impressive of human learning abilities is skill
acquisition-the ability to create
new skills, refine them through practice, and apply them in new task
contexts. Skill acquisition lies
at the heart of two important aspects of human intelligence.
First, humans are able to perpetually improve their solutions to
difficult control tasks through
practice, moving from inefficient, planned movements that require a
great deal of attention, to
smooth, optimized movements that are executed efficiently without
conscious thought. This type of
learning underlies much of human achievement because it supports our
unique ability to specialize
at tasks by devoting time and effort to them. Second, through the
retention and refinement of
solutions to important subproblems, humans become able to solve
increasingly difficult problems
over time. From shortly after birth, we begin assembling a vast library
of motor and cognitive
skills over our lifetimes that gradually enables more and more
activities with less and less cognitive
effort.
My dissertation will focus on developing new methods for
autonomous skill acquisition
in high-dimensional, continuous problems, with the ultimate aim of
achieving autonomous skill
acquisition on a mobile robot.
In the recent past I have worked on transfer in reinforcement
learning, and on robot motivational systems. They are not a
focus right this minute, but I expect they will be again in the future.
Further back, during some of my time at Edinburgh
I worked on the Hydra
project, which aimed at investigating the principles underlying the co-operation
of relative simple units (think cells) to construction more complex units
(think arms and legs) and to generate complex behaviour, using only
local communication. That experience has left me with a secondary but
enduring
interest in Artificial Morphogenesis and Self-Assembly Robotics.
I also used to occasionally dabble in Artificial Evolution
and Evolutionary Methods, where I was interested
in the properties of the evolutionary process, rather than the use of genetic
algorithms to solve hard optimisation problems.
During my undergraduate years at Wits my research was mainly
in Computational Geometry, but I don't do that
at all anymore. I have, however, retained a strong interest
in the problems facing Computer Science Education
in South Africa.
|
