Human-Robot Interaction and Predictive Displays
	Telerobotics, i.e., human supervisory control of robots, remains perhaps the most effective means for solving many robotics tasks beyond the factory floor. The advantage of telerobotics over automatic control is that a human operator remains in the robot's control loop to provide either fine-grained motor commands, intelligent decision-making, or both. As a long-term solution, however, current forms of telerobotics suffer from the operator's inability to control a robot as easily as his or her own limbs. This leads to movements that are fatiguing for the operator and that fail to utilize the robot's full potential, especially in terms of speed. The goal of this project is to combine a learned user model with a predictive display in order to facilitate human-robot interaction as both operator and machine acquire new skills.

Related Publications
Remote supervisory control of a humanoid robot
	M.T. Rosenstein, A.H. Fagg, R. Platt, J.D. Sweeney, and R.A. Grupen. In Proceedings of the Twentieth National Conference on Artificial Intelligence, pp. 1702-1703, 2005. [pdf]
User intentions funneled through a human-robot interface
	M.T. Rosenstein, A.H. Fagg, S. Ou, and R.A. Grupen. In Proceedings of the 10th International Conference on Intelligent User Interfaces, pp. 257-259, 2005. [pdf]
Robot learning with predictions of operator intent
	M.T. Rosenstein, A.H. Fagg, and R.A. Grupen. In Papers from the 2004 AAAI Fall Symposium on The Intersection of Cognitive Science and Robotics: From Interfaces to Intelligence, pp. 107-108, 2004. [pdf]
Supervised actor-critic reinforcement learning
	M.T. Rosenstein and A.G. Barto. In J. Si, A. Barto, W. Powell, and D. Wunsch, eds., Learning and Approximate Dynamic Programming: Scaling Up to the Real World. John Wiley & Sons, Inc., New York, 359-380, 2004. [pdf]

Movies
File	Description	Download
shell_game	Demonstration of a user interface that predicts operator intentions. The task for the operator is find an object hidden under one of several cans. The movie shows two versions of the task: the first one under full manual control by the operator and the second one under traded control by the operator and an autonomous grasping routine. For the latter, red circles superimposed on the video display indicate the ranked predictions of operator intent, and the object can associated with the largest circle is the one grasped by the autonomous system. Notice that full manual control takes considerably longer and is more error-prone.	15.2 MB
bolts1	Demonstration of a user interface that predicts operator intentions. The task for the operator is to dock a cordless drill held by the UMass humanoid with one of several bolts arranged on a workpiece. (See photos above.) Manual docking is difficult and fatiguing, but a fully automated approach ignores the strengths of a human participant. As a way to balance these extremes, the user interface allows the operator to make relatively gross movements when selecting landmarks, and then a simple gesture (closing the hand) initiates the transfer from manual control to automated docking. Candidate landmarks are highlighted with red circles, the size of which indicate the ranked predictions of user intent.	4.9 MB
pegsim	Learning to solve a simulated peg insertion task with a human operator. Red indicates autonomous control by a machine learning system and blue indicates supervisory control by the human operator. As learning progresses, the predictive display (short red bars as in the animated GIF above) becomes a useful tool for the operator when deciding either to control the manipulator or to rest while the learning system takes charge. The target slot, marked by a blue arrow, is hidden from the learning system and so ongoing supervision by the operator is required for this task.	8.2 MB
Note: To view the movies, use a QuickTime player (Mac/Win), MPlayer (Mac/ Linux), or at least version 2.80 of XAnim (Unix).

updated 13-Dec-2005 mtr@cs.umass.edu