Robot learning by demonstration
From Scholarpedia
| This article has not been peer-reviewed or accepted for publication yet; It may be unfinished, contain inaccuracies, or unapproved changes. | ||||||||||||||||||||
Author: Dr. Aude Billard, EPFL Ecole Polytechnique Federale de Lausanne
Dr. Aude Billard accepted the invitation on 14 July 2007 (self-imposed deadline: 14 January 2008).
This article will briefly cover: Research on robot programming by demonstration from the standpoint of machine learning and computational neuroscience.
WORK IN PROGRESS!!
Contents |
Introduction
Robot programming by demonstration develops algorithms by which a human can teach a robot a task without explicitly programming the robot, but rather by simply "showing" to the robot how to do the task. The robot learns from observing the human. This way of teaching resembles the way human learn. The method is thus deemed to provide a powerful way for lay people (workers in factories or end-users) to teach robots. Robot programming by demonstration requires usually less than ten demonstrations of the same task to make the training sessions bearable for the user.
Research on robot programming by demonstration is also known as imitation learning, apprenticeship learning or learning from human advice.
Background and History
Robot programming by demonstration started in the 80's. The rationale for moving from purely pre-programmed robots to flexible user-based interfaces for training robots was based on the idea that such a "natural" means of teaching the machine would minimize or even eliminate the explicit and tedious programming of a task by a human user. With the recognition that not all events the robot would face when interacting with the real world could be foreseen by the user, robot learning became a central preoccupation of robots; and with the introduction of various tools for robot learning, programming by demonstration was viewed as a powerful mechanism for reducing the complexity of search spaces for learning. Observing either good or bad examples reduces the search for a possible solution by either starting the search from the observed good solution (local optima), or conversely, by learning from a subset of the original search space that exclude known bad solutions. Note that, at the time of the writing of this review, the vast majority of work in PbD have sofar considered learning only from good examples. Learning from bad examples remains an open but very promising research route.
PbD promises are thus multiple. On the one hand, one hopes that it will make learning faster, in contrast to tedious reinforcement learning methods or trials-and-error learning. On the other hand, one expects that the methods, being user-friendly, would enhance the application of robots in human daily environments. Robot Programming by demonstration has by now become a central topic of robotics that spans across general research areas such as human-robot interaction, machine learning, machine vision and motor control.
For reviews of current advances in the field, see (Argall et al 09, Billard et al 08, Billard 02, Breazeal & Scasselatti 02, Schaal et al 04-99)#References_PBD .
Interfaces and data representation
The type of data on which programming by demonstration learns depends on the interface used to teach the robot. Methods such as motion sensors, kinesthetic teaching and vision, provide kinematics (position, velocity) information on the motion of the limbs of the teacher, whereas haptics and exoskeleton provide information on the dynamics (force, torques) underlying body motion. Kinematics data are sufficient to control robots that rely on position and velocity control (e.g. through low-level PID controllers). Dynamics is crucial for providing compliant control.
Motion sensors and vision provide data from the view point of the teacher. Since teacher and robot differ in their body structure and dynamics, transferring motion from one to the other is non trivial and is known as the correspondence problem [NehanivDautenhahn99]. Part of the correspondence problem is resolved automatically when teaching is done through tele-operated devices, such as haptics or exoskeleton, or through direct contact, as in kinesthetic teaching, as these means of teaching provide information from the robot's stand point.
 Kinesthetic Teaching consists in guiding the robot through the task's steps by moving passively its limbs |
 Motion Sensors worn by the trainer allows the robot to track the motion of each of the user's limbs. These may include [Dillman] |
 thumb| Information on the dynamics of the task can be transmitted .]] |
using motion sensors and having the robot mirroring the motion on the fly.
Teaching can be either done in batch, i.e. by showing all the demonstrations at once and then get the robot to replicate a "generalized" version of the demonstration or incrementally, by showing the robot
Formulation of optimal control problems
Dynamical Systems's Approach
To be done
Symbolic Approach
To be done
Scaffolding, Moulding and Incremental Teaching
To be done
References
- Textbooks
- A. Billard, S. Calinon, R. Dillmann and S. Schaal (2008). Robot Programming by Demonstration. Handbook of Robotics, : MIT Press, 2008.
- A. Billard. Imitation. Handbook of Brain Theory and Neural Networks, : MIT Press, 2002.
- Calinon, S. (2009), "Robot Programming by Demonstration: A Probabilistic Approach". EPFL/CRC Press.
(http://www.routledge.co.uk/books/Robot-Programming-by-Demonstration-isbn978 1439808672)
- 'Reviews
- B.D. Argall, S. Chernova, M. Veloso, and B. Browning. A Survey of Robot Learning from Demonstration. To appear, Robotics and Autonomous Systems
- C. Breazeal and B. Scassellati (2002). "Robots that imitate humans," Trends in Cognitive Science, 6, pp. 481-487.
- Schaal, S.;Ijspeert, A.;Billard, A. (2004). Computational approaches to motor learning by imitation, in: Frith, C. D.;Wolpert, D. (eds.), The Neuroscience of Social Interaction, 1431, pp.199-218, Oxford University Press.
- S. Schaal. Is imitation learning the route to humanoid robots? Trends in Cognitive Sciences, 3(6):233{242, 1999.
Links
- Research Laboratories
- item1
- item1
- item1
Source Codes
- Learning a dynamical system [Author:Mohammad Khansari]
- Encoding a N-Dim. Trajectory in a Gaussian Mixture Model and retrieving a Generalized version [Author:Sylvain Calinon]
| Invited by: | Dr. Jan Peters, Max-Planck Institute, Germany & University of Southern California, USC |
