I1-[OntoSpace] - Research

Human-Robot Interaction

The goal of this part of the project I1-[OntoSpace] is to explore how humans interact with robots in order to communicate spatially relevant information. In this field, the wider area of variability in spatial reference meets the peculiarities of communicating with artificial interlocutors. Spatial reference can occur with a variety of dialogic strategies, reflecting diverse levels of description varying between goal-based, path-based, and action-based instructions. Goal-based reference builds on principles and preferences in achieving spatial localisation and identification. Results of our experiments show that speakers establish spatial object reference by contrasting the goal object sufficiently from competing objects in a scenario, using a broad range of reference systems underlying a considerable variability in linguistic forms employed for the description. Interpretation problems arise mainly because of underspecification, ambiguity, and many-to-many mappings between linguistic forms and underlying concepts, which furthermore differ between English and German.

In addition to the complexities involved here, preliminary findings indicate that there are different ways of conceptualising both the spatial environment and the artificial communication partner (viz. the robot). That is, the robot may be conceptualized as a tool or as an embodied human-like interlocutor. Also, speakers treat their communication partner as someone who they do not know enough about. Finally, the results show that we can find frequent questions, both directed to the leader of the experiments and to the system itself, regarding those kinds of information speakers consider as prerequisites for formulating their instructions. These clarification questions may cause serious problems for dialogue systems because they can occur basically anytime, and their contents often involve out-of-domain vocabulary. Starting points and a view of the state of the art in dialog modelling are available here.

Here is a demo video with extracts of our Wizard-of-Oz experiments.

Robots

Aibo
Rolland
Box
Scorpion see A6-[ReactiveSpace]
The Pioneer, "speedy" (with laser scanner)
The Pioneer, "giraffe"(with camera/tripod)

Experiments

No. Name Date Type Participants, corpus size Mode Task Description Download
I1-7 Rolland_2 Jan/ Feb 2006 HRI using Rolland 51 participants speaking native German and native & non-native English spoken Object localisation (static and dynamic); route instruction Verbal robot reactions; Robot movements are controlled by participant sample, sound1, sound2 (1.3MB)
I1-6 Aibo_2 Jun/ Jul/ Nov 2004 & Jun/ Jul 2005 HRI using Aibo, Box 37 German approx. 25 min. each, 20 English spoken (& some written) Object identification, partly sequential; movement Verbal robot reactions and movement; Comparison between two robots sample
I1-5 Rolland May 2004 HRI using Rolland 27 German approx. 20 min. each, 11 English spoken Object localisation (static and dynamic); route instruction Only input by participant, no reaction by robot; Robot movements are controlled by participant sample
I1-4 Web-experiment Oct 2003 - Oct 2004 Web-based corpus collection >180 native German, ~ 3000 contrib.; >200 native English, ~ 3500 contrib.; >100 nonnative English, ~ 1700 contrib.; approx. 10-15 min. each typed Object identification; route instruction Participants type spatial descriptions according to the given task with respect to various scenarios sample
I1-3 October Oct/ Nov 2003 HRI using Aibo, Scorpion, and Pioneer Speedy Aibo: 11 English, 23 German; Scorpion: 8 English, 22 German; Pioneer: 8 English, 33 German; approx. 20 min. each typed and spoken Object identification, partly sequential; movement Robot sometimes gives scene description; No detailed verbal feedback; Validation of improved implemented computational model of spatial relations; Additional test in HHI (a human is instructed) with 9 participants transcription movie1
movie2 (4.6M)
I1-2 Pioneer Experiment 2 Jul 2003 HRI using Pioneer Speedy 29 German approx. 15 min. each, 482 instructions Complete corpus (xml) Complete extract of all dialogues (html) typed Object identification; movement Robot always gives scene description; Fairly detailed verbal feedback depending on modules; Validation of improved implemented computational model of spatial relations transcription movie
Documentation file (pdf)
I1-1 Pioneer Experiment 1 Feb 2003 HRI using Pioneer Speedy 25 German approx. 25 min. each, 1382 instructions Complete corpus (xml) Complete extract of the human-robot interaction data (html) typed and spoken Object identification; movement Robot sometimes gives scene description; No detailed verbal feedback; Validation of improved implemented computational model of spatial relations; Additional test in HHI (a human is instructed) with 9 participants transcription movie Documentation file (pdf)
  Erlangen 2003 (elicited at Er-langen) HRI using Aibo 51 German approx. 30 min. each spoken Object identification, partly sequential; movement Cooperation with Erlangen research group (Anton Batliner); participants: schoolchildren; Only non-verbal robot reactions (movement) sample
  Nats 2002 (prior to official project start) HRI using Pioneer Speedy 21 German approx. 15 min. each, 738 instructions typed Object identification; measure Pre-determined automatic verbal robot reactions sample
  WSV 2000 (prior to official project start) HRI using Pioneer Giraffe 15 German, approx. 15 min. each, 467 instructions typed Object identification; movement Exploratory study; Only non-verbal robot reactions (movement); Validation of implemented computational model of spatial relations sample