Natural language is one of the pivotal achievements of human intelligence. We have not understood one of the most fundamental aspects of human nature until we can build artificial communication systems similar in robustness, flexibility, and open-endedness, I am dedicating my research to solving the elementary puzzles that surround natural language. How do we put the world into words? How is language processed? How is language acquired by learners and how does it change on short and long timescales?

I explore these issues using autonomous robots that develop and evolve aspects of human language such as spatial language, action language, and tense-aspect systems. My work leads to artificial systems remarkably resilient to noise, perceptual deviation, ambiguity and unorthodox language use. Results of my research are applied in building artificial assistants, and improving human-computer interaction and gaming. Recently I am working on digital communication partners people can trust.

The following few paragraphs highlight some of my recent research in more detail.

Development of Communication (Developmental A.I.)

Infants around the age of 10 months start actively developing symbolic communication skills, because sensory-motor intelligence (visual perception, body movement, navigation, object manipulation, auditory perception and articulatory control) is already beginning to reach a high level of competence. Then three major transitions can be seen:

  1. The discovery of gestural symbols with increasing vocal productions;
  2. The vocabulary spurt (18-24 months) during which children’s lexicon rapidly grows at the same time the start of multi-word utterances
  3. The development of fully productive grammar (starting around the age 3)

We try to emulate these stages in robot experiments. Results have been published in various papers

Evolution of Language

Spatial language is an ideal testbed for instantiating theories and models of language evolution on embodied artificial agents, i.e. robots. The reason is that spatial language is directly related to objects in world. Despite being so perceptually grounded and cognitively central, spatial language is to a large extent culturally conditioned. This not only true for the lexicon and grammatical aspects but extends all the way to the conceptual structures a language affords. Tzeltal, a Mayan, language impressively demonstrates this through its sole reliance on absolute spatial relations such as uphill/downhill and lack of projective categories such as front/left. I have developed a series of models researching individual aspects of spatial language. Together they form a comprehensive theory of the evolution of a key aspect of human language.

Grounded spatial language processing

Formation of spatial category systems

Co-Evolution of syntax and semantics

There are also a number of shorter overview articles discussing the research approach in general. See for example the following article