20 March 2014, Ruhr University Bochum, Germany
Prof. Yiannis Aloimonos talk at 5th EUCogIII Members Conference
Abstract:
I introduce a minimalist grammar for human manipulation action. It is a context free grammar involving hands, objects, tools and their movements and relationships. Using this grammar together with signal processing operators for attention, tracking, segmentation and recognition, I show how to create semantic descriptions of complex human activity from perceptual data and then how to create natural language describing the activity. The signals representing the activity are parsed in a way similar to the manner natural language is parsed. I conclude with two experimental demonstrations of this theory using robots in the real world: (a) from language to action – robots executing verbal commands and (b) from action to language – robots observing a human activity and producing a description of the activity in natural language (English).
[Work funded in part by the European Union under the FP 7 Cognitive Systems Program (Project POETICON) , by The National Science Foundation and the National Institutes of Health (USA) and Qualcomm Corporation under an Innovation Prize Grant]
Bio:
The research of Professor Aloimonos is devoted to the principles governing the design and analysis of real-time systems that possess perceptual capabilities, for the purpose of both explaining animal vision and designing seeing machines. Such capabilities have to do with the ability of the system to control its motion and the motion of its parts using visual input (navigation and manipulation) and the ability of the system to break up its environment into a set of categories relevant to its tasks and recognize these categories (categorization and recognition). The work is being done in the framework of Active and Purposive Vision, a paradigm also known as Animate or Behavioral Vision.
Since the early 2000 he has been working on the integration of sensorimotor information with the conceptual system, bridging the gap between signals and symbols. This led to the introduction of language tools into the Robotics community. During the past five years, his research is supported by the European Union under the cognitive systems program in the projects POETI- CON and POETICON++ , by the National Science Foundation under the Cyber Physics Systems Program in the project Robots with Vision that find objects and by the National Institues of Health in the project Human Activity Languages.