Funded by EU's

7th framework programme
A robotic agent, built by an international team lead by researchers at Reykjavik University in Iceland, is pushing the boundaries of artificial intelligence by automatically learning socio-communicative skills. It’s a big step towards the ultimate goal of creating intelligence that is both self-sufficient and adaptable in a wide varety of environments.

1 3 next –>

Artificial Interviewer

The system the HUMANOBS team have created is a comprehensive cognitive architecture, named AERA (Auto-catalytic Endogenous Reflective Architecture), that achieves higher levels of autonomy than prior architectures. One way to measure this is with the ratio between the amount of things learned and the amount of a-priori knowledge its human designers must provide. The HUMANOBS team decided to put their system to test based on the scenario of a TV interview: “We needed a task that would be viewed as relatively complicated – impossible for current AI and complex for even a human to learn,” explains Kristinn. Socio-communicative skills were chosen as a way to evaluate the power of the system, a complex enough task to convince anyone that their new AI was presenting some new and powerful principles worth taking note of. “We didn’t want our AI to require extensive hand-coding like the expert systems of the past, but instead be able to acquire the data for their programming on their own, and then to manage their quote about learningown growth through self-programming.” To demonstrate the abilities of the system, the system observes an interview between two humans engaged in a mock-up TV interview about waste recycling. The interviewer asks the expert interviewee to talk about the objects on the table in front of them.

“After about two or three minutes of watching the humans do this,” says Kristinn, “the system starts to understand what’s going on, how to structure and conduct such an interview, and has generalized some of the main principles of human communication to a point that it can be asked to take over the interview, either in the role of the interviewer or interviewee, and continue to interact with the other person.”

As giving computers vision through cameras is an unsolved problem, human movement and speech is tracked with special high-accuracy sensors and microphones, S1 agent interviewing a humanwhich capture human interaction in realtime for replication in a virtual world. Two avatars represent the interacting humans, each one seeing the other's avatar on their screen, not unlike a video conference call if you were to see the other person as a three-dimensional graphical avatar that precisely mimics everything the other does. "This setup allows us to represent the interaction as a stream of accurate digital data, which the AI observes in realtime, as the interview unfolds” explains Kristinn. The virtual scene captures human natural behavior in all all important spatio-temporal details, down to arm, head, and body movements, including hand gestures.”

Achieving Domain Independence

Kristinn says it is clear that the system is able to learn “many nuances” of human social behavior “such as the co-ordination of gaze and head movement and speech in the service of conducting an interview”. However, the system is not specifically designed to learn socio-communicative skills – it is general enough to learn any skill of similar complexity. While capturing the behavioral signals of human communication was certainly one of the challenges of the project, an even greater challenge was to build the system so that the learning mechanisms would not be limited to a particular domain but rather work for any task in any scenario.

Achieving domain independence required new levels of system integration and synthesis of ideas. Although the foundation of the project is software engineering and computer science, significant shortcomings in the accepted methodological approaches these fields offer were not sufficient to enable the project to realize these goals. To push further a wide array of unorthodox ideas from various sources, including cybernetics, mathematics, and non-axiomatic logic, were pulled into the mix. The result is an unusual system architecture that breaks with prior software architecture traditions in many ways. “We have come out with a very un-modular system,” explains Kristinn. “It is fairly difficult to explain because doing so one must build on concepts that are not part of any scientific field's vernacular except perhaps biology. In some ways our system is like a car engine. If you take anything away the rest will not work. While you could say a car engine is modular because you can take pieces out and look at them, well, so can you with our system. But as far as the operation is concerned, getting from A to B, that requires everything under the hood as well as the axle and the wheels. But because the system must autonomously acquire vast amounts of knowledge – in a way program itself, an analogy to an automobile engine falls really far off the mark. Coming up with a unified solution has been the greatest challenge,” he continues, “but a necessary part of the solution, as anything else would introduce debilitating slowdown in performance, architectural complexity, or both.”

At the heart of this success is AERA’s ability to develop its own level of understanding, through both accumulated experience and internally simulated predictions of how the world works.

“Our system can accept goals from its designers and then, depending on how much extra information these designers give, it will come up with sufficient understanding of the phenomenon in question to actually meet those goals in a complex scenario. S1 full screen

This is a key feature of AERA and it’s a very practical one because now you can automate tasks that couldn’t be automated before. You essentially give it a small example description of a problem domain and the goals that you want it to achieve in it, and hit ‘run’.”

Kristinn and his collaborators are surprised by the level of domain-independence they have achieved with the AERA system. “Find anything in the real world that is as complex as a human to human interview," says Kristinn, “such as ploughing fields or picking lemons - actions that are fairly complex and operate in under real time constraints in the real world – and our system can be applied to that.”

An Intelligent Future

Despite all of its challenges the success of the project has been, according to team members, beyond everyone's expectations. “The ambitious goals originally set for the project have all been met. We are achieving all our hopes and dreams for this project,” says Kristinn. “The system that we have come up has already demonstrated the ability to do a small-scale version of a real human-human interview, a task that no prior system could even hope to achieve, as existing methodological and theoretical assumptions simply don't allow it. We have very high hopes for scaling this up to very complex human interaction and, more importantly, other complex tasks in a vast number of other domains.” The potential application for the system developed by the HUMANOBS project is extensive, not least because of the system’s ability to deal with distractions and unexpected situations. Development of this system could be applied to a myriad of complex situations such as underwater exploration or manufacturing.

With the HUMANOBS team riding high on their success with AERA, Kristinn says they are keen to build on this to keep their lead. “We are currently looking at the next steps – we would like to see the technology developed further within academia, as this is a platform that can shed light on a number of complex cognitive processes such as learning, attention, autonomy, and intelligence itself. We would also like to see its application in various domains, sooner rather than later, realizing the next generation of automation systems, and possibly bringing us one step closer to a science fiction future of very capable robots able to help with complex real-world tasks – in medicine, in manufacturing, in disaster relief – in all sorts of important situations.”

1 3 next –>



RU logo

cadia logoCenter for Analysis and Design
of Intelligent Agents
Reykjavik University, Iceland

aslab logo
Autonomous Systems Laboratory
University of Madrid, Spain

roboticslab logo
Robotics Laboratory
Department of Engineering
& Informatics
University of Palermo, Italy

cmlabs logo
Communicative Machines

Edinburgh, United Kingdom

cnr logo
Institute of Cognitive Sciences
& Technologies

National Research Council, Italy

idsia logo
Dalle Molle Institute for Artificial Intelligence

University of Applied Sciences
and Arts of Southern Switzerland (SUPSI) and the University of Lugano (USI), Manno-Lugano, Switzerland

cognitive systems and robotics


humanobs banner
RU logo 2010