Multisensory guidance of goal-oriented behaviour of legged robots

Publication: Research - peer-reviewArticle in proceedings


View graph of relations

Biological systems often combine cues from two different sensory modalities to execute goal-oriented sensorimotor tasks, which otherwise cannot be accurately executed with either sensory stream in isolation. When auditory cues alone are not sufficient to accurately localise an audio-visual target by orienting towards it, visual cues can complement their auditory counterparts and improve localisation accuracy. We present a multisensory goal-oriented locomotion control architecture that uses visual feedback to adaptively improve acoustomotor orientation response of the hexapod robot AMOS II. The robot is tasked with localising an audio-visual target by turning towards it. The architecture extracts sound direction information with a model of the peripheral auditory system of lizards to modulate locomotion control parameters driving the turning behaviour. The visual information adaptively changes the strength of the acoustomotor coupling to adjust turning speed of the robot. Our experiments demonstrate improved orientation towards the audio-visual target emitting a tone of frequency 2.2kHz located at an angular offset of 45 degrees from the robot.
Original languageEnglish
Title of host publicationHuman-centric Robotics : Proceedings of the 20th International Conference on CLAWAR 2017
EditorsManuel F. Silva, Gurvinder S. Virk, Mohammad O Tokhi, Benedita Malheiro, Paulo Ferreira, Pedro Guedes
Number of pages9
PublisherWorld Scientific Publishing Co Pte Ltd
Publication date2017
ISBN (print)978-981-3231-03-0
ISBN (electronic)978-981-3231-05-4 , 978-981-3231-04-7
StatePublished - 2017
Event20th International Conference on Climbing and Walking Robots and Support Technologies for Mobile Machines - Porto, Portugal


Conference20th International Conference on Climbing and Walking Robots and Support Technologies for Mobile Machines
LocationInstituto Superior de Engenharia do Porto


  • lizard peripheral auditory system, audio-visual localisation, multisensory integration, hexapedal locomotion