VI Lab's Dr Dima Damen awarded prestigious 5-year EPSRC Early Career Fellowship
16 December 2019
The EPSRC Fellowship will start in January 2020 and extend VI Lab's strong research track record in understanding object interactions, which includes leadership of the largest dataset in egocentric videos 'EPIC-Kitchens'.
The £1.3M grant entitled ‘UMPIRE’ targets the perception of human-object interactions (HOIs) from visuoauditory wearable sensors. Humans interact with tens of objects daily, manipulating objects or utilising these as tools and equipment. Automatic perception of HOIs is currently hindered by limitations of current computer vision models. Models are lacking in their modularity and accordingly their capabilities. To achieve human-level perception of object interactions, this fellowship focuses on informing deep learning architectures from well-studied cognitive behavioural frameworks.
The fellowship aims to design an ensemble of deep models, based on knowledge of perception modules and their dependencies from multiple cognitive frameworks. The ensemble will be jointly trained from videos of realistic interactions. Dependencies will then be reshaped through data-driven structural evolution, optimised for both passive and actionable perception. This will integrate two disciplines that have been only lightly intermingled, due to the initial focus of deep learning on features and optimisation. The integration will be guided through collaborating with two leading cognitive scientists - Professor Iain Gilchrist from the University of Bristol, an expert in active vision, and Professor Bernhard Hommel from Leiden University, the originator of the Theory of Event Coding.
During the fellowship, Dr Damen will be closely working with industrial collaborators at the forefront of the need for perception of human-object interactions. NVidia will partner to integrate the UMPIRE within their robotics simulation platforms. Samsung AI Centre Cambridge (SAIC) will utilise UMPIRE for assistive technologies within wearable prototypes.
Professor David Bull, Director of Bristol Vision Institute and Head of the Visual Information Laboratory said, “This is a fantastic achievement, built on many years of dedication by Dima in creating a world-leading area of research. It reflects the spirit of collaboration and interdisciplinary research promoted by BVI - embedding perceptual processes into machine vision applications and combining fundamental research with important application drivers.’