The software, which uses an ‘artificial net’ to map key features of the face, can evaluate the intensities of multiple different facial expressions simultaneously. The University of Bristol and Manchester Metropolitan University team worked with Bristol’s Children of the 90s study participants to see how well computational methods could capture authentic human emotions amidst everyday family life. This included the use of videos taken at home, captured by headcams worn by babies during interactions with their parents. 

At the clinic, as a part of the ERC MHINT Headcam Study, parents were provided with two wearable headcams to take home and use during interactions with their babies. Parents and infants both wore the headcams during feeding and play interactions. 

They then used an ‘automated facial coding’ software to computationally analyse parents’ facial expressions in the videos and had human coders analyse the facial expressions in the same videos. 

The team quantified how frequently the software was able to detect the face in the video and evaluated how often the humans and the software agreed on facial expressions. 

Finally, they used machine learning to predict human judgements based on the computer’s decisions. The findings show that scientists can use machine learning techniques to accurately predict human judgements of parent facial expressions based on the computers’ decisions. 

Read the full University of Bristol news item

‘Quantifying the efficacy of an automated facial coding software using videos of parents’ by Romana Burgess et al. in Frontiers.