View all news

I spy with my little eye….peas, chips and pie

Laura Johnson plate of food Catalyst Case Study

Can you identify foods and estimate portion sizes shown in this meal? (see bottom of page for answers)

Laura Johnson Catalyst Case Study

Dr Laura Johnson

25 July 2016

One of the biggest problems in nutrition research is measurement of individual food intake. A novel crowdsourcing approach developed at the University of Bristol allows crowds to identify food groups and portion sizes, from which total meal size and dietary patterns can be computed, in meal photos collected using a smartphone app.

Dietary intake is one of the most difficult factors to accurately quantify in epidemiology and in clinical studies of obesity and diabetes. In most studies food is still reported using pen and paper in diet diaries or food frequency questionnaires which impose a significant participant burden and are subject to non-compliance or problems with recall. Having discovered the idea of crowdsourcing from Professor Ian Craddock (Director of SPHERE), Dr Laura Johnson, Nutritional Epidemiologist and Lecturer in Public Health Nutrition at the University of Bristol, realised that this relatively new concept in science, had never been used for UK dietary assessment.The primary aim of her research was to create a crowdsource platform for coding meal photos. Participants would photograph their meals and a mixture of untrained crowds and expert researchers would visually assess and determine the size and nutritional value of the meal pictured. Whilst in terms of the accuracy of classifying foods and estimating portions sizes this new system would be comparable to existing methods (semi- automated with user feedback or expert researchers), participant burden, cost and time required for analysis will all be reduced. 

With the support from the Elizabeth Blackwell Catalyst Fund, Dr Johnson and her multidisciplinary team of colleagues including nutrition specialists, clinical psychologists and computer scientists built on formative work already completed by US research groups to make UK-specific adaptations to an existing image capture application and develop a new food estimation system to crowdsource dietary information from the images.

The team created the FoodFinder Tool where crowds of 50 people at a time would view meal images and answer questions about what they could see in return for a small payment. FoodFinder was populated with 33 meal images that had undergone expert food analysis and where the weight of the meal was measured. Finally, after linking FoodFinder with Prolific Academic,  a large crowdsourcing platform tailored for research, the wisdom of crowds for identifying foods and portion sizes was tested. Overall the crowd underestimated true meal weight by about 65 grams (15%) and were less accurate than the experts who overestimated by just 3-9%. Crucially the cost of crowdsourcing data from meal photos was just £3.35 and took just 3 minutes per photo (when a crowd of just 5 were used) compared with £19.20 and 1 hour per photo when paying experts. 

An important part of the project was to test the feasibility of a photo-based dietary data capture system. To achieve this, Dr Johnson’s team collaborated with Dr Corby Martin, Clinical Psychologist at the Pennington Biomedical Research Centre and creator of the iPhone Smart-Intake app. The US-developed app was adapted for UK use by first making it available in the UK iTunes store and then integrating the UK National Diet and Nutrition Survey food composition databank thus allowing the Remote Food Photography Method expert team of meal photo food coders to estimate nutrient intakes based on UK foods. Dr Johnson’s team then enlisted 25 pregnant women taking part in the Avon Longitudinal Study of Parents and Children Next Generation Study (ALSPAC-G2) study to photograph and upload their meals using the app. Most participants found installation and set-up (95%), taking photos of meals (70%) and receiving reminders (81%) easy or very easy. In total 484 photos of meals were taken and these are now being analysed by the expert team and FoodFinder. 

Dr Johnson concludes that further development of FoodFinder to improve how well crowds agree on food identification as well as offering a broader range of portion sizes could make rapid low-cost analysis of meal photos via crowdsourcing a feasible method for measuring food intake. Further integrating SmartIntake and FoodFinder would create an end-to-end system of data capture and analysis that could considerably reduce research costs. Two abstracts were presented to the Nutrition Society conference in July 2016, and a paper on the accuracy and reliability of FoodFinder is in progress. Data on the implementation of SmartIntake in the ALSPAC G2 cohort will be used in the development of future grant applications for wider scale data collection. 

Capitalising on the wisdom of crowds could significantly improve the feasibility of collecting diet information from all generations in the ALSPAC study, other large cohorts as well as clinical trials. The support of the EBI catalyst fund was central in helping me bring a multi-disciplinary team together to kick start solving the challenge of knowing what people are eating.  In future, by collecting and analysing lots of meal photos using human intelligence we’ll be able to train computers to accurately recognise foods, crowdsourcing is a crucial step along the way to making that a reality'.

Further information

For more information on research by Dr Laura Johnson  please visit the website

Please visit the EBI Website to learn more about the funding available from the Elizabeth Blackwell Institute, including Catalyst Fund.

Answer: Baked sweet potato 227g; Bean chilli 479g and cheddar cheese 21g

Edit this page