Instagram, Step Aside

A new food photo technology is emerging

Researchers at the Massachusetts Institute of Technology's Computer Science and Artificial Intelligence Laboratory have created an artificial neural network — a computer system modeled after the human brain — to examine those millions and millions of food photos and break them down into recipes. 

NPR reports that Javier Marin, a postdoctoral research associate at CSAIL and co-author of a paper published this July at the Conference on Computer Vision and Pattern Recognition in Honolulu said that the growth of the Internet has supported the ability to collect and publish several large-scale datasets, allowing for great advances in the field of artificial intelligence. "However, when it comes to food, there was not any large-scale dataset available in the research community until now," Marin says. "There was a clear need to better understand people's eating habits and dietary preferences." 

Here’s how they did it.  Researchers feed the computer pairs of photos and their corresponding recipes — about 800,000 of them. The AI network, called Recipe 1M, chews on all of that for a while, learning patterns and connections between the ingredients in the recipes and the photos of food. 

Check it out at Pic2Recipe. Just upload your food photo. The computer will analyze it and retrieve a recipe from a collection of test recipes that best matches the image. 

It’s not foolproof according to NPR.  Sometimes it has trouble making fine distinctions between similar recipes. One example: it may detect a ham sandwich as pastrami or not recognize that brioche contains milk and egg.  Another is that the current model has no explicit knowledge of basic concepts like flavor and texture.   

In the future, the MIT researchers want to do more, they are seeking insight into health and eating habits. "Determining the ingredients — and therefore how healthy they are — of images posted in a specific region, we could see how health habits change through time," says Marin. And that’s just the start -

They would like to take the technology a step farther, and is working on a way to automatically link from an image or ingredient list to nutrition information.