Can a model really tell calories from a picture?

It can name the food reliably and quantify it poorly. In a controlled evaluation of 52 standardised photographs, the two better models estimated energy with a mean absolute percentage error of 35.8%, with errors growing as portions got larger. Identification is the strong half; the amount is the weak half.

Why does a photo miss cooking oil?

Because absorbed fat is not visible. Fried and grilled versions of the same vegetable look alike in an image while differing substantially in energy, since the oil is inside the food rather than on top of it. Saying how the dish was cooked corrects the single largest hidden quantity.

Is photo logging better or worse than writing down what you ate?

Better, on the available evidence. A week of app-based photo logging underestimated intake by 1814 kJ a day against measured expenditure, and two interviewer-led 24-hour recalls underestimated it by 1715 kJ, with no significant difference between the methods.

▸ Article

How accurate is a food photo calorie estimate?

Last updated: 2026-10-22

Expect roughly a third off, biased downwards. Tested against weighed reference meals, the better general-purpose image models estimated food weight with a mean absolute percentage error of 36.3% and 37.3%, and energy with 35.8%, underestimating more as portions grew^[1]. That is close to what dietary assessment methods in research achieve^[3], and much better than describing meals from memory, which was 47% off in the classic study^[4]. A photo identifies food well and measures it poorly, so you supply the portion.

On this page

What a photograph can and cannot contain

A food photo carries two kinds of information. It identifies what is on the plate, which modern image models do well. And it implies how much is there, which is where everything goes wrong.

A flat image has no depth, so the height of a mound of rice is a guess. It has no density, so a dense slab of meat and an airy piece of bread of the same outline are indistinguishable. Above all, it cannot see absorbed fat. Two plates of aubergine can look identical and differ by a large margin in energy, because one was grilled and one came out of a frying pan, and the oil is inside the vegetable rather than on top of it. The same is true of ghee stirred into rice and of the oil in a tray of baked vegetables.

So the honest way to read a photo estimate is as a confident identification attached to a weak measurement.

What the validation studies measured

This has been tested properly. One evaluation gave three general-purpose multimodal models 52 standardised food photographs — single components and complete meals, in three portion sizes — with plates and cutlery visible as size references, and compared their answers with the food's actual weighed values. The two better models estimated food weight with a mean absolute percentage error of 36.3% and 37.3%, and energy with an error of 35.8%; the third was far worse, between 64.2% and 109.9% across nutrients^[1].

Two findings in that work matter more than the headline. First, the errors were systematic, not random: all the models underestimated, and the underestimation grew as the portion got bigger. Second, correlation with the true values sat between 0.65 and 0.81 for the better two^[1] — useful ranking, imprecise magnitudes.

A separate comparison of vision-language models on a large annotated image dataset reached the same conclusion from the other side: the models identified the main components of a dish reliably, and failed on quantification, particularly for composite or visually ambiguous dishes^[2].

Composite dishes are the hard case, and they are our food

That last phrase — composite or visually ambiguous — describes a large share of the Egyptian and Gulf table. Koshari is four foods and a sauce in one bowl, with the lentils and pasta hidden under the rice. Molokhia is a green soup whose energy lives almost entirely in the fat it was finished with, which is invisible. A shared platter of rice and meat does not tell a camera how much of it ended up on your plate, and a stuffed vegetable hides its filling by design.

None of this makes photos useless for these dishes. It means the photo needs one sentence of help: what the dish is, roughly how much of it you ate, and whether it was cooked with fat. That single sentence fixes the part of the estimate the camera is worst at.

Compared with what, though?

A 36% error sounds disqualifying until you ask what the alternative scores. The alternative, for almost everybody, is remembering and describing meals from memory — and that is substantially worse. In the classic study of people who could not lose weight on a reported low-calorie diet, actual intake was underreported by 47% on average, and physical activity over-reported^[4].

Image-based logging has been tested head to head against the research standard too. In a pilot study, a week of app-based photo logging underestimated energy intake relative to measured expenditure by 1814 kJ a day — but so did two interviewer-led 24-hour recalls, by 1715 kJ, and the two methods did not differ significantly from each other^[3].

That is the fair verdict. A photo estimate is not a laboratory measurement. It is roughly as good as the methods nutrition research itself relies on, at a fraction of the effort, which is exactly why it is worth using and exactly why it should not be treated as precise.

How to get the most out of a photo log

Shoot before you eat, from an angle, with a fork, a spoon or your hand in frame. A reference object is the only scale the image has.
Say the portion out loud in words: two ladles, a medium plate, half the tray. You know this; the camera does not.
Always mention cooking fat, because it is the single largest invisible quantity in Arab home cooking.
Weigh the one staple you eat most often — rice, bread, oil — once, on a kitchen scale. After that you can estimate it by eye for months.
Judge the week, not the meal. Systematic errors that repeat every day still show the trend; they just shift the level.

Those five habits turn the weak half of the estimate — the amount — back into something you supply, while the tool handles identification and arithmetic. That is also the practical argument for logging in words and pictures together, which is what an Arabic calorie counter built around chat does, rather than hunting for a dish in a database that does not have it.

When a photo is not enough

If a number is going to drive a clinical decision — an insulin dose, a renal protein limit, a prescribed deficit for someone with very little margin — a photo estimate is the wrong instrument, and so is any estimate. Weigh the components and read the label. The same applies to a competitive athlete in the final weeks of a cut, where the margin for error is smaller than the error.

For everyone else, the useful question is not whether the photo is exact but whether your week's numbers are consistent and moving in the right direction. The guide on how your daily calories are calculated explains what the target is measured against, and how to split your calories between carbohydrate and fat shows how much precision the split actually needs. In both cases the answer is less than people fear.

Sources

Try it on WhatsApp