Had a friend who came to me with the same idea. In challenge of the idea I sent a picture of some chopped cucumbers and tomatoes, and an other picture with the same chopped vegetables drizzled with a copious amount of olive oil. I asked if they can tell which one is which. The difference was hundreds of calories and you could not tell which one is which (least of all how much oil there is on the one with the oil.)
That of course feels like a "weird" edge case, but it illustrates the general problem that butter/oil/sugars can pack a lot of calories and have no or almost no visual signature.
You're right, but I think the general public is largely ignorant about calories and nutrition. Even basics like 4 cals per gram of protein and carbohydrate, 7 for alcohol, 9 for fats. Or even what a carbohydrate really is versus a "sugar", which people talk about as bad.
There's a sweet spot for an app that is inaccurate with a market that wants it but doesn't understand how inaccurate it is.
Kind of like how I could vibe code an app, get it to "work", think it's great and be ignorant of the many ways it will break or isn't working that a knowledgeable developer could.
The general public's understanding of food is still stuck in 1990s "destroy all fats (and completely ignore the sugar we replaced it with)" diet culture.
The fact that people still believe in reducing fat as it's own goal (instead of being an easy way to reduce calorie content) is a testament to how bad the public is at identifying fact from fiction.
Then you have shit like the influencer foods, "Feastables" and "Hydration beverage" Prime, which is just flat soda. It's pathetic.
Or think of all the dude bros who insist on dry scooping cup fulls of protein+caffeine powder, and going home to gorge themselves on two pounds of chicken breast, and yet doing absurdly normal amounts of weightlifting or exercise that requires no modifications to their already protein overloaded American diet.
Diet culture is what is fucking American health. People read fucking tabloids that bad-faith regurgitate poorly done "science", funded by the council for selling more food, and insist that since "Woman's Health" says that scientists say chocolate both kills and saves you, scientists are dumb and know nothing, even though THE ACTUAL SCIENCE NEVER CONCLUDED ANYTHING, because the scientific paper was just an observational study!
Related is the fear some have of Saturated Fats and Meat. So many people don't even realize that the body does need some Saturated Fats, or that fats are often a mix of them and vary by the species, breed and how they're fed.
It doesn't help in how far some foods have been bred in the past century and a half in particular. Or how different people with different genetics may react to certain foods.
Everyone keeps using this example, but you have the exact same problem calorie counting any food that might be lathered in fat. It's why people underestimate their calorie intake by 30%+.
You could level the same criticism at Cronometer and MacroFactor when you try to log food you received at a restaurant. Yet those apps are still useful (and I think requisite) for knowing what you're eating. And you should probably 1.5x the calorie estimation when you eat out.
What's interesting is whether this app can accurately estimate food at all. If it can, then that's a huge win and you can add your own buffer zone for oils like you already have to do when you count calories. ...Or chill on the butter and restaurant food when you're supposedly trying to lose weight.
> Everyone keeps using this example, but you have the exact same problem calorie counting any food that might be lathered in fat.
Not really. In practice you need to know the ingredients to estimate the caloric value. Either because you prepared the meal, or because someone who wrote the recipe of it calculated and wrote it on the packaging/menu.
> If it can, then that's a huge win
But that's the point of the example. That it can't. If it could, that would be good. But it can't do it, and not because the app is deficient in some way, but because the necessary information is not available in the image.
> Or chill on the butter and restaurant food when you're supposedly trying to lose weight.
Yes of course. And that is one of the things you learn when you do calorie counting. The practice drives home that message, and many others. But you are not going to learn that if the app hides the signal from you.
The 1st issue I'd think of would be scale.. eg. identical looking plate of food in EU vs USA. The images might look identical but the USA one would be scaled up and contain much more calories.
Everything that could be in the image for scale reference would also be scaled up along with the food.
That of course feels like a "weird" edge case, but it illustrates the general problem that butter/oil/sugars can pack a lot of calories and have no or almost no visual signature.