Your app kinda sucks now, but the detection is great. You could also do food identification as a service...

I'm trying to use fatsecret/myfitnesspal for counting calories. Something like this could make it much easier!

To me, a better workflow would be: take pictures of everything you eat and classify later (at night, maybe on the PC). Of course, this software would simplify the classification, the user would only fix any errors and adjust sizes.

This workflow would be well suited to your app, which operates on a server? (my understanding).

Congrats, I understand the relevance of AI for simple apps now ;)

Ive been thinking of that 'classify later' workflow. I think of it kind of like facebooks photo tagging, and it might even be better as web app that you get email reminders for to complete entries.

It would also work nicely for background geolocation services, since if the photograph was taken near a restaurant the service could extrapolate those things.

Anyways, I might make this project opensource and build a nice community around it. Depends on whether or not that model would best support my research goals.

Great! And think about using the new data from the users to improve the system (that should probably be in your terms).

How well does it work on plates with multiple foods "kinda mixed"?

Thats another huge design challenge Ive faced, specifically because it really complicates the data model and UI.

Implicitly, since the app presents a list of possible identified foods, and the user can easily check multiple boxes, thats straightforward, but then having the app pull each of those items AND ask the user to estimate portions for each .... gets a little messy.

So I put that complexity off for v2. That use case in particular would also be best off for a web-interface, I think.

What about dividing the whole plate weight by the foods detected? (in Brazil there are many restaurants where you pay by weight, thats why I'm suggesting..)

Theres a company in the bay area trying this approach actually! They look really interesting: https://getsmartplate.com/

Cant compete with that awesomesauce!

You could probably outsource enough of the UI work to Fiverr to make a really decent looking app.

Things like Main screen, App Icon and colour palette.

My color palette was carefully engineered to induce vomiting and weight loss! :p

Laughed for a disconcertingly long time at this.

Wish you luck with the app, really cool idea!

Where'd you get your images? Food-101 plus something else?

We're working on a similar component for our commercial behavioral-economics-driven app suite, targeted toward patients with chronic diseases.

Do you use location as a way to filter the set of possible foods, as in Google's im2calories project/paper last year?

They do some pretty awesome depth calculation stuff too: https://www.google.com/?ion=1&espv=2#q=im2calories+type:pdf

Edit: oops, should have read further. I see that you pulled appropriate terms from WordNet and used ImageNet to gather training images from Flickr. Cool!

I would love a blog post about that!

Any info on the network itself (number of layers, input resolution, pooling, dropout etc)?

Stock Cafenet, trained from scratch.

How much experience with machine learning did you have before you started this project?

Cut my teeth on the netflix prize several years back. Took ML in my graduate coursework, and have dabbled with it on and off since. I love the whole theory of matrix factorization and its myriad uses.

Very cool. What do you think helped you most? Would you recommend grad school?

why caffe ? why not something like h2o or tensorflow ? just curious.

because Im a newb. Looking forward to expand my repitoire!

Is not Caffe actually GoogLeNet? So technically it is Google architecture.

Login via Facebook? Come on! See you next app.