I wonder if Jaques tried to do dropout to increase the efficiency? Also if he made the trained neural net available you could made a lego identifier app.
I've been thinking about making such an app. There are some problems in using the net as it is to power an app: the machine has a uniform background as well as multiple images of the same piece. I don't think the net as it is right now would easily work to recognize images taken with a smartphone. But if I had access to a large number of such smartphone taken images I do think that I could use the method I have right now to help label an initial training set from which a much better network could be derived.
Could you artificially vary the background of your current training set to create a larger and more varied set of backgrounds to train against initially. That is, if it's a uniform color or pattern just treat it like a green screen. Then use real photos with different background and lighting conditions to refine it later?
Yes, this is possible, already I know with fairly large accuracy where the part is in the image and changing or texturing the background is possible. But the optical setup is quite complex and hard to replicate with a smartphone, that would need some work.
You may be able to normalize the labeled images by removing the background. I found some OCR examples of doing that with bluring and subtraction. The same normalization steps could be run on smartphone images taken with a consistent background.
