That was my initial understanding, which left me confused.
But they're taking the top n according to the model, then taking the top according to the proxy, not actual, objective. This avoids the Winner's Curse problem of top model ranking with reasonable probability.
They are then comparing this to the highest scoring actual preference.
The historic picture makes a little more sense (though this is not something a 5yo would understand).
We call these things embeddings because you start with a very high dimensional space (image a space with one dimension per word type, where each word is a unit vector in the appropriate dimension) and then approximate distances between sentences / documents / n-grams in this space using a space with much smaller dimensionality. So we "embed" the high dimensional space in a manifold in the lower dimensional space.
It turns out though that these low dimensional representations satisfy all sorts of properties that we like which is why embeddings are so popular.
I've read a little bit about it. I think it would be a good idea since we don't need to run the subgraphs on parallel or something like that, therefore eliminating the need for a TF session per se.
I'll be helping Diego with some new models, it'd be awesome if you join :)
I think Keras is a real deal framework. It provides a higher-level API than most other frameworks, but it has pretty sweet portability of models across frameworks and platforms and most research papers are implementable in Keras without too much trouble.
In my opinion, the real deal with Pytorch or Chainer, there are similar than numpy API. So the learning curve is flat. The NN construction part and gradiant part are specific but all the glue is regular python unlike Keras, tensorflow ...
Did you try using SavedModel? It should be seamless to use downstream with tensorflow serving and it's not that hard to get estimators to spit those out.
In fact if you dig up the case, then even official support told me that savedmodel needs some freezing using bazel otherwise it doesn't work.
The github page and stackoverflow are full of these. If you can, please take the message to the other side :(
I don't think the cloud guys (where training will happen in distributed mode) talk to the android guys (where models will be used after quantization). There is a huge serialization problem that all of us are currently struggling with.
Ah, I didn't know SavedModel didn't work in android. I think freezing is still the way to go there? I'm sorry, I don't personally work on the mobile side of things.
I should apologize for hijacking this thread(and i'll stop here). But Tensorflow is getting to be unusable because of the serialization story. We don't have such issues on Caffe2 or anywhere else. It essentially means different parts of the tensorflow ecosystem are unable to talk to each other.
I really pray the tensorflow teams give it due importance.
I'm the original author of the freeze_graph script, so I'm to blame for a lot of the on-going mess here. For what it's worth I'm actively working on cleaning this up, since I know what a painful experience it is. Apologies for everyone who's struggled with this, and I will take a look at the case number mentioned above and follow up internally to see if there's anything I can help with.
Thanks for this! I would like to bring two things to your attention :
1. We don't know what to use and its very confusing. For example, now there is https://stackoverflow.com/questions/42216208/should-tensorfl.... Will freeze_graph become canonical and we forget about SavedModel? And everything else deprecated? It should be part of the core API and workable on CloudML, where we don't have a lot of control on running scripts and certainly not Bazel builds.
2. Android/ios story. Now you have the Pixel Visual Core as well... Please make it seamless all the way to Android or Ios or raspberry pi (whatever you guys support).
Hot damn this has got me all giddy. How will this work on single node multi-GPU systems? For example, with PyTorch you have to either use threading, multiprocessing, or even MPI. Can you think of a not-too-scary way to use eager execution with multiple GPUs?
We're still fairly early in the project, so for now threading is the only supported way.
We can do better, however, and we're working on ways to leverage the hardware better (for example, if you have no data-dependent choices in your model we can enqueue kernels in parallel on all GPUs in your machine at once from a single python thread, which will perform much better than explicit python multithreading).
Stay on the lookout as we release new experimental APIs to leverage multiple GPUs and multiple machines.
I write tests not to convince myself my code is correct (it often is, and repl and ad-hoc testing are more than enough to make sure it does the right thing now) but to prevent myself & future others from breaking it as they maintain it in the future.
