Never tried with MsgPack, but I did try Protobuf. The reason I dislike Protobuf is that there is an extra code generation step I need to do. When using a compiled language, I guess you don't really mind the extra code generation step since you also get some safety from the compiler so you don't wind up miss-using the generated code. It is not the same when you are using an interpreted language that doesn't have really good strong typing. You will have to run a static code analyser and be very careful every time you do a change to the interface.
Since in my experiments I was calling a method in Php through RabbitMq from a Rust worker, so it just proved allot simpler to just use json. Also, I measured the time it took to:
1. Make a request to the Php API
2. Php sends the rpc message on the queue
3. Rust processes that message
4. Php catches the response from the worker
5. Php returns the response to the http client
It was <10ms running the cluster locally regardless if I used Protobuf or simply json.
One observation / warning I have WRT msgpack is that I have seen situations where data getting serialized directly to msgpack can't be rendered as proper json. Specifically JSON requires some proper encoding of characters while msgpack is perfectly happy to convey some binary characters.
The specific situation I saw was
1) antique perl application barfs a sql dump into msgpack
2) fluentd takes that, turns it into a thing like json but with a control character in it (think old time cyan blinky happy face or \0x03)
3) that thing gets dumped into kafka
4) logstash pulls that thing out of kafka and tries to feed it to elasticsearch
5) elasticsearch reports that it doesn't like the blinky happy face, generates an extensive log, and stores the event minus the key/value pair with the blinky happy face.
Certainly many of these steps have an implicit "don't do that!" or "Update to a newer thinger!" but the root cause (perl serializer into msgpack renders a thing that's valid msgpack but not valid json) is surprising in an unpleasant way.
