I used grpc and grpc-web in a couple of jobs and found the following:
1) difficult to debug (various) but the over the wire being a binary protocol means you can't just look in dev tools and figure out what happened with your request.
2) reinvents everything http already has - literally in the source of the GRPC library it renames and groups all the HTTP error codes that can happen into GRPC error codes that are meaningless.
3) Versioning hell - Protobuf or whatever it uses does not seem to make sure the message and the schema match so if you happen to send or receive the wrong message format after a schema change you might just get weird errors - even a rudimentary checksum could have helped here.
4) What problem are you solving that you need GRPC - the only reason I can see to use it is scale (are you Google?) and fashionability.
5) Bad documentation - there are known ways to do everything with rest or graphql but often you are rolling your own way with GRPC and I was never sure if what I was doing was idiomatic.
What are the industry standard ways to get around these issues with GRPC?
EDIT: sorry to be so negative about GRPC, I personally think it's a missed opportunity is all, in theory it could be amazing. And Sonora looks like an awesome implementation.
gRPC is very opinionated. Protobuf allows some things that tend to be design mistakes (e.g. required fields). A lot of this has been corrected with proto3, with subsequent field behavior annotations described here: https://google.aip.dev/203
The whole of aip.dev is a collection of API design principles published by Google that are externally relevant. Many of the proposals pertain specifically to gRPC-style APIs.
I've personally found gRPC APIs to be pleasant relative to many HTTP APIs since the use of an IDL can be helpful when interacting with such APIs. I don't have enough GraphQL experience to make any conclusions on that comparison. But the sense that I get is that GraphQL may be more appropriate once you reach a certain scale but is probably overused when a simpler technology would suffice.
Disclaimer: I work for Google but the views I express are my own and not necessarily those of my employer.
> I personally think it's a missed opportunity is all
I don't agree with all your criticisms but Google utterly botched the development of gRPC, it could have been much better.
I had conversations with some of the developers about this in it's early days and they seemed totally unconcerned with actually making it usable by people outside of the very specific environments they had in mind.
Sorry you had some bad experiences. Here are some things you could look into if giving gRPC another shot:
1) Its a fault of a team's infrastructure if the binary requests aren't logged, and there is no is no easy way to link up with the message schemas to decode the messages. Make it configurable to log things, and learn the available command line tools to read messages and make gRPC calls. It is readable because text_format exists: https://googleapis.dev/python/protobuf/latest/google/protobu...
2) gRPC error codes are for specific application errors like your tweet failed to send since you said a bad word, or you sent too many tweets in a minute. The semantics of what you need for error handling in your app may not map directly to HTTP error codes which are quite limited. If you need very detailed error handling there is a standard way to attach your own opaque message types for errors: https://github.com/googleapis/googleapis/blob/107a322d4c61b8.... Think saying what offset and length where the bad word occurred so you can highlight it in a UI. What's the standard way to attach any number of errors to a HTTP/REST(/JSON) request? There is none! Just burn developer time inventing a new way each time when you write REST by hand.
3) Sorry can't guess what you experienced regarding versioning, but a common mistake is to reuse and not deprecate field numbers. If one doesn't trust user input, then that goes along with making sure the message makes sense first before using it to affect a change -- never trust a message even if it parses with no error, etc.
4) Single source of truth, both code and documentation, for your API calls, data payloads, and perhaps even logging messages. Code generate the message types /clients for many programming languages. Get text representation for free if needed as well (text_format). Save developer time.
5) Google has an API design guide: https://cloud.google.com/apis/design. Oh, and if it doesn't mention it there, at multiple companies I've worked at there was a rule to always use an enum instead of a boolean type, and the default value should be 0 which should have an "unknown" identifier. More often and not you'll need to record something additional like whether or not the field was set in the first place.
Pretty cool! Honestly, I've found the challenges related to standing up Python-based gRPC services to be super complicated and not worth the hit. I'm not sure this solves that in particular, because most of the pain is around things like TLS, HTTP2, proxy and loadbalancer configuration, and other things I'd love to not have to spend hours worrying about just to massage the bytes right.
This is a long-winded way of saying that I used to really love protocol buffers and gRPC but at this point am a little soured on them in practice.
gRPC-Web is actually exactly what you want if you want gRPC but none of the TLS, HTTP2 etc pain. It "just works" on HTTP/1.1 like any other server.
The Google libraries are definitely a pain to use in Python and even if you've got all the HTTP2 stuff sorted already it doesn't play that nicely with existing code bases.
Hey, I haven't tried using gRPC with Python yet but I had been considering it recently and now I'm worried. Could you give me an example of what sort of trouble I should expect? What sort of trouble is it going to give my existing code base?
The first issue I've seen people struggle with are to do with generating the Python bindings in the right modules. Getting the right combination of arguments to protoc isn't always obvious.
The main complexity with an existing code base is that gRPC runs its own ioloop so it controls the server entirely so you can't glue it into an existing Python webapp easily. You can call all your existing Python functions etc just fine though.
Nifty. I assume this still means using some combination of Envoy + Nginx to "downgrade" to 1.1 in a traditional web service environment if the client defaults to HTTP/2. Is that right?
I don't think there is actually a good solution for connecting a regular H2 gRPC client to a gRPC-Web server currently. Envoy's gRPC-Web filter only seems to go the other way. I haven't tested it but the old HTTP/1.1 reverse bridge might work. [0]
I'm currently working on a python-based gRPC project. I use envoy to handle all the stuff you mentioned. I do like gRPC compared to REST, although I wish there were more native data types for gRPC, e.g. UUID and Decimal/Numeric - I create types for these ala google's string marshalling for decimals, but it means therefore I must do my own data validation.
So, how reliable is gRPC-web? Is it production ready? Can it be a good replacement for AJAX? I have been itching to use gRPC because, I just hate REST and everything JSON.
1) difficult to debug (various) but the over the wire being a binary protocol means you can't just look in dev tools and figure out what happened with your request.
2) reinvents everything http already has - literally in the source of the GRPC library it renames and groups all the HTTP error codes that can happen into GRPC error codes that are meaningless.
3) Versioning hell - Protobuf or whatever it uses does not seem to make sure the message and the schema match so if you happen to send or receive the wrong message format after a schema change you might just get weird errors - even a rudimentary checksum could have helped here.
4) What problem are you solving that you need GRPC - the only reason I can see to use it is scale (are you Google?) and fashionability.
5) Bad documentation - there are known ways to do everything with rest or graphql but often you are rolling your own way with GRPC and I was never sure if what I was doing was idiomatic.
What are the industry standard ways to get around these issues with GRPC?
EDIT: sorry to be so negative about GRPC, I personally think it's a missed opportunity is all, in theory it could be amazing. And Sonora looks like an awesome implementation.