But your services have to support the Connect protocol. You can also ship services that support gRPC-Web directly too & run without a proxy.
This is definitely a step up, nicely convenient. But also, it has downsides too. gRPC is incredibly well supported by a huge number of observability tools and service-layers/service-mesh systems. Few have support for gRPC-Web. Which if any have support for Connect-Web?
In general, it's just massively sad that the browser never got around to making HTTP Push useful for protocols. gRPC should have been positioned to do great things, was a very common-sense overlay atop HTTP2 that provided just a little more semantics to say how things ought work. It triangulated nicely on where we were going, at it's inception, back in 2014/2015. But the browser basically never evolved, never gave developers access to do the good stuff we thought we'd start to have access to. I still see brand new specs that shout out to the Low Level Extensibility Manifesto[1], but for some reason, in terms of the most base layer of the web, HTTP, this simple guidance has been roundly ignored & the capabilities were never delivered. So gRPC has been stuck with a pretty ok protocol, & frustratingly discompatible browser.
Yeah this is the deal breaker for me as well. This smells like an embrace-extend-extinguish campaign frankly?
1. Embrace gRPC and protobuf
2. Extend with Connect, get everyone using this in their APIs and web apps
3. Drift away from established gRPC and/or protobuf standards, build platforms/business around this and trap as many folks as possible
As silly as it may seem, one thing that really sends this signal for me is the nice trademark symbol on the Buf name. their intention for all of this stuff is to build a business.
In many ways, you're spot on. We are trying to build a business: a schema registry for Protobuf. We think that schema-driven APIs makes a lot of sense, that Protobuf is the de facto best schema language, and that the biggest gap is good tooling to share and leverage the schemas.
To retire in Scrooge McDuck-sized vaults of gold,* though, we need to make Protobuf super awesome so that individual developers and companies are willing to use it. It's not super-awesome today, and gRPC is often even worse. To everyone's benefit (I hope), we have the funding and runway to tackle this problem.
From our perspective, your plan looks more like:
(1) Meet everyone where they are, which is mostly Protobuf v3 and gRPC. Adopting a new implementation of a standardized wire protocol in some portion of your system is risky, but not crazy.
(2) Offer a graceful upgrade path to an RPC protocol we think is better, and genuinely solves many pain points with the current status quo.
(3) If we're wildly successful, influence the Protobuf language community whenever a v4 is under discussion. There are certainly warts to fix and nice features to add, but stability and widespread support have a powerful magic of their own.
Good intentions are hard to prove, of course. Hopefully releasing good, well-supported OSS code can be a first step in building some trust.
* Kidding about the piles of gold. I wouldn't say no, but this is the most fun I've had at work in a decade :)
gRPC is also just HTTP too, yet these tools go much much further to provide good observability of the specific gRPC transactions happening over HTTP. With this application level protocols flowing atop the hypertext transfer protocol, being able to reach in & see what the app protocol doing is invaluable for observability. So no, just about none of these tools will have actually good Connect-Web support. Being able to see streams, parameters, a host of other capabilities is unlikely to just work fine.
Unless you're all-in on server streaming, the Connect protocol is indistinguishable from RESTful HTTP to this class of L7 proxies - it's just a POST to a Protobuf-looking path, with an `application/json` or `application/proto` Content-Type. Most of the proxies/service mesh things that provide rich gRPC instrumentation are also capable of providing rich instrumentation for this style of HTTP.
I dunno. I'd like to see some comparisons. HTTP is so broad. Observability tools do good with it. But I think it's deceitful & disingenuous to blow this topic off like this.
Streams use features like HTTP2+ Push, & there's all manner of semantic meaning implicit in headers & schemas that regular http wont see that these tools pick up on. I want to be sold that the complex multi-directional multi-stream works fine, are fully observable, but I suspect seriously we're overreducing, not being honest, by discarding the difficulties of tuning observability to specific protocols here & saying general http support will buy us everything.
I'm honestly confused as to what your gripe is. I've done it all, OpenAPI, graphql, gRPC -- what exactly is difficult observability wise? Can you provide some specifics as to what is problematic?
HTTP is broad, and some things are easier to observe than others. Something like streaming or websockets is by definition more complex than the most basic possible HTTP request/response. Your complaint boils down to "using features makes observability difficult", which is obviously true in some sense, but you're dismissive of what those features unlock.
> Your complaint boils down to "using features makes observability difficult", which is obviously true in some sense, but you're dismissive of what those features unlock.
It's not a complaint, just a fact that gRPC has fantastic integration with very deep telemetry, and that anything off the beaten path (such as this alternate protocol) is probably going to require significant handholding/customization to make visible. Connect-web wont, out of the box, have any of this deep built-in customization, because it's right now a tiny player, and no one is going to do the work for you, unlike the couple big names.
It's ironic that you mention GraphQL. Yeah, to be honest, I find graphql to be barberic. Submitting all your requests to /graphql is a dirtbag piece of shit regression that only an idiot too stupid to remember how terrible & shitty SOAP was would ever have suggested. It'd offensively bad a premise, & it makes looking at HTTP basics like the path useless. But I use it at work & dont complain. It's well supported by most observability tools.
I have faith Connect-Web hasnt fucked up nearly as bad, but it's still not going to have anywhere near the sweet out of box L7 observability grpc nor graphql has and that's just obvious, uncontestable, clear, & it matters, a lot, & no protest or dislike of what Im saying alters that.
Graphql, in spite of it's dis-http piss-around attitude, is also the exact proof of how valuable popularity is; something connect-web protocol doesnt have now. I can search for specific graphql queries or mutations, or even specific arguments passed in to these queries/mutations- in nearly any major observability platform, in spite of this being fairly obfuscated data deep inside the http posted body envelope, because it's popular, because the work has been done to probe in.
You talk about respecting the features of new protocols. Im not trying to dis on Connect-Web- I have beef but it's elsewhere- but Connect-Web doesnt actually have new features. It's a new 3rd way to do what we already do. One that that will definitely clearly obviously uncontestably have less good tooling support than the we supported official protocol, & which exists only because the browser fucked shit up & failed to deliver, across 3/4 of a decade. It matters that the solutions we built around this shitshow are not as popular, not as supported, and while I like & respect connect-web protocol, I think it's necessary & truthful to view it as a relatively unsupported ideally-not-needed-at-all upstart; the very need for it is terrible versus just using the real, very well supported, highly observbable thing.
I genuinely dont get the objection. Being not-the-mainstream, being an alternate upstart alternative will almost certainly have less awesome support. That's a fact that needs to be faced, not smokescreened.
We’d really recommend checking out the Connect protocol in this case - we totally understand that a new custom protocol would induce a few eye rolls, but it’s about as simple as it gets, especially with unary - it’s just HTTP POST at /service/method, switching on Content-Type between application/json and application/proto (for JSON vs binary payloads). All your existing tooling for HTTP observability should mostly “just work”. https://connect.build/docs/protocol
I love the idea of a more http-native protocol, and I think ya'll are doing saintly work, and I hope observability tools close remaining gaps fast.
I still think we're a little short on candor though. Streams are a bitch in grpc. I have a very very hard time believing plain-old http observability is going to be good enough to help me make sense of multiple cross-directional streams stemming from a single request. gRPC just has a lot to it. Being more plain http-centric is applause worthy. But grpc has too much going on for plain http support to be enough, when it comes to really pleasant observability & tooling.
I dont want to discourage but I dont see how we could expect tl have near as good an experience as what great real grpc tooling supports. I love these workarounds to browser enable, but they have real tradeoffs & sacrifices. That alas the browser hss forced us all into by it's utter lack of giving a shit, caring, doing anything good with making http useful to us.
This is definitely a step up, nicely convenient. But also, it has downsides too. gRPC is incredibly well supported by a huge number of observability tools and service-layers/service-mesh systems. Few have support for gRPC-Web. Which if any have support for Connect-Web?
In general, it's just massively sad that the browser never got around to making HTTP Push useful for protocols. gRPC should have been positioned to do great things, was a very common-sense overlay atop HTTP2 that provided just a little more semantics to say how things ought work. It triangulated nicely on where we were going, at it's inception, back in 2014/2015. But the browser basically never evolved, never gave developers access to do the good stuff we thought we'd start to have access to. I still see brand new specs that shout out to the Low Level Extensibility Manifesto[1], but for some reason, in terms of the most base layer of the web, HTTP, this simple guidance has been roundly ignored & the capabilities were never delivered. So gRPC has been stuck with a pretty ok protocol, & frustratingly discompatible browser.
[1] https://github.com/extensibleweb/manifesto