I touch upon this a little in the epilogue section https://sunilpai.dev/posts/seven-ways/#epilogue-ok-but-doesn... I was thinking what work there really is for "juniors" in this new future, and I suspect the answer then that their "job" is to develop the understanding of the codebase, approaching changes from the edges, and learning how to introduce new changes. (very similar to how people would onboard to a codebase pre-llms: by writing tests, doing smaller changes, poking about the code by adding console statements, etc). The good news is that the coding agent is a very good tool to do that as well. I plan on writing more about this soon.
In general, if you're changing database schemas, you need a code version running that supports old and new. If you have some old code running to serve in-flight requests, you probably need to wait for those requests to complete. This is one of the main areas that makes me feel that requests > a few hours are a real problem (and that we shouldn't be writing workflows this way)
I'm of the opinion that actual database schemas should be a private implementation detail of any given service within an organization, whereas workflows are not services themselves, but instead are clients of the services and so should only ever interact with services via their API (be it a web-service or otherwise (no love for CORBA, eh?) and never directly talking to an RDBMS.
(obviously the rules are different for KV/object/blob-stores and don't apply here).
-----
That said, if you need a stopgap solution for a problem like that, then take advantage of existing compatibility-shim features in a decent RDBMS: things like VIEWs and SYNONOYMs exist for this reason (and yes, you can INSERT INTO-a-VIEW).
-----
Another option is to constrain your DB designs such that all DDL/schema changes are always backwards-compatible (i.e. disallow dropping anything, disallow alter-column except to widen an existing column (e.g. int-to-bigint is okay but not string-to-UUID), all domain-model attributes must be initially modelled with a m:m multiplicity whereever possible, etc etc) - and you can enforce these rules using a DDL trigger too - and any changes made by a workflow's DML can be repaired by a background job.
+1 on views, for those things than need direct DB access.
One effective pattern I’ve seen in large DB deployments is to separate the write schema from the read schema. That is, treat what is allowed to be written into the DB separately from the shapes of any views that exist. The views themselves are a tightly coupled client of the DB log - by constraining writers, you can migrate/rebuild views, then point services to read from the migrated views, and retire old views.
This allows you to keep accepting writes - you never have to shut down the write path. If you’re introducing new shapes or the DB, you’d prepare a new view, the “widen” the write schema, and begin accepting writes in the new shape, and only then re-point clients to read from the new view.
To drop elements of your read schema, you do the dance in reverse. First, constrain your writes. Then, build new views that don’t require the elements you removed. Gradually, update application code to work on the new, reduced views. When you’re done reading from the original views, you can drop them.
This is inherently much less efficient than online or offline DB migrations. But it’s a sane strategy for wrangling very large systems with very low risk.
Versioned workflows are in practice distributed entities that interact with their peers, themselves defined by versioned interfaces. By tracking the versions of handlers touched by any given execution, we can imagine a similar experience for deployed code - including garbage-collecting unused versions.
It is indeed excellent. y-partykit is a library for building a backend on PartyKit, but PartyKit itself isn't tied to Y.js; you can deploy any tech you want on to it.
Hello! I'm on the team that implemented this. I'll be writing a longer post on the design considerations and implementation details, but roughly - rust wasn't a good case for something that was mostly held back by i/o to the edge; further, the actual cpu/mem intensive part was handed off to webpack which was written in javascript. So wrangler v2 flips that design around, using typescript for the outer shell, which makes it easier for our users and community to contribute to the tool, and esbuild for the core part of parsing/bundling code (we may switch esbuild to a rust based bundler like swc in the future).
Fun fact, that post 3 months ago was what got us folks at cloudflare in touch with the author, who then interned with us for 2 months (today was his last day!)
Errors - We're getting better at this. You can now use `--inspect` and use chrome devtools which should log proper exceptions (details to setup https://blog.cloudflare.com/profiling-your-workers-with-wran...)
Console.logs - same as above! Real devtools, interactive and everything.
Timeouts and crashes - This should be better now, timeouts and sessions expires now transparently heal and restart, so you shouldn't face errors (Please file an issue if you see different!)
Overall we're working hard on making this experience better, please feel free to file issues if you see anything that's not expected. Thank you!
That bad, huh :( But I hear you. I'm working on making this better right now, I'd be happy to hear any complaints you have, and I hope we can get you back.
Hi there! I'm on the team, and we're going to invest a bunch of time and effort on making this experience much better. Happy to hear any specific feedback if you'd like, and I'll prioritise it. Feel free to reply here, or DM me on twitter @threepointone.
`wrangler login` doesn't work if I'm not already logged in to CloudFlare website. If I have to login first, it doesn't do whatever necessary redirect it should do so most of the time I have to run `wrangler login` twice.
I also feel that the interaction between DNS settings and workers is under-documented, probably because it crosses responsibility of two different teams.
For example, it took me way too long to figure out that if I have a workers-only website for foo.bar.org, I have to put dummy DNS entry for foo.bar.org
Frankly, despite reading wrangler docs multiple times, I don't think I fully understand the interactions between what's in the DNS, what's in the workers route and further more if you add pages to the mix.
Docs should explicitly spell out DNS / routes setup for most common scenarios like: 1) workers-only site (no proxying to any other server) 2) workers in front of a proxied website
Workers should be seamlessly integrated with pages i.e. pages should just automatically recognize and use workers-site/ directory on deploy.
I don't think setting up pages automatically configures DNS settings for its domain, which it obviously should (might be mis-remembering).
What I constantly ran into was disconnecting from the server I was connected to and wrangler not being able to handle that. At the time I had a somewhat spotty connection (nothing terrible), but it made wrangler dev very painful - so much so that I ended up writing my own node.js wrapper around it for local development. The lock-in to webpack also isn't great when I am nowadays accustomed to esbuild which is so much faster.
Ohhh I remember this too, I hated how it consistently dropped connections. A community member fixed this for us (here https://github.com/cloudflare/wrangler/pull/2008) so that it transparently and automatically reconnects now and the experience has gotten so much better.
And of course, miniflare reduces even the chance of that to zero :)
We also don't have a lockin to webpack anymore (you can choose custom builds with the "javascript" type, which I admit is a little confusing). We're going to move away from even this dependency and make everything much faster and lighter, stay tuned.
I've faced this too, where you'll save, then hit refresh, and it show the previous version; despite the fact that we "know" that the code is changed. SO we should be able to queue requests when we have a new version uploading (and of course, make the whole thing faster). Thanks for your feedback!
I personally never really figured out how to get it working properly when I was using it. I resorted to just returning data as the response page to see what I was debugging and testing "in production". This was a few months ago, so I don't remember the details of what didn't work or why, just that I gave up on it altogether.
I'm so sorry you had to deal with that. We're working on removing a lot of friction from the onboarding process, and making it easier to introspect what everything's doing. Thank you for your feedback!
That's good to hear! I still use and like cloudflare workers, but at the time, earlier this year, the debugging experience was still a bit rough. So if that's improving, then I'm happy.
- [Not relevant] There's this limitation about fetching host+port [0] in the worker. It works fine in wrangler dev but doesn't work on Cloudflare worker. I don't mind that it's a limitation of the platform but there should be documentation and wranlger dev should be consistent with the actual worker.
- It's not relevant now but I couldn't figure out how to connect cloudflared with wrangler tail on windows. Wrangler tail points to the cloudflared docs but the cloudflared docs doesn't mention wrangler at all. I was pretty lost for what to do.
- [Not relevant] I haven't investigated this properly but Promise.then.catch doesn't seem to work? async/await works though.
- I'm pretty sure URL.createObjectURL in this example [1] doesn't work on the worker.
- Wrangler dev doesn't have the request.cf object. I see why but a dummy one would be cool.
None is deal-breaker but they wear you down when you walk through the documentation.
It sounds like you may be thinking of `wrangler preview`, not `wrangler dev`. `wrangler preview` runs the worker on a service that operates outside Cloudflare infrastructure, so a lot of things aren't realistic, like support for host+port and request.cf. `wrangler dev` runs the worker inside the real Cloudflare stack on the real edge, which should solve those problems. Our plan is to sunset `wrangler preview` in favor of `wrangler dev`.
> It's not relevant now but I couldn't figure out how to connect cloudflared with wrangler tail on windows.
Good news, this has been fixed recently -- `wrangler tail` no longer requires `cloudflared`.
> - I haven't investigated this properly but Promise.then.catch doesn't seem to work? async/await works though.
Promises are implemented inside V8, so they should work exactly the same on Workers as in Chrome and Node. I'd love to see an example of what's not working if you have one.
> - I'm pretty sure URL.createObjectURL in this example [1] doesn't work on the worker.
This is indeed an API we don't support, and realistically we probably can't support. I'd be interested to understand your use case for this.
This is amazing feedback, thank you very much! I'm going to work on each of these; but quickly - we definitely shouldn't have something that works in `wrangler dev` but not in the platform, and we should warn/error on that, our docs shouldn't have examples that don't work :facepalm:, and we should be spec compliant where we can. Thank you so much!
Could you add a validate switch? Something I can integate into a CI pipeline that basically takes a look at whether it is a valid CF worker. Obviously running a linter over it already but presumably there are CF specific checks that could be added. Thanks
Could you share more detail about your usecase? What kind of checks are you looking for? I'd be happy to have a look at that (and presumably if we could do it before it gets uploaded, that would be great)
A very basic rudementary static code analysis that can check whether this is a valid CF worker configuration.
>Could you share more detail about your usecase?
In a CI configuration like gitlab CI. e.g. Currently I'm using an incredibly hacky setup that loads the function into miniflare locally, hits then endpoint with a curl and checks that it comes back with 200 HTTP.
If yes then assumption is that it's sound & next stage of CI can actually deploy the function.
Not super high priority though - please prioritise other requests above this since I've already got a working duct tape fix for my problem
My thing, which I asked about in the Discord as well, was how to line up environments for local Dev, cloud staging (e.g. a preview from a pull request) and production, across workers, pages and kv. Having a preview mode is fine but I struggled to know what should map to what in a consistently configured way, for a monorepo setup that's updating everything in the same PR.
Yeah agreed, this is pretty ad-hoc right now, smeared across different commands and config. We're working on a better solution, will share more details when we have something. Thank you for your feedback!
It's a pleasure. I'm a big fan; that's just one of a couple of things that I'm struggling with. The other is, "this smells like extremely well-made proprietary software that I will have to hope a Cloudflare product manager never deprecates if I commit to it".
