I'm curious why they didn't use JSONSchema or OpenAPI to define lexicon schemas, and instead created another schema system for defining what a valid record looks like
The TLDR is that we started with JSONSchema and then evolved it. OpenAPI didn't quite fit our needs either. Lexicon doesn't just describe APIs; it also describes records stored in user's data repositories.
Have you looked at CUE? I saw you mentioned the idea of closedness at the end of the ATProto doc. It certainly
seems like there is influence, but great minds also think alike! There are some interesting use cases around detecting backward (in)compatibility. Might not make sense to define them in CUE, but maybe have CUE around to enhance the DX. Happy to toss around ideas off of HN if it interests you.
I'm sure the CUE team would join the conversation too. We were talking just last week about CUE schemas for the ATProto lexicon and some hard parts around cyclic references
Another angle might be for a future lexicon schema syntax to be encode/decode friendly with CUE, which aims to have two-way interoperability with a number of schema systems (type systems too).