In almost all cases the party who writes the drivers it's the SoC vendor which is either Samsung themselves for their own chipsets or for everyone else Qualcomm, Mediatek or one of the less popular upstarts (Allwinner, Rockchip). For everything used by Google and Samsung, both do open source the kernel modules. Almost everything by Mediatek leaks sooner or later somewhere.
The problem IMHO isn't driver source code availability itself, rather their often shoddy quality and the fact that SoC vendors drop support for updates very fast.
The core problem is Linux' lack of a stable internal API for drivers by design, which was thought of as an incentive for SoC and other HW vendors to upstream their drivers so that they need not care about keeping them updated.
What happened instead was that, at least in the embedded world where the customers are not the mass market directly that votes with their wallets but "suits" without much technical knowledge, SoC vendors came up with bundling a private fossilized fork of the Linux kernel, u-boot and a bunch of tooling and calling it a "BSP" (board support package).
How hard would it be to stabilise that API? My line of thinking is : sure maintaining a fork of the Linux kernel would be a huge amount of work, but would it be less than writing a new one and then maintaining that?
Then the majority voice here would be crucifying Google for forking Linux. "Hurr durr embrace extend extinguish amirite????" See ashmem, binder, wakelocks, etc...
Ashmem is especially great example because upstream, having refused to take it for so long, ended up pulling their own NIH when they made the exact same thing but called it memfd.
At least with a new kernel they could also fix various mistakes that have long since been enshrined in Linux. Like the dreadful approach to permissions & sandboxing. Not that Linux has anything it could easily do here, that's the curse of legacy. But still, it's a fundamentally wrong approach to how systems are used today vs. 50 years ago when many of these semantics were established.
This is what the Android Common Kernel is, it already exists as a Linux fork to provide a stable ABI. They are creating their alternative kernel Zircon so they can instead have a less free ecosystem like Apple, steps to do this may be deprecating open-source Android and continuing on with Google Play Services mandatory closed-source fork.
The problem IMHO isn't driver source code availability itself, rather their often shoddy quality and the fact that SoC vendors drop support for updates very fast.