This sounds exactly like what I was looking for. I settled on htbuilder[1], but it certainly does not feel right as it requires a fair bit of wrangling in order to fit with Django.

I'd love to help you with documentation and such; hit me up at smart.tent1246@fastmail.com if you'd like a partner(:

[1] https://github.com/tvst/htbuilder

EDIT: Actually, scrolling further in this thread, it looks like https://htpy.dev fits the bill? It has explicit integration with Django, which is what I was looking for.