Swift on the server might make sense for an Apple-only full-stack developer, who wants to remain Apple-only, who is happy with the Swift development experience on the frontend, and who is willing to bet their backend on Apple's developer support.

(I'd use Rust, Flask, Node, Go, Scheme, Elixir, or a variety of other platforms/approaches first.)