As someone currently running a Navidrome instance with 40k songs...yeah, I wouldn't wish that on anyone(horrible performance), but it is an option I suppose.
As someone also hosting thousands of songs who has been using Navidrome for months I can't say I experienced any performance issues.
The codebase is fairly small so it should be fairly straight forward to investigate and narrow down the performance issues you're experiencing.
In two places. The web client takes _minutes_ to load the artist page. It's slow enough that I get the Firefox warning stating that the page is slowing down Firefox. The web client (seemingly) doesn't load the song/tracklist in chunks and attempts to load every song at once(at least on the artist page).
The second issue isn't specifically a "Navidrome" problem, but every iOS and Mac desktop client I've used(and I've tried _every_ one that Navidrome lists on their site) attempts to load every song on load and basically becomes unusable.
I'm about two notches away from writing my own music streaming server. Navidrone is barely functional for me...and falls under software I hate, but there are no better options.
I have no such issues using Navidrome (docker image from linuserver.io) with 150k songs. On the client i use sonixd which also have no problem running. It might be something on your server