Yeah that seems bad. If you choose low contrast fonts, reduce the contrast of photographs to the background by making them darker, etc, it's just like turning down the brightness and contrast of your whole site. If the user thinks the content of your site is still too bright with dark mode on, maybe they should turn down the brightness of their device?
By making everything on your site dark and low-contrast, the implication is that the user wants to see other things on the screen in higher contrast alongside your site, like the UI chrome. That seems unlikely - your site is the most important content that they have on the screen at that time and it should probably have the highest brightness or contrast of anything on the screen, not the lowest.
Some global CSS themes change them, but then restore original ones on hover. Not perfect, but generally fine.
Though regular photos (and videos) are rarely so bright that it hurts the eyes while in a dark environment, unlike diagrams and other kinds of drawings, which can easily be mostly white.
This is an interesting choice. While it might be easier on the eyes, personally it feels kind of icky to change colors on images…