I would guess it has not been given much thought, as the initial comment suggests, is most developers and users don't care at all. The most used web sites look terrible but people are using them for function, like searching, shopping and paying their bills.
The frustrating thing is that users do care, at least up to a point. There have been success stories in the modern era where a well-designed, well-executed product has dethroned a long-standing incumbent (or at least provided credible competition despite being David against Goliath) and the slick presentation seems likely to have been a competitive advantage. There's no shortage of complaints about sites or apps with poor design either.
However we know that most users will prioritise other aspects, such as functionality or security, over aesthetics. If the advantages of a nice design are purely cosmetic and don't also improve factors like the usability of an app or the credibility of a marketing site then that's probably going to be less important than missing some key feature the user wants or lacking the network effects that existing products in the market have established or convincing a potential customer that you're trustworthy before they had over their card details.
What do you think an example of a site where a new player has come along with a much better looking site than the incumbent and beat them?
Not trying to be negative, I just can't think of any and that would be excellent evidence that it does matter. Just bringing up a list of the top 200 web sites and they are the reference gallery of visually terrible sites.
The first example that comes to mind is banking. In my country (UK) there seems to be a clear gap now between the old school "high street banks" that have been around forever and a new group that have entered the market more recently.
Both types of bank offer similar basic accounts and services. Interest rates and fees also tend to be similar at the moment.
What does separate them is that the established "big names" almost universally have terrible online banking and mobile apps while the newer alternatives tend to focus on these facilities (I think some of them don't even have physical branches) and they have slick, modern UIs aimed squarely at younger generations and the digital native market.
I don't have any hard data to cite but the number of younger people I see using these services suggests they are having some significant success at breaking into even such a heavily regulated and competitive market.
