Perhaps I am thinking too narrowly, but I think the algorithm-driven approach is much less hubris than someone at the company making a decision. These models reflect what users do -- isn't that much more democratic than employees deciding?
There is an orthogonal argument that this is also harmful to some people's aspect, but "give the people what they want" doesn't seem like it's reductive. Especially when it seems like people in our society can be clustered and have some clear groupings occur.
>These models reflect what users do -- isn't that much more democratic than employees deciding?
What users do is different from what users would like to do. If people's aspirations lined up with their real behaviors, Planet Fitness wouldn't have a business model. But when navigating the world people often want to improve themselves into something they want to be, not fall deeper into the worst tendencies they exhibit in any given moment.
Is the algorithm-driven approach really any better, though? If our UI leads users to accidentally take an action, our data will actively mislead us into thinking it's what they want!
Designing clear interfaces, measuring things and coming up with models are all really hard. Data needs to be paired with human judgment and decision making.
You aren't wrong, but that's presumptive on humans being able to create algorithms that aren't implicitly bias. 2, hopefully not overly simplistic examples, A) Automatic Paper Towel dispensers that don't work nearly as well with people with dark skin. (Unintentional bias on the hands of the designers) B) People getting google banned and then being totally helpless and useless and not being able to get themselves unbanned (an algorithm determined there was abuse, and there is no recourse to tell the machine that it was mistaken, the entire thing was automated away via algorithms.)
There is an orthogonal argument that this is also harmful to some people's aspect, but "give the people what they want" doesn't seem like it's reductive. Especially when it seems like people in our society can be clustered and have some clear groupings occur.