A good analogy might be a shareholder corporation: each one began as a tool of human agency, and yet a sufficiently mature corporation has a de-facto agency of its own, transcending any one shareholder, employee, or board member.
The more AI/ML is woven into our infrastructure and economy, the less it will be possible to find an "off switch", anymore than we can (realistically) find an off switch for Walmart, Amazon, etc.
> a sufficiently mature corporation has a de-facto agency of its own, transcending any one shareholder, employee, or board member.
No, the corporation has an agency that is a tool of particular humans who are using it. Those humans could be shareholders, employees, or board members; but in any case they will have some claim to be acting for the corporation. But it's still human actions. Corporations can't do anything unless humans acting for them do it.
Any instance of an individual person, at any level, deviating from the mandate of the corporate machine is eventually removed from the machine. A CEO who puts the environment before profit, without tricking the machine into thinking that it's a profit-generating marketing move; an engineer refusing to implement a feature they feel is unethical; a call center employee deviating too long from script to help a customer.
All are human actions. "Against corporate policy." Go ahead, exercise your free will. As a shareholder, an employee, hell as CEO. You will find out how much control a human has.
The more AI/ML is woven into our infrastructure and economy, the less it will be possible to find an "off switch", anymore than we can (realistically) find an off switch for Walmart, Amazon, etc.