Humans (and their creativity) have special status and privileges in law.

Machines don't. It doesn't matter how fancy they are. The law doesn't care.

So yes, a human in a black box is different from a machine in a black box until laws change.