The point of Chesterton's fence is that the fence was created for a specific reason and then was removed without addressing that. Replacing that fence with something built for an entirely unrelated purpose isn't replacing that fence. The closest equivalent would be replacing an electric fence with a small stone retaining wall.
They do different things. The electric fence is for keeping animals in/out and the retaining wall is for keeping soil from moving. Sure you may add the retaining wall but you've still removed the electric fence so the foxes can now get into your chicken coops or your cows are running free. That's chesterton's fence. Even if well meaning, making a change that fails to replicate/fulfil the original purpose of the original fence causes the issues to return.
"Limiting government" ≠ "no fence". It is a fence on government regulation. A "meta-regulation" if you will.