I wonder if you could do this with multiple alignment training passes, where you extract the refusal direction each time, and suppress it in future training passes.