Negative - you can only remove audio devices with the audio-midi-setup that have either been added from the feature itself, or are using the underlying APIs for audio. None of my 3rd party audio devices (Sony MX-3000, Sony MX-4000, hdmi audio out, thunderbolt doc audio out) leverage those APIs, leading me to suspect this is actually fairly rare.
You can map default audio input/outputs with the tool, but removing the audio output is a different story.
