But that 0.1% may actually need that setting for the product to work for them. By all means hide the settings, but don't remove them entirely.

A lot of evidence seems to support advanced users having a proclivity toward tweaking, so something like about:config in the browser seems to work fine (as an example).