That's a fair point. We do support agents and much of ssh_config. I'll look into what it would take to patch scp when I get the chance.
For folks who have setups where iTerm2's scp implementation can't connect, you can always download files over an existing ssh connection with this script:
Upload is a little trickier, but you can copy a file to the pasteboard (e.g., in Finder) and then paste it base64-encoded (Edit > Paste Special > Paste File Base64-Encoded) into base64 -D >> filename.
For folks who have setups where iTerm2's scp implementation can't connect, you can always download files over an existing ssh connection with this script:
https://raw.githubusercontent.com/gnachman/iTerm2/master/tes...
Upload is a little trickier, but you can copy a file to the pasteboard (e.g., in Finder) and then paste it base64-encoded (Edit > Paste Special > Paste File Base64-Encoded) into base64 -D >> filename.