Yeah, for me this would be one of the more important features, since I've on more than one occasion had to access FB to get a phone number.

There is a Sony Ericsson application in Facebook which retrieves all your friends' phone numbers and upload it to their servers which then can be synced with any Sony Ericsson phone . Then I can download it as CSV by connecting the phone to the PC. A round about way but it works. So I guess there might be other apps which does this.

wouldn't this be in violation of the tos and/or technically impossible? If not, why can't I write a fb app that syncs all my friends' data to my server?

No, because FB has allowed them access to these data.