I tried this a couple of times but the main problem I encountered was: on Android, it doesn't seem to be possible to delete a conversation without also deleting the contact.