We had a massive gypsy moth outbreak this year. They destroy oak trees and no birds will eat them. It's absolutely a good idea to destroy their nests. Obviously this entails knowing what they look like...