Inthe end, it's all about money - and wasting the time of the person being paid to call you is wasting the money of the person paying them.
Turning off spam filters will have no effect on the time of the person writing them. Thanks to the magic of computers, when the message is written once, it can be sent as many times as necessary, with a much smaller incremental cost than a human on a phone. The cost to a spammer to send a message that bounces off a spam filter is actually higher (due to the need to buy/find open/vulnerable STMP relays when their reputation falls or they're blacklisted) than to send mail that is delivered, so keep those spam filters on!
Can you go into more detail? For my personal mail, my default approach is to toss spam client-side (via automatic filters or otherwise), so the spammer gets no signal about whether or not it is delivered. Does reducing the reputation of their relays hurt them much more? If so, which mail hosts support doing this?
I use a greylist daemon (http://www.x-grey.com/) as a first line filter and it easily stops 50% of spam right there (the spammer will get a transient delivery error---most don't even bother with trying again). Last year I did look into using some realtime black lists, but there were a few critical false positives that kept me from using them (and there's quite a few not worth using---details http://boston.conman.org/2015/05/11.1).
I am not sure about what level you have to reach to get the mail rejected at source relays, but I know that on my Rackspace email system we forward spam mail back to appspam@rackspace.com to add it to Cloudmark, a distributed anti-spam system. Something similar happens with gmail. Dropping mail client-side doesn't inform any of the major relays, and it doesn't spread the workload - you're on your own.
Turning off spam filters will have no effect on the time of the person writing them. Thanks to the magic of computers, when the message is written once, it can be sent as many times as necessary, with a much smaller incremental cost than a human on a phone. The cost to a spammer to send a message that bounces off a spam filter is actually higher (due to the need to buy/find open/vulnerable STMP relays when their reputation falls or they're blacklisted) than to send mail that is delivered, so keep those spam filters on!