Based on someone else's comment I plan to revisit OpenBSD, because previously I wasn't able to get gigabit speeds out of a low-end appliance, even though FreeBSD and Fedora both could.
Things have definitely improved over time, but some of the best improvements are also related to utilizing advanced functionality from hardware that supports it.
Are you doing over 10Gbps? A lot has been done in this space.