why NICs couldn't support the crypto in a way that it can just send the packet straight to the wire after encryption.
That's how the Chelsio T6 works. On the transmit path the NIC needs to perform encryption, TLS framing, and TCP/IP/Ethernet segmentation in that order; on receive it needs to perform TCP reordering and such before decrypting. Most NICs just don't want to add that much functionality.
Thanks, that's looks pretty good. I wonder if it will work with TLS 1.3 as well (from my limited understanding it looks like it might).
Here's dreaming of a future where ED25519/Chacha20/Poly1305 is supported in hardware (it's much cheaper for clients that doesn't have AES so I want servers to use it).
That's how the Chelsio T6 works. On the transmit path the NIC needs to perform encryption, TLS framing, and TCP/IP/Ethernet segmentation in that order; on receive it needs to perform TCP reordering and such before decrypting. Most NICs just don't want to add that much functionality.