It doesn't matter the MTU setting on your end for WAN and ISP / interlink grade networks is meaningless they don't use Ethernet, FDDI frame size is 4500 (ATM is about double that) bytes (minus what ever overhead, but usually 4200 and change) ISP/WAN routers don't care about how many mbit/s they transfer but how many packets they route at per given unit of time, as packets get packed into a single frame the smaller the packet the more packets they can transfer each frame.

Also from a more high level point of view if you think about it the small packets are the most critical ones and they are at least as far as responsiveness goes DNS is limited to 512bytes over UDP, TCP 3-way handshake packets are tiny and those are the packets that need to get to and back from their destination as fast as possible, delays in data transfers means slower speeds, delays of handshakes mean that your application can fail or hang. Other important traffic such as VOIP[0] also uses very small packet sizes for this same reason most critical services need to transfer very little data (per given unit of time) but need to update data as frequently as possible to provide the illusion of real time and to mask the latency, same goes for other things like online/multi-player gaming and so on and on and on. Pretty much if you want your service to be as responsive as possible limit your packet size to the smallest size possible and increase your PPS this will ensure that your packets get quicker to their destination. [0]VOIP Packet Sizes http://www.cisco.com/c/en/us/support/docs/voice/voice-qualit...

The only time you would want to use large packets is pretty much when you can have a buffer, this means that you need to handle less packets per second which lowers CPU consumption (across the entire path) so video streaming and such can use pretty much as large of an MTU as they want unless they start getting fragments.

> It doesn't matter the MTU setting on your end for WAN and ISP / interlink grade networks is meaningless [as] they don't use Ethernet...

It absolutely does matter and is quite meaningful. :)

If you set your edge router's Internet-facing MTU to 9k, and the upstream equipment's MTU is smaller than that, then either your packets will be dropped, or PTMU Discovery will try to figure out the MTU of the path. (Better hope everyone along the path is correctly handling ICMP! :) )

> The only time you would want to use large packets is pretty much when you can have a buffer...

Or if you have high-volumes of data to move and want to dramatically increase the data:Ethernet_frame_boilerplate ratio. :)

> Also ... if you think about it the small packets are the most critical ones... [because they need to be dispatched as quickly as possible.]

Yes, but a larger MTU shouldn't affect this. Set whatever socket options are required to get those packets on their way as soon as they're created, and your system shouldn't wait to fill an Ethernet frame before sending that packet.

Poor choice of words on my part, if you configure jump frames on your uplink you are going to kill your network stack, if you limit it too much you'll have a huge overhead. The point being is that for transferring data especially when responsiveness is important if not paramount utilizing the maximum potential frame size you can push without fragmentation would generally yield a poorer result in real world applications.

> [I]f you configure [jumbo] frames on your uplink you are going to kill your network stack...

I can't agree with that statement. If upstream devices support larger than 1500 byte MTU, OR PTMU works correctly, then you are absolutely not going to "kill your network stack". At worst, (in the PMTU discovery phase) you'll see poor performance for a few moments while the MTU for the path is worked out, and then nothing but smooth sailing from then on.

> The point being is that for transferring data especially when responsiveness is important if not paramount utilizing the maximum potential frame size you can push without fragmentation would generally yield a poorer result in real world applications.

I'm not sure what you're saying here. Are you saying:

"If you configure your networking equipment to always wait to fill up a full L2 frame before sending it off, you'll harm perf on latency-sensitive applications."?

If you're not, would you be so kind as to rephrase your statement? I may be particularly dense today. :)

However, if you are, then that statement is pretty obvious. I expect that few people configure their networks to do that. However, I don't see what that has to do with the link's MTU. Just because you have a 9k+ MTU, doesn't mean that you have to transmit 9k of data at a time. :)

I work for an ISP and it is all ethernet on the interior. Both for residential and commercial customers. The small amount of frame relay and things that are requested are on the ethernet network from edge to edge.