Re: [HSSG] Topics for Consideration: Jumbo Frames

[Pat Thaler <pthaler@xxxxxxxxxxxx>]

Title:

Marcus,

 

Jumbo frames do increase latency including the ability to provide very low latency for some classes of service. You can't send a small higher priortity frame until the jumbo has finished. Because of that they also increase the RTT for congestion control and flow control - you can't send the message telling the source or link partner to slow down until the jumbo has ended. Some of the same application areas that are pushing for higher speed also want lower latency. Yes, latency does get lower at higher speeds but customers are continuing to ask for lower and lower latencies particularly for data center networks with applications such as IPC.

 

Some networks that support very large packet sizes mitigate the latency/QOS issues by allowing for pre-emption, but I doubt that we want to go down that route.

 

Jumbo's don't help design much - even if we support jumbos we are expected to support high flow rates of small packets. You can't count on getting jumbos all or most of the time since the payload isn't always that big and since you don't even know if it will be enabled. Also since you will often have to carry some traffic that is bridged to equipment that doesn't support jumbos (or have them enabled).

 

When a new speed comes out, there is always a period where some devices can't do link speed with normal size packets but this is a temporary transition. For example see Bidging the Ethernet-Ethernot Performance Gap; Balaji, P et al, IEEE Micro, Volume 26 Issue 3, May-June 2006 pages 24-40.  It and similar papers show that jumbos have little effect on throughput and CPU utilization for end nodes with TOEs.

 

Pat

---

From: Marcus Duelk [mailto:duelk@xxxxxxxxxx]
Sent: Thursday, August 10, 2006 8:21 AM
To: STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
Subject: Re: [HSSG] Topics for Consideration: Jumbo Frames

Hi Arthur,

there is a nice little "tutorial" about the Pros and Cons
of jumbo frames at http://sd.wareonearth.com/~phil/jumbo.html,
it also includes actual frame size distribution measurement on the
MCI backbone from 1998 in which you can see that there are a lot
of frames >1500B from FDDI and other sources, they mention that
more than 50% of all traffic (measured in Bytes and not in packets)
come from packets >1500B. But, larger packets also have a clear
benefit to TCP/IP performance. The difference in store-and-forward
latency between 1.5kB and 9kB is negligible at higher speeds, time
of flight is much, much higher ...

Marcus


Arthur Marris wrote:

Joel, Mike,

   Don't jumbo frames increase latency in links where there is store and forward?

 

Arthur.  

Both my position and that of the customers I deal with is that higher speeds, 10gbps included, is more efficient with the larger packets.

no argument from me on this one, although the actual efficiency gained comparing 1.5 K frames to 9 K frames is only ~ 2%, but when you're on the cutting/bleeding edge of technology, every bit counts.

-- 
___________________________
Marcus Duelk
Bell Labs / Lucent Technologies
Data Optical Networks Research

Crawford Hill HOH R-237
791 Holmdel-Keyport Road
Holmdel, NJ 07733, USA
fon +1 (732) 888-7086
fax +1 (732) 888-7074