Arthur:
Yes, I agree with your statements below in that the concepts you
detail out could
be architect and implemented! In a very rough sense these concepts are
embodied
in Infiniband, and we all know how well that turned out!.
The question is that of cost and complexity! Being a veteran of
these types of
developments in the past I am concerned that people tend to under
estimate the
real cost and complexity and over estimate the real value to the
customer. For
example consider the number of verification cases required to debug to
a high
level of confidence the concepts that you present below. An example
from the
past comes to mind of no less that two vendors that spent literally
years debugging
their ATM SAR ( and still do not have right ).
Another more pertinent issue that comes to mind is that of culture.
If the architecture
that you describe below were brought to 802.3 they would have a cow!
This is just
to big a change for this fiercely xenophobic group to handle.
Thomas Dineen
Arthur Marris wrote:
Tom,
Preemption can be specified in such a way that the preempted
frames are not dropped. The transmission of the of the preempted
frames would be suspended to allow higher priority frames to pass
by and then resume without resending the previously sent frame data.
This could be done by specifying two (or more) channels in
the MAC for different priority levels. The higher priority channel
could preempt the lower priority channel. The priority level of
the frame being transmitted and the preemption control would be
communicated between MACs through the PHY using different codes
for start of packet and end of packet. Using this mechanism means
there would be no need for the MACs to examine the innards of the
frame to discover the frame's priority.
Arthur.
-----Original Message-----
From: owner-stds-802-3-cm@listserv.ieee.org [mailto:owner-stds-802-3-cm@listserv.ieee.org] On Behalf Of Thomas Dineen
Sent: Tuesday, May 04, 2004 7:57 PM
To: STDS-802-3-CM@listserv.ieee.org
Subject: Re: [8023-CMSG] Purpose
Gentle People:
An aspect of preemption that was not discussed below has just
come to mind. What would be the effect on both overall link utilization
and the low priority preempted flows?
First of all I assume that the preempted partial frames are just dropped
and thus must be retransmitted later. The entrenched 803.2 mind set prevents
any other viewpoint. As I see it this would in some cases reduce the
effective
link bandwidth for low priority flows by 50%. This would have a devastating
effect on overall link utilization if preemption were constantly occurring.
Next the low priority preempted flows would suffer greatly in a
preemption
scheme due to the constant drop and retransmission. This would in effect
be a
form of double discrimination, first they are low priority at queuing
and second
they are constantly being dropped and retransmitted.
Thomas Dineen
Hugh Barrass wrote:
Arthur,
I agree that preemption is a fine idea, but in my view it falls into the
"not worth the effort" category. Assuming that any new definition that
we could make will not be standardized until 2006 & will be commonly
available in silicon at least a year later, I think we can safely ignore
any Ethernet interfaces below 1Gbps. Even Gigabit Ethernet seems
somewhat pedestrian for high-end data center applications and I would
suggest that anyone concerned about the latency penalty of the frame in
progress at Gigabit speed would be well advised to migrate to 10G before
2007.
In that timeframe, a user will have the choice of 10GBASE-CX4 and
10GBASE-T for (cheap) copper interfaces. The former seems ideal for data
center as it is extremely low latency and targeted at the shorter
distances necessary for system-system communication. If the distances
involved force a requirement of distances up to 100m, making 10GBASE-T a
necessity, then the latency budget will be swamped by the physical
distance (500ns @ 100m) and the PMA/PCS latency of 10GBASE-T (probably
~1uS).
A maximum length frame in progress at 10Gbps will take ~1.2uS, making
the average gain due to pre-emption ~600uS (ignoring packet mix and link
utilization). Even taking the maximum delay (which will map to the delay
jitter component), the order of magnitude is similar to the fixed delay
of 10GBASE-T and therefore cannot possibly lead to a significant
reduction for systems using that technology.
Assuming that the speed-crazed implementor chooses 10GBASE-CX4 and
wishes to eliminate the 1.2uS max jitter then there are two options. The
first is preemption - which can significantly reduce this (depending on
the definition) but will involve significant new work. The alternative
is to reduce the MTU - which involves no new work. Changing the MTU from
1500 bytes to 500 bytes reduces the maximum jitter to 400nS at the
expense of ~3% extra overhead. Further reductions can be achieved for
larger overheads - which is a tradeoff that can be made at system
configuration time. I'm fairly sure that some will argue that the MTU
needs to be increased (to 9k, 16k, 64k or higher) because
software/firmware based NICs cannot encapsulate small frames at line
speed and 1982 vintage routers cannot switch line rate streams of
minimum size packets. I would suggest that anyone who is serious enough
to be asking for a new standard to improve latency should be using
hardware acceleration for packetization and true wire speed switch
fabrics.
Assuming that the MTU has been reduced to 400nS, smart switch fabric
designers might wish to employ some techniques which can reduce the
jitter further at the expense of an increase in fixed latency. Given
that the fixed latency of the copper interconnect is approaching the
same magnitude, this seems like a reasonable tradeoff to make for system
performance (assuming that delay variation is the problem).
In summary, the net gain that can be achieved by preemption is too small
to make a difference except in the most extreme circumstances. For most
applications, current standards can be utilized (at layer 1 & 2) to
attain acceptable performance therefore the demand for silicon
implementing a new standard will be limited to a niche of a niche. If
the application area is sufficiently small then more exotic (or
targeted) technologies may have a competitive edge - there will be no
"Ethernet advantage."
Hugh.
Arthur Marris wrote:
Jonathan,
The presentation you gave in March at the Data Center Ethernet CFI
suggested preemption as an area for exploration.
Preemption would require a minor change to the PCS to support extra
control-codes.
Supporting preemption seems like a worthwhile objective as every
microsecond is precious in cluster computing.
Arthur.
|