Re: [8023-CMSG] Server/NIC analogy
Ben,
I think we can safely assume that, in the not too distant future,
servers will be capable of consuming 10 Gbs links on real applications.
When these servers are used to construct clusters for parallel
processing applications (database processing for instance), they will be
capable of creating frequent congestion with the cluster interconnect.
The traffic loads will include both bursty traffic and high priority
control traffic.
Prioritization will help the high priority traffic get through an
Ethernet cluster interconnect in a timely manner. The rest of the
traffic (that causing all the congestion) is likely to all be
transferred at a low priority. Dropping some portion of it to relieve
congestion is simply unacceptable. During times of heavy loading,
congestion will happen often enough to bring the cluster to its knees
doing retransmissions.
IMO static rate control is also unacceptable because the loading
requirements are not that predictable and not that managable. I just
can't see the database industry buying into static rate control
managment just so they can use Ethernet in backplanes, clusters, and
SANs.
I think the key to making Ethernet acceptable as a short range
interconnect for backplanes and clusters is to support dynamic rate
control. The use of a good implementation of 802.3x (within a limited
enough scope) would be preferrable to drops. If we define the target
scope of the CMSG to include multiple stages of switching, multiple
hops, and different link speeds in the same interconnect (or subnet),
then 802.3x just isn't going to hack it (and neither is static rate
control or packet dropping). I think we need to be thinking in terms of
an improved dynamic rate control, which requires feedback.
I think defining the scope of interconnects we want to target with CM
solutions will go a long way toward framing the range of acceptable
solutions.
Gary
-----Original Message-----
From: owner-stds-802-3-cm@listserv.ieee.org
[mailto:owner-stds-802-3-cm@listserv.ieee.org] On Behalf Of Benjamin
Brown
Sent: Friday, June 04, 2004 2:20 PM
To: STDS-802-3-CM@listserv.ieee.org
Subject: [8023-CMSG] Server/NIC analogy
All,
During a private discussion this afternoon regarding the results of last
week's meeting, the concept of feedback came up - whether it was
necessary or not. There was some level of discussion about this during
the meeting but no one seemed to be able to provide an adequate
justification for providing congestion feedback and why the more common
approach of packet drop wasn't adequate.
During this afternoon's discussion, I came up with something that I
think might be justification. I'm probably just painting a big target on
my chest but let's see how this goes.
Consider a stand-alone server with a 1G Ethernet NIC. Today's CPUs could
easily generate enough traffic to swamp the 1G Ethernet link (okay this
is a bit of an assumption on my part but if they can't today they will
be able to tomorrow). I don't build these things, nor have I looked at
their architecture all that closely in a number of years, but I'll step
out on a limb and state that there's a (most likely proprietary)
mechanism for the NIC to tell the CPU that the link is too slow to
handle all the packets that it is trying to transmit. I'll step even
farther out on that same limb and state that the mechanism is not packet
drop.
Now, let's use this analogy to consider a server card in a backplane
that communicates to the world via a port interface line card. The
server card communicates to the port interface line card using a link
compliant with the newly emerging Backplane Ethernet standard. (Okay, so
I'm looking a little into the future.) If you consider the entire
chassis analogous to the server/NIC in my initial example then it would
seem plausible that you would want to communicate buffer congestion on
the port interface line card back to the server card using a mechanism
other than packet drop.
I'll just close my eyes now. Fire at will.
Ben
--
-----------------------------------------
Benjamin Brown
178 Bear Hill Road
Chichester, NH 03258
603-491-0296 - Cell
603-798-4115 - Office
benjamin-dot-brown-at-ieee-dot-org
(Will this cut down on my spam???)
-----------------------------------------