Re: [8023-CMSG] Server/NIC analogy
Ben,
What happens tomorrow to your ultra-simple one-hop network? Today,
you have edge blades communicating with each other through a switch
blade. Tomorrow, and I mean that quite literally, you will have
edge chips communicating with each other through a switch chip on
one blade, and you then will need to connect *those* blades with a
switch blade. Been there, done that, in two different companies
in three different technologies. It is not in IEEE 802's long-term
interest to define a foot bridge across your back yard.
Defining the switch that you want may or may not be appropriate
for IEEE 802; I have not formed an opinion on that question. But,
it is certainly not appropriate for 802.3, a link group.
-- Norm
Benjamin Brown wrote:
> Hugh,
>
> Wow, you just took a leap off the edge of the grand canyon,
> all while I was simply suggesting a way to build a foot bridge
> across the brook in my back yard.
>
> I was not suggesting NIC-switch-NIC, merely switch-NIC.
> A NIC can push back to the switch if it wants to but my idea
> was that, since the switch was not the source of any data, the
> switch would ignore the requests. My idea was that this was
> an edge switch only kind of thing, or the port interface line
> card (switch) sending back to the server card (NIC) across
> a BPE link in a chassis.
>
> I'm not sure I fully understand "what is being requested". I also
> can't quite understand (I'm agreeing with you here) how the
> kind of idea you suggested can be scalable beyond a single
> hop due to the problem of "moving the choke point". An
> exception, as you suggested, would be to do this outside of
> 802.3 (IETF?) and communicates from destination back to
> source but that doesn't resolve the issue of the round
> trip response time.
>
> I won't say it can't be done since I'm certain there are people
> out there more clever than I am that might come up with a way
> to do this. I'm simply saying I don't know how to go beyond
> what I've already described. However, within what I've
> described, it seems like a very reasonable idea to pursue.
>
> Regards,
> Ben
>
> Hugh Barrass wrote:
>
>> Jonathan and Ben,
>>
>> Jonathan's summary matches my perception of the problem, please correct
>> us if we're wrong. So a few points:
>>
>> 1. NIC - switch - NIC is not 1 hop it is 2. If you say that the
>> backpressure must only traverse 1 hop then that precludes a destination
>> pushing back to the source.
>>
>> 2. On the other hand, I believe that what is being requested is a
>> mechanism to signal congestion from the destination to the source. It
>> may be limited to structures with only one bridging element but I think
>> it runs into a number of problems. The first (and IMHO most important)
>> is that such a scheme (effectively) prohibits scaleability - there is no
>> possibility of moving to a multi-stage or multi-path fabric. How could
>> such a closed system be linked together transparently with another
>> system?
>>
>> 3. LANs defined by IEEE 802 are (generally) connectionless. This is
>> particularly the case for 802.3/802.1 "Ethernet networks." This means
>> that there is no specific relationship between the destination and the
>> source that can be exploited for such a backpressure mechanism.
>>
>> 4. For backpressure mechanisms to work they require that the congestion
>> is pushed back to the source and that the backpressuring device can
>> accurately predict the future. This second part is difficult to achieve
>> with current technology... Imagine a situation where device B is
>> receiving too much traffic from device A. Device B sends a message to
>> device A to tell it to limit its transmit rate. However device A is just
>> about to finish its transmission to device B and has a large
>> transmission pending for device C - which is currently uncongested. In
>> the same network at another time when device B is receiving too much
>> traffic from device A. Device B sends a message to device A to tell it
>> to limit its transmit rate. In this situation, device D is just about to
>> start transmitting to device B - causing the overcongestion that we
>> tried to avoid. The solution requires that all devices have to maintain
>> separate input queues for all sources and output queues for all
>> destinations. Such a "virtual circuit" architecture has already been
>> standardized and I suggest that it would not be in the best interest of
>> the networking industry to redefine it inside Ethernet.
>>
>> 5. Finally, 802.1 defines queues for LANs, 802.3 does not. The queue
>> definitions required for any such mechanism would have to be defined for
>> end-to-end operation and would therefore be out of scope for 802.3.
>> Given that such a mechanism would operate at the endpoints but might not
>> have any effect on the intermediate network elements, I think it might
>> even be out of scope for 802.1 bridging. I suggest that interested
>> parties might be well advised to consider a definition in IETF (or
>> elsewhere if appropriate) for the transport layer protocol that includes
>> congestion management.
>>
>> Hugh.
>>
>> Jonathan Thatcher wrote:
>>
>>> Ben,
>>>
>>> If I get this right, you are painting a picture where the
>>> switch/bridge and
>>> the server(s)/NIC(s) are integrated into a single system.
>>>
>>> The feedback from the switch/bridge would extend back solely to the
>>> server(s)/NIC(s).
>>>
>>> It is presumed that the NIC already has an implementation specific
>>> means to
>>> throttle the processor. It is presumed that an implementation
>>> specific means
>>> will be created to tie the feedback mechanism to the throttle.
>>>
>>> It is further presumed that the switch/bridge/line-card can readily
>>> identify
>>> the source(s) of the traffic that are causing congestion.
>>>
>>> Finally, it is presumed that the congestion is, in Bob Grow's words,
>>> transitory. As Hugh implies below, if this problem is a subscription
>>> problem, then rate limiting is an adequate, if not ideal, solution.
>>>
>>> Did I capture this correctly?
>>>
>>> Presuming so, you have defined the problem as local to the specific
>>> system.
>>>
>>> You have also taken an interesting twist on Hugh's point of moving the
>>> "choke point" by putting it back to where it would have gone anyway,
>>> to the
>>> source. In short, as there is only one hop, there is no other place
>>> for it
>>> to migrate to.
>>>
>>> This is a curious concept in that there is no communication between
>>> bridges,
>>> nor is there an implied bridge in the NIC. If so, there is no
>>> question about
>>> ownership of the problem :-)
>>>
>>> Hmmmmmmmmm.
>>>
>>>
>>>
>>>
>>
>
> --
> -----------------------------------------
> Benjamin Brown
> 178 Bear Hill Road
> Chichester, NH 03258
> 603-491-0296 - Cell
> 603-798-4115 - Office
> benjamin-dot-brown-at-ieee-dot-org
> (Will this cut down on my spam???)
> -----------------------------------------
>