[EFM] Re: Pause frame usage in transport networks
Roy,
I suggest that we shy away from generalizations that arise from a broad brush
discussion. I made very specific comments on the mailing list and would be happy to
discuss specifics. Here is a brief summary (Please note that discussion was
focused on EoS for Ethernet Private Line type services. So both ends of the service
would have the same traffic parameters. More involved service definitions were not
part of this discussion):
1. Network to subscriber Pause has advantages over shaping at subscriber
2. Subscriber to network Pause is UNlikely to be invoked as most often the backbone
link is the bottleneck. However if it does network would buffer and subsequently
drop packets. Sending Pause to the far end is not practical as it would require
large buffers to absorb the wide area round trip delays.
3. Pause would not interfere with OAM, if anything it avoids loss of OAM frames and
the resulting time outs.
Regards, Siamack
Roy Bynum wrote:
> Ben, Siamack,
>
> In these comments I am seeing the movement toward an link facility that can
> be labeled as "Ethernet", but has little or none of the inherent
> characteristics of reliability and low latency variance. Any Ethernet
> service that claims to be full duplex, but drops frames without generating
> a "collision" when congested will fail meet the basic reliability
> characteristic of any 802.3 standard that I am aware of. This is NOT Ethernet.
>
> The ONU should be allowed to "pause" the GFP/OLT on any one link, and the
> GFP/OLT, should be allowed to "pause" the ONU on any one link. With proper
> configuration of the operands, an ONU at one end of an end to end "link"
> should be able to "pause" the ONU at the other end. Without that
> capability, what is being defined is no more reliable that what exists
> today, and is some respects is less reliable than the alternative.
>
> When an end to end link starts "dropping" frames, the data packets get
> retransmitted in new frames which now adds to the congestion of a link and
> thus lowers the effective access bandwidth that can be utilized. This has
> the effect of lowering the effective committed information rate even
> more. This is one of the primary reasons that experienced WAN networking
> architects design IP networks to run as a nominal 30% utilization. (I
> remember reading something by David Boggs about ~30% utilization being the
> effective performance ceiling of congestion domain networks. I turns out
> that he knew what he was talking about.)
>
> The current defined X.86/OLT does make use of "pause" functionality, and
> will allow an ONU at one end of an end to end link "pause" the ONU at the
> other end. By properly use of active flow control, the service
> communications link can perform as a non-congestion domain link. With
> properly configured operands at the ONUs, this would be highly reliable at
> the cost of lower predetermined bandwidth utilization, but without
> retransmissions. In experiments performed in the 1998-2000 time frame,
> effective utilization was found to be a direct ratio of link/circuit speed
> to distance (I am having to write this from memory because I no longer have
> access to the data.) Properly configured use of active flow control can
> allow architecture designs with much higher effective bandwidth utilization
> than 30%.
>
> Thank you,
> Roy Bynum
>
> At 10:15 PM 2/18/2003 -0500, Siamack Ayandeh wrote:
> >Ben,
> >
> >Please see my comments interleaved.
> >
> >Regards, Siamack
> >
> >Ben Brown wrote:
> >
> > > Siamack,
> > >
> > > This comment is way off track of the original question but I
> > > feel a need to ask this question. That's why I changed the
> > > subject. I'll even redraw the network so that we're all using
> > > the same context.
> > >
> > > ONU1 ------ OLT1/GFP ------------------- GFP/OLT2 ------ ONU2
> > > Ethernet SONET Ethernet
> > >
> > > Why are Pause frames used on the Ethernet links? The ONU should
> > > never be allowed to Pause the OLT as that would back-pressure
> > > the entire WAN. Since the WAN doesn't support back-pressure,
> > > packets over the SONET link that exceed the OLT's egress buffers
> > > would wind up being dropped at the OLT.
> >
> >That's basically what I said "OLT can simply buffer and subsequently drop
> >packets."
> >
> > >
> > >
> > > The OLT could Pause the ONU but for what reason? The only reason
> > > that I can think of would be to enforce the ONU's SLA. The point
> > > of putting an SLA in place is to enforce some set of rules,
> > > usually having to do with the minimum and maximum throughput
> > > guaranteed at the OLT for the ONU. The reason an SLA needs to
> > > be in place is because each side doesn't really trust the
> > > other's "handshake" and a legal document of sorts is needed.
> > > So, if neither side "trusts" the other, why do you rely on
> > > Pause frames to enforce the SLA? If the ONU will attempt to
> > > use as much bandwidth as possible, it will likely do so by
> > > ignoring the Pause frames from the OLT. This means that the
> > > only way the OLT can truly enforce the SLA is to be able to
> > > discard the frames that exceed the bandwidth agreed to in
> > > the SLA. If the OLT is capable of this, why even bother with
> > > Pause frames?
> >
> >You forget two things:
> >1. The bursty nature of traffic and the fact that peak is greater than the
> >committed
> >rate of service
> >2. That networks by definition protect themselves i.e. can not rely on
> >subscriber
> >alone
> >
> >If the subscriber exceeds its SLA then yes, frames would be dropped. But
> >if it's just
> >a burst, i.e. on average the subscriber is in compliance then buffering
> >and Pause
> >should do the job.
> >
> > >
> > >
> > > Sorry for being long winded but I'm trying to make a logical
> > > argument. What assumptions did I make that aren't valid?
> > >
> > > Thanks,
> > > Ben
> > >
> > > Siamack Ayandeh wrote:
> > > >
> > > > Shahram,
> > > >
> > > > It would be helpful to this discussion if you indicate what you had
> > in mind
> > > > with regard to OAM frames so one can think of a pragmatic
> > answer. For example
> > > > if you are worried about Pause frames: In this case the SONET/SDH
> > (OLT1-OLT2)
> > > > is often the bottleneck link. So Pause is used on the local link to
> > protect the
> > > > OLT-1/2 buffers. If the egress ONU back pressures the network, then
> > OLT can
> > > > simply buffer and subsequently drop packets. Otherwise fairly large
> > buffers
> > > > would be required to absorb the round trip time of the wide area. If
> > you think
> > > > about it as a two port bridge, again Pause is not propagated over the
> > wide
> > > > area. If you look at various IETF Pseudowire flavors, again following
> > the lead
> > > > of IEEE, Pause is not propagated over the wide area.
> > > >
> > > > Siamack
> > > >
> > > > Geoff Thompson wrote:
> > > >
> > > > > Matt-
> > > > >
> > > > > To further enrich the information that you provided below....
> > > > >
> > > > > 802.3 has a long, well established tradition of not supporting media
> > > > > converters.
> > > > >
> > > > > It would be my opinion that any "features" designed to specifically
> > support
> > > > > media converters were out of scope unless they were specifically
> > mentioned
> > > > > in the PAR.
> > > > >
> > > > > Geoff
> > > > >
> > > > > At 01:05 AM 2/12/2003 -0500, Matt Squire wrote:
> > > > >
> > > > > >My 2 cents.
> > > > > >
> > > > > >If there is a MAC layer in the ONU, then OAM terminates there. If a
> > > > > >vendor builds something without a MAC layer, then it doesn't terminate
> > > > > >there. If you put MACs in your ONU, you're really building something
> > > > > >akin to a 2-port bridge (likely with STP disabled
> > permanently). If you
> > > > > >don't, then its more along the lines of a media converter. Both
> > models
> > > > > >can work. Both models can interoperate. So do it anyway you
> > want, and
> > > > > >maybe the market will agree with you.
> > > > > >
> > > > > >- Matt
> > > > > >
> > > > > >Shahram Davari wrote:
> > > > > > >
> > > > > > > Roy,
> > > > > > >
> > > > > > > Since EOS is a private line service, It seems that you agree
> > with me
> > > > > > and Chak that for the EOS it seems from architectural point of
> > view the
> > > > > > OAM MAC frames should not be terminated at OLTs. Is that correct?
> > > > > > >
> > > > > > > Yours,
> > > > > > > -Shahram
> > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: Roy Bynum [mailto:rabynum@xxxxxxxxxxxxxx]
> > > > > > > > Sent: Tuesday, February 11, 2003 12:26 PM
> > > > > > > > To: Chau, Chak; stds-802-3-efm@ieee.org
> > > > > > > > Subject: RE: [EFM] Question regarding OAM in 802.3ah D1.3
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Chak,
> > > > > > > >
> > > > > > > > Depending on the service definition and who owns the service
> > > > > > > > facilities,
> > > > > > > > end to end OAM functionality would not necessarily
> > available. In all
> > > > > > > > packet services, the service provider would own at least a
> > > > > > > > portion, if not
> > > > > > > > all of the communications facilities. The OAM functionality
> > > > > > > > would then be
> > > > > > > > for the use of the service provider, not the customer. Only
> > > > > > > > with a Leased
> > > > > > > > Circuit type of service (also referred to as "Private Line")
> > > > > > > > would end to
> > > > > > > > end, OLT1 to OLT2 OAM functionality exist. For all other
> > types of
> > > > > > > > services, the OAM for Link 1 would not be the OAM for Link 2.
> > > > > > > > Other than
> > > > > > > > with a Leased Circuit service, the services are defined to
> > > > > > > > alter, filter,
> > > > > > > > or drop customer originated frames/packets in one way or
> > > > > > > > another, including
> > > > > > > > the OAM frames. This is the gist of the work that is being
> > done by
> > > > > > > > SG15/Q.10 under E.Ethna. Other than with a Leased Circuit
> > > > > > > > service, true
> > > > > > > > "transparency" does not exist.
> > > > > > > >
> > > > > > > > Thank you,
> > > > > > > > Roy Bynum
> > > > > > > >
> > > > > > > > At 10:15 AM 2/11/2003 -0600, Chau, Chak wrote:
> > > > > > > > >Hi David and All,
> > > > > > > > >
> > > > > > > > >I though for a PtP application the OAMPDUs message should be
> > > > > > > > transparent
> > > > > > > > >form OLT1 to OLT2.
> > > > > > > > >Or else, this will defeat the purpose of PtP, i.e., no L2 frame
> > > > > > > > >processing. Which means that OAM for Link1 can be OAM for
> > > > > > > > Link2, is that
> > > > > > > > >correct? This topic may be reviewed outside of EFM if prefered.
> > > > > > > > >
> > > > > > > > >Kind Regards,
> > > > > > > > >Chak
> > > > > > > > >
> > > > > > > > >Chak Chau
> > > > > > > > >FUJITSU, Transmission Development
> > > > > > > > >Phone: (972) 479-2795
> > > > > > > > >chak.chau@xxxxxxxxxxxxxxx
> > > > > > > > >chakavuth.chau@xxxxxxxxxxxx
> > > > > > > > >-----Original Message-----
> > > > > > > > >From: David Martin [mailto:dwmartin@xxxxxxxxxxxxxxxxxx]
> > > > > > > > >Sent: Monday, February 10, 2003 2:59 PM
> > > > > > > > >To: stds-802-3-efm@ieee.org
> > > > > > > > >Subject: RE: [EFM] Question regarding OAM in 802.3ah D1.3
> > > > > > > > >
> > > > > > > > >Shahram,
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >Agreed, the EoS (e.g. GFP-F) mapping is a simple
> > > > > > > > port-to-port mapping and
> > > > > > > > >doesn't include the full MAC sublayer processing (i.e. only
> > > > > > > > terminates
> > > > > > > > >IPG, preamble, SFD). Inspecting the MAC DA and filtering off
> > > > > > > > EFM OAMPDUs
> > > > > > > > >and processing them is required by the network application,
> > > > > > > > since the
> > > > > > > > >Ethernet link / PHY to which they apply is terminated. OAM
> > > > > > > > for link 1
> > > > > > > > >cannot be mixed with OAM for link 2 on the other side of the
> > > > > > > > provider's
> > > > > > > > >network.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >ONU1 -------------- OLT1/GFP------------------ GFP/OLT2
> > > > > > > > ------------ ONU2
> > > > > > > > > Ethernet SONET
> > > > > > > > Ethernet
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >It might be more appropriate to continue this privately, or
> > > > > > > > on the Q.12/15
> > > > > > > > >reflector.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >...Dave
> > > > > > > > >David W. Martin
> > > > > > > > >Nortel Networks
> > > > > > > > ><mailto:dwmartin@xxxxxxxx>dwmartin@xxxxxxxx
> > > > > > > > >+1 613 765-2901 (esn 395)
> > > > > > > > >~~~~~~~~~~~~~~~~~~~~
> > > > > > > > >-----Original Message-----
> > > > > > > > >From: Shahram Davari [mailto:Shahram_Davari@xxxxxxxxxxxxxx]
> > > > > > > > >Sent: Monday, February 10, 2003 3:16 PM
> > > > > > > > >To: Martin, David [SKY:QW10:EXCH]; stds-802-3-efm@ieee.org
> > > > > > > > >Subject: RE: [EFM] Question regarding OAM in 802.3ah D1.3
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >David,
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >I like to agree with you, but from layering architectural
> > > > > > > > point of view,
> > > > > > > > >the EOS box does not have to implement MAC
> > > > > > > > >layer (i.e., do any MAC lookup), rather a P-2-P EOS is a
> > > > > > > > kind of port
> > > > > > > > >transport in which all traffic coming form an Ethernet port
> > > > > > > > are send over
> > > > > > > > >a specific SONET channel.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >Please see further comments in-line:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >Thanks,
> > > > > > > > >-Shahram
> > > > > > > > >-----Original Message-----
> > > > > > > > >From: David Martin [mailto:dwmartin@xxxxxxxxxxxxxxxxxx]
> > > > > > > > >Sent: Monday, February 10, 2003 3:00 PM
> > > > > > > > >To: stds-802-3-efm@ieee.org
> > > > > > > > >Subject: RE: [EFM] Question regarding OAM in 802.3ah D1.3
> > > > > > > > >Shahram,
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >This question is somewhat out of scope wrt EFM, but the
> > > > > > > > answer is yes, the
> > > > > > > > >EFM OAMPDU flow must be terminated at the Provider Edge.
> > > > > > > > Otherwise it
> > > > > > > > >would flow through the provider's SONET network and get
> > > > > > > > mixed in with a
> > > > > > > > >separate EFM OAMPDU flow at the far end.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >SD=> Which separate OAMPDU flow? do you mean from ONU2 to OLT2?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Note that you have the terms ONU / OLT reversed.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >So the ONU is the customer side?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >This "filter function" is being defined in the ITU-T SG15 /
> > > > > > > > Q.12 work in
> > > > > > > > >draft G.ethna (was G.etna) and in the OIF UNI v2.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >Thanks I will have a look.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >The PE-PE, or SONET portion is not an EFM link, but rather a
> > > > > > > > SONET path,
> > > > > > > > >which has its own OAM (i.e. POH).
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >Agree.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >...Dave
> > > > > > > > >David W. Martin
> > > > > > > > >Nortel Networks
> > > > > > > > ><mailto:dwmartin@xxxxxxxx>dwmartin@xxxxxxxx
> > > > > > > > >+1 613 765-2901 (esn 395)
> > > > > > > > >~~~~~~~~~~~~~~~~~~~~
> > > > > > > > >-----Original Message-----
> > > > > > > > >From: Shahram Davari [mailto:Shahram_Davari@xxxxxxxxxxxxxx]
> > > > > > > > >Sent: Monday, February 10, 2003 2:13 PM
> > > > > > > > >To: stds-802-3-efm@ieee.org
> > > > > > > > >Subject: [EFM] Question regarding OAM in 802.3ah D1.3
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >Hi,
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >802.3ah section 57 says that the OAM defined is for single
> > link (or
> > > > > > > > >emulated link), and should not be forwarded by bridges/switches.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >My question is, in case of Ethernet over SONET transport
> > > > > > > > (GFP + Virtual
> > > > > > > > >concatenation), should the OAMPDU be terminated at the EOS
> > > > > > > > device or it
> > > > > > > > >should be transparently transported?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >Consider this example:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >OLT1 -------------- ONU 1------------------ ONU 2
> > ------------ OLT 2
> > > > > > > > > Ethernet SONET
> > Ethernet
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >Assume OLT1 and OLT2 are the customer equipments and ONU1
> > > > > > > > and ONU2 are
> > > > > > > > >provider
> > > > > > > > >transport equipments that transport Ethernet over SONET (but
> > > > > > > > don't do any
> > > > > > > > >switching/bridging).
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >In this case should ONU1 terminate OAMPDUs from OLT1 or it
> > > > > > > > should sent
> > > > > > > > >them transparently to OLT2?
> > > > > > > > >In other words is OLT1--ONU1 considered a single link? what
> > > > > > > > about ONU1 to
> > > > > > > > >ONU2, is this also a link?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >Thanks in advance,
> > > > > > > > >Shahram Davari
> > > > > > > > >Senior Product Research Engineer
> > > > > > > > >R&D Research Ottawa
> > > > > > > > >PMC-Sierra, Inc.
> > > > > > > > >Phone: (613) 271-4018
> > > > > > > > >Fax: (613) 271-6468
> > > > > > > > >
> > > > > > > >
> > >
> > > --
> > > -----------------------------------------
> > > Benjamin Brown
> > > AMCC
> > > 200 Minuteman Rd
> > > 3rd Floor
> > > Andover, MA 01810
> > > 978-247-8022 - Work
> > > 603-491-0296 - Cell
> > > 978-247-0024 - Fax
> > > 603-798-4115 - Home Office/Fax
> > > bbrown@xxxxxxxx
> > > -----------------------------------------
begin:vcard
n:Ayandeh;Siamack
tel;fax:781 271 9988
tel;work:781 276 4192
x-mozilla-html:FALSE
url:www.txc.com
org:Transwitch Corporation (Onex)
adr:;;34 Crosby Drive;Bedford;MA;01730;USA
version:2.1
email;internet:sayandeh@xxxxxxx
title:Senior Consulting Engineer
end:vcard