RE: [EFM] RE: Pause frame usage in transport networks
Siamak and all,
I was just talking to a major carrier and in their initial deployment
of EOS they are relying on customer shaping/rate limiting. They don't send
any PAUSE at all.
Yours,
Shahram
> -----Original Message-----
> From: Shahram Davari
> Sent: Thursday, February 20, 2003 2:21 PM
> To: 'Siamack Ayandeh'; Roy Bynum
> Cc: Ben Brown; Geoff Thompson; mattsquire@xxxxxxx; Chau Chak;
> stds-802-3-efm@ieee.org
> Subject: [EFM] RE: Pause frame usage in transport networks
>
>
>
> Siamak and all,
>
> Thanks for starting discussion on my original question.
> But my original question was more related to 802.3ah OAM.
> Do you think that the OLT should terminate 802.3ah OAM
> frames received from ONU?
>
> Please see some comments in-line.
>
> Yours,
> -Shahram
>
>
> > -----Original Message-----
> > From: Siamack Ayandeh [mailto:sayandeh@xxxxxxx]
> > Sent: Thursday, February 20, 2003 10:49 AM
> > To: Roy Bynum
> > Cc: Ben Brown; Geoff Thompson; mattsquire@xxxxxxx; Shahram
> > Davari; Chau
> > Chak; stds-802-3-efm@ieee.org
> > Subject: Re: Pause frame usage in transport networks
> >
> >
> > Roy,
> >
> > I suggest that we shy away from generalizations that arise
> > from a broad brush
> > discussion. I made very specific comments on the mailing list
> > and would be happy to
> > discuss specifics. Here is a brief summary (Please note that
> > discussion was
> > focused on EoS for Ethernet Private Line type services. So
> > both ends of the service
> > would have the same traffic parameters. More involved service
> > definitions were not
> > part of this discussion):
> >
> > 1. Network to subscriber Pause has advantages over shaping at
> > subscriber
>
> I wouldn't call it advantage. I think both are acceptable. In one case
> the ONU drops packet while in other case the OLT.
>
>
> > 2. Subscriber to network Pause is UNlikely to be invoked as
> > most often the backbone
> > link is the bottleneck. However if it does network would
> > buffer and subsequently
> > drop packets.
>
>
> Agree. The link BW is usually more than the SONET VC. Besides
> OLT can't
> back pressure the SONET.
>
> >Sending Pause to the far end is not practical
> > as it would require
> > large buffers to absorb the wide area round trip delays.
>
> We have heard some customers requiring this functionality. The
> problem is, how can we have both ONU-OLT pause and ONU-ONU pause?
>
> > 3. Pause would not interfere with OAM, if anything it avoids
> > loss of OAM frames and
> > the resulting time outs.
>
> Actually if you read 802.3ah section 55.1.6.3 it says that
> PAUSE frames
> may delay/prevent signaling of critical events. I think the
> reason is that
> PAUSE frames are treated the same as user MAC frames.
>
> >
> > Regards, Siamack
> >
> > Roy Bynum wrote:
> >
> > > Ben, Siamack,
> > >
> > > In these comments I am seeing the movement toward an link
> > facility that can
> > > be labeled as "Ethernet", but has little or none of the inherent
> > > characteristics of reliability and low latency variance.
> > Any Ethernet
> > > service that claims to be full duplex, but drops frames
> > without generating
> > > a "collision" when congested will fail meet the basic reliability
> > > characteristic of any 802.3 standard that I am aware of.
> > This is NOT Ethernet.
> > >
> > > The ONU should be allowed to "pause" the GFP/OLT on any one
> > link, and the
> > > GFP/OLT, should be allowed to "pause" the ONU on any one
> > link. With proper
> > > configuration of the operands, an ONU at one end of an end
> > to end "link"
> > > should be able to "pause" the ONU at the other end. Without that
> > > capability, what is being defined is no more reliable that
> > what exists
> > > today, and is some respects is less reliable than the alternative.
> > >
> > > When an end to end link starts "dropping" frames, the data
> > packets get
> > > retransmitted in new frames which now adds to the
> > congestion of a link and
> > > thus lowers the effective access bandwidth that can be
> > utilized. This has
> > > the effect of lowering the effective committed
> information rate even
> > > more. This is one of the primary reasons that experienced
> > WAN networking
> > > architects design IP networks to run as a nominal 30%
> > utilization. (I
> > > remember reading something by David Boggs about ~30%
> > utilization being the
> > > effective performance ceiling of congestion domain
> > networks. I turns out
> > > that he knew what he was talking about.)
> > >
> > > The current defined X.86/OLT does make use of "pause"
> > functionality, and
> > > will allow an ONU at one end of an end to end link "pause"
> > the ONU at the
> > > other end. By properly use of active flow control, the service
> > > communications link can perform as a non-congestion domain
> > link. With
> > > properly configured operands at the ONUs, this would be
> > highly reliable at
> > > the cost of lower predetermined bandwidth utilization, but without
> > > retransmissions. In experiments performed in the 1998-2000
> > time frame,
> > > effective utilization was found to be a direct ratio of
> > link/circuit speed
> > > to distance (I am having to write this from memory because
> > I no longer have
> > > access to the data.) Properly configured use of active
> > flow control can
> > > allow architecture designs with much higher effective
> > bandwidth utilization
> > > than 30%.
> > >
> > > Thank you,
> > > Roy Bynum
> > >
> > > At 10:15 PM 2/18/2003 -0500, Siamack Ayandeh wrote:
> > > >Ben,
> > > >
> > > >Please see my comments interleaved.
> > > >
> > > >Regards, Siamack
> > > >
> > > >Ben Brown wrote:
> > > >
> > > > > Siamack,
> > > > >
> > > > > This comment is way off track of the original question but I
> > > > > feel a need to ask this question. That's why I changed the
> > > > > subject. I'll even redraw the network so that we're all using
> > > > > the same context.
> > > > >
> > > > > ONU1 ------ OLT1/GFP ------------------- GFP/OLT2 ------ ONU2
> > > > > Ethernet SONET Ethernet
> > > > >
> > > > > Why are Pause frames used on the Ethernet links? The
> ONU should
> > > > > never be allowed to Pause the OLT as that would back-pressure
> > > > > the entire WAN. Since the WAN doesn't support back-pressure,
> > > > > packets over the SONET link that exceed the OLT's
> egress buffers
> > > > > would wind up being dropped at the OLT.
> > > >
> > > >That's basically what I said "OLT can simply buffer and
> > subsequently drop
> > > >packets."
> > > >
> > > > >
> > > > >
> > > > > The OLT could Pause the ONU but for what reason? The
> only reason
> > > > > that I can think of would be to enforce the ONU's
> SLA. The point
> > > > > of putting an SLA in place is to enforce some set of rules,
> > > > > usually having to do with the minimum and maximum throughput
> > > > > guaranteed at the OLT for the ONU. The reason an SLA needs to
> > > > > be in place is because each side doesn't really trust the
> > > > > other's "handshake" and a legal document of sorts is needed.
> > > > > So, if neither side "trusts" the other, why do you rely on
> > > > > Pause frames to enforce the SLA? If the ONU will attempt to
> > > > > use as much bandwidth as possible, it will likely do so by
> > > > > ignoring the Pause frames from the OLT. This means that the
> > > > > only way the OLT can truly enforce the SLA is to be able to
> > > > > discard the frames that exceed the bandwidth agreed to in
> > > > > the SLA. If the OLT is capable of this, why even bother with
> > > > > Pause frames?
> > > >
> > > >You forget two things:
> > > >1. The bursty nature of traffic and the fact that peak is
> > greater than the
> > > >committed
> > > >rate of service
> > > >2. That networks by definition protect themselves i.e. can
> > not rely on
> > > >subscriber
> > > >alone
> > > >
> > > >If the subscriber exceeds its SLA then yes, frames would
> > be dropped. But
> > > >if it's just
> > > >a burst, i.e. on average the subscriber is in compliance
> > then buffering
> > > >and Pause
> > > >should do the job.
> > > >
> > > > >
> > > > >
> > > > > Sorry for being long winded but I'm trying to make a logical
> > > > > argument. What assumptions did I make that aren't valid?
> > > > >
> > > > > Thanks,
> > > > > Ben
> > > > >
> > > > > Siamack Ayandeh wrote:
> > > > > >
> > > > > > Shahram,
> > > > > >
> > > > > > It would be helpful to this discussion if you
> > indicate what you had
> > > > in mind
> > > > > > with regard to OAM frames so one can think of a pragmatic
> > > > answer. For example
> > > > > > if you are worried about Pause frames: In this case
> > the SONET/SDH
> > > > (OLT1-OLT2)
> > > > > > is often the bottleneck link. So Pause is used on the
> > local link to
> > > > protect the
> > > > > > OLT-1/2 buffers. If the egress ONU back pressures the
> > network, then
> > > > OLT can
> > > > > > simply buffer and subsequently drop packets.
> > Otherwise fairly large
> > > > buffers
> > > > > > would be required to absorb the round trip time of
> > the wide area. If
> > > > you think
> > > > > > about it as a two port bridge, again Pause is not
> > propagated over the
> > > > wide
> > > > > > area. If you look at various IETF Pseudowire flavors,
> > again following
> > > > the lead
> > > > > > of IEEE, Pause is not propagated over the wide area.
> > > > > >
> > > > > > Siamack
> > > > > >
> > > > > > Geoff Thompson wrote:
> > > > > >
> > > > > > > Matt-
> > > > > > >
> > > > > > > To further enrich the information that you provided
> > below....
> > > > > > >
> > > > > > > 802.3 has a long, well established tradition of not
> > supporting media
> > > > > > > converters.
> > > > > > >
> > > > > > > It would be my opinion that any "features" designed
> > to specifically
> > > > support
> > > > > > > media converters were out of scope unless they were
> > specifically
> > > > mentioned
> > > > > > > in the PAR.
> > > > > > >
> > > > > > > Geoff
> > > > > > >
> > > > > > > At 01:05 AM 2/12/2003 -0500, Matt Squire wrote:
> > > > > > >
> > > > > > > >My 2 cents.
> > > > > > > >
> > > > > > > >If there is a MAC layer in the ONU, then OAM
> > terminates there. If a
> > > > > > > >vendor builds something without a MAC layer, then
> > it doesn't terminate
> > > > > > > >there. If you put MACs in your ONU, you're really
> > building something
> > > > > > > >akin to a 2-port bridge (likely with STP disabled
> > > > permanently). If you
> > > > > > > >don't, then its more along the lines of a media
> > converter. Both
> > > > models
> > > > > > > >can work. Both models can interoperate. So do it
> > anyway you
> > > > want, and
> > > > > > > >maybe the market will agree with you.
> > > > > > > >
> > > > > > > >- Matt
> > > > > > > >
> > > > > > > >Shahram Davari wrote:
> > > > > > > > >
> > > > > > > > > Roy,
> > > > > > > > >
> > > > > > > > > Since EOS is a private line service, It seems
> > that you agree
> > > > with me
> > > > > > > > and Chak that for the EOS it seems from
> > architectural point of
> > > > view the
> > > > > > > > OAM MAC frames should not be terminated at OLTs.
> > Is that correct?
> > > > > > > > >
> > > > > > > > > Yours,
> > > > > > > > > -Shahram
> > > > > > > > >
> > > > > > > > > > -----Original Message-----
> > > > > > > > > > From: Roy Bynum [mailto:rabynum@xxxxxxxxxxxxxx]
> > > > > > > > > > Sent: Tuesday, February 11, 2003 12:26 PM
> > > > > > > > > > To: Chau, Chak; stds-802-3-efm@ieee.org
> > > > > > > > > > Subject: RE: [EFM] Question regarding OAM in
> > 802.3ah D1.3
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Chak,
> > > > > > > > > >
> > > > > > > > > > Depending on the service definition and who
> > owns the service
> > > > > > > > > > facilities,
> > > > > > > > > > end to end OAM functionality would not necessarily
> > > > available. In all
> > > > > > > > > > packet services, the service provider would
> > own at least a
> > > > > > > > > > portion, if not
> > > > > > > > > > all of the communications facilities. The
> > OAM functionality
> > > > > > > > > > would then be
> > > > > > > > > > for the use of the service provider, not the
> > customer. Only
> > > > > > > > > > with a Leased
> > > > > > > > > > Circuit type of service (also referred to as
> > "Private Line")
> > > > > > > > > > would end to
> > > > > > > > > > end, OLT1 to OLT2 OAM functionality exist.
> > For all other
> > > > types of
> > > > > > > > > > services, the OAM for Link 1 would not be the
> > OAM for Link 2.
> > > > > > > > > > Other than
> > > > > > > > > > with a Leased Circuit service, the services
> > are defined to
> > > > > > > > > > alter, filter,
> > > > > > > > > > or drop customer originated frames/packets in
> > one way or
> > > > > > > > > > another, including
> > > > > > > > > > the OAM frames. This is the gist of the work
> > that is being
> > > > done by
> > > > > > > > > > SG15/Q.10 under E.Ethna. Other than with a
> > Leased Circuit
> > > > > > > > > > service, true
> > > > > > > > > > "transparency" does not exist.
> > > > > > > > > >
> > > > > > > > > > Thank you,
> > > > > > > > > > Roy Bynum
> > > > > > > > > >
> > > > > > > > > > At 10:15 AM 2/11/2003 -0600, Chau, Chak wrote:
> > > > > > > > > > >Hi David and All,
> > > > > > > > > > >
> > > > > > > > > > >I though for a PtP application the OAMPDUs
> > message should be
> > > > > > > > > > transparent
> > > > > > > > > > >form OLT1 to OLT2.
> > > > > > > > > > >Or else, this will defeat the purpose of
> > PtP, i.e., no L2 frame
> > > > > > > > > > >processing. Which means that OAM for Link1
> > can be OAM for
> > > > > > > > > > Link2, is that
> > > > > > > > > > >correct? This topic may be reviewed outside
> > of EFM if prefered.
> > > > > > > > > > >
> > > > > > > > > > >Kind Regards,
> > > > > > > > > > >Chak
> > > > > > > > > > >
> > > > > > > > > > >Chak Chau
> > > > > > > > > > >FUJITSU, Transmission Development
> > > > > > > > > > >Phone: (972) 479-2795
> > > > > > > > > > >chak.chau@xxxxxxxxxxxxxxx
> > > > > > > > > > >chakavuth.chau@xxxxxxxxxxxx
> > > > > > > > > > >-----Original Message-----
> > > > > > > > > > >From: David Martin
> > [mailto:dwmartin@xxxxxxxxxxxxxxxxxx]
> > > > > > > > > > >Sent: Monday, February 10, 2003 2:59 PM
> > > > > > > > > > >To: stds-802-3-efm@ieee.org
> > > > > > > > > > >Subject: RE: [EFM] Question regarding OAM in
> > 802.3ah D1.3
> > > > > > > > > > >
> > > > > > > > > > >Shahram,
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >Agreed, the EoS (e.g. GFP-F) mapping is a simple
> > > > > > > > > > port-to-port mapping and
> > > > > > > > > > >doesn't include the full MAC sublayer
> > processing (i.e. only
> > > > > > > > > > terminates
> > > > > > > > > > >IPG, preamble, SFD). Inspecting the MAC DA
> > and filtering off
> > > > > > > > > > EFM OAMPDUs
> > > > > > > > > > >and processing them is required by the
> > network application,
> > > > > > > > > > since the
> > > > > > > > > > >Ethernet link / PHY to which they apply is
> > terminated. OAM
> > > > > > > > > > for link 1
> > > > > > > > > > >cannot be mixed with OAM for link 2 on the
> > other side of the
> > > > > > > > > > provider's
> > > > > > > > > > >network.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >ONU1 --------------
> > OLT1/GFP------------------ GFP/OLT2
> > > > > > > > > > ------------ ONU2
> > > > > > > > > > > Ethernet SONET
> > > > > > > > > > Ethernet
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >It might be more appropriate to continue
> > this privately, or
> > > > > > > > > > on the Q.12/15
> > > > > > > > > > >reflector.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >...Dave
> > > > > > > > > > >David W. Martin
> > > > > > > > > > >Nortel Networks
> > > > > > > > > > ><mailto:dwmartin@xxxxxxxx>dwmartin@xxxxxxxx
> > > > > > > > > > >+1 613 765-2901 (esn 395)
> > > > > > > > > > >~~~~~~~~~~~~~~~~~~~~
> > > > > > > > > > >-----Original Message-----
> > > > > > > > > > >From: Shahram Davari
> > [mailto:Shahram_Davari@xxxxxxxxxxxxxx]
> > > > > > > > > > >Sent: Monday, February 10, 2003 3:16 PM
> > > > > > > > > > >To: Martin, David [SKY:QW10:EXCH];
> > stds-802-3-efm@ieee.org
> > > > > > > > > > >Subject: RE: [EFM] Question regarding OAM in
> > 802.3ah D1.3
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >David,
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >I like to agree with you, but from layering
> > architectural
> > > > > > > > > > point of view,
> > > > > > > > > > >the EOS box does not have to implement MAC
> > > > > > > > > > >layer (i.e., do any MAC lookup), rather a
> > P-2-P EOS is a
> > > > > > > > > > kind of port
> > > > > > > > > > >transport in which all traffic coming form
> > an Ethernet port
> > > > > > > > > > are send over
> > > > > > > > > > >a specific SONET channel.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >Please see further comments in-line:
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >Thanks,
> > > > > > > > > > >-Shahram
> > > > > > > > > > >-----Original Message-----
> > > > > > > > > > >From: David Martin
> > [mailto:dwmartin@xxxxxxxxxxxxxxxxxx]
> > > > > > > > > > >Sent: Monday, February 10, 2003 3:00 PM
> > > > > > > > > > >To: stds-802-3-efm@ieee.org
> > > > > > > > > > >Subject: RE: [EFM] Question regarding OAM in
> > 802.3ah D1.3
> > > > > > > > > > >Shahram,
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >This question is somewhat out of scope wrt
> > EFM, but the
> > > > > > > > > > answer is yes, the
> > > > > > > > > > >EFM OAMPDU flow must be terminated at the
> > Provider Edge.
> > > > > > > > > > Otherwise it
> > > > > > > > > > >would flow through the provider's SONET
> > network and get
> > > > > > > > > > mixed in with a
> > > > > > > > > > >separate EFM OAMPDU flow at the far end.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >SD=> Which separate OAMPDU flow? do you mean
> > from ONU2 to OLT2?
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Note that you have the terms ONU / OLT reversed.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >So the ONU is the customer side?
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >This "filter function" is being defined in
> > the ITU-T SG15 /
> > > > > > > > > > Q.12 work in
> > > > > > > > > > >draft G.ethna (was G.etna) and in the OIF UNI v2.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >Thanks I will have a look.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >The PE-PE, or SONET portion is not an EFM
> > link, but rather a
> > > > > > > > > > SONET path,
> > > > > > > > > > >which has its own OAM (i.e. POH).
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >Agree.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >...Dave
> > > > > > > > > > >David W. Martin
> > > > > > > > > > >Nortel Networks
> > > > > > > > > > ><mailto:dwmartin@xxxxxxxx>dwmartin@xxxxxxxx
> > > > > > > > > > >+1 613 765-2901 (esn 395)
> > > > > > > > > > >~~~~~~~~~~~~~~~~~~~~
> > > > > > > > > > >-----Original Message-----
> > > > > > > > > > >From: Shahram Davari
> > [mailto:Shahram_Davari@xxxxxxxxxxxxxx]
> > > > > > > > > > >Sent: Monday, February 10, 2003 2:13 PM
> > > > > > > > > > >To: stds-802-3-efm@ieee.org
> > > > > > > > > > >Subject: [EFM] Question regarding OAM in
> 802.3ah D1.3
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >Hi,
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >802.3ah section 57 says that the OAM defined
> > is for single
> > > > link (or
> > > > > > > > > > >emulated link), and should not be forwarded
> > by bridges/switches.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >My question is, in case of Ethernet over
> > SONET transport
> > > > > > > > > > (GFP + Virtual
> > > > > > > > > > >concatenation), should the OAMPDU be
> > terminated at the EOS
> > > > > > > > > > device or it
> > > > > > > > > > >should be transparently transported?
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >Consider this example:
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >OLT1 -------------- ONU 1------------------ ONU 2
> > > > ------------ OLT 2
> > > > > > > > > > > Ethernet SONET
> > > > Ethernet
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >Assume OLT1 and OLT2 are the customer
> > equipments and ONU1
> > > > > > > > > > and ONU2 are
> > > > > > > > > > >provider
> > > > > > > > > > >transport equipments that transport Ethernet
> > over SONET (but
> > > > > > > > > > don't do any
> > > > > > > > > > >switching/bridging).
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >In this case should ONU1 terminate OAMPDUs
> > from OLT1 or it
> > > > > > > > > > should sent
> > > > > > > > > > >them transparently to OLT2?
> > > > > > > > > > >In other words is OLT1--ONU1 considered a
> > single link? what
> > > > > > > > > > about ONU1 to
> > > > > > > > > > >ONU2, is this also a link?
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >Thanks in advance,
> > > > > > > > > > >Shahram Davari
> > > > > > > > > > >Senior Product Research Engineer
> > > > > > > > > > >R&D Research Ottawa
> > > > > > > > > > >PMC-Sierra, Inc.
> > > > > > > > > > >Phone: (613) 271-4018
> > > > > > > > > > >Fax: (613) 271-6468
> > > > > > > > > > >
> > > > > > > > > >
> > > > >
> > > > > --
> > > > > -----------------------------------------
> > > > > Benjamin Brown
> > > > > AMCC
> > > > > 200 Minuteman Rd
> > > > > 3rd Floor
> > > > > Andover, MA 01810
> > > > > 978-247-8022 - Work
> > > > > 603-491-0296 - Cell
> > > > > 978-247-0024 - Fax
> > > > > 603-798-4115 - Home Office/Fax
> > > > > bbrown@xxxxxxxx
> > > > > -----------------------------------------
> >
>