RE: Wide Area Networking for the Rest of US - the debate on BER and other issues

["Bill St. Arnaud" <bill.st.arnaud@xxxxxxxxxx>]

Larry:

What you say may be true for other types of networks.  But dropped packets
and re-transmissions are an essential feature of Internet networks.  The
TCP/IP congestion control mechanisms uses dropped packets as a mechanism to
signal the source to throttle back the data flow.

In fact many ISPs use a utility called RED ( Random Early Discard ) or WRED
( Weighted Early Random Discard) to deliberately drop packets as a mechanism
to throttle traffic on congested links.   Yes this does cause a
re-transmission, but TCP automatically drops down to a lower speed when this
happens.  As a result on most Internet links about 1-3% of the traffic is
dropped packet and re=transmissions.  However, most of these dropped packets
are not due to RED but to buffer overflow at the destination receiver.
SIGCOMM'98 has some excellent papers documenting this behaviour on the
Internet.

If I have to do packet discard in any event I might as well do it a layer 1
just as well as at layer 3.  More importantly if I am already dropping
packets for other reasons, then as long as the number of dropped packets
from BER is less than the number of dropped packets from TCP congestion
control then the actual BER (whether it is 10^-15 or 10^-8) is irrelevant to
me.

I am assuming that if 10XGbE is used in the long haul the primary
application will be to carry Internet traffic.  That is why it would be nice
to have an option for those of use who are running Internet networks to have
a BER Knob.  With a BER knob I may be able to extend my repeater distance,
use lower cost lasers, etc etc.  However, as I said before this may still
may not be practical because of other issues particularly with respect to
the non-linear factors that affect BER.  But it still might be worth a
cursory investigation.


Bill



-------------------------------------------
Bill St Arnaud
Director Network Projects
CANARIE
bill.st.arnaud@xxxxxxxxxx
http://tweetie.canarie.ca/~bstarn

 

 



> -----Original Message-----
> From: owner-stds-802-3-hssg@xxxxxxxxxxxxxxxxxx
> [mailto:owner-stds-802-3-hssg@xxxxxxxxxxxxxxxxxx]On Behalf Of Larry
> Miller
> Sent: Wednesday, June 02, 1999 11:38 AM
> To: stds-802-3-hssg@xxxxxxxx
> Subject: Re: Wide Area Networking for the Rest of US - the debate on BER
> and other issues
>
>
>
> I think the bit is that when you report bad frames upward to higher layers
> they have to do some work to re-request those frames and that takes much
> longer than the time actually burned by the dropped frames. Hence, if you
> get too low of a raw BER you spend all (or maybe more than all)
> of your time
> with higher layer thrashing and never get through with the (say) file
> transfer.
>
> This, I think, is the fallacy in Mr St. Arnaud's notion.
>
> Larry Miller
> Nortel Networks
>
>
> -----Original Message-----
> From: Mike Dudek <mdudek@xxxxxxxxxxxx>
> To: Chang, Edward S <Edward.Chang@xxxxxxxxxx>
> Cc: bin.guo@xxxxxxx <bin.guo@xxxxxxx>; bill.st.arnaud@xxxxxxxxxx
> <bill.st.arnaud@xxxxxxxxxx>; rtaborek@xxxxxxxxxxxxxxxx
> <rtaborek@xxxxxxxxxxxxxxxx>; dwmartin@xxxxxxxxxxxxxxxxxx
> <dwmartin@xxxxxxxxxxxxxxxxxx>; stds-802-3-hssg@xxxxxxxx
> <stds-802-3-hssg@xxxxxxxx>; sachs@xxxxxxxxxxxxxx <sachs@xxxxxxxxxxxxxx>
> Date: Tuesday, June 01, 1999 5:42 PM
> Subject: Re: Wide Area Networking for the Rest of US - the debate
> on BER and
> other issues
>
>
> >
> >Agreed, but the percentage of good frames stays the same.  ie the
> percentage
> >bandwidth used for retransmissions is the same.
> >
> >"Chang, Edward S" wrote:
> >
> >> Mike:
> >>
> >> If the BER is maintained the same for both GbE and 10xGbE and assume
> >> everything is equal, the frequency of getting error from 10GbE is 10
> times
> >> than GbE from PHY.  Of course, the whole system has other factors to be
> >> included to find the final throughput.  In another word, the occurrence
> of
> >> frame error will be much more for 10GbE than GbE.
> >>
> >> I may present mathematical analysis in July, if my time is allowed.
> >>
> >> Ed Chang
> >> Unisys Corporation
> >>
> >> -----Original Message-----
> >> From: Mike Dudek [mailto:mdudek@xxxxxxxxxxxx]
> >> Sent: Tuesday, June 01, 1999 10:07 AM
> >> To: Chang, Edward S
> >> Cc: bin.guo@xxxxxxx; bill.st.arnaud@xxxxxxxxxx;
> >> rtaborek@xxxxxxxxxxxxxxxx; dwmartin@xxxxxxxxxxxxxxxxxx;
> >> stds-802-3-hssg@xxxxxxxx; sachs@xxxxxxxxxxxxxx
> >> Subject: Re: Wide Area Networking for the Rest of US - the
> debate on BER
> >> and other issues
> >>
> >> I do not agree that the BER must be improved with data rate increase in
> >> order to
> >> obtain the higher throughput.  At least for packet based transmission
> with
> >> retransmission of errored packets, the throughput increases in
> proportion
> to
> >> the
> >> data rate for the same BER, assuming that the packet length (in bytes)
> >> remains
> >> fixed.  I do not think that anyone has proposed changing the packet
> length,
> >> but
> >> if they did then the BER might have to be improved.  The
> throughput is of
> >> course
> >> the number of good packets in any interval of time.
> >>
> >> "Chang, Edward S" wrote:
> >>
> >> > Bin:
> >> >
> >> > Yes, I agree.  The BER should be improved with data rate increase, if
> the
> >> > through put gained from higher data rate is to be maintained.  In
> addition
> >> > to the retry times wasted, the external sources of noise remain the
> same,
> >> > which further requires the lower BER.  These are the correct design
> goals
> >> we
> >> > should work on.  Although, we also should keep the cost-effectiveness
> in
> >> > mind to maintain optimum balance between performance and cost.
> >> >
> >> > Ed Chang
> >> > Unisys Corporation
> >> >
> >> > -----Original Message-----
> >> > From: bin.guo@xxxxxxx [mailto:bin.guo@xxxxxxx]
> >> > Sent: Friday, May 28, 1999 4:57 PM
> >> > To: Edward.Chang@xxxxxxxxxx; bill.st.arnaud@xxxxxxxxxx;
> >> > rtaborek@xxxxxxxxxxxxxxxx; dwmartin@xxxxxxxxxxxxxxxxxx
> >> > Cc: stds-802-3-hssg@xxxxxxxx; sachs@xxxxxxxxxxxxxx;
> "widmer@xxxxxxxxxx
> >> > widmer@xxxxxxxxxx widmer"@us.ibm.com
> >> > Subject: RE: Wide Area Networking for the Rest of US - the debate on
> BER
> >> > a nd other issues
> >> >
> >> > Ed,
> >> >
> >> > If the specified BER for 1000BASE-X is 10^ -12, then to have
> the equal
> >> > error-free period the specified BER for 10G should be at
> least 10^ -13.
> >> > Based on Rich T and Rich S's BER number:
> >> >
> >> > A system BER of 10 E - 8 @  10 Mbps = a bit error every 10 seconds.
> >> > (10BASE-T)
> >> > A system BER of 10 E-12 @ 100 Mbps = a bit error every 166
> minutes, 40
> >> > seconds.        (100BASE-X)
> >> > A system BER of 10 E-10 @     1 Gbps = a bit error every 1
> minutes, 40
> >> > seconds.                (1000BASE-T)
> >> > A system BER of 10 E-12 @     1 Gbps = a bit error every 16
> minutes, 40
> >> > seconds.        (1000BASE-X)
> >> > A system BER of 10 E-12 @   10 Gbps = a bit error every 1 minutes, 40
> >> > seconds.
> >> > A system BER of 10 E-13 @   10 Gbps = a bit error every 16
> minutes, 40
> >> > seconds.
> >> >
> >> > If the TCP/IP is the only protocol 10G PHY needs to support, then the
> >> above
> >> > specified BER may be more than enough.  Moving from 1G to
> 10G, the bit
> >> > period is scaled 10X smaller while jitter and noise from some sources
> are
> >> > not scaled the same way -- much tight control should be applied to
> achieve
> >> > even the same BER.
> >> >
> >> > Bin
> >> >
> >> > ADL,AMD
> >> >
> >> >
> >> >
> >> >
> >> > > -----Original Message-----
> >> > > From: Chang, Edward S [SMTP:Edward.Chang@xxxxxxxxxx]
> >> > > Sent: Friday, May 28, 1999 12:44 PM
> >> > > To:   bill.st.arnaud@xxxxxxxxxx; Guo, Bin;
> rtaborek@xxxxxxxxxxxxxxxx;
> >> > > dwmartin@xxxxxxxxxxxxxxxxxx
> >> > > Cc:   stds-802-3-hssg@xxxxxxxx; sachs@xxxxxxxxxxxxxx;
> "widmer@xxxxxxxxxx
> >> > > widmer@xxxxxxxxxx          widmer"@us.ibm.com
> >> > > Subject:      RE: Wide Area Networking for the Rest of US - the
> debate
> >> on
> >> > > BER a nd other issues
> >> > >
> >> > > Bill:
> >> > >
> >> > > I like your idea of implementing native 10xGBE for
> intermediate long
> >> haul
> >> > > and WAN, which is a good move.  The advantage you are
> mentioning will
> >> > > greatly reduce the cost to users.
> >> > >
> >> > > It is true, in a TCP/IP links, the TCP flow control causes more
> >> > > retransmission than BER. Therefore, the extremely low BER, 10^-15,
> does
> >> > > not
> >> > > necessarily gain any more advantage than the specified BER
> of 10^-12.
> >> > >
> >> > >
> >> > > Ed Chang
> >> > >
> >> > > -----Original Message-----
> >> > > From: Bill St. Arnaud [mailto:bill.st.arnaud@xxxxxxxxxx]
> >> > > Sent: Friday, May 28, 1999 8:52 AM
> >> > > To: bin.guo@xxxxxxx; rtaborek@xxxxxxxxxxxxxxxx;
> >> > > dwmartin@xxxxxxxxxxxxxxxxxx
> >> > > Cc: stds-802-3-hssg@xxxxxxxx; sachs@xxxxxxxxxxxxxx;
> "widmer@xxxxxxxxxx
> >> > > widmer@xxxxxxxxxx widmer"@us.ibm.com
> >> > > Subject: Wide Area Networking for the Rest of US - the
> debate on BER
> and
> >> > > other issues
> >> > >
> >> > >
> >> > >
> >> > > All:
> >> > > I have been following the interesting debate about BER.
> Let me bring
> >> some
> >> > > further issues into the debate.
> >> > >
> >> > > I am assuming that on WAN and long haul GbE the upper
> layer protocol
> >> will
> >> > > only be IP.
> >> > >
> >> > > On most IP links, even ones with BERs of 10^-15 there is about 1-3%
> >> packet
> >> > > loss and retransmission.  This is due to a number of
> factors but most
> >> > > typically it relates to TCP flow control mechanism from
> server bound
> >> > > congestion (not network congestion) and the use of WRED in routers.
> >> > >
> >> > > So, on most IP links the packet loss due to BER is
> significantly less
> >> than
> >> > > that due to normal TCP congestion.  As long as that ratio is
> maintained
> >> it
> >> > > is largely irrelevant what the absolute BER value is.
> There will be
> >> many
> >> > > more retransmissions from the IP layer than there will be at the
> >> physical
> >> > > layer due to BER.
> >> > >
> >> > > Other protocols like Frame Relay and SNA are a lot more
> sensitive to
> >> high
> >> > > BERs.  IP ( in particular TCP/IP) is significantly more robust and
> can
> >> > > work
> >> > > quite effectively in high BER environments e.g. TCP/IP over barbed
> wire.
> >> > >
> >> > > I would like to suggest that the 802.3 HSSG group consider an 2
> >> solutions
> >> > > for 10xGbE WAN:
> >> > > (1) native 10xGbE using 8b/10b; and
> >> > > (2)10xGbE mapped to a SONET STS OC-192 frame
> >> > >
> >> > > For extreme long haul solutions SONET makes a lot of sense as a
> >> transport
> >> > > technology.  However for intermediate long haul (up to 1000 km) and
> WAN
> >> > > native 10xGbE is more attractive. Native GbE can be either
> transported
> >> on
> >> > > a
> >> > > transparent optical network or carried directly on a CWDM
> system with
> >> > > transceivers. In medium range networks coding efficiency is not as
> >> > > important
> >> > > as it is in long haul networks. If coding efficiency is important
> then
> >> in
> >> > > my
> >> > > opinion, it does not make sense to invent a new coding scheme for
> 10xGbE
> >> > > when it would be just as easy to map it to a SONET frame.
> >> > >
> >> > > The attraction of native 10xGbE for the WAN is that it is a "wide
> area
> >> > > networking solution for the rest of us".  You don't need to hire
> >> > > specialized
> >> > > SONET engineers to run and manage your networks.  The 18
> year old kid
> >> who
> >> > > is
> >> > > running your LAN can now easily learn to operate and manage a WAN.
> >> > >
> >> > > In Canada and the US, there are several vendors who are willing to
> sell
> >> > > dark
> >> > > fiber at a very reasonable cost.  Right now the cost of building a
> WAN
> >> > > with
> >> > > 10xGbE and CWDM is substantially less (for comparable data rates)
> than
> >> > > using
> >> > > SONET equipment.
> >> > >
> >> > > Bill
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > -------------------------------------------
> >> > > Bill St Arnaud
> >> > > Director Network Projects
> >> > > CANARIE
> >> > > bill.st.arnaud@xxxxxxxxxx
> >> > > http://tweetie.canarie.ca/~bstarn
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > > -----Original Message-----
> >> > > > From: owner-stds-802-3-hssg@xxxxxxxxxxxxxxxxxx
> >> > > > [mailto:owner-stds-802-3-hssg@xxxxxxxxxxxxxxxxxx]On Behalf Of
> >> > > > bin.guo@xxxxxxx
> >> > > > Sent: Thursday, May 27, 1999 7:28 PM
> >> > > > To: rtaborek@xxxxxxxxxxxxxxxx; dwmartin@xxxxxxxxxxxxxxxxxx
> >> > > > Cc: stds-802-3-hssg@xxxxxxxx; sachs@xxxxxxxxxxxxxx;
> "widmer@xxxxxxxxxx
> >> > > > widmer@xxxxxxxxxx widmer"@us.ibm.com
> >> > > > Subject: RE: 1000BASE-T PCS question
> >> > > >
> >> > > >
> >> > > >
> >> > > > Rich,
> >> > > >
> >> > > > The DC balance can be directly translated into jitter
> (when timing
> is
> >> > > > concerned) and offset (when threshold slicing is concerned).  You
> >> > > > only need
> >> > > > to deal with the former if the signal is 2-level NRZI, while you
> need
> >> to
> >> > > > deal with both if multi-level signal modulation is used.
> >> > > >
> >> > > > For long term DC imbalance, it translates into low
> frequency jitter
> >> and
> >> > > if
> >> > > > it's low enough(<1 KHz ?), it's called baseline wonder.  For
> >> > > > short term, it
> >> > > > relates to Data Dependent Jitter, which is more difficult for
> timing
> >> > > > recovery to handle since it's not from system or channel
> imparity,
> and
> >> > > > therefore it's harder to compensate.
> >> > > >
> >> > > > When you have a lot of jitter margin, for example in lower speed
> >> > > clocking,
> >> > > > the amount of jitter, translated from DC drift resulted from data
> >> > > > imbalance
> >> > > > coupled by AC circuit, percentage wise is a small portion of the
> clock
> >> > > > period and therefore does not contribute to much of the eye
> >> > > > closing.  On the
> >> > > > other hand, for high speed clocking at 10G (100 ps?), the jitter
> >> > > > translated
> >> > > > from the same amount of DC drift can be a significant portion of
> the
> >> > > clock
> >> > > > period, so contributes to much large percentage wise jitter which
> >> > > > results in
> >> > > > reduced eye opening -- higher BER.
> >> > > >
> >> > > > Dave said in his mail that "The limiting factor is enough RX
> optical
> >> > > power
> >> > > > to provide a sufficiently open eye." but you still have to deal
> with
> >> the
> >> > > > data dependent jitter due to DC imbalance generated
> after O/E, that
> >> can
> >> > > > close the eye further again.
> >> > > >
> >> > > > Bin
> >> > > >
> >> > > > ADL, AMD
> >> > > >
> >> > > > > -----Original Message-----
> >> > > > > From:     Rich Taborek [SMTP:rtaborek@xxxxxxxxxxxxxxxx]
> >> > > > > Sent:     Thursday, May 27, 1999 3:23 PM
> >> > > > > To:       David Martin
> >> > > > > Cc:       HSSG_reflector; Sachs,Marty; Widmer,Albert_X
> >> > > > > Subject:  Re: 1000BASE-T PCS question
> >> > > > >
> >> > > > >
> >> > > > > Dave,
> >> > > > >
> >> > > > > Do you know of any research or other proofs in this
> area? You say
> >> that
> >> > > > > lower speed SONET links regularly achieves BERs of < 10 E-15. I
> have
> >> > > > > substantial experience with mainframe serial links such as
> ESCON(tm)
> >> > > > > where the effective system BERs are in the same ballpark. SONET
> uses
> >> > > > > scrambling with long term DC balance and ESCON uses 8B/10B with
> >> short
> >> > > > > term DC balance. The following questions come to mind:
> >> > > > >
> >> > > > > - How important is DC balance?
> >> > > > > - How does this importance scale in going to 10 Gbps?
> >> > > > >
> >> > > > > I'll see if I can get some 8B/10B experts to chime in
> here if you
> >> can
> >> > > > > get scrambling experts to bear down on the same problem.
> >> > > > >
> >> > > > > --
> >> > > > >
> >> > > > > >(text deleted)
> >> > > > > >
> >> > > > > >The point here is that the SONET scrambler is not the limiting
> >> issue
> >> > > in
> >> > > > > >achieving low error rates. The issue is having enough
> photons/bit,
> >> or
> >> > > > > >optical SNR (eye-Q) to accurately recover the data.
> >> > > > > >
> >> > > > > >...Dave
> >> > > > > >
> >> > > > > >David W. Martin
> >> > > > > >Nortel Networks
> >> > > > > >+1 613 765-2901
> >> > > > > >+1 613 763-2388 (fax)
> >> > > > > >dwmartin@xxxxxxxxxxxxxxxxxx
> >> > > > > >========================
> >> > > >
> >
> >
>
>
>