RE: A telephony carrier industry perspective
- To: Andrew Smith <andrew@xxxxxxxxxxxxxxxxxxx>
- Subject: RE: A telephony carrier industry perspective
- From: Roy Bynum <RBYNUM/0004245935@xxxxxxxxxxx>
- Date: Tue, 18 May 1999 09:46:28 -0500 (EST)
- Cc: IEEE HSSG <stds-802-3-hssg@xxxxxxxx>
- Sender: owner-stds-802-3-hssg@xxxxxxxxxxxxxxxxxx
Andrew,
I have been designing and implementing all sorts of data networking
environments, LAN/MAN/WAN, for some time. One of the headaches is
understanding the business application requirements and cost trade-off
on link circuit bandwidth. Because intermediate/gateway systems
historically do not have the ability to signal back when an output
circuit becomes overloaded (the new term is "over-subscribed"), queue
buffering has been a major issue. The problem is a network that has
circuits that consistently become overloaded and do so for extended
lengths of time, such that the queues become overloaded. Data network
architects tend to design large enterprise WAN networks based on a
consistent loading with an attempt to limit the peaks to close to the
maximum circuit bandwidth.
With "flow-control" using GbE I can design WAN data networks closer to
the consistent loading with less concern for peak traffic. I have set
up a test WAN network using Native data GbE over Optical DWDM and
SONET DATS to test this. When the L2/L3 GbE switches are properly
configured, I can only attempt to overload the network "circuits".
When the flow-control is applied all the way to the data source
systems, the flow-control will continue to cascade back such that
over-subscription does not occur. Unlike a traditional TDM WAN circuit
based design, no data is lost during peak "overloading", which tends
to prevent more problems because of re-transmissions. As this is
better understood and implemented, it will change the bandwidth
allocation process of designing enterprise networks.
Thank you,
Roy Bynum
Date: Mon May 17, 1999 5:27 pm CST
Source-Date: Mon, 17 May 1999 16:26:58 -0700
From: Andrew Smith
EMS: INTERNET / MCI ID: 376-5414
MBX: andrew@xxxxxxxxxxxxxxxxxxx
TO: * ROY BYNUM / MCI ID: 424-5935
CC: IEEE HSSG
EMS: INTERNET / MCI ID: 376-5414
MBX: stds-802-3-hssg@xxxxxxxx
Subject: RE: A telephony carrier industry perspective
Message-Id: 99051723270571/INTERNETGWDN1IG
Source-Msg-Id: <D0805D3B448BD211A7990008C7B18130142CFC@xxxxxxxxxxxxxxxxxxxxxxx>
U-Content-return: allowed
U-X-Mailer: Internet Mail Service (5.5.2448.0)
Roy,
At the risk of waking some sleeping dogs, could I ask you to elaborate a
little on your statement:
'In addition to being less expensive, GbE over Native Data SONET DATS will
provide "subscription control" though "flow control" That "subscription
control" will prevent over-subscribing the WAN links, which is a big problem
for enterprise data network designers and architects.'
Andrew
****************************************************************
Andrew Smith tel: +1 (408) 579-2821
Extreme Networks fax: +1 (408) 579-3000
3585 Monroe St. http://www.extremenetworks.com
Santa Clara CA 95051-1450 em: andrew@xxxxxxxxxxxxxxxxxxx
****************************************************************
> -----Original Message-----
> From: Roy Bynum [mailto:RBYNUM/0004245935@xxxxxxxxxxx]
> Sent: Monday, May 17, 1999 4:12 PM
> To: bill.st.arnaud
> Cc: IEEE HSSG
> Subject: RE: A telephony carrier industry perspective
>
>
>
> Bill,
>
> Internet IP will continue to be what the market requirements reflect.
> Slow restoration times are acceptable in that environment. The UUNet
> Long Haul GbE is part of what I am doing. I am the one that put the
> Long Haul Optical switching/Metro DWDM/SONET DATS evaluation of GbE
> together.
>
> As a bit irony, dependable GbE is turning out to be less expensive
> than undependable IP over TDM, ATM, or POS. Unless MPLS comes in a
> respectable price break, GbE over Native Data SONET DATS will still be
> less expensive.
>
> In addition to being less expensive, GbE over Native Data SONET DATS
> will provide "subscription control" though "flow control" That
> "subscription control" will prevent over-subscribing the WAN links,
> which is a big problem for enterprise data network designers and
> architects. Combine that with "priority queueing" and most of what
> MPLS was supposed to do has already been accomplished by GbE. You
> guys as IEEE have done a greater job than you knew.
>
> Dependable, high quality transport of Native data traffic such as GbE
> and 10GbE is probably going to be a different market, one that "best
> effort" is unacceptable to. If there is a market that provides the
> profit margin that will sustain 10GbE, it will not be the Internet as
> it is today.
>
> I do know that I have been given the requiement that carriers can not
> support a data service over long haul systems that does not provide
> "SONET like" functionality. The reason that I joined this study group
> is to provide that insight to the standards developers. If that is GbE
> or 10GbE over SONET then the issue is already resolved. All that
> remains is to determine what the LAN application requirements are,
> then the standard can be defined.
>
>
> Date: Mon May 17, 1999 3:23 pm CST
> Source-Date: Mon, 17 May 1999 17:17:51 -0400
> Fromm: bill.st.arnaud
> EMS: INTERNET / MCI ID: 376-5414
> MBX: bill.st.arnaud@xxxxxxxxxx
>
> TO: * ROY BYNUM / MCI ID: 424-5935
> Subject: RE: A telephony carrier industry perspective
> Message-Id: 99051721234042/INTERNETGWDN1IG
> Source-Msg-Id:
> <NBBBJIMEPHPGCNGAHPMFIEIJELAA.bill.st.arnaud@xxxxxxxxxx>
> U-Importance: Normal
> U-X-Mailer: Microsoft Outlook IMO, Build 9.0.2212 (4.71.2419.0)
> U-X-MIMEOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
> U-X-MSMail-priority: Normal
> U-X-Priority: 3 (Normal)
>
> Roy:
>
> Again no disagreement. I don't think traditional SONET or
> ATM networks will
> disappear. The model we advocate is the same that Frontier is
> now deploying:
> IP/DWDM for best efforts, slow restoral traffic on one set of
> wavelengths,
> IP over SONET on another set of wavelengths for those
> services that need
> fast restoral and security of SONET, and IP over ATM over
> SONET on another
> set of wavelengths for fine grained QoS services.
>
> I agree with you that the driving force for GbE is cost. It makes a
> dramatic difference to the overall cost of the network.
>
> But I believe GbE can also make an equal dramatic difference on the
> transport side on medium, long haul links up to 1000 km. Your sister
> company UUNet has already demonstrated that on some long haul
> GbE systems.
> But I agree with you this type of link is probably only good for best
> efforts IP traffic.
>
> Bill
>
> -------------------------------------------
> Bill St Arnaud
> Director Network Projects
> CANARIE
> bill.st.arnaud@xxxxxxxxxx
> http://tweetie.canarie.ca/~bstarn
>
>
>
>
>
>
>
> > -----Original Message-----
> > From: Roy Bynum [mailto:RBYNUM/0004245935@xxxxxxxxxxx]
> > Sent: Monday, May 17, 1999 4:54 PM
> > To: bill.st.arnaud
> > Cc: IEEE HSSG
> > Subject: RE: A telephony carrier industry perspective
> >
> >
> > Bill,
> >
> > I have an IP base video conferencing demonstration
> application that is
> > full motion, full resolution. It uses MPEG2 compression and
> drives the
> > IP data at 7Mbs bidirectional. Part of the demonstration of the
> > reliability of GbE over optical and SONET Native Data
> systems is to do
> > a simulated fiber break. Over a normal IP network, there is a major
> > loss of data and thus video synchronization, sometimes the "call" is
> > even dropped. Over a Native data network, optical or SONET DATS, if
> > you blink, you miss the cut. It is that kind of traffic path
> > restoration quality that will be required by future, real time,
> > visual, and virtual applications. This is the quality of
> traffic path
> > restoration that needs to be implemented in Metro/MAN and WAN
> > environments. I do not believe that any one protocol and/or
> system can
> > do that and still be cost effective.
> >
> > Fromm looking at Cisco's DPT description, it looks a lot like a
> > SONET/SDH BLSR ring. In duplicating the alternate path coupled
> > architecture of SONET/SDH, Cisco could well duplicate the
> restoration
> > functionality of SONET/SDH. Although what value it will
> have over long
> > haul systems that are geared for very high bandwidth, I am not sure.
> > Within a LAN environment, that type of restoration is probably not
> > required. Within a Metro MAN environment, we have found that GbE
> > combined with optical path protected DWDM is very cost effective.
> > Cisco will have to come in at a VERY "cheep" cost in order
> to justify
> > the deployment of their systems. I expect to have some test systems
> > before too long. I am looking forward to finding out.
> >
> > The bottom line to this is cost. Preliminary evaluations are showing
> > that GbE already is so cost effective that it is less expensive to
> > have 10 GbE interfaces on a router than it is to have 4 OC48 POS
> > interfaces. Combine that with the less expensive interfaces on the
> > DWDM and SONET DATS equipment it becomes even more attractive. The
> > capital cost of IP over "Ethernet" or what I am now calling
> Native IP
> > is less expensive over a MAN or WAN environment than the
> existing TDM
> > WAN systems. 10GbE must compete in this environment. In order for it
> > to be deployed, it must provide high native data bandwidth,
> very cost
> > effectively with the specific service, maintenance, and operations
> > support that each very different environment requires.
> >
> >
> > Thank you,
> > Roy Bynum
> >
> >
> >
> > Date: Mon May 17, 1999 1:07 pm CST
> > Source-Date: Mon, 17 May 1999 15:02:01 -0400
> > Fromm: bill.st.arnaud
> > EMS: INTERNET / MCI ID: 376-5414
> > MBX: bill.st.arnaud@xxxxxxxxxx
> >
> > TO: * ROY BYNUM / MCI ID: 424-5935
> > Subject: RE: A telephony carrier industry perspective
> > Message-Id: 99051719074549/INTERNETGWDN3IG
> > Source-Msg-Id:
> <NBBBJIMEPHPGCNGAHPMFAEHPELAA.bill.st.arnaud@xxxxxxxxxx>
> > U-Importance: Normal
> > U-X-Mailer: Microsoft Outlook IMO, Build 9.0.2212 (4.71.2419.0)
> > U-X-MIMEOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
> > U-X-MSMail-priority: Normal
> > U-X-Priority: 3 (Normal)
> >
> > Roy:
> >
> > I agree with you that for traditional telecommunications
> traffic SONET
> > restoral makes sense. For IP traffic it makes less sense.
> >
> > I can assure you we can already do 500 msec restoration on an IP
> > network by
> > simpling cranking down the I/F timers. Peter Lotheberg at Sprint
> > claims he
> > will be doing 300 msec, again by adhusting the OSPF and I/F
> timers. CISCO
> > claims, although we have to yet to test it, that they can
> do 50 msec with
> > their new DPT product. However, I tend to believe the
> CISCO claims as I
> > know most of the engineering team, mostly former SONET
> engineers from
> > Nortel.
> >
> > Juniper has also promised similar values, but as yet unproven.
> >
> > Bill
> >
> > -------------------------------------------
> > Bill St Arnaud
> > Director Network Projects
> > CANARIE
> > bill.st.arnaud@xxxxxxxxxx
> > http://tweetie.canarie.ca/~bstarn
> >
> >
> >
> >
> >
> >
> >
> > > -----Original Message-----
> > > From: Roy Bynum [mailto:RBYNUM/0004245935@xxxxxxxxxxx]
> > > Sent: Monday, May 17, 1999 12:01 AM
> > > To: bill.st.arnaud
> > > Cc: IEEE HSSG; Roy A. Bynum
> > > Subject: RE: A telephony carrier industry perspective
> > >
> > >
> > > Bill,
> > >
> > > I have had some of the "IP over DWDM" systems in a lab at
> MCI. They
> > > are not able to restore at SONET speeds because they do
> not have the
> > > tightly coupled framing and link maintenance that is
> incorporated in
> > > SONET. You would be surprised, as I was, at how much time
> was added to
> > > traffic restoration, in addition to the DWDM/Optical path
> restoration
> > > time. The fastest traffic restoration was by a "cut-through" GbE
> > > switch. An IP switch did not come close enough to even be
> considered
> > > as "SONET Like" even though it was running POS over DWDM.
> > >
> > > Long haul, carrier grade, optical networking requires the same
> > > traffic protection as well as the maintenance and
> operations support
> > > that currently exists in SONET. This is a different
> requirement from
> > > the IT industry that drove the existing 802.3 standards.
> > >
> > > What some of the other carriers are implementing is
> "transparent" data
> > > services. There was a recent magazine that showed that the AT&T,
> > > Sprint, and GTE offerings are actually ATM with LAN
> emulation. This
> > > is not the same as Native data exemplified by GbE.
> > >
> > > Don't count too much on the "promises" of IP level restoration. At
> > > present, 2 minutes path restoration is considered fast. Tightly
> > > coupling IP traffic restoration to layer two changes the
> nature of the
> > > routing protocols as they exist in a mesh or semi-mesh
> architecture.
> > > Other than MPLS, which is more for traffic bandwidth reservation,
> > > there have not been any proposals accepted that change
> the existing
> > > protocols. I may be mistaken, but I believe that PNNI Augmented
> > > Routing (PAR) did not make it beyond RFC Draft.
> > >
> > > I hope that this group continues to consider 10GbE as an
> independent
> > > protocol. The upper layer protocols, such as IP or IPX, will take
> > > care of themselves.
> > >
> > > Thank you,
> > > Roy Bynum
> > >
> > >
> > > Date: Sun May 16, 1999 6:04 pm CST
> > > Source-Date: Sun, 16 May 1999 19:58:37 -0400
> > > Fromm: bill.st.arnaud
> > > EMS: INTERNET / MCI ID: 376-5414
> > > MBX: bill.st.arnaud@xxxxxxxxxx
> > >
> > > TO: * ROY BYNUM / MCI ID: 424-5935
> > > TO: IEEE HSSG
> > > EMS: INTERNET / MCI ID: 376-5414
> > > MBX: stds-802-3-hssg@xxxxxxxx
> > > CC: "Roy A. Bynum"
> > > EMS: INTERNET / MCI ID: 376-5414
> > > MBX: roy.bynum@xxxxxxx
> > > Subject: RE: A telephony carrier industry perspective
> > > Message-Id: 99051700041309/INTERNETGWDN2IG
> > > Source-Msg-Id:
> <NBBBJIMEPHPGCNGAHPMFKEFIELAA.bill.st.arnaud@xxxxxxxxxx>
> > > U-Importance: Normal
> > > U-X-Mailer: Microsoft Outlook IMO, Build 9.0.2212 (4.71.2419.0)
> > > U-X-MIMEOLE: Produced By Microsoft MimeOLE V5.00.2314.1300
> > > U-X-MSMail-priority: Normal
> > > U-X-Priority: 3 (Normal)
> > >
> > > Roy:
> > >
> > > Some excellent comments and observations.
> > >
> > > One comment I would make is that many carriers are moving away
> > from SONET
> > > for restoral and protection for IP traffic. CANARIE,
> Enron, Sprint,
> > > Froontier, Teleglobe and many others are building IP/DWDM
> networks where
> > > restoral is done at layer 3. MPLS (multi Protocol Label
> > Switching) is the
> > > most common implementation of supporting restoral and protection
> > > at layer 3.
> > > The vendors claim that we will be able to do restoral at the
> > same speed as
> > > SONET - 50 msec for up to 14 nodes.
> > >
> > > With layer 3 restoral, the type of transport protocol becomes
> > > less critical.
> > > Thus a simple and cost effective data protocol like GbE may be
> > all that is
> > > required.
> > >
> > > FFor more detailed information on layer 3 restoral please
> see the white
> > > papers on our web site at www.canet3.net
> > >
> > > Bill
> > >
> > > -------------------------------------------
> > > Bill St Arnaud
> > > Director Network Projects
> > > CANARIE
> > > bill.st.arnaud@xxxxxxxxxx
> > > http://tweetie.canarie.ca/~bstarn
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: owner-stds-802-3-hssg@xxxxxxxxxxxxxxxxxx
> > > > [mailto:owner-stds-802-3-hssg@xxxxxxxxxxxxxxxxxx]On Behalf Of
> > Roy Bynum
> > > > Sent: Sunday, May 16, 1999 10:55 AM
> > > > To: IEEE HSSG
> > > > Cc: Roy A. Bynum
> > > > Subject: A telephony carrier industry perspective
> > > >
> > > >
> > > >
> > > > All,
> > > >
> > > > As an introduction, my name is Roy Bynum. I work for
> MCI WorldCom in
> > > > the Data and Optical Network Technology Development
> organization. I am
> > > > late coming to this discussion because of a failure by
> the telephony
> > > > industry to recognize GbE as a defacto optical
> networking technology.
> > > > My charter was to work on a native optical networking
> standard for IP
> > > > services over carrier systems. I came across GbE by accident and
> > > > friends that work for data systems vendors.
> > > >
> > > > Over the last several months, I have been involved in evaluating
> > > > Gigabit Ethernet (GbE) as a viable data service. The
> outcome of that
> > > > is the following observations (I apologize for the
> length of this
> > > > memo.):
> > > >
> > > > 1. Depending on whom you talk to, 80% to 85% of all data
> > > > communications traffic in the world originates on
> "Ethernet" (802.3).
> > > > The reason for this is a combination of almost
> commodity prices on the
> > > > interfaces and very simple operational support
> requirements, which
> > > > translates into very low cost of ownership for the return on
> > investment.
> > > >
> > > > 2. Internet Protocol (IP) can operate on most any layer
> two protocol,
> > > > and does. However, (again depending on whom you talk
> to) up to 95% of
> > > > all IP communications traffic originates on Ethernet. This makes
> > > > Ethernet the defacto native data communications
> protocol for IP. The
> > > > reason for this is economics, as stated above.
> > > >
> > > > 3. GbE is following the precedence of Ethernet in that
> it is very cost
> > > > effective to deploy compared to other high bandwidth
> technologies. The
> > > > cost for GbE, per bandwidth, is anywhere from one forth
> to one tenth
> > > > of that of ATM or Packet Over SONET (POS).
> > > >
> > > > 4. Dense Wavelength Division Multiplexing (DWDM) is
> being developed
> > > > and deployed in the Metropolitan carrier and other fiber optic
> > > > systems. These DWDM systems have up to 32 wavelengths, with path
> > > > protection for each. Metro DWDM uses single mode fiber
> (SMF) over
> > > > "short" distances, 200km or less. The economics for
> these systems is
> > > > turning out to be very favorable as well.
> > > >
> > > > 5. Another technology standard is being proposed in the
> telephony
> > > > industry that provides for transportation of native
> data, such as
> > > > Ethernet, directly over SONET facilities. Data Aware
> Transmission over
> > > > SONET (DATS) is the name of that proposal. DATS comes
> in two types,
> > > > transparent data, and Native data. The transparent data
> technology
> > > > puts ATM SAR switches directly on SONET transport
> nodes. Native data
> > > > technology puts Ethernet switches directly on SONET
> nodes. Native data
> > > > over SONET was demonstrated last year at Interopt and
> is also turning
> > > > out to have some economic benefits.
> > > >
> > > > 6. Telephony carrier data communications standards
> (WAN) today are
> > > > very different from Information Technology (IT) data
> communications
> > > > (LAN/MAN) standards. Telephony standards are circuit
> based and are
> > > > concerned with maintaining traffic connection path integrity and
> > > > quality. IT standards are based on connectionless data
> with a major
> > > > emphasis on cost of ownership. Telephony standards have
> been based on
> > > > Time Division Multiplexing (TDM) of voice rate (modulo 64kbs)
> > > > circuits. IT Ethernet standards have developed
> independently and are
> > > > based on native data requirements (modulo 10Mbs). Up
> until recently,
> > > > the two standards only came together at a
> router/gateway device that
> > > > removed the different standards at layer two, leaving
> the upper layer
> > > > (layer three and above) data to be communicated. It
> also means that
> > > > data traffic path restoration has been dealt with
> differently by the
> > > > two industries and standards. This is changing.
> > > >
> > > > 7. Telephony carriers have recognized that in a few
> years the massive
> > > > bulk of the traffic on their systems will be
> connectionless oriented
> > > > native data, not connection oriented voice. Some have
> also recognized
> > > > that the services on this traffic are abstracted from
> software on the
> > > > end systems, not the facilities based services that provides the
> > > > profits of today. This means that telephony carriers
> are looking for a
> > > > very cost-effective alternative to the TDM systems that
> they have been
> > > > using. Many are working, along with vendors on what
> they refer to as
> > > > "Optical Networking". This is a combination of very high DWDM
> > > > wavelength systems (up to 160 wavelengths) and optical
> switching which
> > > > provides for direct optical transport of data. I will
> not go into the
> > > > economics of what has been developed so far, but it is
> sufficient to
> > > > know that this work is being done.
> > > >
> > > > 8. SONET/SDH is a very resilient communications
> standard. It provides
> > > > for very tightly coupled traffic path restoration,
> which prevents the
> > > > unnecessary loss of data traffic connectivity. It
> provides for very
> > > > high bandwidth of channelized and concatenated traffic.
> It provides
> > > > for operational and maintenance support for 365 day x 24 hour
> > > > communications services. It is also very expensive, but
> justified in
> > > > the many customers, much circuit oriented, bulk traffic services
> > > > provided by the telephony carrier industry.
> > > >
> > > > 9. The loss of traffic path connectivity for high
> bandwidth, bulk data
> > > > communications has a much more massive impact than it
> does with lower
> > > > or moderate bandwidth communications. As more and more
> applications
> > > > utilize more and more data communications bandwidth, the loss of
> > > > traffic path connectivity will have a business,
> economic, and personal
> > > > impact that it did not have on lower or moderate bandwidth data
> > > > communications.
> > > >
> > > > 10. The nature of wide area networking protocols such
> as IP's OSPFF
> > > > changes when you move from a telephony circuit based
> WAN to a common
> > > > virtual circuit based, layer two switching/bridging WAN. The
> > > > implications of traffic restoration timers and timing
> requirements
> > > > change when moved from a non-broadcast, multiple circuit path
> > > > architecture to a broadcast domain, single segment
> architecture. This
> > > > is not very well understood at the present time. It
> will take a while
> > > > for WAN data communications engineers to work out these
> changes. This
> > > > will delay, for a short while, the deployment of GbE or
> 10GbE for
> > > > enterprise WAN systems.
> > > >
> > > > These observations should help provide some insight
> into the some of
> > > > the issues being discussed by the HSSG. Whatever is
> developed must be
> > > > economical for it to survive. 10GbE is approaching the
> bandwidth and
> > > > functionality requirements of the telephony carrier
> systems. Where it
> > > > is deployed will have a major impact on what the
> requirements for it
> > > > will be. Depending on where it is deployed, it needs to
> be traffic
> > > > path resilient and have operational and maintenance support
> > > > functionality directly within 10GbE. This does not mean
> that 10GbE
> > > > could not defined with two standards, one for LAN/MAN,
> and another
> > > > that encapsulates the LAN/MAN framing in a WAN
> transport standard. A
> > > > LAN standard does not have the requirements of a WAN
> standard. Many of
> > > > the WAN requirements can be incorporated in the Metro
> DWDM systems for
> > > > MAN services. This could simplify many of the issues of
> wavelength,
> > > > power, distance, synchronous or block coding, fiber
> type, and others.
> > > >
> > > > I hope that I can be of some help with the development
> of this or
> > > > these standards. It is very critical to the future
> economics of the
> > > > carrier data communications industry.
> > > >
> > > > Thank you,
> > > > Roy Bynum roy.bynum@xxxxxxx
> > > > Sr. Engineer, Data and Optical Networking Technology Development
> > > > MCI WorldCom
> > > > (972) 729-7249
> > > >
> > > >
> > >
> >
>