Re: [HSSG] Reliability or relative reliability objective?
Drew,
Thank you very much for elaborating on this and sharing results of your LH fiber inter-lane propagation delay difference measurements.
Indeed actual field measured MTBF can be quite high for a properly designed system. In fact in most cases I am familiar with, the calculated MTBF was much lower than the actual one experienced in the field.
I agree that proper design can address MTBF.
Having said that, I believe it is still important to keep in mind that when it comes to some of the proposals discussed on this reflector (using existing 10G modules and increment link capacity by adding these one at a time) the MTBF will decrease as the link capacity increases. If the comparison needs to be with LAG we are not worse off. If the comparison is with our tradition (going from 1 to 10G by designing new serial transmission modules) we are worse off.
If you can share data on the reliability of Photonic Integrated Circuits that would be great. Actually, if you plan to present something why not give us a mini-tutorial in general including the state of the industry, will PICs likely be available from 5-10 suppliers in the near future (I am aware only of Luxtera at this time) etc.
Thanks very much.
Menachem
Menachem Abraham
Columbus Advisors
-----Original Message-----
From: Drew Perkins <dperkins@xxxxxxxxxxxx>
Date: Wed, 23 Aug 2006 17:54:59
To:STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
Subject: Re: [HSSG] Reliability or relative reliability objective?
Menachem,
I'm personally in favor of an NxL (L = 10 Gb/s for example though it could be 20 Gb/s or higher in the future) scalable approach with graceful degradation as you describe. As you grow N by adding links -- this would be a very useful LAG-like feature -- you are right regarding the probability of failures in this context. FITs is likely to be close to linear with N resulting in an equally linear 1/N reduction in MTBF and link availability as you grow.
However, this is not necessarily true for an MxNxL approach where there is at least the option of scaling in larger units (for example N=10 and L=10 Gb/s). Of course this depends on the actual design of the system or component. Reliability is not something that just happens by accident and you take what you get; it is designed very carefully into components. It is possible to design a component with 10 x 10G lanes that does not have N times as many FITs as one with a single 10G lane. Failures tend to be highly correlated with some manufacturing processes so that FITs is highly sublinear with increasing channel count. For instance, Infinera's 10x 10G Photonic Integrated Circuits have seen millions of hours of field use and we've never seen a single PIC fail. I'm trying not to sound like a commercial here, but the point is that devices can be designed with very high reliability that doesn't have the implications for reliability that you are concerned about. It is highly desira!
ble that reliability is thought about up front in the architecture, but there are also a lot of knobs that can be turned in individual implementations to provide additional protection.
Drew
_____________________________
Drew Perkins
Chief Technology Officer
Infinera Corporation
1322 Bordeaux Drive
Sunnyvale, CA 94089
Phone: 408-572-5308
Cell: 408-666-1686
Fax: 408-904-4644
Email: dperkins@xxxxxxxxxxxx
WWW : http://www.infinera.com
_____________________________
-----Original Message-----
From: Menachem Abraham [mailto:mabraham@xxxxxxxxxxxxxxxxxxxx]
Sent: Wednesday, August 23, 2006 11:10 AM
To: STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
Subject: Re: [HSSG] Reliability or relative reliability objective?
Jugnu,
I agree that it has value in this context.
Thanks,
Menachem
-----Original Message-----
From: OJHA,JUGNU [mailto:jugnu.ojha@xxxxxxxxxxxxx]
Sent: Wednesday, August 23, 2006 2:04 PM
To: mabraham@xxxxxxxxxxxxxxxxxxxx; STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
Subject: RE: [HSSG] Reliability or relative reliability objective?
Menachem,
I think this is a very good reason to have a graceful degradation mechanism
- i.e., if one channel fails, continue to operate over the others at reduced
bit rate. This would come for free in a scalable solution.
Jugnu
-----Original Message-----
From: Menachem Abraham [mailto:mabraham@xxxxxxxxxxxxxxxxxxxx]
Sent: Tuesday, August 22, 2006 5:00 PM
To: STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
Subject: [HSSG] Reliability or relative reliability objective?
All,
Has IEEE 802 set reliability (MTBF) objectives in any of the past projects?
The multi-lane approach assumed in HSSG raises some questions in this regard
when compared to our tradition.
When we went from 1G to 10G optical interfaces we still had one laser, one
PIN diode, one transimpedance receiver preamplifier etc etc. These
components moved up in their cutoff frequencies but the number of components
was in the same order of magnitude and therefore the MTBF and reliability
was expected to be similar.
In contrast, going from 10G to 100G by using 10 x 10G lanes implies a
reduction in reliability by a factor of 10.
This consideration combined with the cost concern related to 10 x 10G lane
approach (10 lasers, 10 PIN diodes etc) causes me to think that it is
important to keep an open mind regarding the number of lanes and the speed
of each one of these lanes.
Thanks,
Menachem
Menachem Abraham
Columbus Advisors
-----Original Message-----
From: "Drew Perkins" <dperkins@xxxxxxxxxxxx>
Date: Wed, 23 Aug 2006 00:01:45
To:<mabraham@xxxxxxxxxxxxxxxxxxxx>, <STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx>
Subject: RE: [HSSG] Reach Objectives
Menachem,
I think we need to differentiate between what PMDs we specify and what other
PMDs we enable. For instance, I don't think it is the IEEE's place to
specify an ULH PMD in terms of the optical specifications. However, this
could be one of the more important applications of a HS Ethernet. So I think
it would be worthwhile for us to enable vendors to develop it in a
straightforward fashion. Thus I think we should get into some of the details
that you mention including:
1. PHY layer - what degree of compatibility with LAN-PHY, WAN-PHY
(SONET/SDH), and/or G.709 is desired?
2. What amount of differential delay (skew) will be allowed? What will be
mandated for all conformant implementation?
It is clearly desirable to maintain compatibility with today's DWDM
transponders. This is a specific goal of some carriers that are
participating in this process. Carriers would love to have a PMD option that
leverages the 10G LAN-PHY or WAN-PHY. Of course this will depend on the
answers to these questions and other decisions we make.
Many (I believe most) DWDM systems on the market now support the LAN-PHY
natively by simply speeding up the G.709 OUT to run at ~11Gb/s instead of
10.7Gb/s rather than by doing some sort of overhead compression into
SONET/SDH or the G.709 digital wrapper.
Drew
_____________________________
Drew Perkins
Chief Technology Officer
Infinera Corporation
1322 Bordeaux Drive
Sunnyvale, CA 94089
Phone: 408-572-5308
Cell: 408-666-1686
Fax: 408-904-4644
Email: dperkins@xxxxxxxxxxxx
WWW : http://www.infinera.com
_____________________________
-----Original Message-----
From: Menachem Abraham [mailto:mabraham@xxxxxxxxxxxxxxxxxxxx]
Sent: Tuesday, August 22, 2006 5:00 PM
To: STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
Subject: Re: [HSSG] Reach Objectives
Geoff,
Thanks for your comments.
I also believe that our efforts should focus on distances no higher than
10's of Km (up to and including metro).
If we decide as a group that it is an objective to make it easy to hook into
LH and ULH transport systems in the installed base, we will have to study a
number of issues such as:
(A) should we pick a data rate that is matching SONET rates?
(B) should we design our 802.3 std so that it tolerates a much larger
inter-lane differential delay than what would be expected in a metro
application of the standard?
(C) should we assume we never go through existing LH transponders and just
have to COEXIST on the same fiber, optical amplifiers, dispersion
compensators located in the huts, optical mux demux at both ends etc.
Etc. In this case we would assume a new type of LH tranponder purpose built
for HSSG applications.
If this is the case, SONET rate compatibility would not be important.
Today's 10G LH and ULH system run mostly at OC-192 rates plus FEC overhead.
Chip vendors were creative and managed to find ways to build devices that
pack into these solutions the full 10G LAN data rate even though OC-192 is
less than 10G.
I think (but I may be wrong) that they use available bandwidth in the
management bits available in Digital Wrapper or something like that.
Not clean but seems to work...
Cheers,
Menachem
Menachem Abraham
Columbus Advisors
-----Original Message-----
From: "Geoff Thompson" <gthompso@xxxxxxxxxx>
Date: Tue, 22 Aug 2006 16:09:11
To:mabraham@xxxxxxxxxxxxxxxxxxxx
Cc:STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
Subject: Re: [HSSG] Reach Objectives
Menachem-
Thanks for your much more specific answer to the question. I'm afraid that
my earlier answer was handicapped by my ignorance of the specifics of that
market.
Based on what you said, I believe the questions for us to consider or not
are:
a) Will we consider long haul solutions.
OR
b) Will we limit ourselves to metro solutions and "transport end" (i.e.
stuff that hooks into the transport infrastructure) solutions.
Back in the old days of 10Gig we spent an awful lot of time discussing the
need for the WAN PHY to address case "b)". I think most of us didn't get it
then. I would hope that with a different cast of characters involved in the
discussions that we (or at least I, for one) could come out with a clear
rationale for what we choose.
(Just FYI, I believe the crux of the issue came down to whether or not one
could have a 2 port bridge, as opposed to an Optical-Electrical-Optical
repeater in a Transport Chassis.)
None the less, I believe that my proposed answer stands. We don't need to
tackle this issue in the first set of objectives and projects.
I do remain interested (old repeater hack that I am) in looking into an
O-E-O repeater that does not necessarily come all the way back up to the
level of reassembling the full packet.
Geoff
At 01:30 PM 8/22/2006 , Menachem Abraham wrote:
All,
If we decide to include in our reach objectives Long Haul (e.g. 1000 km
with
optical amps placed at 80 Km spacing) and Ultra Long Haul (e.g. 3000 Km
with
optical amps at 80 Km spacing, without Optical-Electrical-Optical
regeneration), we need to keep in mind that modulation/encoding/FEC choices
play an important role in how far we can go on an optical amplifier based
line system. Such PMD designs may be too costly for our < 80Km
applications/objectives so we may end up with more PMDs.
While there are some examples of Routers / Switches which have LH or ULH
optical interfaces built in, most systems use Routers / Switches with
shorter reach interfaces connected to separate Transport Chassis that house
proprietary LH or ULH solutions. As far as I know the LH and ULH world does
not have interoperable standard solutions today in terms of the signaling
on
the fiber.
My input for our activities in HSSG is to optimize for cost and not require
that one of our PMDs be directly useable as part of a LH or ULH line system
(unless that is doable without incremental cost).
Having said that, I believe we should debate the need to address "ease of
HSSG data transport" on top of existing and emerging LH and ULH transport
systems. If this debate already happened as part of the 10G 802.3 standard
development and the conclusions apply here, perhaps somebody can educate
those of us who were not involved at that time.
Thanks,
Menachem
-----Original Message-----
From: Aaron Dudek [mailto:adudek@xxxxxxxxxx: <mailto:adudek@xxxxxxxxxx> ]
Sent: Tuesday, August 22, 2006 12:50 PM
To: STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
Subject: Re: [HSSG] Reach Objectives
Geoff,
Shouldn't the migration to ULH systems have any impact on the spacing
and hence be taken into consideration? Or is that beyond the scope for
now?
Aaron Dudek
(703) 689-6879
Sprintlink Engineering
adudek@xxxxxxxxxx
On Tue, 22 Aug 2006, Geoff Thompson wrote:
> Roger-
>
> At 03:47 AM 8/22/2006 , Roger Merel wrote:
>
> Agree with Drew. Have a few additional comments on other reachs:
>
> For reach objectives, we should start with customer based needs
(for
broad market potential) and only amend if an
> obvious technical limitation with compelling economics can t
readily
meet the broad customer need.
>
> Specifically:
>
> - Long Reach probably should be set at 80km rather than 100km (as
this is the common hut-to-hut amplifier spacing
> in telecom)
>
> - While 50m does serve a useful portion of the market (smaller
datacenters and/or the size of a large computer
> cluster), it is somewhat constraining as I ve been lead to
understand that the reach needed in larger datacenters
> is continuing to out-grow the 100m meter definition but the 100m
definition at least serves the customer well.
> Certainly 10G-BaseT worked awfully hard to get to 100m (for
Datacenter interconnect).
>
>
> I wouldn't attach a lot of creedence to the 10GBASE-T goal for 100
meters.
It was, I believe, mainly driven by the
> traditional distance in horizontal (i.e. wiring closet to desktop)
distances rather than any thorough examination of data
> center requirements.
>
> Geoff
>
>
> - For both in-building reaches (50m & 300m; or 100m & 300m), the
bigger issue which affects the PMD is the loss
> budget arising from the number of patch panels. The shorter /
datacenter reach should include a budget for 1
> patch panel. The longer / enterprise reach should include a budget
for 2 patch panels (one in the datacenter and
> 1 in the remote switch closet).
>
>
>
>
> From: Drew Perkins [mailto:dperkins@xxxxxxxxxxxx:
<mailto:dperkins@xxxxxxxxxxxx> ]
> Sent: Tuesday, August 22, 2006 1:24 AM
> To: STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
> Subject: Re: [HSSG] Reach Objectives
>
>
>
> John,
>
>
>
> I suggest dividing Metro into Metro Short Reach at 10 km
(equivalent
application to 10GBASE-LR) and Metro
> Intermediate Reach at 40 km (equivalent application to 10GBASE-ER).
>
>
>
> Drew
>
> _____________________________
>
>
>
> Drew Perkins
>
> Chief Technology Officer
>
> Infinera Corporation
>
> 1322 Bordeaux Drive
>
> Sunnyvale, CA 94089
>
>
>
> Phone: 408-572-5308
>
> Cell: 408-666-1686
>
> Fax: 408-904-4644
>
> Email: dperkins@xxxxxxxxxxxx
>
> WWW : http://www.infinera.com: <http://www.infinera.com/>
>
>
>
>
>
> _____________________________
>
>
>
>
___________________________________________________________________________
_
________________________________________________________
> From: John DAmbrosia [mailto:jdambrosia@xxxxxxxxxxxxxxxxxxx:
<mailto:jdambrosia@xxxxxxxxxxxxxxxxxxx> ]
> Sent: Monday, August 21, 2006 9:38 PM
> To: STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
> Subject: [HSSG] Reach Objectives
>
>
>
> All,
>
> We have had some conversation on the reflector regarding reach
objectives. Summarizing what has been discussed
> on the reflector I see the following
>
>
>
> Reach Objectives
>
> Long-Haul --> 100+ km
>
> Metro --> 10+ km
>
> Data Center --> 50m & 300m
>
>
>
> Data Center Reach Segregation
>
> Intra-rack
>
> Inter-rack
>
> Horizontal runs
>
> Vertical risers
>
>
>
> Use this data to identify a single low-cost solution that would
address a couple of the reach objectives
>
>
>
> Other Areas
>
> During the course of the CFI there were individuals who wanted
Backplane Applications kept in for consideration,
> but I have not heard any further input in this area. Are there
still individuals who wish to propose Backplane
> as an objective?
>
>
>
> John
>
>
>
>
>