Re: [HSSG] Reliability or relative reliability objective?
Piers,
I think it is about all of these things.
If your new interface uses 4 or 10 transceivers of a similar technology as the previous generation, the FIT count is higher.
If your new interface uses components that run hotter or have a higher gate count than your previous generation interface, the FIT count is higher.
In fact, I would be surprised if there is any way one can design this new high speed interface in a way that doesn't have more FITs than a 10G interface.
So how to mitigate this? If you have an interface of 10 lanes that can operate at 90% of the throughput if one of the trasceivers fails, or an interface of 4 lanes that can operate at 75% of the throughput if one of the transceivers fails, and if the customer will still consider the interface to be in service operating at 90% or 75% of capacity, and if you have a maintenance strategy that allows for replacing the failed transceiver without having to take the entire interface out of service (at least for more than a brief period of time) and a sufficiently low MTTR, then the expected outage of the new interface can be as low, or even lower than the expected outage of the previous generation technology even though total FITs are higher.
Regards,
Steve
-----Original Message-----
From: DAWE,PIERS [mailto:piers.dawe@xxxxxxxxxxxxx]
Sent: Thursday, August 24, 2006 8:42 AM
To: STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
Subject: Re: [HSSG] Reliability or relative reliability objective?
> (using existing 10G modules and increment link capacity by adding
> these one at a time) the MTBF will decrease as the link capacity
> increases. If the comparison needs to be with LAG we are not worse
> off. If the comparison is with our tradition (going from 1 to 10G by
> designing new serial transmission modules) we are worse off.
Quite the opposite! If you build with that's been tested and field proven, you have some idea what your reliability will be like. If you build with new, unproven stuff, likely running hotter and more stressed than previous generations, you don't know what will happen. Drew mentions a correlation effect; that still applies if one puts components from the same batch into different boxes as much as if they are left integrated and within one box. But fundamentally, it's about careful design and manufacture, not counting parts.
Piers
> -----Original Message-----
> From: Menachem Abraham [mailto:mabraham@xxxxxxxxxxxxxxxxxxxx]
> Sent: 24 August 2006 01:00
> To: STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
> Subject: Re: [HSSG] Reliability or relative reliability objective?
>
>
> Drew,
>
> Thank you very much for elaborating on this and sharing results of
> your LH fiber inter-lane propagation delay difference measurements.
>
> Indeed actual field measured MTBF can be quite high for a properly
> designed system. In fact in most cases I am familiar with, the
> calculated MTBF was much lower than the actual one experienced in the
> field.
>
> I agree that proper design can address MTBF.
>
> Having said that, I believe it is still important to keep in mind that
> when it comes to some of the proposals discussed on this reflector
> (using existing 10G modules and increment link capacity by adding
> these one at a time) the MTBF will decrease as the link capacity
> increases. If the comparison needs to be with LAG we are not worse
> off. If the comparison is with our tradition (going from 1 to 10G by
> designing new serial transmission modules) we are worse off.
>
> If you can share data on the reliability of Photonic Integrated
> Circuits that would be great. Actually, if you plan to present
> something why not give us a mini-tutorial in general including the
> state of the industry, will PICs likely be available from 5-10
> suppliers in the near future (I am aware only of Luxtera at this time)
> etc.
>
> Thanks very much.
> Menachem
> Menachem Abraham
> Columbus Advisors
>
>
>
> -----Original Message-----
> From: Drew Perkins <dperkins@xxxxxxxxxxxx>
> Date: Wed, 23 Aug 2006 17:54:59
> To:STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
> Subject: Re: [HSSG] Reliability or relative reliability objective?
>
> Menachem,
>
> I'm personally in favor of an NxL (L = 10 Gb/s for example though it
> could be 20 Gb/s or higher in the future) scalable approach with
> graceful degradation as you describe. As you grow N by adding links --
> this would be a very useful LAG-like feature -- you are right
> regarding the probability of failures in this context. FITs is likely
> to be close to linear with N resulting in an equally linear 1/N
> reduction in MTBF and link availability as you grow.
>
> However, this is not necessarily true for an MxNxL approach where
> there is at least the option of scaling in larger units (for example
> N=10 and L=10 Gb/s). Of course this depends on the actual design of
> the system or component. Reliability is not something that just
> happens by accident and you take what you get; it is designed very
> carefully into components. It is possible to design a component with
> 10 x 10G lanes that does not have N times as many FITs as one with a
> single 10G lane.
> Failures tend to be highly correlated with some manufacturing
> processes so that FITs is highly sublinear with increasing channel
> count. For instance, Infinera's 10x 10G Photonic Integrated Circuits
> have seen millions of hours of field use and we've never seen a single
> PIC fail. I'm trying not to sound like a commercial here, but the
> point is that devices can be designed with very high reliability that
> doesn't have the implications for reliability that you are concerned
> about. It is highly desira!
> ble that reliability is thought about up front in the architecture,
> but there are also a lot of knobs that can be turned in individual
> implementations to provide additional protection.
>
> Drew
> _____________________________
>
> Drew Perkins
> Chief Technology Officer
> Infinera Corporation
> 1322 Bordeaux Drive
> Sunnyvale, CA 94089
>
> Phone: 408-572-5308
> Cell: 408-666-1686
> Fax: 408-904-4644
> Email: dperkins@xxxxxxxxxxxx
> WWW : http://www.infinera.com
>
>
> _____________________________
>
>
> -----Original Message-----
> From: Menachem Abraham [mailto:mabraham@xxxxxxxxxxxxxxxxxxxx]
> Sent: Wednesday, August 23, 2006 11:10 AM
> To: STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
> Subject: Re: [HSSG] Reliability or relative reliability objective?
>
> Jugnu,
>
> I agree that it has value in this context.
>
> Thanks,
> Menachem
>
> -----Original Message-----
> From: OJHA,JUGNU [mailto:jugnu.ojha@xxxxxxxxxxxxx]
> Sent: Wednesday, August 23, 2006 2:04 PM
> To: mabraham@xxxxxxxxxxxxxxxxxxxx; STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
> Subject: RE: [HSSG] Reliability or relative reliability objective?
>
> Menachem,
>
> I think this is a very good reason to have a graceful degradation
> mechanism
> - i.e., if one channel fails, continue to operate over the others at
> reduced bit rate. This would come for free in a scalable solution.
>
> Jugnu
>
> -----Original Message-----
> From: Menachem Abraham [mailto:mabraham@xxxxxxxxxxxxxxxxxxxx]
> Sent: Tuesday, August 22, 2006 5:00 PM
> To: STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
> Subject: [HSSG] Reliability or relative reliability objective?
>
> All,
>
> Has IEEE 802 set reliability (MTBF) objectives in any of the past
> projects?
>
> The multi-lane approach assumed in HSSG raises some questions in this
> regard when compared to our tradition.
>
> When we went from 1G to 10G optical interfaces we still had one laser,
> one PIN diode, one transimpedance receiver preamplifier etc etc. These
> components moved up in their cutoff frequencies but the number of
> components was in the same order of magnitude and therefore the MTBF
> and reliability was expected to be similar.
>
> In contrast, going from 10G to 100G by using 10 x 10G lanes implies a
> reduction in reliability by a factor of 10.
>
> This consideration combined with the cost concern related to 10 x 10G
> lane approach (10 lasers, 10 PIN diodes etc) causes me to think that
> it is important to keep an open mind regarding the number of lanes and
> the speed of each one of these lanes.
>
> Thanks,
> Menachem
> Menachem Abraham
> Columbus Advisors
>
>
>
> -----Original Message-----
> From: "Drew Perkins" <dperkins@xxxxxxxxxxxx>
> Date: Wed, 23 Aug 2006 00:01:45
> To:<mabraham@xxxxxxxxxxxxxxxxxxxx>,
> <STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx>
> Subject: RE: [HSSG] Reach Objectives
>
> Menachem,
>
>
>
> I think we need to differentiate between what PMDs we specify and what
> other PMDs we enable. For instance, I don't think it is the IEEE's
> place to specify an ULH PMD in terms of the optical specifications.
> However, this
> could be one of the more important applications of a HS Ethernet. So I
> think it would be worthwhile for us to enable vendors to develop it in
> a straightforward fashion. Thus I think we should get into some of the
> details that you mention including:
>
> 1. PHY layer - what degree of compatibility with LAN-PHY, WAN-PHY
> (SONET/SDH), and/or G.709 is desired?
>
> 2. What amount of differential delay (skew) will be allowed? What
> will be mandated for all conformant implementation?
>
>
>
> It is clearly desirable to maintain compatibility with today's DWDM
> transponders. This is a specific goal of some carriers that are
> participating in this process. Carriers would love to have a PMD
> option that leverages the 10G LAN-PHY or WAN-PHY. Of course this will
> depend on the answers to these questions and other decisions we make.
>
>
>
> Many (I believe most) DWDM systems on the market now support the
> LAN-PHY natively by simply speeding up the G.709 OUT to run at ~11Gb/s
> instead of 10.7Gb/s rather than by doing some sort of overhead
> compression into SONET/SDH or the G.709 digital wrapper.
>
>
>
> Drew
>
> _____________________________
>
>
>
> Drew Perkins
>
> Chief Technology Officer
>
> Infinera Corporation
>
> 1322 Bordeaux Drive
>
> Sunnyvale, CA 94089
>
>
>
> Phone: 408-572-5308
>
> Cell: 408-666-1686
>
> Fax: 408-904-4644
>
> Email: dperkins@xxxxxxxxxxxx
>
> WWW : http://www.infinera.com
>
>
>
>
>
> _____________________________
>
>
>
>
>
> -----Original Message-----
> From: Menachem Abraham [mailto:mabraham@xxxxxxxxxxxxxxxxxxxx]
> Sent: Tuesday, August 22, 2006 5:00 PM
> To: STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
> Subject: Re: [HSSG] Reach Objectives
>
>
>
> Geoff,
>
>
>
> Thanks for your comments.
>
>
>
> I also believe that our efforts should focus on distances no
> higher than
> 10's of Km (up to and including metro).
>
>
>
> If we decide as a group that it is an objective to make it
> easy to hook into
> LH and ULH transport systems in the installed base, we will
> have to study a
> number of issues such as:
>
>
>
> (A) should we pick a data rate that is matching SONET rates?
>
> (B) should we design our 802.3 std so that it tolerates a much larger
> inter-lane differential delay than what would be expected in a metro
> application of the standard?
>
> (C) should we assume we never go through existing LH
> transponders and just
> have to COEXIST on the same fiber, optical amplifiers, dispersion
> compensators located in the huts, optical mux demux at both ends etc.
>
> Etc. In this case we would assume a new type of LH tranponder
> purpose built
> for HSSG applications.
>
> If this is the case, SONET rate compatibility would not be important.
>
>
>
>
>
> Today's 10G LH and ULH system run mostly at OC-192 rates plus
> FEC overhead.
> Chip vendors were creative and managed to find ways to build
> devices that
> pack into these solutions the full 10G LAN data rate even
> though OC-192 is
> less than 10G.
>
> I think (but I may be wrong) that they use available bandwidth in the
> management bits available in Digital Wrapper or something like that.
>
> Not clean but seems to work...
>
>
>
> Cheers,
>
> Menachem
>
> Menachem Abraham
>
> Columbus Advisors
>
>
>
>
>
>
>
> -----Original Message-----
>
> From: "Geoff Thompson" <gthompso@xxxxxxxxxx>
>
> Date: Tue, 22 Aug 2006 16:09:11
>
> To:mabraham@xxxxxxxxxxxxxxxxxxxx
>
> Cc:STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
>
> Subject: Re: [HSSG] Reach Objectives
>
>
>
> Menachem-
>
>
>
> Thanks for your much more specific answer to the question.
> I'm afraid that
> my earlier answer was handicapped by my ignorance of the
> specifics of that
> market.
>
>
>
> Based on what you said, I believe the questions for us to
> consider or not
> are:
>
>
>
> a) Will we consider long haul solutions.
>
> OR
>
> b) Will we limit ourselves to metro solutions and "transport
> end" (i.e.
> stuff that hooks into the transport infrastructure) solutions.
>
> Back in the old days of 10Gig we spent an awful lot of time
> discussing the
> need for the WAN PHY to address case "b)". I think most of us
> didn't get it
> then. I would hope that with a different cast of characters
> involved in the
> discussions that we (or at least I, for one) could come out
> with a clear
> rationale for what we choose.
>
>
>
> (Just FYI, I believe the crux of the issue came down to
> whether or not one
> could have a 2 port bridge, as opposed to an
> Optical-Electrical-Optical
> repeater in a Transport Chassis.)
>
>
>
> None the less, I believe that my proposed answer stands. We
> don't need to
> tackle this issue in the first set of objectives and projects.
>
>
>
> I do remain interested (old repeater hack that I am) in
> looking into an
> O-E-O repeater that does not necessarily come all the way
> back up to the
> level of reassembling the full packet.
>
>
>
> Geoff
>
>
>
> At 01:30 PM 8/22/2006 , Menachem Abraham wrote:
>
> All,
>
>
>
> If we decide to include in our reach objectives Long Haul
> (e.g. 1000 km
> with
>
> optical amps placed at 80 Km spacing) and Ultra Long Haul
> (e.g. 3000 Km
> with
>
> optical amps at 80 Km spacing, without Optical-Electrical-Optical
>
> regeneration), we need to keep in mind that
> modulation/encoding/FEC choices
>
> play an important role in how far we can go on an optical
> amplifier based
>
> line system. Such PMD designs may be too costly for our < 80Km
>
> applications/objectives so we may end up with more PMDs.
>
>
>
> While there are some examples of Routers / Switches which
> have LH or ULH
>
> optical interfaces built in, most systems use Routers / Switches with
>
> shorter reach interfaces connected to separate Transport
> Chassis that house
>
> proprietary LH or ULH solutions. As far as I know the LH and
> ULH world does
>
> not have interoperable standard solutions today in terms of
> the signaling
> on
>
> the fiber.
>
>
>
> My input for our activities in HSSG is to optimize for cost
> and not require
>
> that one of our PMDs be directly useable as part of a LH or
> ULH line system
>
> (unless that is doable without incremental cost).
>
>
>
> Having said that, I believe we should debate the need to
> address "ease of
>
> HSSG data transport" on top of existing and emerging LH and
> ULH transport
>
> systems. If this debate already happened as part of the 10G
> 802.3 standard
>
> development and the conclusions apply here, perhaps somebody
> can educate
>
> those of us who were not involved at that time.
>
>
>
> Thanks,
>
> Menachem
>
>
>
>
>
> -----Original Message-----
>
> From: Aaron Dudek [mailto:adudek@xxxxxxxxxx:
> <mailto:adudek@xxxxxxxxxx> ]
>
> Sent: Tuesday, August 22, 2006 12:50 PM
>
> To: STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
>
> Subject: Re: [HSSG] Reach Objectives
>
>
>
> Geoff,
>
> Shouldn't the migration to ULH systems have any impact on
> the spacing
>
> and hence be taken into consideration? Or is that beyond the
> scope for
>
> now?
>
>
>
> Aaron Dudek
>
> (703) 689-6879
>
> Sprintlink Engineering
>
> adudek@xxxxxxxxxx
>
>
>
>
>
> On Tue, 22 Aug 2006, Geoff Thompson wrote:
>
>
>
> > Roger-
>
> >
>
> > At 03:47 AM 8/22/2006 , Roger Merel wrote:
>
> >
>
> > Agree with Drew. Have a few additional comments on
> other reachs:
>
> >
>
> > For reach objectives, we should start with customer
> based needs
> (for
>
> broad market potential) and only amend if an
>
> > obvious technical limitation with compelling economics can t
> readily
>
> meet the broad customer need.
>
> >
>
> > Specifically:
>
> >
>
> > - Long Reach probably should be set at 80km rather
> than 100km (as
>
> this is the common hut-to-hut amplifier spacing
>
> > in telecom)
>
> >
>
> > - While 50m does serve a useful portion of the
> market (smaller
>
> datacenters and/or the size of a large computer
>
> > cluster), it is somewhat constraining as I ve been lead to
>
> understand that the reach needed in larger datacenters
>
> > is continuing to out-grow the 100m meter definition
> but the 100m
>
> definition at least serves the customer well.
>
> > Certainly 10G-BaseT worked awfully hard to get to 100m (for
>
> Datacenter interconnect).
>
> >
>
> >
>
> > I wouldn't attach a lot of creedence to the 10GBASE-T goal for 100
> meters.
>
> It was, I believe, mainly driven by the
>
> > traditional distance in horizontal (i.e. wiring closet to desktop)
>
> distances rather than any thorough examination of data
>
> > center requirements.
>
> >
>
> > Geoff
>
> >
>
> >
>
> > - For both in-building reaches (50m & 300m; or 100m
> & 300m), the
>
> bigger issue which affects the PMD is the loss
>
> > budget arising from the number of patch panels. The
> shorter /
>
> datacenter reach should include a budget for 1
>
> > patch panel. The longer / enterprise reach should
> include a budget
>
> for 2 patch panels (one in the datacenter and
>
> > 1 in the remote switch closet).
>
> >
>
> >
>
> >
>
> >
>
> > From: Drew Perkins [mailto:dperkins@xxxxxxxxxxxx:
> <mailto:dperkins@xxxxxxxxxxxx> ]
>
> > Sent: Tuesday, August 22, 2006 1:24 AM
>
> > To: STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
>
> > Subject: Re: [HSSG] Reach Objectives
>
> >
>
> >
>
> >
>
> > John,
>
> >
>
> >
>
> >
>
> > I suggest dividing Metro into Metro Short Reach at 10 km
> (equivalent
>
> application to 10GBASE-LR) and Metro
>
> > Intermediate Reach at 40 km (equivalent application
> to 10GBASE-ER).
>
> >
>
> >
>
> >
>
> > Drew
>
> >
>
> > _____________________________
>
> >
>
> >
>
> >
>
> > Drew Perkins
>
> >
>
> > Chief Technology Officer
>
> >
>
> > Infinera Corporation
>
> >
>
> > 1322 Bordeaux Drive
>
> >
>
> > Sunnyvale, CA 94089
>
> >
>
> >
>
> >
>
> > Phone: 408-572-5308
>
> >
>
> > Cell: 408-666-1686
>
> >
>
> > Fax: 408-904-4644
>
> >
>
> > Email: dperkins@xxxxxxxxxxxx
>
> >
>
> > WWW : http://www.infinera.com: <http://www.infinera.com/>
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > _____________________________
>
> >
>
> >
>
> >
>
> >
>
>
> ______________________________________________________________
> _____________
> _
>
> ________________________________________________________
>
> > From: John DAmbrosia [mailto:jdambrosia@xxxxxxxxxxxxxxxxxxx:
> <mailto:jdambrosia@xxxxxxxxxxxxxxxxxxx> ]
>
> > Sent: Monday, August 21, 2006 9:38 PM
>
> > To: STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
>
> > Subject: [HSSG] Reach Objectives
>
> >
>
> >
>
> >
>
> > All,
>
> >
>
> > We have had some conversation on the reflector
> regarding reach
>
> objectives. Summarizing what has been discussed
>
> > on the reflector I see the following
>
> >
>
> >
>
> >
>
> > Reach Objectives
>
> >
>
> > Long-Haul --> 100+ km
>
> >
>
> > Metro --> 10+ km
>
> >
>
> > Data Center --> 50m & 300m
>
> >
>
> >
>
> >
>
> > Data Center Reach Segregation
>
> >
>
> > Intra-rack
>
> >
>
> > Inter-rack
>
> >
>
> > Horizontal runs
>
> >
>
> > Vertical risers
>
> >
>
> >
>
> >
>
> > Use this data to identify a single low-cost solution
> that would
>
> address a couple of the reach objectives
>
> >
>
> >
>
> >
>
> > Other Areas
>
> >
>
> > During the course of the CFI there were individuals
> who wanted
>
> Backplane Applications kept in for consideration,
>
> > but I have not heard any further input in this area.
> Are there
>
> still individuals who wish to propose Backplane
>
> > as an objective?
>
> >
>
> >
>
> >
>
> > John
>
> >
>
> >
>
> >
>
> >
>
> >
>
>