I actually think we’re all saying the same thing.
Breakout has become a increasingly popular deployment model in certain networks. It doesn’t necessarily affect the specific Task Force work in defining an interoperable specification for a PHY. It could however be a consideration in
the Study Group phase to help the group figure out its objectives. If there is agreement on an objective for some higher speed PHY that could be broken out, then the potential lower speed PHYs that make up the subsets of the higher speed PHY could also be
added as an objective. And the breakout deployment model can be used to help justify either or both PHY objectives.
But a word of caution, “breakout” isn’t some magic bean that justifies everything, we would still need to have the usual technical and economic feasibility and broad market potential analysis on each PHY. Breakout deployment use cases
could augment those justifications.
This does create more complex projects though. A single rate project can be complex enough, but what I’m suggesting here means the scope of a new project touches across more clauses in our standard. While many people might not worry about
that, editorial teams know this isn’t a trivial shift and it puts a lot of pressure on schedule and timely progress. I don’t think that should affect a SG selecting objectives, but it should be a consideration on people’s expectations for how quick things
can get done.
Mark
From: Steve Trowbridge <steve.trowbridge@xxxxxxxxx>
Reply-To: Steve Trowbridge <steve.trowbridge@xxxxxxxxx>
Date: Friday, August 21, 2020 at 8:59 AM
To: "802.3 NGECDC" <STDS-802-3-NGECDC@xxxxxxxxxxxxxxxxx>
Subject: Re: [802.3_NGECDC] Definition of "Breakout"?
Hi John,
No, Mark is right. This is a total distraction.
Obviously something people love to talk about, but it doesn’t serve any purpose.
If what you need is not 800GbE or 1.6TbE, but rather a dual or quad 400GbE module, great. Go build that. You don’t need any new 802.3 project or standard to allow you to do this.
I think I can assert now that there will not be an 800GbE or 1.6TbE PHY type with broad market potential that uses the lane technology of today’s 200G and 400G PHYs. Sure, there might be one defined so people can debug their MAC/PCS (like
the 400GBASE-SR16), but forget that from a broad market potential perspective.
Will the lane technology (e.g., 200G lanes with a stronger FEC) that emerges from an 800GbE or 1.6TbE project be something you can leverage to do a later P802.3cd type project to define new 200G and 400G PHY types with fewer lanes? Possibly.
Maybe even probably.
So if and when that all happens, there emerges (whether you write it down or not), that people will build stuff that does breakout of these new 200G and 400G PHY types from the modules you built for 800GbE or 1.6TbE.
That might be something that lets you claim broad market potential for your new 200G and 400G PHY types to justify the future P802.3cd type project but it is
not something that establishes broad market potential for your 800GbE or 1.6TbE. So let’s not talk about it in the context of your new CFI or study group.
I would not like to see this called out as something for the Study Group to talk about, because as this email thread proves, people have the capacity to spend infinite amounts of time talking about it, and it doesn’t help at all in establishing
broad market potential for 800GbE or 1.6TbE.
So yes, I think distraction is exactly the right word.
Regards,
Steve
From: John D'Ambrosia <jdambrosia@xxxxxxxxx>
Sent: Thursday, August 20, 2020 7:29 PM
To: STDS-802-3-NGECDC@xxxxxxxxxxxxxxxxx
Subject: Re: [802.3_NGECDC] Definition of "Breakout"?
Mark
I am going to disagree with you. I don’t see this conversation as a distraction at all. The fact that we are having this conversation, which has been quite healthy and revealing, is a great reason for us to exercise it the form of a study
group.
Please note you state –
- “Breakout” typically means an specific implementation that supports more than one of these specific individual PHY specifications so that for example a 400G pluggable optical module
you could buy today can support a single 400GBASE-DR4 interface and also support 4 individual 100GBASE-DR interfaces that are “broken out” from that module.
The discussion about what is breakout has been helpful
As noted – one could note a single PHY consists of “n” lanes where “n” could be an existing lower ethernet speed. In this instance the physical lower speed specification would leverage quite heavily the existing “lane” specification of
the higher speed port.
The instance that you have noted – where an implementation has been broken up may or may not meet what I noted above. For example, the LR example noted earlier – where it is very clear the PMD specification would be quite different because
of the obvious elimination of the mux / demux.
So very good conversation.
Now I note you state –
Looking forward as we start considering new PHYs and possibly new lane rates, we may want to develop our set of objectives in a project so that the if a PHY is being proposed that could allow a breakout into another PHY then we might want
to include all related PHYs into the same project. By separating 400GBASE-DR4 into .3bs and 100GBASE-DR into .3cd we caused ourselves some challenges. But by including 50GBASE-CR, 100GBASE-CR2 and 200GBASE-CR4 together into .3cd we avoided potential challenges.
As I noted at beginning of thread – this conversation was started to help me define the chartering motion of the study group I am proposing. Based on what you have stated above, it appears you would support having the discussion in the
study group, and letting the study group determine what objectives go in or not. This means I need to properly scope it, which means what is meant by “breakout” becomes very important.
Regards,
John
From: Mark Nowell (mnowell) <00000b59be7040a9-dmarc-request@xxxxxxxxxxxxxxxxx>
Sent: Thursday, August 20, 2020 9:17 PM
To: STDS-802-3-NGECDC@xxxxxxxxxxxxxxxxx
Subject: Re: [802.3_NGECDC] Definition of "Breakout"?
Late coming to the thread…
I’ve thought through this many times over the many years that this discussion flares up in these forums. I have come to a fairly simple perspective in my mind:
- In IEEE we define individual PHY specifications to be interoperable which are independent of implementation
- “Breakout” typically means an specific implementation that supports more than one of these specific individual PHY specifications so that for example a 400G pluggable optical module
you could buy today can support a single 400GBASE-DR4 interface and also support 4 individual 100GBASE-DR interfaces that are “broken out” from that module.
- While “breakout” was never an objective in any of the standards work developing these PHYs specs, everyone was aware of the implementation considerations and the specs were written
to a) ensure interop as we alwaysdo and b) be inclusive of some of the extra losses/noises etc that might occur in the breakout implementation that might not exist in the single implementation
- Examples here might be the connector losses of an MPO vs an LC or the crosstalk/noise in a copper cable from extra adjacent conductors.
I therefore think that from an IEEE standardization perspective that “breakout” is a bit of a distraction that causes these email threads but doesn’t really factor into the specifics of defining a single interoperable PHY specification.
Looking forward as we start considering new PHYs and possibly new lane rates, we may want to develop our set of objectives in a project so that the if a PHY is being proposed that could allow a breakout into another PHY then we might want
to include all related PHYs into the same project. By separating 400GBASE-DR4 into .3bs and 100GBASE-DR into .3cd we caused ourselves some challenges. But by including 50GBASE-CR, 100GBASE-CR2 and 200GBASE-CR4 together into .3cd we avoided potential challenges.
Long winded way of saying that I don’t think there needs to be a formal definition of “breakout” in IEEE 802.3 as I don’t think it affects our work. We write these specs with the knowledge that we want to use these specs to build products
in implementations that we know (or hope) the industry wants to use. If we did develop a formal definition of “breakout” I’m not sure where it would go and I can guarantee that once we write it down, we’ll come up with something that wants to violate it…
Mark
From: "Gary Nicholl (gnicholl)" <00000bb92642e11e-dmarc-request@xxxxxxxxxxxxxxxxx>
Reply-To: Gary Nicholl <gnicholl@xxxxxxxxx>
Date: Thursday, August 20, 2020 at 5:56 PM
To: "802.3 NGECDC" <STDS-802-3-NGECDC@xxxxxxxxxxxxxxxxx>
Subject: Re: [802.3_NGECDC] Definition of "Breakout"?
Fixing my typo below
😊
From: Gary Nicholl (gnicholl)
Sent: Thursday, August 20, 2020 5:55 PM
To: Trowbridge, Steve (Nokia - US) <steve.trowbridge@xxxxxxxxx>;
STDS-802-3-NGECDC@xxxxxxxxxxxxxxxxx
Subject: RE: [802.3_NGECDC] Definition of "Breakout"?
Let me add to my comment below to pick up on the point that I think Jeff may be making.
If I implement a single 100GBASE-DR in a QSFP28 module, it is likely to be using a duplex LC connector as the MDI.
If I implement four instances of 100GBASE-DR in a QSFP-DD module , it is likely to be using two lanes of an MPO-12 connector as the MDI.
So the two cases are using different optical connectors.
Now in the IEEE specification we don’t dictate a specific optical connector (as that is an implementation choice). However if you look at the 802.3cd spec below (section 140.10.3) , we do define the performance specifications that any
optical connector has to meet.
So irrespective of whether I am implementing a single 100GBASE-DR in a QSFP28 or four 100GBASE-DRs in a QSFP-DD , then in both cases whatever optical connector I decide to use has to meet the same performance specifications that are defined
in 140.10.3.
Gary …
From: Gary Nicholl (gnicholl)
Sent: Thursday, August 20, 2020 5:40 PM
To: Trowbridge, Steve (Nokia - US) <steve.trowbridge@xxxxxxxxx>;
STDS-802-3-NGECDC@xxxxxxxxxxxxxxxxx
Subject: RE: [802.3_NGECDC] Definition of "Breakout"?
I agree with Steve. To me “breakout” has always been an “implementation” decision, and is not something that should be part of an IEEE 802.3 PHY specification.
I do agree that there has been a lot of confusion over the term in the industry. For example I have often heard people talk about “400GBASE-DR4 breakout”. There is technically no such thing. Now as an implementation choice a clever module
designer might design a single 400G QSFP-DD module that can support either a single instance of a 400GBASE-DR4 PHY or four separate instances of a 100GBASE-DR PHY.
Gary
From: Trowbridge, Steve (Nokia - US) <steve.trowbridge@xxxxxxxxx>
Sent: Thursday, August 20, 2020 12:51 PM
To: STDS-802-3-NGECDC@xxxxxxxxxxxxxxxxx
Subject: Re: [802.3_NGECDC] Definition of "Breakout"?
Hi John,
I think you know my views on breakout, since I’ve expressed them when the discussion has come up at the start of previous projects.
Breakout is something that the clever designer does opportunistically in the case where you have PHYs that operate over parallel physical lanes (e.g., different strands of fiber or traces), and you have different MAC rates that operate
over the same lane rates & technologies that differ only in the number of lanes they use.
So just as an example, you have your 400GBASE-DR4 which could be used as a quad 100GBASE-DR(1) if your chip supports that mode of operation. But that mode of operation is something that a designer chooses to put into their chip. It isn’t
“free” other than possibly from a cabling perspective, since you need to build four clause 82 PCSs and clause 91 FEC sublayers into your chip as well as the clause 119 400G FEC/PCS. No standard tells you whether or how to do this.
But from a project perspective, what you really need or want is a dual or quad 400G module and not an 800GbE or 1.6TbE PHY, then go build that dual or quad module. You don’t need any new 802.3 project to do this. Only do the new project
if you need (have broad market potential for) the 800GbE or 1.6TbE PHY, and if such a thing has technical and economic feasibility.
While you do that project, do you keep your eyes open for breakout opportunities? Sure, who wouldn’t? But to try to relate it to applications in the past:
For 400GbE, there was ONE PHY type (400GBASE-SR16) which was based on previous generation lane technology, so sure, one might want to have built a chip that could also do a quad 100GBASE-SR4. But the lane rate isn’t even the same (since
the 400GBASE-SR16 uses RS(544) FEC and 100GBASE-SR4 uses (RS528)), so not quite as simple as it sounds. Plus, 400GBASE-SR16 was just a super early adopter/lab type interface, never a widely deployed 400G interface. The way 400G was written for eventual high-volume
applications was optimized for (choosing RS(544) FEC ubiquitously) 50G and 100G lanes, and not optimized for the previous generation lane technology.
The 400GBASE-DR4/100GBASE-DR is probably a more widely used (or will be more widely used) breakout application, however, this was dependent on specifying a new 100G PHY type using the next gen lane technology in the P802.3cd project.
So to muse about the future, one can assume that a future 800GbE or 1.6TbE standard isn’t going to be optimized for 100G lanes, as we historically don’t get to high volumes with 8 or 16 lane implementations. Surely, we’ll optimize around
something like a 200G lane. Such a lane technology may well require more than RS(544) FEC.
So if you posit that you want to use these future PHYs for breakout, this leads to some future project that will define 200G and 400G PHY types based on this new 200G per lane technology and new FEC, and you’ll break out to this new PHY
type, not to any existing one.
But that future project again is opportunistic, saying I can build a better 200G or 400G PHY because of the technology developed for 800GbE or 1.6TbE.
So in summary, breakout should not be the consideration for why you do the next gen 800G or 1.6T rate project. Breakout
may be the reason you do the next “802.3cd” type project, using the lane technology of your 800G or 1.6T PHYs to build better or smaller 200G or 400G PHYs.
Regards,
Steve
From: John D'Ambrosia <jdambrosia@xxxxxxxxx>
Sent: Thursday, August 20, 2020 10:04 AM
To: STDS-802-3-NGECDC@xxxxxxxxxxxxxxxxx
Subject: Re: [802.3_NGECDC] Definition of "Breakout"?
David –
I don’t consider the second case breakout of an 800GbE PHY. Using prior standards as an example – one would expect 800GBASE-LRx to be a WDM solution, where “x” wavelengths are optically muxed / demuxed. What I believe you are proposing
is an implementation where “x” # of the lower speed are packaged together. So the 800GbE PHY is not being broke out – the port implementation, which is not the focus of the standard, is.
John
From: David Ofelt <ofelt@xxxxxxxxxxx>
Sent: Thursday, August 20, 2020 11:50 AM
To: jdambrosia@xxxxxxxxx;
STDS-802-3-NGECDC@xxxxxxxxxxxxxxxxx
Subject: Re: [802.3_NGECDC] Definition of "Breakout"?
I think there are two different concepts that end up being colloquially referred to as breakout. The first is that case you detaiil below- a set of parallel media lanes that can be grouped in various ways- either as a single unified PHY
or as a number of slower PHYs. The other case is a module that happens to hold a number of PHYs that are completely independent- like a QSFP-DD/OSFP module that has 4 x 100GBASE-LR optics.
I’m Ok with declaring that “breakout” just covers the first case and your list is a good start at scoping the definition, but if we do that, I’d like us to figure out a name for the other case. I find that when I talk to people about modules-
it is important to clearly address both cases, since many folks have only one of the cases in mind and conversations can get confusing.
I am also Ok with defining “breakout” to cover both cases, but then we can make that explicit in the definition.
--
DaveO
From: John D'Ambrosia <jdambrosia@xxxxxxxxx>
Reply-To: "jdambrosia@xxxxxxxxx" <jdambrosia@xxxxxxxxx>
Date: Thursday, August 20, 2020 at 06:03
To: "STDS-802-3-NGECDC@xxxxxxxxxxxxxxxxx" <STDS-802-3-NGECDC@xxxxxxxxxxxxxxxxx>
Subject: [802.3_NGECDC] Definition of "Breakout"?
[External Email. Be cautious of content]
All,
As I explore the scope for the Beyond 400 GbE effort, I have been having a number of conversations related to “breakout”
While we all discuss it – I have never seen some actual formal definition that is agreed upon within 802.3. So I would like to get some input.
I am going to start with breakout actually does and solicit input before proposing some definition to potentially use.
I see break out of the following –
- AUI
- Related PHYs
- Backplane
- Twin-ax cabling based on multiple different pairs
- SR optics based on parallel MMF
- DR optics based on parallel SMF
FR / LR / ER optics – I don’t see as being part of breakout.
Thoughts?
John
To unsubscribe from the STDS-802-3-NGECDC list, click the following link:
https://listserv.ieee.org/cgi-bin/wa?SUBED1=STDS-802-3-NGECDC&A=1
To unsubscribe from the STDS-802-3-NGECDC list, click the following link:
https://listserv.ieee.org/cgi-bin/wa?SUBED1=STDS-802-3-NGECDC&A=1
To unsubscribe from the STDS-802-3-NGECDC list, click the following link:
https://listserv.ieee.org/cgi-bin/wa?SUBED1=STDS-802-3-NGECDC&A=1
To unsubscribe from the STDS-802-3-NGECDC list, click the following link:
https://listserv.ieee.org/cgi-bin/wa?SUBED1=STDS-802-3-NGECDC&A=1
To unsubscribe from the STDS-802-3-NGECDC list, click the following link:
https://listserv.ieee.org/cgi-bin/wa?SUBED1=STDS-802-3-NGECDC&A=1
To unsubscribe from the STDS-802-3-NGECDC list, click the following link:
https://listserv.ieee.org/cgi-bin/wa?SUBED1=STDS-802-3-NGECDC&A=1
To unsubscribe from the STDS-802-3-NGECDC list, click the following link: https://listserv.ieee.org/cgi-bin/wa?SUBED1=STDS-802-3-NGECDC&A=1
To unsubscribe from the STDS-802-3-NGECDC list, click the following link: https://listserv.ieee.org/cgi-bin/wa?SUBED1=STDS-802-3-NGECDC&A=1