David,
I don’t disagree, but a key tenet of these specs is to not constrain implementation and we therefore need to think through how to propose something.
IEEE has separate specifications for each PMD, so it is not clear to me where we would add this “breakout guidance” in the specification ?
Also ‘breakout” is not just an optical issue but it also impacts all of the upper layers. If you look at the 400GBASE-DR4 PMD clause (Clause 124) it calls out all of the higher levels in the stack that are required to work with this specific
optical PMD (see below). You will notice that what is called out here is completely different than what is required for a 100GBASE-DR PMD (different FEC, PCS, etc, etc) .
So if we do want to add some “breakout guidelines” then we should probably put them in an informative Annex , as we do for PMA sublayer positioning (see Annex 120A as an example).
Gary
Ps. I wanted to respond to you’re a few of your specific comments below:
“How else would (not so clever) ASIC designers (or their marketing overlords) know that a
DR4 should break out into four DR(1)s" and “Lots extra money was spent and time was lost because
IEEE did not guide the implementers to include the breakout applications“
We need to be very careful here. The IEEE should absolutely
NOT be mandating that a 400G-DR4 solution has to include 4x100G-DR breakout. A designer should be free to implement an optimized 400G-DR4 port (ASIC + optics) without having to add all of the extra circuitry/complexity to support 4x100G-DR breakout
(and note even in the optics there is additional complexity here to support asynchronous clocking for the individual 100G lanes, which is not required for the 400G application).
From: Piehler, David <David.Piehler@xxxxxxxx>
Sent: Thursday, August 20, 2020 5:54 PM
To: Gary Nicholl (gnicholl) <gnicholl@xxxxxxxxx>; STDS-802-3-NGECDC@xxxxxxxxxxxxxxxxx
Subject: RE: [802.3_NGECDC] Definition of "Breakout"?for
I think it would be very useful for IEEE to be at least guiding towards the breakout application. How else would (not so clever) ASIC designers (or their marketing overlords) know that a DR4 should break out into four DR(1)s. Technically
the ASIC designers were correct, the 400GbE PHY does not support a breakout. Unfortunately many, if not most, of their customers, just *assumed* that DR4 would break out into four DR(1)s. Lots extra money was spent and time was lost because IEEE did
not guide the implementers to include the breakout application.
David P
From: Gary Nicholl (gnicholl) <00000bb92642e11e-dmarc-request@xxxxxxxxxxxxxxxxx>
Sent: Thursday, 20 August, 2020 14:40
To: STDS-802-3-NGECDC@xxxxxxxxxxxxxxxxx
Subject: Re: [802.3_NGECDC] Definition of "Breakout"?
[EXTERNAL EMAIL]
I agree with Steve. To me “breakout” has always been an “implementation” decision, and is not something that should be part of an IEEE 802.3 PHY specification.
I do agree that there has been a lot of confusion over the term in the industry. For example I have often heard people talk about “400GBASE-DR4 breakout”. There is technically no such thing. Now as an implementation choice a clever module
designer might design a single 400G QSFP-DD module that can support either a single instance of a 400GBASE-DR4 PHY or four separate instances of a 100GBASE-DR PHY.
Gary
From: Trowbridge, Steve (Nokia - US) <steve.trowbridge@xxxxxxxxx>
Sent: Thursday, August 20, 2020 12:51 PM
To: STDS-802-3-NGECDC@xxxxxxxxxxxxxxxxx
Subject: Re: [802.3_NGECDC] Definition of "Breakout"?
Hi John,
I think you know my views on breakout, since I’ve expressed them when the discussion has come up at the start of previous projects.
Breakout is something that the clever designer does opportunistically in the case where you have PHYs that operate over parallel physical lanes (e.g., different strands of fiber or traces), and you have different MAC rates that operate
over the same lane rates & technologies that differ only in the number of lanes they use.
So just as an example, you have your 400GBASE-DR4 which could be used as a quad 100GBASE-DR(1) if your chip supports that mode of operation. But that mode of operation is something that a designer chooses to put into their chip. It isn’t
“free” other than possibly from a cabling perspective, since you need to build four clause 82 PCSs and clause 91 FEC sublayers into your chip as well as the clause 119 400G FEC/PCS. No standard tells you whether or how to do this.
But from a project perspective, what you really need or want is a dual or quad 400G module and not an 800GbE or 1.6TbE PHY, then go build that dual or quad module. You don’t need any new 802.3 project to do this. Only do the new project
if you need (have broad market potential for) the 800GbE or 1.6TbE PHY, and if such a thing has technical and economic feasibility.
While you do that project, do you keep your eyes open for breakout opportunities? Sure, who wouldn’t? But to try to relate it to applications in the past:
For 400GbE, there was ONE PHY type (400GBASE-SR16) which was based on previous generation lane technology, so sure, one might want to have built a chip that could also do a quad 100GBASE-SR4. But the lane rate isn’t even the same (since
the 400GBASE-SR16 uses RS(544) FEC and 100GBASE-SR4 uses (RS528)), so not quite as simple as it sounds. Plus, 400GBASE-SR16 was just a super early adopter/lab type interface, never a widely deployed 400G interface. The way 400G was written for eventual high-volume
applications was optimized for (choosing RS(544) FEC ubiquitously) 50G and 100G lanes, and not optimized for the previous generation lane technology.
The 400GBASE-DR4/100GBASE-DR is probably a more widely used (or will be more widely used) breakout application, however, this was dependent on specifying a new 100G PHY type using the next gen lane technology in the P802.3cd project.
So to muse about the future, one can assume that a future 800GbE or 1.6TbE standard isn’t going to be optimized for 100G lanes, as we historically don’t get to high volumes with 8 or 16 lane implementations. Surely, we’ll optimize around
something like a 200G lane. Such a lane technology may well require more than RS(544) FEC.
So if you posit that you want to use these future PHYs for breakout, this leads to some future project that will define 200G and 400G PHY types based on this new 200G per lane technology and new FEC, and you’ll break out to this new PHY
type, not to any existing one.
But that future project again is opportunistic, saying I can build a better 200G or 400G PHY because of the technology developed for 800GbE or 1.6TbE.
So in summary, breakout should not be the consideration for why you do the next gen 800G or 1.6T rate project. Breakout
may be the reason you do the next “802.3cd” type project, using the lane technology of your 800G or 1.6T PHYs to build better or smaller 200G or 400G PHYs.
Regards,
Steve
From: John D'Ambrosia <jdambrosia@xxxxxxxxx>
Sent: Thursday, August 20, 2020 10:04 AM
To: STDS-802-3-NGECDC@xxxxxxxxxxxxxxxxx
Subject: Re: [802.3_NGECDC] Definition of "Breakout"?
David –
I don’t consider the second case breakout of an 800GbE PHY. Using prior standards as an example – one would expect 800GBASE-LRx to be a WDM solution, where “x” wavelengths are optically muxed / demuxed. What I believe you are proposing
is an implementation where “x” # of the lower speed are packaged together. So the 800GbE PHY is not being broke out – the port implementation, which is not the focus of the standard, is.
John
From: David Ofelt <ofelt@xxxxxxxxxxx>
Sent: Thursday, August 20, 2020 11:50 AM
To: jdambrosia@xxxxxxxxx;
STDS-802-3-NGECDC@xxxxxxxxxxxxxxxxx
Subject: Re: [802.3_NGECDC] Definition of "Breakout"?
I think there are two different concepts that end up being colloquially referred to as breakout. The first is that case you detaiil below- a set of parallel media lanes that can be grouped in various ways- either as a single unified PHY
or as a number of slower PHYs. The other case is a module that happens to hold a number of PHYs that are completely independent- like a QSFP-DD/OSFP module that has 4 x 100GBASE-LR optics.
I’m Ok with declaring that “breakout” just covers the first case and your list is a good start at scoping the definition, but if we do that, I’d like us to figure out a name for the other case. I find that when I talk to people about modules-
it is important to clearly address both cases, since many folks have only one of the cases in mind and conversations can get confusing.
I am also Ok with defining “breakout” to cover both cases, but then we can make that explicit in the definition.
--
DaveO
From: John D'Ambrosia <jdambrosia@xxxxxxxxx>
Reply-To: "jdambrosia@xxxxxxxxx" <jdambrosia@xxxxxxxxx>
Date: Thursday, August 20, 2020 at 06:03
To: "STDS-802-3-NGECDC@xxxxxxxxxxxxxxxxx" <STDS-802-3-NGECDC@xxxxxxxxxxxxxxxxx>
Subject: [802.3_NGECDC] Definition of "Breakout"?
[External Email. Be cautious of content]
All,
As I explore the scope for the Beyond 400 GbE effort, I have been having a number of conversations related to “breakout”
While we all discuss it – I have never seen some actual formal definition that is agreed upon within 802.3. So I would like to get some input.
I am going to start with breakout actually does and solicit input before proposing some definition to potentially use.
I see break out of the following –
- AUI
- Related PHYs
- Backplane
- Twin-ax cabling based on multiple different pairs
- SR optics based on parallel MMF
- DR optics based on parallel SMF
FR / LR / ER optics – I don’t see as being part of breakout.
Thoughts?
John
To unsubscribe from the STDS-802-3-NGECDC list, click the following link:
https://listserv.ieee.org/cgi-bin/wa?SUBED1=STDS-802-3-NGECDC&A=1
To unsubscribe from the STDS-802-3-NGECDC list, click the following link:
https://listserv.ieee.org/cgi-bin/wa?SUBED1=STDS-802-3-NGECDC&A=1
To unsubscribe from the STDS-802-3-NGECDC list, click the following link:
https://listserv.ieee.org/cgi-bin/wa?SUBED1=STDS-802-3-NGECDC&A=1
To unsubscribe from the STDS-802-3-NGECDC list, click the following link:
https://listserv.ieee.org/cgi-bin/wa?SUBED1=STDS-802-3-NGECDC&A=1
To unsubscribe from the STDS-802-3-NGECDC list, click the following link: https://listserv.ieee.org/cgi-bin/wa?SUBED1=STDS-802-3-NGECDC&A=1