|
Just a thought on this...
Latency will depend on the application that the network must support. HPC, storage and inference can benefit from lower latency, yet require a higher level of reliability. Standard compute and training operations are typically more latency tolerant, but training
operates much like inference when it comes to reliability. Generally, I've noticed that most networks are more concerned about tail latency, but that is rarely related to anything 802.3 can fix. I agree that the operators are unlikely to be happy if the network
autonomously adjusts the FEC (and thereby adjusts the latency), and it is likely that there will be a preference for a simpler and well-defined latency and reliability specification.
Cheers,
Brad
From: Chris Cole <chris@xxxxxxxxxxxxxxx>
Sent: Wednesday, July 5, 2023 12:08 PM
To: STDS-802-3-B400G@xxxxxxxxxxxxxxxxx <STDS-802-3-B400G@xxxxxxxxxxxxxxxxx>
Subject: Re: [802.3_B400G] [E] Re: [802.3_B400G] [802.3_B400G_LOGIC] Files posted: P802.3dj Joint Logic/Optics Track June 29 ad hoc meeting
Adee,
In Ethernet, every rate has its own specification. There are no mystery-performance operating rates.
How will an operator know if a link can operate in FEC bypass mode if we don't specify the performance?
Unfortunately, I was not involved in RJ45 Ethernet so can't comment. However, I am having trouble seeing operators today not caring about link latency before designing their datacenter network. Which operator to your knowledge doesn't need to know this?
Chris
On Wed, Jul 5, 2023 at 9:56 AM Adee Ran (aran) < aran@xxxxxxxxx> wrote:
Chris,
Ethernet has a long history of enabling plug and play and automatic configuration. Auto negotiation has been part of Ethernet since the first time there were two data rates, 10 Mb/s and “Fast Ethernet”
100 Mb/s. It is an extremely useful feature for anyone who has ever plugged an Ethernet cable into a RJ45 jack.
I’m not questioning the claim that many operators will want to have the power to configure the endpoints to their desired speed. But that can also be achieved with the auto negotiation framework. And
some operators/users may want to have the freedom to plug in a module and get the “it just works” experience. I think Nicklous has mentioned that in his response, I concur with that part.
In the case of inner FEC, we are not even talking about data rate negotiation. Whether or not the inner FEC is bypassed does not change the data rate, and for the host (PCS and upwards) it only shows
up as different latency. Now, this is not the first time we have latency that depends on FEC choice; 802.3by had three FEC modes with different latencies, and the mode was negotiated between the two sides using clause 73 AN. This was 25G copper cable driven
by data center applications, not end-user laptops. Even in the recent 802.3ck there is a 100G interleaved FEC mode that is negotiated using clause 73 AN. In all cases, anyone who wants to force as host to work in only one mode can do that.
The decision on FEC bypass should be independent of the question of whether or how the modules negotiate the right mode. Whatever “intelligence” the module has, it has to be overridable by management
– this goes without saying.
</Adee>
That seems sensible, but does bring us back to the difficult problem of how to do the autonomous mode.
Chris,
I concur. Many, if not most, operators will want to have the power to configure as opposed to autonomous negotiation. That being said a default to autonomous should be assumed as not every operator will look
to define it.
Dear 802.3dj TF Participants,
During the Ad Hoc call, Mike Dudek presented an important proposal for PMD reuse.
There was broad recognition of the benefits of reducing latency by bypassing the inner FEC. However, there was also concern about autonomously switching modes.
It may benefit to look back at how modes are switched in communications and Ethernet. The first widespread digital communication devices were voiceband modems which
initiate at low rate guaranteed to close the link, and increase the rate as progressively more powerful equalization enables faster and higher order modulation, i.e. same basic idea as proposed by Mike. The drawback is that one doesn't know until after the
highest rate is established, how fast the link will be. In contrast, Ethernet has always guaranteed a fixed rate. Even though many optical modules ship as multirate, each rate is a separate PDM spec. The datacenter operator decides when configuring their network
which rate to operate at based on what performance the PDM specs guarantee.
This suggests that the desired behavior for selecting whether the inner FEC is bypassed or not, is by the datacenter operator when configuring their network, not autonomously
in real time after the module is plugged in. If a shorter latency is not guaranteed, then the link must be assumed to operate at a longer latency for the purposes of defining the network configuration. What the operator needs is information on capability,
which means two independent PMD specs, one with inner FEC turned on, and one with inner FEC turned off, available before the network is configured. The BER rate can be one of the many real time diagnostics provided to the operating system, just like optical
power and temperature. It is unlikely that the operator would want the module to configure the network.
It would be helpful if any datacenter operator on the reflector chimed in and commented whether there is a benefit for optical modules to configure a network autonomously.
If yes, sharing the application scenario would further help understand the need.
Hi everyone,
The files for tomorrow’s P802.3dj joint optics and logic ad hoc are posted here:
https://www.ieee802.org/3/dj/public/adhoc/optics/0623_OPTX/index.html
The Presentations we will be reviewing are:
- "By-Passing the DR/FR inner FEC code for 200G/Lane" presented by Mike Dudek, Marvell
- "Module output BER requirements with inner FEC" presented by Adee Ran, Cisco
- "TDECQ Analysis with Increased SER Targets" presented by Greg LeCheminant, Keysight (*not posted yet, will be posted asap)
A calendar invite was sent out to both reflectors. But call details can be found here:
https://www.ieee802.org/3/calendar.html
I want to remind all teleconference meeting participants to review the following documents prior to participation in an IEEE 802.3 meeting teleconference:
- IEEE SA patent policy
- IEEE SA Copyright Policy
- IEEE SA Participation Policy
All of these policies may be found at http://ieee802.org/3/policies.html
Thanks,
Mark N
IEEE P802.3dj optics track leader
And
Mark G
IEEE P802.3dj architecture and logic track leader
To unsubscribe from the STDS-802-3-B400G-LOGIC list, click the following link:
https://listserv.ieee.org/cgi-bin/wa?SUBED1=STDS-802-3-B400G-LOGIC&A=1
To unsubscribe from the STDS-802-3-B400G list, click the following link:
https://listserv.ieee.org/cgi-bin/wa?SUBED1=STDS-802-3-B400G&A=1
To unsubscribe from the STDS-802-3-B400G list, click the following link:
https://listserv.ieee.org/cgi-bin/wa?SUBED1=STDS-802-3-B400G&A=1
To unsubscribe from the STDS-802-3-B400G list, click the following link: https://listserv.ieee.org/cgi-bin/wa?SUBED1=STDS-802-3-B400G&A=1
To unsubscribe from the STDS-802-3-B400G list, click the following link: https://listserv.ieee.org/cgi-bin/wa?SUBED1=STDS-802-3-B400G&A=1
|