Rich,
You have fully missed my point.
This is not an issue of objectives – it is an issue of what is within scope for this project, as granted by the 802.3bs PAR. My point to the others is if you want to talk about 50GbE, 100GbE, and 200GbE, the Task Force for 400GbE
is not the place to hold these discussions. One should use the DIALOG reflector. As I noted - Hence, my request to take 50GbE, 100GbE, or 200GbE discussion onto DIALOG, where it would belong.
IEEE 802.3 dialog reflector <stds-802-3-dialog@xxxxxxxx>
Location for appropriate non project related discussion
This is not a question of not having discussions – but where those discussions are appropriate.
Regards
John
From: Mellitz, Richard [mailto:richard.mellitz@xxxxxxxxx]
Sent: Wednesday, January 07, 2015 5:38 PM
To: DAmbrosia, John; STDS-802-3-400G@xxxxxxxxxxxxxxxxx
Subject: RE: 400G Electrical Channels
I fully understand that 400G AUI is in the objectives. Implementation is not. There is context in the CSD to guide us on this bandwidth march. If 50G by something migrates downward you will get your next
bandwidth tick too. (1.6T maybe?)
I get the impression from some discussions that 400G is drifting towards “a cost is no object” implementation. For 8x50G to work, you still need 50G. Perhaps, if at all possible, we should steer away
from known predicaments as this technology migrates down to lower cost boards and other port allocations. I understand about schedule, but is there a middle ground to playing ostrich?
… Rich
Chris,
We are no longer a study group, we are a task force with stated approved objectives. Much of this conversation in your prior email was primarily not about 400GbE, but about 50G and other rates that might
use it. Hence, my request to take 50GbE, 100GbE, or 200GbE discussion onto DIALOG, where it would belong.
Regards,
John
John
The thread is about 8x50G CDAUI-8 specification and how it should be written. It is entirely within scope to consider full range of applications. We have learned from past projects that when we don’t consider
the full range of applications, we cause ourselves grief.
This thread is also consistent with discussion during Study Group when developing our project objectives. We agreed that lower speed break-out applications are important and can be considered as part of
the specifications even if they are not formal objectives.
Chris
All,
This thread started out talking about topics related to 400GbE and an 8x50 approach, but then digressed into 1x50 and 4x50 (200GbE) discussions.
If the conversation is going to move onto these topics (which are not part of the scope of the 802.3bs project), I request that you move such threads over onto the 802-3 DIALOG reflector. See http://www.ieee802.org/3/dialog_reflector.html
for further information on subscribing to the Dialog reflector.
Regards,
John D’Ambrosia
Chair, IEEE P802.3bs 400GbE Task Force
Hi Scott,
You raise a valid question as to whether CAUI-2 (2x50G) electrical I/O will be interesting.
ASICs with 50Gb/s SerDes can support 100Gb/s Ethernet with two configurations.
1. Reduce the SerDes rate to 25Gb/s and support 100G with 4x25G ( CAUI-4) I/O
2. Keep the SerDes rate at 50Gb/s and use new 2x50G (CAUI-2) I/O
Reducing the SerDes rate to 25Gb/s results in half the 100G ports supportable by operating at 50Gb/s rate. This wastes half the ASIC switching bandwidth. Therefore if ASICs with 50Gb/s SerDes do not use
2x50G I/O for 100G ports, 200G ports become dominant and 100G becomes legacy. This is similar to how ASICs with 25G SerDes are now used to support 40G. The 25G SerDes are operated at 10G to support 4x10G legacy links and not as a 2x20G I/O to double 40G ports.
There is no disagreement that SFP has lower cross-talk than QSFP. However, the issue is how do we develop a 50Gb/s per single lane spec. If the specification only supports SFP level cross-talk, than 4x break-out using CFP4/QSFP or 8x break-out using CFP2 applications
are precluded. This is too restrictive. The more general specification assumes multi-channel crosstalk (either 4x or 8x), which support 4x and 8x break-out modules. SFP applications can then take advantage of the loss allocated to multi-channel cross-talk to
support longer host reach.
So future 1x50G, 2x50G (if of interest), and 4x50G electrical specifications should be similar or same as 8x50G CDAUI-8 specification we develop in 802.3bs.
Chris
Chris,
I haven’t heard of much interest in 2X50G 100GbE, but IEEE could consider a future project that includes 1X50, 2X50 and 4X50. I’d be glad to hear if others are interested in 2X50G 100GbE. 200GbE, based
on 4X50G, has been added to the Ethernet Alliance roadmap that can be viewed at:
My claim is that there is more crosstalk in a QSFP28 than an SFP28. I think this is backed up in all simulations that have been submitted to this group. In the case of 32GFC vs 128GFC, Avago has proposed
that there is a 2.5dB electrical crosstalk penalty for using QSFP28 vs an SFP28. Others, like John Petrilla and Jonathan King, are more versed in these crosstalk simulations and I hope that they can comment on the differences. Molex or TE should also be able
to discuss the differences in the crosstalk between the two connectors. I didn’t think this assumption that crosstalk is worse in a QSFP28 was really up for debate, the question is how much worse it is.
I’ll try and get some data that shows that electrical channels from the same SERDES in an ASIC support different lengths of traces depending on if they channels are routed through a QSFP28 or an SFP28.
The SFP28 should support longer channels with the same BER than a QSFP28 on a similar board.
My second claim is that the crosstalk penalties between 1X50G in SFP56 and 8X50G in CFP2 will be even larger than 2.5dB. I don’t have data to back this assumption up yet and would like to hear if anyone
has such data.
I agree that the standard should specify loss and not distance.
Kind regards,
Scott
Hi Scott,
Thank you for bringing into the discussion the perspective that we should be cognizant of future 1x50G and 4x50G I/O when developing 8x50G CDAUI-8 specifications. 2x50G should also be included as this will
double ASIC and front panel 100G port density.
However, the conclusion that 1x50G I/O will have lower cross-talk penalty than 8x50G is not supported by our experience with 1x10G and 1x25G I/O. 10G I/O is now used in break-out configurations, like 4x10G
QSFP+ and 10x10G CFP2. 25G I/O is now used in break-out configurations like 4x25G CFP4 followed by QSFP28, and may be used in 16x25G CDFP. This strongly suggests that we are likely to see 50G I/O break-out configurations like 4x50G CFP4 followed by QSFP28,
and 8x50G CFP2. Therefore the same cross-talk penalty is appropriate for 1x50G, 2x50G, 4x50G and 8x50G specifications.
With respect to material loss, the normative specification should specify loss, allowing the designer to trade-off material quality (and cost) against reach. This means we should write the 8x50G specification
so that it can be used for a variety of applications. Precluding 8x50G I/O Switch applications until 4x100G I/O becomes available seems too restrictive.
Chris
400G Task Force,
In the electrical ad hoc yesterday, I objected to using the materials loss tool because I thought it could be misused. After Beth explained how people should use the tool, I could not retract my objection before the conversation
went out of control. I agree to use the tool to predict losses, but would like for people to use the tool appropriately.
I would like to build consensus to what kinds of electrical channels this project should support. The question that arises is:
Should we define the same electrical channels for 1X50G, 4X50G and 8X50G when they are serving different markets and volumes?
I propose that 8X50G (CDAUI-8) electrical channels should be different than channels that will eventually be defined for 1X50GbE (LAUI) because:
1. Crosstalk - Crosstalk will impact the channel by more than 3dB. I base this off some excellent work that Avago
presented to T11 earlier this month. In 14-409v0, Avago showed how going from 1X28G (32GFC) to 4X28G (128GFC) caused a 2.5dB decrease in the channel mainly due to crosstalk in the QSFP28 connector. I would expect more crosstalk than 2.5dB when the speed and
number of channels are doubled.
2. Small market - The 400GbE market based on 802.3bs standards will be small compared to single lane 50GbE channels
that won’t be standardized for a few years. 802.3bs needs to recognize that this project is targeting a very small market for high-end router interconnects. The complete router market was less than 2M ports/year according to leading analyst firms and not
growing much. 16X25G and 8X50G will never be deployed in millions of ports/year because it will be made obsolete by a future 4X100G 400GbE project. 50GbE will go after a larger market and be designed into high-volume, server interconnects by the end of the
decade while 4X100G technology will be in the next decade.
3. High cost – Because of the low volume of 8X50G and the high quality required by the router market, the cost will
be high for 400GbE. Because of the premium nature of the router market, electrical channels will be made of high quality, high cost materials that can be absorbed in high end router products.
4. Better Materials – The group should model channels based on the better materials supported by the high-end router
market in 2018. As others said, the materials being targeted in 802.3bs should be different than the materials targeted for high volume servers in 802.3bj and a future 50GbE project.
Based on these 4 factors that are related, we need to agree on a distance that needs to be supported and consider crosstalk and the loss/inch of better materials. Based on these difference, I bet that CDAUI-8 electrical
channels will be 5dB different than the eventual LAUI-1 electrical channels.
We should make sure that CDAUI-8 is optimized for our application, not for LAUI-1 that might be optimized for server and switches.
I’d like to hear if you disagree with these claims.
Kind regards,
Scott Kipp