Re: [STDS-802-3-400G] one comment about 4x100G breakout RE: IEEE P802.3bs 400 Gb/s Ethernet Task Force Logic Ad Hoc
Joel,
Thanks for bringing up these pragmatic issues. They pertain not only to ToR switches but to all uses.
If running many close parallel traces is problematic, for ToR or any layout that does not place transceivers immediately adjacent to each other, there is the option to spread the PCB traces out a bit to alleviate the crosstalk and ground plane saturation. But that spread would be only minor relative to the variation required to reach to multiple transceivers.
Paul
-----Original Message-----
From: Joel Goergen (jgoergen) [mailto:jgoergen@xxxxxxxxx]
Sent: Wednesday, June 24, 2015 1:03 PM
To: STDS-802-3-400G@xxxxxxxxxxxxxxxxx
Subject: Re: [STDS-802-3-400G] one comment about 4x100G breakout RE: IEEE P802.3bs 400 Gb/s Ethernet Task Force Logic Ad Hoc
Hi Paul
I don’t disagree with your point on the electrical lanes, but there are
some disadvantages that have not been completely discussed and addressed.
It is true that the length will be minimal for that I/O going to a 16 lane
or 8 lane port and it is true that nets for a port type are more
controlled because they are not running all over the board.
But …
We are driving 16 lanes plus 16 lanes on the circuit board at 25Gbps … so
basically 32 diff pairs. Routing will be wide and it will cover layers,
widening the impedance discontinuity distribution for the board tolerance.
The mass number of pairs routed together will also saturate ground and
power planes, increase NEXT / FEXT, and basically wreck havoc with SI in
general. Even the compliance boards seem to have some interesting
characteristics to them based on the fact that compliance boards today are
difficult at best to build by anyone but a few that have spent an
incredible amount of time paying attention to each and every tiny detail.
We will have the same issues to address with 8 lanes of 50Gbps. Diff pair
spacing requirements and diff delay between +/- and pair to pair will be
challenging. This is one reason that I have been an advocate of 4 lanes.
Wether 4by25 or 4by50 or 4by100 or 4by200 or 4by?? … 4 lanes times two is
much easier to control and contain within the circuit board parameters.
Always has been. Though 5 lanes for 5by20 would have been a nice
compromise early on and gotten us to wide spread 20G solutions much faster
… but that discussion was years ago and I shouldn’t have brought it up.
But to that point … there were good reasons for the 4by approach even
though 20G would have made SI a bit easier and controlled materials a bit
easier.
What I haven’t seen is a lot of work on what needs to change in the tx,
rx, channel, and compliance board parameters to serve a 16by interface
(times two). I don’t know if my work will be finished in time for a
presentation in July, but I will try.
So in short … yup, somethings are easier and better … and somethings just
got a lot harder …
Take care
Joel
On 6/24/15, 7:25 AM, "Kolesar, Paul" <PKOLESAR@xxxxxxxxxxxxx> wrote:
>Du Wenhua,
>I thanks for your thoughts. This is a valuable discussion.
>
>I do not disagree with your points on KP4 and KR4 FEC at 100G, but I have
>a different view on your conclusion point 3). I think 16x25G and 8x50G
>break-outs can play a role in ToR switch ports facing the servers. A
>port that delivers 16 lanes of 25G can provide transceiver cost
>advantages compared to discrete transceivers. It also provides electrical
>signal routing advantages because all electrical lanes can be of nearly
>the same minimal length rather than spread out to reach ports across the
>entire switch faceplate which alleviates the need for repeater chips
>saving cost and power.
>
>Regards,
>Paul Kolesar
>
>-----Original Message-----
>From: Duwenhua [mailto:duwenhua@xxxxxxxxxxxxx]
>Sent: Wednesday, June 24, 2015 3:36 AM
>To: STDS-802-3-400G@xxxxxxxxxxxxxxxxx
>Subject: [STDS-802-3-400G] one comment about 4x100G breakout RE: IEEE
>P802.3bs 400 Gb/s Ethernet Task Force Logic Ad Hoc
>
>Hi, Martin
>
>400GE FEC Breakout: (langhammer_02_0615_logic.pdf) pag5 said:
>1x400G KP4 ASIC: 55645
>1x400G KP4 & 4x100G KR4 ASIC: 61518
> 11% area increase
>
>My comments:
> 1) What is the size of 1x400G KP4 & 4x100G KP4 FEC?
> 2) today 100GbE choose KR4，but I think that the breakout 100GbE will
>need KP4 FEC if the 100GbE SMF PMD choose 100G PAM4 in future.
> --In long term, serial is better than parallel. For 100GbE, 100G
>PAM4 SMF PMD is better than 4x25G SMF PMD.
> 3) same idea can be seen at page29 of
>http://www.ieee802.org/3/bs/public/14_05/maki_3bs_01a_0514.pdf QSFP28
>has a DSP&FEC which convert 4x25G to 1x100G.
>
>My conclusions:
> 1) 4x100G breakout, the FEC should be able to dynamic changed: 4x100G
>KR4, or 4x100G KP4.
> 2) 4x100G breakout is very important in data center switch at the
>positions of Spine switch and Core switch, because face port density and
>flexibility.
> 3) 8x50G and 16x25G breakout is not important, because only TOR switch
>need 50G/25GE, but TOR switch do not have 400GE port.
>
> --Du Wenhua
>
>
>> -----Original Message-----
>> From: Mark Gustlin [mailto:mark.gustlin@xxxxxxxxxx]
>> Sent: Saturday, June 20, 2015 8:09 AM
>> To: STDS-802-3-400G@xxxxxxxxxxxxxxxxx
>> Subject: Re: [STDS-802-3-400G] IEEE P802.3bs 400 Gb/s Ethernet Task
>> Force Logic Ad Hoc
>>
>> All,
>>
>> Presentations are now up on the logic ad hoc web site from today's ad
>>hoc call:
>> http://www.ieee802.org/3/bs/public/adhoc/logic/index.shtml
>>
>> Thanks, Mark
>>
>> > -----Original Message-----
>> > From: Mark Gustlin
>> > Sent: Monday, June 15, 2015 10:59 AM
>> > To: STDS-802-3-400G@xxxxxxxxxxxxxxxxx
>> > Subject: RE: IEEE P802.3bs 400 Gb/s Ethernet Task Force Logic Ad Hoc
>> >
>> > All,
>> >
>> > Just a reminder of the upcoming logic ad hoc call.
>> > The current agenda is:
>> > 400GE FEC Implementation - Martin Langhammer Considerations for
>> > breakout - Martin Langhammer 1x400G vs 4x100G FEC Implications -
>> > Bill Wilkie
>> >
>> > Requests are due by end of day Wednesday. There is room for only one
>> > more presentation for this call, so the next request will get the
>>last slot.
>> > There is another opportunity on June 29th.
>> >
>> > Thanks, Mark Gustlin
>> >
>> >
>> > From: Mark Gustlin [mailto:mark.gustlin@xxxxxxxxxx]
>> > Sent: Thursday, June 04, 2015 2:49 PM
>> > To: STDS-802-3-400G@xxxxxxxxxxxxxxxxx
>> > Subject: [STDS-802-3-400G] IEEE P802.3bs 400 Gb/s Ethernet Task
>> > Force Logic Ad Hoc
>> >
>> > All,
>> >
>> > This is an announcement of the next IEEE P802.3bs 400 Gb/s Ethernet
>> > Task Force Logic Ad Hoc conference call opportunity.
>> >
>> > The next call will take place on Friday June 19th from 8am-10am PDT.
>> >
>> > If you are interested in presenting, please request a timeslot by
>> > the end of Wednesday June 17th. A separate calendar invite will also
>>follow.
>> >
>> > In addition there will be another opportunity on Monday June 29th at
>> > 8am- 10am PDT.
>> >
>> > Thanks, Mark Gustlin
>> >
>> >
>> > -- Do not delete or change any of the following text. --
>> >
>> > Join WebEx meeting
>> > Meeting number: 492 414 237
>> > Meeting password: four123!
>> > Join by phone
>> > Call-in toll-free number: 1-(877) 582-3182 (US) Call-in number:
>> > 1-(770) 657-
>> > 9791 (US) Show global numbers Conference Code: 568 245 6486