Re: [STDS-802-3-400G] one comment about 4x100G breakout RE: IEEE P802.3b

Thread Links	Date Links
Thread Prev	Thread Next	Thread Index	Date Prev	Date Next	Date Index

Dan, Brad, Ali, Du,

I think there may be a fundamental disconnect in our discussion. Here you all seem to be viewing the introduction of 400G-SR16 PMD in ToR switches as simultaneously delivering 400G uplinks. While it certainly is a 400G interface, it also has at least two additional utilities: 1) as a 4x100G-SR4 interface and 2) as a 16x25G interface. It is not necessary for ToR switches to offer 400G uplinks in order for the 400G-SR16 transceiver to be useful for cost reduction of the 25G server-side ports. This is a wider and faster extension of the already-proven-successful deployment of 40G-SR4 as 4x10G, or the use of 100G-SR4 for 4x25G. I am sure this is already obvious to you, but I need to make this distinction for clarity.

If 25G servers will exist side by side in the market with 10G servers and 50G servers, then switches optimized for each make sense. Each of these rates will have its own significant lifespan, serving the various needs of the diverse market. It may be that 100G-SR4 is the go-to solution for increasing density and decreasing cost for 25G server-side switch ports because it is a mirror of 40G-SR4 deployments. If that is the case, please explain why integration of 4 ports takes preference over integration of 16. Does it come back to Joel’s implementation issues, or is it something else?

Regards,

Paul

From: Dan Dove [mailto:dan.dove@xxxxxxxxxxxxxxxxxx]
Sent: Wednesday, June 24, 2015 7:35 PM
To: Kolesar, Paul; STDS-802-3-400G@xxxxxxxxxxxxxxxxx
Subject: Re: [STDS-802-3-400G] one comment about 4x100G breakout RE: IEEE P802.3bs 400 Gb/s Ethernet Task Force Logic Ad Hoc

Hi Paul,

I agree with Brad, Ali on this. While "25G is an important and lasting lane rate", it will rapidly shift to 50G serial rates when they become available at an approximate cost (1.5x or less) for higher volume applications.

Its all about timeframes.

I agree about your point of MicroSoft vs the broader market. It sounds like they are driving server I/O bandwidth faster than the enterprise and will be for a long time to come. So you aren't missing the target on volume demand, but as for time-frame I think that by the time 50G I/O hits the enterprise, 50G serial will be mainstream.

That said, the enterprise market is not going to be putting 400G on TOR switches for a "long-time". 4x50G QSFP-28 is a more likely solution in that market space and in that time frame and thus 16x25G for TOR applications doesn't make sense. And this 400G project is the 1st of more to come, so pushing a 400G breakout to servers makes no sense.

The original question of supporting KP4 FEC on a 400G port in case 100G PAM-4 needs the FEC is a good one IMO. Its a tough one as well. Bet on the come? Bet on the pass? I think we can assume that future projects for 400G can address it in higher-density, lower cost timeframes. I would bet on the pass for this project. Leave it to the next generation 400G project.

Dan Dove
Chief Consultant
Dove Networking Solutions
530-906-3683 - Mobile

On 6/24/15 4:44 PM, Kolesar, Paul wrote:

Brad,

I am not trying to dispute what you already know as an employee of Microsoft. But the 50G you speak of is multi-lane, meaning 2x25G. This only emphasizes my point that 25G is an important and lasting lane rate because it has a lot of utility. Just because 50G servers may exist does not mean 25G servers disappear. That certainly has not been the case with 1G servers in the wake of 10G servers. The redesign of switches is not only driven by lane rate increases, but also by cost reductions. There will remain a lot of customers interested in non-bleeding edge solutions even as hyperscale DCs move upward. So while your view may be spot on for Microsoft, that does not mean it reflects the total market. As a recurring theme it has become obvious that one size increasingly fails to fit all.

Regards,

Paul

From: Brad Booth [mailto:bbooth@xxxxxxxx]
Sent: Wednesday, June 24, 2015 5:52 PM
To: STDS-802-3-400G@xxxxxxxxxxxxxxxxx
Subject: Re: [STDS-802-3-400G] one comment about 4x100G breakout RE: IEEE P802.3bs 400 Gb/s Ethernet Task Force Logic Ad Hoc

There will be 50 GbE on the server before there is 400G on the TOR and even before 400G task force gets to working group ballot. :-) It won't be 50G on a single lane though, but it will be 50G Ethernet.

Also, by the time 400G is on the TOR, there will be end users considering 100G (again, not on a single lane) to the server.

Cheers,
Brad

On Wed, Jun 24, 2015 at 1:27 PM, Ali Ghiasi <aghiasi@xxxxxxxxx> wrote:

Paul

I said by "the time server and silicon technology catches where 400 GbE is integrated on the TOR”! In 1-2 silicon generation when 400 GbE is integrated on the TOR then you will also see 50 GbE on the server.

We are developing 25GbE because 10G is not sufficient for a segment of market and 25 GbE cost less than the alternative the 40 GbE. The same economic will apply to 50 GbE later on!

Thanks,

Ali Ghiasi

Ghiasi Quantum LLC

On Jun 24, 2015, at 12:31 PM, Kolesar, Paul <PKOLESAR@xxxxxxxxxxxxx> wrote:

Ali,

In order for 50G lanes to make sense for ToR switches, servers must be able to use that rate. I think the market will put 25G servers to several years of use before 50G servers overtake them. Witness the time lag for 10G servers to overtake sub-10G servers. 10G is now in its prime with 25G up and coming. If this is not true, why are we bothering to develop 25GE?

Paul

From: Ali Ghiasi [mailto:aghiasi@xxxxxxxxx]
Sent: Wednesday, June 24, 2015 1:59 PM
To: Kolesar, Paul
Cc: STDS-802-3-400G@xxxxxxxxxxxxxxxxx
Subject: Re: [STDS-802-3-400G] one comment about 4x100G breakout RE: IEEE P802.3bs 400 Gb/s Ethernet Task Force Logic Ad Hoc

Paul

Initial application of 400 GbE will follow 100 GbE early deployment SMF ports on Routers and Spin switches. Du Wenhua statement is correct to say 400 GbE will not be integrated initially on TOR. By the time server and silicon technology catches where 400 GbE will be integrated on TOR then we will have 50 GbE on single lane of SMF/MMF.

So 16x25G MMF has limited life/application and soon to be obsolete with emergence of 50G eco-system.

Thanks,

Ali Ghiasi

Ghiasi Quantum LLC

<image001.png>

On Jun 24, 2015, at 7:25 AM, Kolesar, Paul <PKOLESAR@xxxxxxxxxxxxx> wrote:

Du Wenhua,
I thanks for your thoughts. This is a valuable discussion.

I do not disagree with your points on KP4 and KR4 FEC at 100G, but I have a different view on your conclusion point 3). I think 16x25G and 8x50G break-outs can play a role in ToR switch ports facing the servers. A port that delivers 16 lanes of 25G can provide transceiver cost advantages compared to discrete transceivers. It also provides electrical signal routing advantages because all electrical lanes can be of nearly the same minimal length rather than spread out to reach ports across the entire switch faceplate which alleviates the need for repeater chips saving cost and power.

Regards,
Paul Kolesar

-----Original Message-----
From: Duwenhua [mailto:duwenhua@xxxxxxxxxxxxx]
Sent: Wednesday, June 24, 2015 3:36 AM
To: STDS-802-3-400G@xxxxxxxxxxxxxxxxx
Subject: [STDS-802-3-400G] one comment about 4x100G breakout RE: IEEE P802.3bs 400 Gb/s Ethernet Task Force Logic Ad Hoc

Hi, Martin

400GE FEC Breakout: (langhammer_02_0615_logic.pdf) pag5 said:
1x400G KP4 ASIC: 55645
1x400G KP4 & 4x100G KR4 ASIC: 61518
  11% area increase

My comments:
1) What is the size of 1x400G KP4 & 4x100G KP4 FEC?
2) today 100GbE choose KR4，but I think that the breakout 100GbE will need KP4 FEC if the 100GbE SMF PMD choose 100G PAM4 in future.
     --In long term, serial is better than parallel. For 100GbE, 100G PAM4 SMF PMD is better than 4x25G SMF PMD.
3) same idea can be seen at page29 of http://www.ieee802.org/3/bs/public/14_05/maki_3bs_01a_0514.pdf   QSFP28 has a DSP&FEC which convert 4x25G to 1x100G.

My conclusions:
1) 4x100G breakout, the FEC should be able to dynamic changed: 4x100G KR4, or 4x100G KP4.
2) 4x100G breakout is very important in data center switch at the positions of Spine switch and Core switch, because face port density and flexibility.
3) 8x50G and 16x25G breakout is not important, because only TOR switch need 50G/25GE, but TOR switch do not have 400GE port.

                         --Du Wenhua

-----Original Message-----
From: Mark Gustlin [mailto:mark.gustlin@xxxxxxxxxx]
Sent: Saturday, June 20, 2015 8:09 AM
To: STDS-802-3-400G@xxxxxxxxxxxxxxxxx
Subject: Re: [STDS-802-3-400G] IEEE P802.3bs 400 Gb/s Ethernet Task
Force Logic Ad Hoc

All,

Presentations are now up on the logic ad hoc web site from today's ad hoc call:
http://www.ieee802.org/3/bs/public/adhoc/logic/index.shtml

Thanks, Mark

-----Original Message-----
From: Mark Gustlin
Sent: Monday, June 15, 2015 10:59 AM
To: STDS-802-3-400G@xxxxxxxxxxxxxxxxx
Subject: RE: IEEE P802.3bs 400 Gb/s Ethernet Task Force Logic Ad Hoc

All,

Just a reminder of the upcoming logic ad hoc call.
The current agenda is:
400GE FEC Implementation - Martin Langhammer Considerations for
breakout - Martin Langhammer 1x400G vs 4x100G FEC Implications -
Bill Wilkie

Requests are due by end of day Wednesday. There is room for only one
more presentation for this call, so the next request will get the last slot.
There is another opportunity on June 29th.

Thanks, Mark Gustlin

From: Mark Gustlin [mailto:mark.gustlin@xxxxxxxxxx]
Sent: Thursday, June 04, 2015 2:49 PM
To: STDS-802-3-400G@xxxxxxxxxxxxxxxxx
Subject: [STDS-802-3-400G] IEEE P802.3bs 400 Gb/s Ethernet Task
Force Logic Ad Hoc

All,

This is an announcement of the next IEEE P802.3bs 400 Gb/s Ethernet
Task Force Logic Ad Hoc conference call opportunity.

The next call will take place on Friday June 19th from 8am-10am PDT.

If you are interested in presenting, please request a timeslot by
the end of Wednesday June 17th. A separate calendar invite will also follow.

In addition there will be another opportunity on Monday June 29th at
8am- 10am PDT.

Thanks, Mark Gustlin

-- Do not delete or change any of the following text. --

Join WebEx meeting
Meeting number: 492 414 237
Meeting password: four123!
Join by phone
Call-in toll-free number: 1-(877) 582-3182 (US) Call-in number:
1-(770) 657-
9791 (US) Show global numbers Conference Code: 568 245 6486