Kent,
Thank you for making the case for a single PHY. As I mentioned at the adhoc meeting on 2/25, the SFP+ (aka SFF-8431) exceeded success expectations because
it galvanized on a single port multiple optical reaches plus it introduced the usage of twin-ax cable as a viable copper link. Sure, as my good friend George Zimmerman mentioned in his comment, it had some initial shortcomings but it did not stop it from becoming
the de-facto interface for 10Gb/s Ethernet ports. Let's take this example as a direction to meet a broad market potential by facilitating the customer ease of selection and flexibility.
Cordially,
Rick Rabinovich
Principal Hardware Design Engineer
Distinguished Member of the Technical Staff
Alcatel-Lucent Enterprise
T: +1 818 878 4584
26801 West Agoura Road
Calabasas, CA 91301
rick.rabinovich@xxxxxxxxxxxxxxxxxx
www.alcatel-lucent.com/enterprise
From: Lusted, Kent C [mailto:kent.c.lusted@xxxxxxxxx]
Sent: Friday, February 27, 2015 11:23 AM
To: STDS-802-3-25G@xxxxxxxxxxxxxxxxx
Subject: Re: [STDS-802-3-25G] Market Fragmentation
Hi,
This is my perspective as an applications engineer that deals with the customer support calls.
J When x86 class servers with Ethernet LOM go into production, there are usually >100 different
customer designs at the same time, going to a wide range of customers. These server customers will have a breadth of Ethernet experience – from none all the way to much. The support burden is
significant.
First and foremost to me personally is interoperability. Nay, let’s call it “link-ability”. Specifically, when something is plugged in between 2 Ethernet
devices, the priority is to bring up _a_ link at that speed regardless of the latency or power implication or FEC type or cable length. Given that I don’t know in advance what the end customer’s link partner may be, I strongly desire to have all choices
available on both sides in order to have the greatest chance at getting link up.
Then let a savvy customer with intimate knowledge of their infrastructure optimize the link for their particular environment (latency, power, reach, etc.).
Second, it is my experience that server system designers are not always aware of what Ethernet capabilities will be required in their end-user network(s). Quite
commonly, a server designer chooses product #1 because it is much “better” than product #2 in some way only to find out much later that it isn’t what their customer wanted/needed. I am concerned that the same thing will happen in this 25G project if there
is a multiple PHY type designations. My server customers generally don’t know if they want a CR-S PHY or a CR-L PHY or a CR-N PHY or some combination of these but they do want “25 Gig Ethernet”!
Third, many of my server customers currently have challenges distinguishing connector types and the associated Ethernet port capabilities. The twin-ax connectors
no longer explicitly indicate the available speed (a la the BASE-T conundrum as George Zimmerman mentioned). As a recent example, a customer built a system with a QSFP+ connector. It didn’t link with a QSFP switch. :( This particular server system only
provided four 10G SFI DAC links via a breakout cable while the switch was one 40GBASE-CR4 link. This port identity issue is exacerbated if we specify different PHY types for 25G and the end user can't tell which 25G PHY sub-type is on the other side of the
connector.
It is for these reasons, I support a single 25GBASE-CR PHY type with 3 mandatory FEC types (no FEC, BASE-R FEC and RS-FEC) and the option to enable them selectively.
Dan, I would gladly collaborate with you for Berlin.
With regards,
-Kent
From: Dan Dove [mailto:dan.dove@xxxxxxxxxxxxxxxxxx]
Sent: Thursday, February 26, 2015 7:20 AM
To: STDS-802-3-25G@xxxxxxxxxxxxxxxxx
Subject: Re: [STDS-802-3-25G] Market Fragmentation
Hi Rob,
I have truly walked into this with an open mind and have been open to consider the alternatives. I've heard some who appear to have drawn a conclusion that removing RS-FEC provides a measurable benefit, but have yet to see a real benefit that outweighs
what I perceive to be a negative impact.
The process should drive the answer. The process should be to start off with a goal of broad market acceptance for a 25G direct attach solution. Whether that turns into two or three different PHYs is an outcome, not a goal.
To figure out how to get to the outcome, we look at the total % difference of silicon/cost/complexity of a 3m PHY with a 5m PHY. If there is a substantial difference, you lean toward multiple PHYs (at the expense of losing plug/play). If there is not a
substantial difference, you lean toward a single PHY to achieve plug/play at the expense of minor optimization.
So far the data presented, appears to show that while 20% of a PHY may be allocated just to the RS-FEC (which seems like a substantial amount), in practice these PHYs are going to have other silicon wrapped around them. My experience in ASIC products was
that the PHY was typically 10-20% of the overall ASIC. This makes the differential impact now 4%.
In addition, at a given process node, 20% may seem large but at the next process node, its going to become a smaller overall portion relative to the analog circuits that don't shrink substantially. Recognizing this future scenario just leads to more data
leaning toward a single PHY.
For 4x25G breakout from 100G ports which must include RS-FEC, there is no benefit to taking it out. For NIC ports that must support MMF, there is no benefit to having a PHY w/o RS-FEC capability. Will NIC vendors have to implement two solutions? One for
MMF applications, one for CR applications?
So the data is leading me toward a single PHY with RS-FEC, BASE-R FEC, and the ability to Auto-Negotiate FEC on/off as well as changing priority based on FEC vs reach.
The TCO for customers who seek low power can be achieved by careful cable selection and network layout. The cost differential of a non-RS-FEC solution (if there even was one) would rapidly disappear against the broader adoption and success offered by the
single plug-n-play PHY solution.
Conversely, creating two PHYs that don't interoperate at the same speed on the same type of cable can create a real problem for customers who just want to buy equipment and plug it in with the expectation that if its 25G, has the same type of plug, it will
just work.
I'm planning to put a presentation together for Berlin that lays this out. Anyone who wishes to collaborate, please contact me.
Dan Dove
Chief Consultant
Dove Networking Solutions
530-906-3683 - Mobile
On 2/25/15 3:00 PM, Rob Stone wrote:
Hi Rick and Dan
I recognize that we need to make sure we are addressing BMP, but I don’t believe it is valid to equate creation of separate PHY types with market fragmentation. There seems to be support within the group for implementing a PHY (or set of PHYs) which support CL108, 74 and no FEC). I think what we are discussing is how you AN these modes with minimal customer confusion and maximum flexibility. I see the perceived drawback of creating three PHY types therefore somewhat artificial as it’s just another way of meeting those end goals.
Remember, we have received feedback that there is a desire from the end customers that they want maximum flexibility to not design in redundant modes for a particular use case, as well as making the solution maximally plug and play interoperable. I would argue that the three PHY solution achieves both these goals. If you want maximum interoperability, implement all three PHYs. If you want to do a subset, then that’s OK too, and if you do this it will be very clear what FEC modes and channels are and are not supported. I don’t believe this would be the case if we went for single PHY solution, especially if you want AN to resolve in a single pass.
Thanks
Rob
From: RABINOVICH, Rick (Rick)** CTR ** [mailto:rick.rabinovich@xxxxxxxxxxxxxxxxxx]
Sent: Wednesday, February 25, 2015 11:17 AM
To: STDS-802-3-25G@xxxxxxxxxxxxxxxxx
Subject: Re: [STDS-802-3-25G] Market Fragmentation
Hi Dan,
I do concur with your comments.
As a switch manufacturer I believe that fragmenting the market into niche ports defeats the broad market potential. Directly or indirectly the feedback I always get from end-users is to deliver plug-and-play solutions rather than pick and choose what you specifically need today even though it may become useless when different needs appear next year. Other system manufactures may get a different feedback and their comments are welcome. I rather move in the direction of a coherent and comprehensive solution.
Thank you,
Rick Rabinovich
Principal Hardware Design Engineer
Distinguished Member of the Technical Staff
Alcatel-Lucent Enterprise
T: +1 818 878 4584
26801 West Agoura Road
Calabasas, CA 91301
rick.rabinovich@xxxxxxxxxxxxxxxxxx
www.alcatel-lucent.com/enterprise
From: Dan Dove [mailto:dan.dove@xxxxxxxxxxxxxxxxxx]
Sent: Wednesday, February 25, 2015 9:29 AM
To: STDS-802-3-25G@xxxxxxxxxxxxxxxxx
Subject: [STDS-802-3-25G] Market Fragmentation
All,
As we discuss potential market fragmentation for 25GBASE-CR PHYs based upon reach (CR-S, CR-L, etc)...
We should also keep in mind that MMF optics will require RS-FEC.
So, if we choose to create a PHY type that does not support RS-FEC, we are not just dividing it from the relatively small 3-5m copper market. (I assume its small relative to 0-3m market) but we are also dividing it away from the larger body of 100G devices that would break out into 4x25G devices, and dividing it away from 25G MMF devices which may have a reasonable market size.
There was a comment made in today's adhoc that plug-and-play leads to happy customers. I would take it one step further, plug-and-play leads to more rapid market adoption and conversely, lack of plug-and-play can lead to substantial market disruption.
Historically, standards that have had trouble getting off the ground due to interoperability problems, lagged and sometimes never gained market acceptance. In some cases, MSAs or other means were used to achieve the desired customer satisfaction. We could literally "snake-bite" the standard in customer's eyes if we don't have plug-and-play, or at least clearly defined PHYs so that customers don't attempt to gain interoperability where it's not intended.
I believe we should take all of this into account when making our decision on which way to proceed and am willing to collaborate on a presentation to this effect if someone wishes to join me. I don't have a conclusion at this point, but am leaning toward a single PHY with both FECs and the ability to disable FEC for low-latency applications. This leaning comes from the data presented by Jeff Slavik combined with comments that in a typical ASIC the overall impact of the PHY gate counts for RS-FEC will be relatively small. I also agree with Eric's idea of remapping priority for low-latency vs cable-reach to ease Auto Negotiation, but would prefer to see it simply CR vs CR w/o FEC. I base this on the idea that only those who are engineering links would disable FEC.--
Dan Dove
Chief Consultant
Dove Networking Solutions
530-906-3683 - Mobile