Hi, Brad,
Thanks for asking the clarification.
For me in the customer support role, it is important to have a path for all of the “25 Gig Ethernet” servers that may exist to connect to all of the “25 Gig
Ethernet” switches that may exist, regardless of the 25 Gig cable used.
I hope this helps.
With regards,
-Kent
From: Brad Booth [mailto:bbooth@xxxxxxxx]
Sent: Sunday, March 01, 2015 9:26 PM
To: STDS-802-3-25G@xxxxxxxxxxxxxxxxx
Subject: Re: [STDS-802-3-25G] Market Fragmentation
I do have a question of clarification. When you stated, "priority is to bring up _a_ link at that speed regardless of the latency or power implication or FEC type or
cable length" did you mean that a link is up and running at better than the required BER or did you mean that the devices attempt to establish a link? To me, the first requirement of a link
up and running would mean that FEC and reach are important; the latter would agree with your statement.
What I'm trying to wrap my head around is the "3 mandatory FEC types." What if we specified only one mandatory FEC type (no FEC) and its channel specification. Then, the KRFEC and RSFEC could be optional to achieve the 3 m and 5 m reaches,
respectively.
On Fri, Feb 27, 2015 at 11:23 AM, Lusted, Kent C <kent.c.lusted@xxxxxxxxx> wrote:
Hi,
This is my perspective as an applications engineer that deals with the customer support calls.
J When x86 class servers with Ethernet LOM go into production, there are usually >100 different
customer designs at the same time, going to a wide range of customers. These server customers will have a breadth of Ethernet experience – from none all the way to much. The support burden is
significant.
First and foremost to me personally is interoperability. Nay, let’s call it “link-ability”. Specifically,
when something is plugged in between 2 Ethernet devices, the priority is to bring up _a_ link at that speed regardless of the latency or power implication or FEC type or cable length. Given that I don’t know in advance what the end customer’s link
partner may be, I strongly desire to have all choices available on both sides in order to have the greatest chance at getting link up.
Then let a savvy customer with intimate knowledge of their infrastructure optimize the link for their particular environment (latency, power, reach, etc.).
Second, it is my experience that server system designers are not always aware of what Ethernet capabilities
will be required in their end-user network(s). Quite commonly, a server designer chooses product #1 because it is much “better” than product #2 in some way only to find out much later that it isn’t what their customer wanted/needed. I am concerned that the
same thing will happen in this 25G project if there is a multiple PHY type designations. My server customers generally don’t know if they want a CR-S PHY or a CR-L PHY or a CR-N PHY or some combination of these but they do want “25 Gig Ethernet”!
Third, many of my server customers currently have challenges distinguishing connector types and the
associated Ethernet port capabilities. The twin-ax connectors no longer explicitly indicate the available speed (a la the BASE-T conundrum as George Zimmerman mentioned). As a recent example, a customer built a system with a QSFP+ connector. It didn’t link
with a QSFP switch. :( This particular server system only provided four 10G SFI DAC links via a breakout cable while the switch was one 40GBASE-CR4 link. This port identity issue is exacerbated if we specify different PHY types for 25G and the end user
can't tell which 25G PHY sub-type is on the other side of the connector.
It is for these reasons, I support a single 25GBASE-CR PHY type with 3 mandatory FEC types (no FEC,
BASE-R FEC and RS-FEC) and the option to enable them selectively.
Dan, I would gladly collaborate with you for Berlin.
With regards,
-Kent
Hi Rob,
I have truly walked into this with an open mind and have been open to consider the alternatives. I've heard some who appear to have drawn a conclusion that removing RS-FEC provides a measurable benefit, but have yet to see a real benefit that outweighs
what I perceive to be a negative impact.
The process should drive the answer. The process should be to start off with a goal of broad market acceptance for a 25G direct attach solution. Whether that turns into two or three different PHYs is an outcome, not a goal.
To figure out how to get to the outcome, we look at the total % difference of silicon/cost/complexity of a 3m PHY with a 5m PHY. If there is a substantial difference, you lean toward multiple PHYs (at the expense of losing plug/play). If there is not a
substantial difference, you lean toward a single PHY to achieve plug/play at the expense of minor optimization.
So far the data presented, appears to show that while 20% of a PHY may be allocated just to the RS-FEC (which seems like a substantial amount), in practice these PHYs are going to have other silicon wrapped around them. My experience in ASIC products was
that the PHY was typically 10-20% of the overall ASIC. This makes the differential impact now 4%.
In addition, at a given process node, 20% may seem large but at the next process node, its going to become a smaller overall portion relative to the analog circuits that don't shrink substantially. Recognizing this future scenario just leads to more data
leaning toward a single PHY.
For 4x25G breakout from 100G ports which must include RS-FEC, there is no benefit to taking it out. For NIC ports that must support MMF, there is no benefit to having a PHY w/o RS-FEC capability. Will NIC vendors have to implement two solutions? One for
MMF applications, one for CR applications?
So the data is leading me toward a single PHY with RS-FEC, BASE-R FEC, and the ability to Auto-Negotiate FEC on/off as well as changing priority based on FEC vs reach.
The TCO for customers who seek low power can be achieved by careful cable selection and network layout. The cost differential of a non-RS-FEC solution (if there even was one) would rapidly disappear against the broader adoption and success offered by the
single plug-n-play PHY solution.
Conversely, creating two PHYs that don't interoperate at the same speed on the same type of cable can create a real problem for customers who just want to buy equipment and plug it in with the expectation that if its 25G, has the same type of plug, it will
just work.
I'm planning to put a presentation together for Berlin that lays this out. Anyone who wishes to collaborate, please contact me.
On 2/25/15 3:00 PM, Rob Stone wrote:
Hi Rick and Dan
I recognize that we need to make sure we are addressing BMP, but I don’t believe it is valid to equate
creation of separate PHY types with market fragmentation. There seems to be support within the group for implementing a PHY (or set of PHYs) which support CL108, 74 and no FEC). I think what we are discussing is how you AN these modes with minimal customer
confusion and maximum flexibility. I see the perceived drawback of creating three PHY types therefore somewhat artificial as it’s just another way of meeting those end goals.
Remember, we have received feedback that there is a desire from the end customers that they want
maximum flexibility to not design in redundant modes for a particular use case, as well as making the solution maximally plug and play interoperable. I would argue that the three PHY solution achieves both these goals. If you want maximum interoperability,
implement all three PHYs. If you want to do a subset, then that’s OK too, and if you do this it will be very clear what FEC modes and channels are and are not supported. I don’t believe this would be the case if we went for single PHY solution, especially
if you want AN to resolve in a single pass.
Thanks
Rob
Hi Dan,
I do concur with your comments.
As a switch manufacturer I believe that fragmenting the market into niche ports defeats the broad
market potential. Directly or indirectly the feedback I always get from end-users is to deliver plug-and-play solutions rather than pick and choose what you specifically need today even though it may become useless when different needs appear next year. Other
system manufactures may get a different feedback and their comments are welcome. I rather move in the direction of a coherent and comprehensive solution.
Thank you,
All,
As we discuss potential market fragmentation for 25GBASE-CR PHYs based upon reach (CR-S, CR-L, etc)...
We should also keep in mind that MMF optics will require RS-FEC.
So, if we choose to create a PHY type that does not support RS-FEC, we are not just dividing it from the relatively small 3-5m copper market. (I assume its small relative to 0-3m market) but we are also dividing it
away from the larger body of 100G devices that would break out into 4x25G devices, and dividing it away from 25G MMF devices which may have a reasonable market size.
There was a comment made in today's adhoc that plug-and-play leads to happy customers. I would take it one step further, plug-and-play leads to more rapid market adoption and conversely, lack of plug-and-play can lead
to substantial market disruption.
Historically, standards that have had trouble getting off the ground due to interoperability problems, lagged and sometimes never gained market acceptance. In some cases, MSAs or other means were used to achieve the
desired customer satisfaction. We could literally "snake-bite" the standard in customer's eyes if we don't have plug-and-play, or at least clearly defined PHYs so that customers don't attempt to gain interoperability where it's not intended.
I believe we should take all of this into account when making our decision on which way to proceed and am willing to collaborate on a presentation to this effect if someone wishes to join me. I don't have a conclusion
at this point, but am leaning toward a single PHY with both FECs and the ability to disable FEC for low-latency applications. This leaning comes from the data presented by Jeff Slavik combined with comments that in a typical ASIC the overall impact of the
PHY gate counts for RS-FEC will be relatively small. I also agree with Eric's idea of remapping priority for low-latency vs cable-reach to ease Auto Negotiation, but would prefer to see it simply CR vs CR w/o FEC. I base this on the idea that only those who
are engineering links would disable FEC.
|