| Thread Links | Date Links | ||||
|---|---|---|---|---|---|
| Thread Prev | Thread Next | Thread Index | Date Prev | Date Next | Date Index |
|
"the SERDES designs are complete and we are just cleaning up the COM code (Afe and SNDR changes are just fixes) and taking out some of the margin" I beg to differ – (a) SERDES designs are still ramping up, future designs that comply to the standard will be more power optimized, and possibly save power
when FEC is enabled and provide more margin. (b) The discussions led by Joel proved that there is very little margin to be gained from COM parameters assuming the AFE parameters are not improved (which may cause higher power consumption). We were not even
shown any data to convince that no-FEC on 3 m is feasible with worst-case instances of current products. Having said that, the numbers shown yesterday, and especially with Scott's addition below, make "power saving" a very unconvincing argument. We could continue
discussing power as a strawman to prove that no-FEC is bad for energy saving. So I suggest that we stop even mentioning power. I was convinced from the discussion yesterday that latency is a metric that the market looks at, and therefore is important. Let's concentrate on that. </Adee> From: Jonathan King [mailto:jonathan.king@xxxxxxxxxxx]
Thanks Tom, good point, power is not the justification, the impact of latency is. From: Tom Palkert [mailto:tpalkert@xxxxxxxx]
Jonathan, One of the points that I tried to make during the discussion was that we can easily achieve 3m no FEC with no changes to the existing 25G SERDES. This means
that although in a perfect world you might be able to take advantage of FEC to design a lower power SERDES, reality is that the SERDES designs are complete and we are just cleaning up the COM code (Afe and SNDR changes are just fixes) and taking out some of
the margin. I think we need to move the discussion away from power and over to latency. Does anyone have an idea on the impact of added latency? I looked up low latency
ethernet switches and found specs ranging from 190ns to 350ns. Adding 82ns of latency (x2 for in and out) looks like a doubling of latency to me. Tom From: Jonathan King [mailto:jonathan.king@xxxxxxxxxxx]
Hi Brad, I don’t disagree with any of your points 1 to 4 and…. if you remove the FEC gain, the SNR has to be made up in other ways – FEC is an efficient way of buying SNR. Agree, using FEC means adding the power for implementing the code/decode
circuitry, but it allows power reduction in other parts of the link. Net result, less power burn for ‘with FEC’ links, leaving
more power for the things customers really care about etc… best wishes jonathan From: Brad Booth [mailto:bbooth@xxxxxxxx]
I think there a few factors that have to be considered, but unfortunately there was so much discussion about the validity of the data, that the key points were truly missed. 1) Datacenters have a power constraint. There is not unlimited power. 2) Even if the power is 25 mW for FEC, FEC has to be used on both ends of the link; therefore, the power number is double. 3) The power supplied is about 5-6x the power used due to losses inherent in electronics, transformers, etc. 4) Power is amplified by the scale. What seems small at a single point in the architecture is amplified when you move to hyper-scale. At 1M servers, that's 50 kW for FEC which is equivalent to about 4-5 server racks. Finally, power spent on FEC (if not required) takes power away from other things that customers really care about like VMs, security, etc. Just some extra food for thought. Thanks, On Tue, Jul 14, 2015 at 4:26 PM, Scott Kipp <skipp@xxxxxxxxxxx> wrote: 802.3by, Yesterday, there was considerable debate about power consumption due to the RS-FEC used in 25GbE. In goergen_3by_02_0715.pdf, the power consumption for
RS-FEC was approximated to be 300mW. I did a quick search and found these estimates: http://www.ieee802.org/3/bm/public/sep12/wang_01_0912_optx.pdf
= 45mW on slide 3 http://www.ieee802.org/3/bj/public/mar12/gustlin_01_0312.pdf
= 90 mW on page 6 http://www.ieee802.org/3/bs/public/adhoc/logic/oct21_14/wangz_01_1014_logic.pdf,
Wangz concludes: In practice, either KR4 or KP4 FEC is easy to implement and not much power-consuming. A conservative estimate is 100mW for 100GBASE-KR4. 25GBASE-KR4 will consume 1/4 of this power or 25mW based on a one of the four lanes. I propose that 25mW is a practical power consumption for RS-FEC. Let’s calculate the cost of the power consumption for the server over a 3-year lifespan.
Another correction that I wanted to make is to the estimate is the cost of electricity. Hyperscale data centers that consume megawatts of power buy electricity
at wholesale rates. Many are located near electric power plants so that Google pays about $0.04/kWh according to this article:
http://www.oregonlive.com/silicon-forest/index.ssf/2013/09/google_reaches_new_data_center.html The wholesale electricity industry is regulated and current pricing is available for this site and shows electricity ranges in 2015 has varied from $0.036/MWh
to $0.065/MWh. Read more here: http://www.eia.gov/electricity/wholesale/ I used a conservative $0.05/kWh. My estimate is that the cost of RS-FEC per server is about $0.03 over the life of the server. Three cents is basically in the noise for our purposes and
should not be a fundamental driver for 3m cabling without FEC. Kind regards, Scott Kipp |