Thread Links Date Links
Thread Prev Thread Next Thread Index Date Prev Date Next Date Index

Re: [STDS-802-3-25G] Power Consumption of of 25G RS-FEC



"the SERDES designs are complete and we are just cleaning up the COM code (Afe and SNDR changes are just fixes) and taking out some of the margin"

 

I beg to differ – (a) SERDES designs are still ramping up, future designs that comply to the standard will be more power optimized, and possibly save power when FEC is enabled and provide more margin. (b) The discussions led by Joel proved that there is very little margin to be gained from COM parameters assuming the AFE parameters are not improved (which may cause higher power consumption). We were not even shown any data to convince that no-FEC on 3 m is feasible with worst-case instances of current products.

 

Having said that, the numbers shown yesterday, and especially with Scott's addition below, make "power saving" a very unconvincing argument. We could continue discussing power as a strawman to prove that no-FEC is bad for energy saving. So I suggest that we stop even mentioning power.

 

I was convinced from the discussion yesterday that latency is a metric that the market looks at, and therefore is important. Let's concentrate on that.

 

 

</Adee>

 

From: Jonathan King [mailto:jonathan.king@xxxxxxxxxxx]
Sent: Wednesday, July 15, 2015 9:29 AM
To: STDS-802-3-25G@xxxxxxxxxxxxxxxxx
Subject: Re: [STDS-802-3-25G] Power Consumption of of 25G RS-FEC

 

Thanks Tom, good point, power is not the justification, the impact of latency is.

 

From: Tom Palkert [mailto:tpalkert@xxxxxxxx]
Sent: Wednesday, July 15, 2015 12:21 PM
To: Jonathan King; STDS-802-3-25G@xxxxxxxxxxxxxxxxx
Subject: RE: [STDS-802-3-25G] Power Consumption of of 25G RS-FEC

 

Jonathan,

One of the points that I tried to make during the discussion was that we can easily achieve 3m no FEC with no changes to the existing 25G SERDES. This means that although in a perfect world you might be able to take advantage of FEC to design a lower power SERDES, reality is that the SERDES designs are complete and we are just cleaning up the COM code (Afe and SNDR changes are just fixes) and taking out some of the margin.

 

I think we need to move the discussion away from power and over to latency. Does anyone have an idea on the impact of added latency? I looked up low latency ethernet switches and found specs ranging from 190ns to 350ns. Adding 82ns of latency (x2 for in and out) looks like a doubling of latency to me.

 

Tom

 

 

From: Jonathan King [mailto:jonathan.king@xxxxxxxxxxx]
Sent: Tuesday, July 14, 2015 10:28 PM
To: STDS-802-3-25G@xxxxxxxxxxxxxxxxx
Subject: Re: [STDS-802-3-25G] Power Consumption of of 25G RS-FEC

 

Hi Brad, I don’t disagree with any of your points 1 to 4

and….

if you remove the FEC gain, the SNR has to be made up in other ways – FEC is an efficient way of buying SNR.  Agree, using FEC means adding the power for implementing the code/decode circuitry, but it allows power reduction in other parts of the link.  Net result, less power burn for ‘with FEC’ links, leaving more power for the things customers really care about etc…

 

best wishes

jonathan

 

From: Brad Booth [mailto:bbooth@xxxxxxxx]
Sent: Tuesday, July 14, 2015 6:03 PM
To: STDS-802-3-25G@xxxxxxxxxxxxxxxxx
Subject: Re: [STDS-802-3-25G] Power Consumption of of 25G RS-FEC

 

I think there a few factors that have to be considered, but unfortunately there was so much discussion about the validity of the data, that the key points were truly missed.

 

1) Datacenters have a power constraint. There is not unlimited power.

2) Even if the power is 25 mW for FEC, FEC has to be used on both ends of the link; therefore, the power number is double.

3) The power supplied is about 5-6x the power used due to losses inherent in electronics, transformers, etc.  

4) Power is amplified by the scale. What seems small at a single point in the architecture is amplified when you move to hyper-scale. At 1M servers, that's 50 kW for FEC which is equivalent to about 4-5 server racks.

 

Finally, power spent on FEC (if not required) takes power away from other things that customers really care about like VMs, security, etc.

 

Just some extra food for thought.

 

Thanks,
Brad

 

On Tue, Jul 14, 2015 at 4:26 PM, Scott Kipp <skipp@xxxxxxxxxxx> wrote:

802.3by,

 

Yesterday, there was considerable debate about power consumption due to the RS-FEC used in 25GbE.  In goergen_3by_02_0715.pdf, the power consumption for RS-FEC was approximated to be 300mW.  I did a quick search and found these estimates:

http://www.ieee802.org/3/bm/public/sep12/wang_01_0912_optx.pdf = 45mW on slide 3

http://www.ieee802.org/3/bj/public/mar12/gustlin_01_0312.pdf = 90 mW on page 6

http://www.ieee802.org/3/bs/public/adhoc/logic/oct21_14/wangz_01_1014_logic.pdf, Wangz concludes: In practice, either KR4 or KP4 FEC is easy to implement and not much power-consuming.

 

A conservative estimate is 100mW for 100GBASE-KR4.  25GBASE-KR4 will consume 1/4 of this power or 25mW based on a one of the four lanes.

 

I propose that 25mW is a practical power consumption for RS-FEC.  Let’s calculate the cost of the power consumption for the server over a 3-year lifespan.

 

 

Goergen_3by_02_0715

My proposal

Power of RS-FEC (mW)

200 (value used in calculation)

25

Power Consumption over 3 year lifespan of server (kWh)

5.25

0.66

Cost of kWh ($/kWh)

0.11

0.05

Power cost/server for RS-FEC

$0.58

$0.03

 

Another correction that I wanted to make is to the estimate is the cost of electricity.  Hyperscale data centers that consume megawatts of power buy electricity at wholesale rates.  Many are located near electric power plants so that Google pays about $0.04/kWh according to this article:

http://www.oregonlive.com/silicon-forest/index.ssf/2013/09/google_reaches_new_data_center.html

 

The wholesale electricity industry is regulated and current pricing is available for this site and shows electricity ranges in 2015 has varied from $0.036/MWh to $0.065/MWh.  Read more here:

http://www.eia.gov/electricity/wholesale/

 

I used a conservative $0.05/kWh.

 

My estimate is that the cost of RS-FEC per server is about $0.03 over the life of the server.  Three cents is basically in the noise for our purposes and should not be a fundamental driver for 3m cabling without FEC.

 

Kind regards,

Scott Kipp