Thread Links Date Links
Thread Prev Thread Next Thread Index Date Prev Date Next Date Index

Re: [802.3BA] XR ad hoc Phone Conference Notice


Frank,

I just got the following info on the interconnect length distributions for RoadRunner (1 PetaFlop HPC), the current #1 on the top500 list.

975         7m cables
1218     10m cables
1929      20m cables
286         30m cables
334         50m cables  
96           100m cables

There are also 407 10 GbE links, all few tens of meters. Switch to switch links have peak of the distribution at 20m.

I don't think that for future systems the distances will grow.

Regards,

Peter



Petar Pepeljugoski
IBM Research
P.O.Box 218 (mail)
1101 Kitchawan Road, Rte. 134 (shipping)
Yorktown Heights, NY 10598

e-mail: petarp@xxxxxxxxxx
phone: (914)-945-3761
fax:        (914)-945-4134


From: Frank Chang <ychang@xxxxxxxxxxx>
To: STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
Date: 07/10/2008 10:31 AM
Subject: Re: [802.3BA] XR ad hoc Phone Conference Notice




Petar;
 
My understanding the ad hoc group led by Alessandro/John is giving enough thoughts to address your concern, for example, at least from my interpretation of John's email.
 
Before the ah hoc group provide the recommendation, lets NOT take it granted the cost/power will be significantly impacted or even doubled. So far I donot see any option under discussion even close to double the power consumption.    
 
thanks
Frank

From: Petar Pepeljugoski [mailto:petarp@xxxxxxxxxx]
Sent: Thursday, July 10, 2008 6:23 AM
To: STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
Subject: Re: [802.3BA] XR ad hoc Phone Conference Notice

Steve,

My statement does not preclude extended reach. It only states that the cost should not be shared among all users. Let the ones who need the longer distances pay for it. I am sure that in many cases the extended distances are result of poor space planning.

The other reason, besides the cost, which becomes even more important these days is the power consumption. I see no reason to double the power consumption for large majority because few needed the extended reach that will require also higher power consumption.

Regards,

Peter

Petar Pepeljugoski
IBM Research
P.O.Box 218 (mail)
1101 Kitchawan Road, Rte. 134 (shipping)
Yorktown Heights, NY 10598

e-mail: petarp@xxxxxxxxxx
phone: (914)-945-3761
fax:        (914)-945-4134
From: "Swanson, Steven E" <SwansonSE@xxxxxxxxxxx>
To: STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
Date: 07/10/2008 08:23 AM
Subject: Re: [802.3BA] XR ad hoc Phone Conference Notice



Petar,
 
I am still puzzled by the comment "very few who need extended reach."
 
I think we have misinterpreted the data; my belief is that 100m does in fact cover a significant percentage of the overall links (there is still some debate over the definition of "significant") but that most datacenters have link lengths that exceed 100m. In our survey of customers, not one said that 100m is adequate. Customers want a solution that covers all their needs - it doesn't help them to have a solution that only covers a percentage of their links even if the number exceeding 100m is less than the number of <100m links.
 
How do we reconcile that with our customers?
 
Steve

From: Petar Pepeljugoski [mailto:petarp@xxxxxxxxxx]
Sent: Wednesday, July 09, 2008 10:51 PM
To: STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
Subject: Re: [802.3BA] XR ad hoc Phone Conference Notice


Frank,

If I interpret correctly, you are saying that all users should amortize the cost of very few who need extended reach.
We need to be careful how we proceed here - we should not repeat the mistakes of the past if we want successful standard.

Regards,

Peter

Petar Pepeljugoski
IBM Research
P.O.Box 218 (mail)
1101 Kitchawan Road, Rte. 134 (shipping)
Yorktown Heights, NY 10598

e-mail: petarp@xxxxxxxxxx
phone: (914)-945-3761
fax:        (914)-945-4134
From: Frank Chang <ychang@xxxxxxxxxxx>
To: STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
Date: 07/09/2008 10:29 PM
Subject: Re: [802.3BA] XR ad hoc Phone Conference Notice



Hi Jeff;

Thanks for your comment. You missed one critical point that there is cost increase from OM3 to OM4. If you take ribbon cable cost in perspective, OM4 option is possibly the largest of the 4 options.

Besides, the use of OM4 requires to tighten TX specs which impact TX yield, so you are actually compromising the primary goal.

Frank
From: Jeff Maki [mailto:jmaki@xxxxxxxxxxx]
Sent: Wednesday, July 09, 2008 7:02 PM
To: STDS-802-3-HSSG@xxxxxxxxxxxxxxxxx
Subject: Re: [802.3BA] XR ad hoc Phone Conference Notice

Dear MMF XR Ad Hoc Committee Members,

I believe our current objective of “at least 100 meters on OM3 MMF” should remain as a primary goal, the baseline.  Support for any form of extended reach should be considered only if it does not compromise this primary goal.  A single PMD for all reach objectives is indeed a good starting premise; however, it should not be paramount.  In the following lists are factors, enhancements, or approaches I would like to put forward as acceptable and not acceptable for obtaining extended reach.

Not Acceptable:
1. Cost increase for the baseline PMD (optic) in order to obtain greater than 100-meter reach
2. EDC on the system/host board in any case
3. CDR on the system/host board as part of the baseline solution
4. EDC in the baseline PMD (optic)
5. CDR in the baseline PMD (optic)

Acceptable:
1. Use of OM4 fiber
2. Process maturity that yields longer reach with no cost increase

In summary, we should not burden the baseline solution with cost increases to meet the needs of an extended-reach solution.

Sincerely,

Jeffery Maki


————————————————
Jeffery J. Maki, Ph.D.
Principal Optical Engineer
Juniper Networks, Inc.
1194 North Mathilda Avenue
Sunnyvale, CA  94089-1206
Voice +1-408-936-8575
FAX +1-408-936-3025
www.juniper.net
jmaki@xxxxxxxxxxx
————————————————