RE: [10GBASE-T] latency

["Michael J. Bennett" <mjbennett@lbl.gov>]

Folks,

 

I agree that the discussion of the possibility of lowering latency does not imply impact on the original BMP brought forth by the study group.  That is to say we’ve no less market potential than was presented in the 5 criteria if we do nothing to lower latency.  On the other hand the potential could be greater if the latency can be lowered.  Joel brought up a good point that when end users go looking for equipment, latency is often used by vendors as a lever to edge out their competition.  In the clustering domain, Ethernet is not the only choice.  Geoff pointed out that Ethernet was designed for loosely coupled systems.  Alternative technologies for clustering are more tightly coupled and have significantly less latency than Ethernet. Furthermore, Network Interface Cards (NIC) for these systems are very inexpensive compared to 10 Gigabit Ethernet NICs.  The folks I know who build clusters are willing to pay for the switch ports but they expect the NIC to be close to very low cost, meaning it doesn’t cost much more to get a motherboard with the NIC built-in than without it (similar to 1 GbE now). So this leads to a few questions: is the potential market that could be gained by lowering the latency so significant that it’s worth doing?  Does anyone have market data on this?  Even if the market data suggests it would be a good thing to do, would that justify the trade-offs to make it happen?  Should we just wait until after the Data Canter Ethernet CFI before we even discuss it further in 10GBASE-T?   

 

 

Mike

 

 

-----Original Message-----
From: owner-stds-802-3-10gbt@majordomo.ieee.org [mailto:owner-stds-802-3-10gbt@majordomo.ieee.org] On Behalf Of Jonathan Thatcher
Sent: Monday, February 23, 2004 1:06 PM
To: 'Geoff Thompson'
Cc: 'stds-802-3-10gbt'
Subject: RE: [10GBASE-T] latency

 

Geoff,

 

Me-thinks you are being a little harsh here. I do not believe there is any implication that BMP is not met unless low latency applications are supported. The implication is simply that the BMP is B'er if latency is less.

 

Now, if the cost (C) of achieving B'er BMC is out of balance, then it certainly should not be pursued. But if additional C has the potential to B'en BMC then it is worthy of discussion.

 

If you wish to take the position that no B'er BMC is worth additional C, that is fine.

 

I, for one, would like to hear other opinions. And, without trying to be harsh myself, I think you should also.

 

jonathan

 

p.s. your point about 10GBASE-CX4 is valid, within its distance limits. Of course, one of the major latency aspects of low latency is distance divided by the speed of light. While 10's of km for low latency is absurd, 10's of meters may not be.

 

 

 

 -----Original Message-----
From: owner-stds-802-3-10gbt@majordomo.ieee.org [mailto:owner-stds-802-3-10gbt@majordomo.ieee.org]On Behalf Of Geoff Thompson
Sent: Monday, February 23, 2004 12:33 PM
To: Bruce Tolley
Cc: jonathan.thatcher@ieee.org; 'stds-802-3-10gbt'
Subject: RE: [10GBASE-T] latency

All-

I am having a little trouble with some of the assumptions of this entire thread.

If you have a special need for low latency at 10 Gig for cluster computing then we already have a solution, 10GBASE-CX4.

I would say that it is not particularly appropriate at this time to even presume that we have legitimate requirements for Ethernet where the coding delay in a PHY is significant for a speed for which we are not doing CSMA/CD (whose round-trip time was the traditional driver of the low latency requirements).

There has been a lot of talk about fiddling the latency to meet the requirements for a topic for which there has not even been a Call For Interest!

This leads me to believe that all of the talk in getting this study group approved, in which there were claims of more than adequate "Broad Market Potential" were not quite as true as was depicted. There was no mention of the BMP of 10GBASE-T being latency dependent during the PAR proposals.

Does all this discussion mean that you wish to revisit the basis on which the 10GBASE-T was granted?

Ethernet is at its MOST BASIC level is designed to be a connection for loosely coupled systems. This has been the source of its success. I am perfectly willing to explore new avenues for Ethernet. I am not willing to cross out multiple pieces of its basic nature just to chase every corner of the data transfer market. It is NOT just the name "Ethernet" that is the basis of its success.

The most significant things that 10GBASE-T needs for its success is:
        Silicon processing that can tolerate its speed and complexity requirements
        A large enough market for a general purpose "Ethernet" interconnect at 10G to
                pay for the development and
                drive the part cost down

Geoff

At 11:16 AM 2/23/2004 -0800, Bruce Tolley wrote:

Jonathan

Thanks for the summary

I would argue that early 10GBASE_T switching products should be sold to early adopters at National Labs and other R&D sites building clusters. We need this community to come to the TF and state its latency requirement in the 2006 timeframe and determine the tradeoffs.


Bruce

At 10:54 AM 2/23/2004 -0800, Jonathan Thatcher wrote:

There have been numerous interesting and correct comments made. A subset of these apply only in certain contexts. To that end, I will attempt to add some context.
 
There is little question that lower latency increases the market potential. There is little question that lower prices (read that less complexity), and earlier time to market also increases the potential market. The problem is that these fight against each other, and the optimization point is not clear.
 
I presume that there are two principal application spaces for 10GBASE-T in the near term: data center and enterprise (home and school will probably have to wait a couple of years :-). If you want a strict boundary between these two spaces, I can't provide it. So we will have to deal with some ambiguity. In the enterprise, it is difficult to argue that low latency is as critical as low price. The exception to this would be low latency applications that want to be set up as a "grid computer," which I will lump into the "data center" bucket.
 
The data center, on the other hand, has instances where both low latency is required (clustered computing) and higher latency is acceptable (most file serving). From a parallel computing perspective, there are classes of problems (applications) that range from low latency NUMA to those that are "embarrassingly parallel (e.g. http://setiathome.ssl.berkeley.edu/)."
 
From the perspective of the upcoming "Data Center Ethernet" (may not be the best name) call for interest, the intent is to explore those means that can be used to decrease latency in Ethernet networks. If one is to presume that this should be a key application space for 10GBASE-T, then it would be interesting to understand the trade-off between latency and complexity. It may be the case, that even under the most complex scenario, that 10GBASE-T latency is simply insufficient for entire classes of low latency applications.
 
So, the question remains, what does the complexity vs latency curve look like? I expect that it is something like the left side of a bathtub curve (vertical axis is latency, horizontal axis is complexity). What is the inflection point? What is the slope of the falling portion of the curve? What is the asymptote?
 
jonathan
 
 

Bruce Tolley
Senior Manager, Emerging Technologies
Gigabit Systems Business Unit
Cisco Systems
170 West Tasman Drive
MS SJ B2
San Jose, CA 95134-1706
internet: btolley@cisco.com
ip phone: 408-526-4534

"Don't put your hiking boots in the oven unless you plan on eating them."

Colin Fletcher, The Complete Walker