RE: [10GBASE-T] latency
Michael
I am not sure there is market data that addresses the issue directly.
-- -We could probably size the cluster market as it exists today in the
R&D community. My guess is that half to two thirds of all 10
GbE fiber ports today are sold into this application space. Some market
research house has probably sized the market.
---We could probably size the blade server market as it exists in the
data center today. Last time I checked, one of the market research houses
estimated that 1 to 2% of servers were blade servers and the forecast was
that this would grow to double digit percentages over the next few
years
One thing that is clear is the trend to faster, cheaper, simpler
computing and the reduction in the number of protocols our customer run
in their networks either on the enterprise side or the network operator
side. I think the trend to clusters is perfectly complementary to
Ethernet and some of my first 1000BASE-T customers in 1999 were asking
hard questions about latency in the NICs and through the switches,
I think the fundamental issue before us is how important low latency
might be to enable these markets that are just emerging, for example
clusters in enterprise commercial environments. That the BMP would
be broader I think we can agree. How much broader would involve some
work. The problem whose answer might be more tangible and less open
to debate is what it would cost in terms of power and complexity to
address the market potential.
Bruce
At 06:00 PM 2/23/2004 -0800, Michael J. Bennett wrote:
Folks,
I agree that the discussion of
the possibility of lowering latency does not imply impact on the original
BMP brought forth by the study group. That is to say we ve no less
market potential than was presented in the 5 criteria if we do nothing to
lower latency. On the other hand the potential could be greater if
the latency can be lowered. Joel brought up a good point that when
end users go looking for equipment, latency is often used by vendors as a
lever to edge out their competition. In the clustering domain,
Ethernet is not the only choice. Geoff pointed out that Ethernet
was designed for loosely coupled systems. Alternative technologies
for clustering are more tightly coupled and have significantly less
latency than Ethernet. Furthermore, Network Interface Cards (NIC) for
these systems are very inexpensive compared to 10 Gigabit Ethernet
NICs. The folks I know who build clusters are willing to pay for
the switch ports but they expect the NIC to be close to very low cost,
meaning it doesn t cost much more to get a motherboard with the NIC
built-in than without it (similar to 1 GbE now). So this leads to a few
questions: is the potential market that could be gained by lowering the
latency so significant that it s worth doing? Does anyone have
market data on this? Even if the market data suggests it would be a
good thing to do, would that justify the trade-offs to make it
happen? Should we just wait until after the Data Canter Ethernet
CFI before we even discuss it further in 10GBASE-T?
Mike
-----Original Message-----
From: owner-stds-802-3-10gbt@majordomo.ieee.org
[mailto:owner-stds-802-3-10gbt@majordomo.ieee.org]
On Behalf Of Jonathan Thatcher
Sent: Monday, February 23, 2004 1:06 PM
To: 'Geoff Thompson'
Cc: 'stds-802-3-10gbt'
Subject: RE: [10GBASE-T] latency
Geoff,
Me-thinks you are being a little harsh
here. I do not believe there is any implication that BMP is not met
unless low latency applications are supported. The implication is simply
that the BMP is B'er if latency is less.
Now, if the cost (C) of achieving B'er
BMC is out of balance, then it certainly should not be pursued. But if
additional C has the potential to B'en BMC then it is worthy of
discussion.
If you wish to take the position that
no B'er BMC is worth additional C, that is fine.
I, for one, would like to hear other
opinions. And, without trying to be harsh myself, I think you should
also.
jonathan
p.s. your point about 10GBASE-CX4 is
valid, within its distance limits. Of course, one of the major latency
aspects of low latency is distance divided by the speed of light. While
10's of km for low latency is absurd, 10's of meters may not be.
-----Original
Message-----
From: owner-stds-802-3-10gbt@majordomo.ieee.org
[mailto:owner-stds-802-3-10gbt@majordomo.ieee.org]On
Behalf Of Geoff Thompson
Sent: Monday, February 23, 2004 12:33 PM
To: Bruce Tolley
Cc: jonathan.thatcher@ieee.org; 'stds-802-3-10gbt'
Subject: RE: [10GBASE-T] latency
- All-
- I am having a little trouble with some of the assumptions of this
entire thread.
- If you have a special need for low latency at 10 Gig for cluster
computing then we already have a solution, 10GBASE-CX4.
- I would say that it is not particularly appropriate at this time to
even presume that we have legitimate requirements for Ethernet where the
coding delay in a PHY is significant for a speed for which we are not
doing CSMA/CD (whose round-trip time was the traditional driver of the
low latency requirements).
- There has been a lot of talk about fiddling the latency to meet the
requirements for a topic for which there has not even been a Call For
Interest!
- This leads me to believe that all of the talk in getting this study
group approved, in which there were claims of more than adequate
"Broad Market Potential" were not quite as true as was
depicted. There was no mention of the BMP of 10GBASE-T being latency
dependent during the PAR proposals.
- Does all this discussion mean that you wish to revisit the basis on
which the 10GBASE-T was granted?
- Ethernet is at its MOST BASIC level is designed to be a connection
for loosely coupled systems. This has been the source of its success. I
am perfectly willing to explore new avenues for Ethernet. I am not
willing to cross out multiple pieces of its basic nature just to chase
every corner of the data transfer market. It is NOT just the name
"Ethernet" that is the basis of its success.
- The most significant things that 10GBASE-T needs for its success
is:
- Silicon
processing that can tolerate its speed and complexity requirements
- A
large enough market for a general purpose "Ethernet"
interconnect at 10G to
- pay
for the development and
- drive
the part cost down
- Geoff
- At 11:16 AM 2/23/2004 -0800, Bruce Tolley wrote:
- Jonathan
- Thanks for the summary
- I would argue that early 10GBASE_T switching products should be sold
to early adopters at National Labs and other R&D sites building
clusters. We need this community to come to the TF and state its latency
requirement in the 2006 timeframe and determine the tradeoffs.
- Bruce
- At 10:54 AM 2/23/2004 -0800, Jonathan Thatcher wrote:
- There have been numerous interesting and correct comments made. A
subset of these apply only in certain contexts. To that end, I will
attempt to add some context.
-
- There is little question that lower latency increases the market
potential. There is little question that lower prices (read that less
complexity), and earlier time to market also increases the potential
market. The problem is that these fight against each other, and the
optimization point is not clear.
-
- I presume that there are two principal application spaces for
10GBASE-T in the near term: data center and enterprise (home and school
will probably have to wait a couple of years :-). If you want a strict
boundary between these two spaces, I can't provide it. So we will have to
deal with some ambiguity. In the enterprise, it is difficult to argue
that low latency is as critical as low price. The exception to this would
be low latency applications that want to be set up as a "grid
computer," which I will lump into the "data center"
bucket.
-
- The data center, on the other hand, has instances where both low
latency is required (clustered computing) and higher latency is
acceptable (most file serving). From a parallel computing perspective,
there are classes of problems (applications) that range from low latency
NUMA to those that are "embarrassingly parallel (e.g.
http://setiathome.ssl.berkeley.edu/)."
-
- From the perspective of the upcoming "Data Center Ethernet"
(may not be the best name) call for interest, the intent is to explore
those means that can be used to decrease latency in Ethernet networks. If
one is to presume that this should be a key application space for
10GBASE-T, then it would be interesting to understand the trade-off
between latency and complexity. It may be the case, that even under the
most complex scenario, that 10GBASE-T latency is simply insufficient for
entire classes of low latency
applications.
-
- So, the question remains, what does the complexity vs latency curve
look like? I expect that it is something like the left side of a bathtub
curve (vertical axis is latency, horizontal axis is complexity). What is
the inflection point? What is the slope of the falling portion of the
curve? What is the asymptote?
-
- jonathan
-
-
- Bruce Tolley
- Senior Manager, Emerging Technologies
- Gigabit Systems Business Unit
- Cisco Systems
- 170 West Tasman Drive
- MS SJ B2
- San Jose, CA 95134-1706
- internet: btolley@cisco.com
- ip phone: 408-526-4534
- "Don't put your hiking boots in the oven unless you plan on
eating them."
- Colin Fletcher, The Complete Walker
Bruce Tolley
Senior Manager, Emerging Technologies
Gigabit Systems Business Unit
Cisco Systems
170 West Tasman Drive
MS SJ B2
San Jose, CA 95134-1706
internet: btolley@cisco.com
ip phone: 408-526-4534
"Don't put your hiking boots in the oven unless you plan on
eating them."
Colin Fletcher, The Complete Walker