Re: [802.3BA] 40G/100G SMF distinct identity [WAS: RE: [802.3BA] 5 Criteria mod to support 40 G on SMF]
Sanjeev,
I agree that there are many differences between Ethernet
and Fibre Channel protocols and markets. Both protocols have shared
certain aspect at the physical layers and that is where I think it is
helpful to compare and contrast the two.
Any system or organism that grows exponentially by
doubling will reach limitations. I did not try to create a rule
that FC will continue to double indefinitely without increase in cost. I
was trying to report what happened, not predict the
future.
The cost of 8GFC is less than one half of
the cost of 1GFC in 2001 without even considering
inflation. This is mainly due to the lower volumes of 1GFC when
prices were relatively high. 1GFC ports were shipped in 16 or 32 port
configurations while 8GFC products are now shipping with 768 port
backbone switches to a much larger market in higher volumes.
Ethernet does typically span larger topologies than FC, but
this is not a limit of the Fibre Channel architecture. Fibre Channel has
been successfully mapped to SONET and ATM (Fibre Channel Backbone - FC-BB),
Internet Protocol (FC-BB-2 and RFC 3821), Generic Framing Protocol
(FC-BB-3) and Pseudo Wire (FC-BB-4). Fibre Channel is even now being
mapped to Ethernet in FC-BB-5. As long as response times (round trip
latency) are below Error Detect Time Out Values of 2 seconds, most
applications will work fine on earth. The press and pundits often don't
realize that the Fibre Channel protocols are not limited in distance,
only in the demands that customers place upon it.
I'm not suggesting that Ethernet should have gone to 2G,
but I bet users would have adopted it if it was backward compatible.
FC and Ethernet address different markets and needs as you
say. I agree that 10GE was not designed for laptops or even PCs or
servers. If 10GE was designed for servers, it would have been designed for
a shorter distance than 300 meters. The 300 meter requirement drove the
expensive lasers, tight jitter budgets and difficult testing
requirements. I have quoted Finisar's presentation on this ( http://www.t11.org/t11/docreg.nsf/ufile/06-036v0 )
a number of times and most transceiver companies agree that 10GE is
unnecessarily difficult to manufacture. I have heard that the 10GE
transceiver costs could be reduced by 40% if the distance was decreased to 100
meters.
8GFC PHY was designed to be lower cost and backward
compatible to earlier speeds. 8GFC links only spans 150 meters on OM3
fiber (still longer than 40/100GE) and has better allocation of jitter
budgets than 10GE where too much jitter budget was allocated to fiber at the
detriment of the electronics. Proper definition of 8GFC for a
different market between servers and storage has kept the cost in 8GFC
physical layer costs considerably lower than 10GE.
I know many users would like to move to 10GE for their
server interconnects since they need to use multiple 1GE links now. With
multicore processors and blade server chassis, you'd think that more than
100,000 10GE ports would have shipped on the 7+ million servers shipped in 2007.
If some server vendors could comment on why there is low
adoption of 10GE to the server, I'd like to know why if it isn't
cost.
Agree that cost plays a big role. However, if 2x increments keep the cost
same (or lower)---as you mentioned---then why is 8GFC more expansive than 1G
(rule of transitive; 2G<=1G, 4G<=2G, 8G<=4G)?
I think Ethernet networks span larger topologies than FC (which is
constrained and limited to and by computing/storage topologies). One can't
compare the two directly.
I think 10GE didn't take off in the HBA not because of cost of Ethernet,
but the cost of filling up the 10GE port and perhaps, not being able to fill
that pipe. Since that did not happen the volime did not happen and when that did
not happen cost did not come down as fast.
Just my opinion, I don't think 10GE was envisioned to be in laptop machines
and not sure if there was 2G Ethernet even designed to be compatible with 10/100
how would have added any value to these machines ---which are ading to the
volume---with 1GE link?
Also, if I recall correctly 1G FC was very expansive until 1G Ethernet took
off.
My $0.0.
Thanks,
Sanjeev
On 3/1/08, Scott Kipp
<skipp@brocade.com> wrote:
Mikael,
You
said:
Being a data communications guy, I don't understand why FC seems
to be
increasing by factor 2 each generation, instead of factor 4 or
10.
Representatives for Brocade (in my mind primarily FC vendor)
says
customers want small increases in link speed, which is not
my
experience.
I'm the Brocade representative, so I'll attempt to
answer the question
regarding why FC continues to grow by factors of 2
instead of 4 or 10.
I never said customers want small increases in link
speed, but I'll say
that they want it if its no additional cost or even a
reduction in cost.
Fibre Channel has always attempted to be the low
cost leader in fiber
optic bandwidth. Fibre Channel standardized the first
1 gigabit links
based on VCSELs that were very
successful. Ethernet saw the advantage
of this and soon copied
large parts of the specification and developed
highly successful 1 Gig
fiber optic links using the same technologies.
Most 1 Gig FC and 1 Gig
Ethernet fiber optic transceivers work on either
protocol.
Many
small improvements in transceivers lead to big
differences. Fibre
Channel saw that they could soon advance to 2
Gig and be backward
compatible with 1 gig for basically no additional cost
or even less cost
when new processes were developed. Customers
liked doubling the speed
for no additional cost and it still autonegotiated
down to 1 Gig if
needed. The same shift was seen for 4 Gig when almost all
2 Gig gear was
obsoleted by the new gear. The speed doubling
usually coincided with
new ASIC development and gained other capabilities
with the speed
increase. Speed sells and there were no interoperability
problems
between the speeds.
You see much smaller speed jumps in
microprocessor speeds and disk drive
capacities. Does anyone
complain when a processor increases from 2.1
GHz to 2.4 GHz as long as it
works?
Fibre Channel is currently going through the transition to 8 Gig
and
there is currently a cost premium over 4 Gig solutions, so
the
transition to 8 Gig might not be as fast as from 1GFC to 2GFC or 2GFC
to
4GFC, but the cost difference is diminishing and customers are
adopting
8GFC rapidly in the first quarter of released
products.
Fibre Channel did make the mistake of following Ethernet into
the 10 Gig
space. 10GFC has seen minimal adoption because of the
high cost premium
over even 8GFC. No servers have adopted 10GFC and FC HBAs
are not
expected to move to 10GFC but to 8GFC and then
16GFC. Ethernet has seen
a very similar lack of adoption of 10
Gig because it isn't backward
compatible with 1GE. GigE has been
very successful because of its
backward compatibility with
10/100. In 2007, over 1 million 10GE ports
were shipped compared
to over 100M Gig E ports. Six years after the 10
Gig standard
was complete, less than 1% of Ethernet ports are 10 Gig.
I was at OFC
this week and the high cost of 10 Gig was a big topic of
debate. We're
still in a chicken and egg scenario with 10 Gig. Volumes
could
drive the cost of 10 Gig down, but we haven't seen it
yet. Users
would like 10 Gig on servers, but mainly buy 1
Gig. I've heard that
only about 100,000 servers connected with
10GE last year.
I've tried to make the point that users don't mind
minimal speed
increases for minimal or no cost increases as long as they
are backward
compatible. Fibre Channel is attempting to continue
that tradition with
8GFC and 16GFC. Just in time technologies
that have been optimized
instead of before there time technologies that are
not cost-effective.
My 10 cents,
Scott Kipp
-----Original
Message-----
From: Mikael Abrahamsson [mailto:swmike@SWM.PP.SE]
Sent: Saturday,
February 16, 2008 12:35 PM
To: STDS-802-3-HSSG@LISTSERV.IEEE.ORG
Subject:
Re: [802.3BA] 40G/100G SMF distinct identity [WAS: RE:
[802.3BA] 5 Criteria
mod to support 40 G on SMF]
On Sat, 16 Feb 2008, Alessandro Barbieri
(abarbier) wrote:
> This group has identified a minimum set of
applications that is
> sufficient to justify the DI of 40G and 100G,
that should be enough.
> We can't pretend 40G will not find its way into
the aggregation. Much
> like in 10-12 years 100G will find its way into
the server market.
Sorry for my previous email where I used "price"
instead of "cost". I'll
try to improve.
So, backing up a bit, I'm
trying to see this from a system perspective,
ie in the equipment where
these technologies are going to be used.
I have some thinking I would
like some vendors to give feedback on,
which I think is important in this
decision as well.
Currently, a lot of the current highend platforms use
40 gigabit/s
backplane connectivity. Would they get 40GE support on the
current
chassis/fabric generation in 2010?
By 2010, my guess is that
highend platforms will have backplane
connectivity somewhere in the 100,
120, 160 or 200 gigabit/s speeds.
If 100, this would enable single port
100GE or two port 40GE blades.
If 120, this would enable single port 100GE
or three port 40GE blades.
If 160, this would enable single port 100GE or
four port 40GE blades.
If 200, this would enable dual port 100GE or five
port 40GE blades.
Would the 40GE blades have pluggable optics that
would support all
proposed reaches? If there is a push for 10km SMF reach
for 40GE, why
not 40 or 80km reach there as well? With 3 or 4 port 40GE
blades one
would have capacity to on a single blade create rings and then
use 40GE
as aggregation technology immediately. Then it makes sense to have
10
and 40km SMF reaches (and seeing that 10GBASE-ZR is actually used
quite
a bit, wouldn't 80km as well be of interest to standardize by the
same
logic that vendors will standardize 40GE-10km SMF if IEEE
doesn't)?
I also think that any proponent for 40GE needs to explain
what the next
step after 100GE should be, as 40GE creates a precedent for
250GE or
400GE as next step if the logic is to be intact.
Being a
data communications guy, I don't understand why FC seems to be
increasing
by factor 2 each generation, instead of factor 4 or 10.
Representatives for
Brocade (in my mind primarily FC vendor) says
customers want small
increases in link speed, which is not my
experience.
In the ISP world
people often want to use their equipment for a long
time and write them off
in 3-6 years, as they are usually quite big and
big in capital spending,
thus the need for factor 4 or 10 in speed
increase over the previous
generation.
I also feel that 40GE was pushed as server technology
needed in 2012, in
order to make 40GE worthwile it needs to be here before
100GE, not
after, or show big cost savings so as to be more economical per
bit
compared to 100GE. Being compatible with existing equipment
(new
linecard for existing
platforms) might cause it to be more
economical for end users from a
total expenditure perspective.
Being
an ISP guy, I also feel that any push for 40GE needs to have
broad
switch/router vendor support, because if the core routing
platforms
doesn't support 40GE then there is a big hurdle in how to connect
the
aggregation to the core for any larger sized ISP who has a
WAN.
But then again, if 40GE 10km SMF uses a lot of 10GE technology and
has a
2.5x cost increase over 10GE, and 100GE has 10x cost increase, and
40GE
can be brought quickly to market and 100GE will take more r&d and
is
harder to realise, then 40GE SMF might still have a place.
But I
do have to say again: 40GE cannot delay 100GE effort, it's needed
soon of
not NOW. 40GE is bandaid for ISPs, the solution is 100GE if not
higher
speeds, or we risk stalling the growth of the Internet.
--
Mikael
Abrahamsson email: swmike@swm.pp.se