Re: [HSSG] The List
Paul,
I admire your willingness to take on this difficult task
and your efforts to get the root of the matter. You're right about cost
being an uncomfortable subject, especially in the margin-challenged module
business. But I expect anyone involved with shipping both SFP's and
parallel modules in volume will agree that the SFP platform is the most
cost-effective one to transceive a given number of Gb/s. Just take the
cost of a 4G SFP and ask if a 4Gb/s/ch QSFP (or POP4) can be made for
less than 4X that cost. The same will be true at 10Gb/s/ch, due to very
high volumes in 8GFC. Only with comparable volumes and quite a few years
of technology maturation will parallel modules be able to approach or even
surpass the cost-effectiveness of the SFP platform.
The seemingly inverted TOSA cost (a 1x4 vs 4 singlets) does
NOT apply to the Tx/Rx ICs. A quad or a 12-element driver or receiver IC
utilizes essentially the same process/package/test technology as a singlet, with
less handling and labor and chip area. Thus a NX array IC is
less than N times that of a singlet IC - unless there is a large
disparity in volume/demand/availability. Singlet and array TOSA/ROSA
packages are quite different. You can find dozens of contract
manufacturers throughout the far east that perform high-quality TO-can
packaging for extremely low cost. The techniques are known, equipment is
prelevant, barrier to entry is low. Not yet so for array packaging.
Equipment is custom. CM's with know-how are few and far between; ones with
experience are even fewer. For QSFP, there is the additional task of
mounting the VCSEL and PIN chips with 1's of um registration accuracy (to align
with the same fiber ribbon). Alignment has more degrees of freedom
than for a serial TOSA, which must be dealt with. All this in
volumes that are on the order of 1% (probably far less, actually) than that of
TO-cans.
The above is not meant to paint a dismal picture for
parallel. On the contrary, parallel is the appropriate approach to deliver
the interconnection density and integration required for 100GigEthernet and
many other bandwidth-intensive applications. 4 SFP's occupy roughly 3X
more width than a QSFP; comparing widths between 12 SFP's to a pair of
SNAP12's gets downright ludicrous. But there is the cost barrier to array
packaging discussed above. That barrier occurs as soon as you get away
from a singlet, e.g. a 1x2. As you go to higher integration levels, the
cost barrier is spread out over more channels and per/channel cost
decreases. Compare a "4+4" parallel approach (e.g. QSFP or POP4) vs a
"12x" approach (e.g. SNAP12 or any other that aligns a 1x12
array). Two QSFP modules require the mounting of 4 optical
ICs (2 VCSEL arrays and 2 PIN arrays) and 4 electrical ICs (2 drivers and 2
receivers). In each case, the VCSEL and PIN arrays must be mounted to 1's
of um accuracy in registration. The two QSFPs can deliver 80Gb/s. In
contrast, a Tx/Rx pair of SNAP12's require the mounting of only 2 optical ICs
and 2 electrical ICs. Registration between optical ICs is not an
issue since only one optical IC aligns to a fiber ribbon. The SNAP12 pair
should cost less than the QSFP pair since it involves fewer parts and is
simpler, i.e. it's MORE INTEGRATED. Yet the SNAP12 pair delivers 100Gb/s
(or even 120Gb/s), compared to 80Gb/s. Reduced costs for parallel optical
technology will occur and will arrive sooner if the industry settles on an
appropriate level of integration - not too high and not too low.
Given the (relative) prevalence of 12-fiber ribbons and MTP connectors and
modules in the field and the match to 100Gig Ethernet, I think 12 Tx and 12 Rx
is the appropriate level of integration.
Cheers,
Jack
Jack,
I appreciate your detailed responses.
Regarding your statements that I am filling
a role that does not match my affiliation, I have to agree. But I am
filling a role that needs to be filled. I am trying to stimulate the
discussion in an area that has become one of the remaining factors that will set
direction for the next several years work. There are certain dynamics that
stifle this type of discussion from occurring to the level necessary between the
expected parties so that folks like myself can have sufficient understanding.
For example, cost is not usually a comfortable subject to explore in
detail within a competitive arena. So I offer my analysis/contributions
and questions/comments as a vehicle to draw out controversy and agreement, and I
welcome more transceiver experts to join in.
Regarding your comments on item 5), it is clear that more
work needs to be done on the content of kolesar_01_0507. I will look to
submit a revised version with more explanation for the July meeting. Part
of the problem is that there is no explanation for the approach I took, which I
think has lead to misunderstandings. The first order explanation is that
the XFP column is the basis for the transceiver component comparisons and each
row is independent. That means that each element of the XFP components is
rated 4x (times 4 because there are four XFPs used for 4x10G LAG), regardless of
their relative cost to each other. The only row were these are summed
together is in the "weighted average".
I am surprised regarding your assertions that four
discrete OSAs cost less than one 4-element array OSA, albeit with volume effects
factored in. I sure hope this relationship does not come to haunt the IC
industry, or we'll have quad devices costing more than four times their single
counterparts, and integration will fail to be a means to lower cost.
Regards,
Paul
Kolesar
CommScope Inc.
Enterprise® Solutions
1300 East Lookout Drive
Richardson, TX 75082
Phone: 972.792.3155
Fax:
972.792.3111
eMail:
pkolesar@commscope.com
| "Jack Jewell"
<Jack.Jewell@picolight.com>
06/29/2007 05:58 PM
|
|
To
| <PKOLESAR@SYSTIMAX.COM>,
<STDS-802-3-HSSG@listserv.ieee.org>
|
|
cc
|
|
|
Subject
| RE: [HSSG] The
List |
|
Thanks Paul for undertaking the task of this cost analysis.
I must say though, it's most curious that the task of 40G module cost
analysis has been left to a fiber supplier. You're truly dedicated.
I've been traveling without email access, hence the delayed
response.
My responses are in red. For you CrackBerry
Warriors without colored text, the responses start on the next line following
Paul's questions, preceded by JJ -
.
Jack
From: Paul Kolesar
[mailto:PKOLESAR@SYSTIMAX.COM]
Sent: Wednesday, June 27, 2007 4:57
PM
To: STDS-802-3-HSSG@listserv.ieee.org
Subject: Re: [HSSG]
The List
Jack,
thank you for chiming in. We need more of this type of input.
Your response is provocative on several points, so consequently I have
some questions and comments about your statements:
1) First, SFP+ achieves the
300m SR objective with similar ease as XFP.
What is absorbing or mitigating the jitter that is
handled by the CDR in the XFP? Is it an external CDR, and EDC function,
reduced temperature range, or improved signal quality from the upper level
divides to which the SFP+ interfaces? Or was the presence of the CDR in
the XFP unwarranted in the first place? Or are you saying that SFP+
supports a 300m link despite being noncompliant to 10GBASE-S specs?
JJ - The SFP+ architecture utilizes Tx pre-emphasis
and Rx equalization in the PHY IC, and limits the FR4 traces to ~8" rather than
12". XFP does not need these features due to the CDR. Modern PHY
chip include these features with only incremental cost, but they were not
available when XFP was defined. The comments directly relate to the SFP+
limiting version (no EDC), which is analogous to the XFP. With EDC, the
comments still apply, and additional benefits are available.
2) While a
reduced-reach 10G PMD might reduce costs, it's a retreating approach that
requires the customer to purchase higher-cost transceivers to fill in the reach
gap, e.g. for reaches between 100m and 300m.
I completely agree that this is a
retreating approach that is not a preferred solution if it can be avoided,
especially if all that is required is a simple (low cost and power) EDC function
built into the receiver's TIA.
JJ - Check. EDC functionality however is not in the
receiver's TIA, but on the PHY chip (better yet!). The TIA simply needs a
linear response.
3) A reduced operating temperature range offers immediate cost
savings for some customers without compromising reach and without any new
standards specifications.
I do not think that the 802.3 standard specifies operating
temperature range. If so, this improvement is available without violation
of the 10GBASE-S spec. But then there is the market requirement on temp
range to consider, which can be just as imposing as the standards specs.
This leads back to the questions of item 1.
JJ - That's why it involves no new standards specifications and
also why only "some" customers are able to enjoy this benefit.
4) For QSFP,. Here again, EDC offers great
advantage, and ignoring it would be a mistake. But crosstalk introduces
module-level limitations that might motivate reach reduction or other
modification to the PMD.
The crosstalk in a transceiver is another source of jitter. I
accept that the crosstalk in a single lane transceiver is likely to be easier to
mitigate than that in a multi-lane transceiver that shares a common
housing/circuit board between the transmit and receive functions. The degree of
added impairment is the issue, and whether the jitter impairment causes a
reduction in reach or other parameter trade-offs will need to be sorted out
during the setting of specifications. But if EDC and reduced temp range
are means to lower cost for SFP+, then they will be as well for QSFP. I
think this may need to be sorted out in the lab as companies design these
components into their test fixtures and systems. That is why I encourage
those with experience in this area to afford us the benefit of that experience.
Existence proof is a powerful argument.
JJ - Of course EDC and reduced temp range benefit SFP+ and QSFP (and
anything else) about equally, hence my comment on EDC.
For a parallel module at 10G/ch,
crosstalk and power variation add to the difficulty and motivate some relief
elsewhere. That is why although I'm fine with a 300m reach for 10G serial,
I support a shorter reach objective for 10G parallel without EDC. Adding
EDC provides a tradeoff between further spec relaxation and increased
reach.
5)
Comparing costs, the SFP+ will be lower than QSFP for the foreseeable future
(per lane for the same spec).
Does this claim rely on volumes favoring SFP+?
From what I can tell, the intrinsic costs favor QSFP relative to 4xSFP+.
If you disagree with this, please provide the particulars of where we
differ by addressing the constituent comparisons in kolesar_01_0507.
JJ - This claim is based on the reality of present SFP+ costs
and lack of any
visible path for QSFP to reach parity. Making a cost comparison for equal
volumes is irrelevant when the reality is that one module will continue to have
far higher volumes than the other - again for the forseeable future.
Before there is any 40G Ethernet market, the tsunami of 8G Fibre-Channel,
utilizing 10G technology, will drive down the costs for the serial ICs.
The balance of TOSA/ROSA costs between 4xSFP+ and QSFP is reversed on your
chart. I think you underestimate the impact of product maturity on
TOSA/ROSA packaging costs as well. Also in the 4xSFP+, the PCB and Pkg
costs are too high.
Thanks for
helping to increase understanding of these issues.
Regards,
Paul
Kolesar
CommScope Inc.
Enterprise® Solutions
1300 East Lookout Drive
Richardson, TX 75082
Phone: 972.792.3155
Fax:
972.792.3111
eMail: pkolesar@commscope.com
| Jack Jewell
<Jack.Jewell@PICOLIGHT.COM>
06/27/2007 11:41 AM
|
Please respond
to
Jack Jewell
<Jack.Jewell@PICOLIGHT.COM> |
|
|
To
| STDS-802-3-HSSG@listserv.ieee.org
|
|
cc
|
|
|
Subject
| Re: [HSSG] The
List |
|
Input from a
transceiver vendor with experience and interest in both serial and parallel
modules:
First, SFP+ achieves the 300m SR objective with similar ease as XFP.
While a reduced-reach 10G PMD might reduce costs, it's a retreating
approach that requires the customer to purchase higher-cost transceivers to fill
in the reach gap, e.g. for reaches between 100m and 300m.. A reduced
operating temperature range offers immediate cost savings for some customers
without compromising reach and without any new standards specifications.
Use of a linear receiver and EDC and relaxed Tx specs can also achieve the
300m reach at reduced cost. For this application, the amount of
compensation required is far less than what is required for LRM. EDC is
becoming widely available, to the point of being a standard feature of PHY ICs,
so the cost of this EDC is becoming insignificant. Forward-looking
standards efforts will achieve their full impact by making use of
EDC.
For QSFP,. Here again, EDC offers great advantage, and ignoring
it would be a mistake. But crosstalk introduces module-level limitations
that might motivate reach reduction or other modification to the
PMD.
Comparing costs, the SFP+ will be lower than QSFP for the forseeable
future (per lane for the same spec). The only advantage of QSFP over SFP+
is density and possibly simpler cabling. However, a 12-channel parallel
module pair, e.g. SNAP12, offers greater density than QSFP and 2.5-3X the
bandwidth at <2X the cost.
Jack
From: Dove, Dan [mailto:dan.dove@HP.COM]
Sent: Wednesday, June 27, 2007 9:32 AM
To:
STDS-802-3-HSSG@listserv.ieee.org
Subject: Re: [HSSG] The
List
Paul,
Regarding SFP+, I am very familiar with the technology and
have been tracking the SFF-8431 development. The architecture re-distribution of
cost that SFP+ offers will have a substantial impact on cost, especially when
combined with the higher density we can achieve with smaller geometry ASICs and
multiport PHYs that will come with it.
As for QSFP, I am less
familiar with whether or not it will provide a cost improvement over SFP+ or be
capable of meeting the existing SR spec. This is something for the QSFP experts
to consider, but like I said, a shorter 10G PMD might be the avenue to take
rather than an identity challenged 40G spec.
Regarding LAG, my
conversation with HP Server architects indicates there are a number of avenues
for improvement of LAG under development.
I cited these areas in
my earlier message and would appreciate them being addressed rather than
ignored.
Dan
From: Paul Kolesar
[mailto:PKOLESAR@SYSTIMAX.COM]
Sent: Wednesday, June 27, 2007 8:04
AM
To: STDS-802-3-HSSG@listserv.ieee.org
Subject: Re: [HSSG]
The List
Dan,
I can't tell how successfully SFP+ and QSFP will be at meeting the
existing 10GBASE-S spec. If they can, a new shorter distance 10G PMD would
not be of value. If they can't, then a new PMD spec may be worth while.
Those attempting to implement these lower cost platforms need to weigh in
to provide guidance. In the event that either the QSFP and/or SFP+ can
meet 10GBASE-S specs in multiple vendor's platforms, or that a new shorter
distance spec is developed that allows lower cost, the performance issues of LAG
will remain. I believe Howard's presentations on LAG have indicated that
improving LAG would not be without compromise, leading me to conclude that,
however improved, LAG performance could not become equivalent to a 40G pipe.
Developing a 40G spec would ensure a solution that simultaneously
addresses these cost and performance issues.
Regards,
Paul Kolesar
CommScope
Inc.
Enterprise® Solutions
1300 East Lookout Drive
Richardson, TX
75082
Phone: 972.792.3155
Fax:
972.792.3111
eMail: pkolesar@commscope.com
| "Dove, Dan"
<dan.dove@HP.COM>
06/26/2007 09:24 PM
|
Please respond
to
"Dove, Dan"
<dan.dove@HP.COM> |
|
|
To
| STDS-802-3-HSSG@listserv.ieee.org
|
|
cc
|
|
|
Subject
| Re: [HSSG] The
List |
|
Hi
Paul,
Good
points. I was not really expecting to see a significant cost differential at the
PMD although its a good argument that a 100m PMD would be less expensive. If
this is the case, why not do another 10G PMD focused on lowering the cost of
server interconnect? I believe that would be a smaller project and have a much
less significant impact on 100G development.
Thanks,
Dan
From: Paul Kolesar
[mailto:PKOLESAR@SYSTIMAX.COM]
Sent: Tuesday, June 26, 2007 6:16
PM
To: STDS-802-3-HSSG@listserv.ieee.org
Subject: Re: [HSSG]
The List
Dan,
thanks for your detailed thoughts and
proposals. I appreciate the points you made regarding the volume effect of
10G components on the cost comparison. The presentation I submitted for
the May interim looked at the intrinsic cost factors and did not attempt to
include volume in the equation. But volume certainly can be a significant
factor. Your suggestion to look into its impact when comparing 4x10G LAG
to 40G is reasonable, but complicated at the PMD level. As my May
presentation shows there are a few ways to implement LAG on MMF. One uses
the XFP, another the SFP+, still another the QSFP. Today the XFP is
shipping to the 10GBASE-S spec, and supports 300m transmission. Designs
using SFP+ and QSFP will be more challenged to meet this spec due to jitter, so
it remains to be seen how successfully these lower cost form factors can
substitute for the XFP in 10GBASE-S compliant LAG. However, a reduced
distance requirement, such as that stated in the HSSG objectives, would greatly
improve the chances that QSFP will suffice for "40GBASE-S". So while
volume is important, these unanswered questions on suitability make it
impossible from my vantage point to determine how the volumes for 10GBASE-S will
be divided among XFP, SFP+, and QSFP. And the effects of volume on
production costs are better left to those who manufacture the devices.
Perhaps individuals with such insights will offer some scenarios.
Regards,
Paul Kolesar
CommScope Inc.
Enterprise®
Solutions
1300 East Lookout Drive
Richardson, TX 75082
Phone:
972.792.3155
Fax: 972.792.3111
eMail:
pkolesar@commscope.com
| "Dove, Dan"
<dan.dove@HP.COM>
06/26/2007 02:45 PM
|
Please respond
to
"Dove, Dan"
<dan.dove@HP.COM> |
|
|
To
| STDS-802-3-HSSG@listserv.ieee.org
|
|
cc
|
|
|
Subject
| Re: [HSSG] The
List |
|
My fellow colleagues ,
Last week I sent out a list of items that I felt need to be addressed to
ensure that a 40G PAR would be justified. At a subsequent EA teleconference
intended to build concensus in the HSSG, I offered to review the presentations
made in support of 40G Economic Feasibility and comparing 40G vs 4x10 LAG
performance to ensure that I was not being too harsh in my consideration of the
material that was presented.
Over the weekend, I reviewed every presentation I could find on
these subjects so that I could be comfortable that I was not being unfair in my
concerns. Fortunately, it was not a huge task as there are not that many to
review.
After doing
so, I found myself less convinced in the validity of some presentations
that were made. This statement is not made to criticize my colleagues, but to
honor the concept of peer review which requires that we review and criticize,
otherwise we might as well just upload them to a server and forget about
them.
Specifically, I
disagreed with cost arguments made on the assumption that 10G cost remains a
constant, when in fact I anticipate substantial reductions in 10G cost over the
next few years at a rate much faster than today due to a few
factors;
1) Higher
density/lower cost optical form factors (SFP+) allowing better utilization of
switch infrastructural cost and QSFP for NICs.
2) Smaller geometry CMOS allowing higher port densities to
work in synergy with PMD cost reductions.
3) Integration of XFI / SFI interfaces directly into ASICs
or multi-port PHYs driving 10G cost further downward.
4) Higher volumes / commoditization of 10G
driving cost down much faster than the current trajectory.
While 40G can leverage some of these
elements, it cannot leverage the volume that feeds the downward cost spiral. So
in 4 years, a 40G switch port cost is going to be based on low-volume, freshly
designed and un-amortized silicon used primarily for server interconnect,
whereas a 10G port cost will be based on amortized, high-volume silicon being
used in a huge array of applications. Having different trajectories, the
relative cost for 40G will be higher than presented. This is true for 100G as
well, but who is arguing a need for 100G based on cost? It is bandwidth that
drives 100G demand.
In addition, I found presentations claiming that LAG was
insufficient to address server I/O bandwidth needs, yet those presentations
failed to address upcoming technology enhancements like TRILL and its impact
combined with I/O Virtualization, perhaps with a physical manifestation of QSFP
and MPO optics which I believe can lead to graceful performance scaling for
servers that does not demand an intermediate IEEE standard. In other words,
activities and technologies are advancing which will parse server network access
into multiple conversations that can then be put onto a LAG group with much
higher than presented performance levels.
Now, I realize that I am swimming upstream here by
asking that the proponents for "40G now" to complete a task that took the
100G proponents almost a year to accomplish, in less than 6 months, but then I
am not asking them to do that. My first choice, the one I proposed in
Geneva, was that we move 100G forward (because it is DONE) and that we continue
to work on 40G (until it is done).
This appears to be a minority
position because apparently some people will accept an unproven 40G proposal
rather than risk 100G. Others think that 40G is proven sufficiently and are
demanding "40G now" or they will not allow a 100G PAR to go forward. Those in
the latter camp must either be unconvinced of my concerns, or they think my
concerns are insufficient to justify any further work being done to justify a
40G project.
I can accept differences of opinion.
What I cannot do, however, is pretend
that these issues do not exist, or that the work we would have to spend getting
a 40G standard done is not going to delay the much needed 100G aggregation
solution our customers demand. I cannot ignore what I perceive as holes in the
40G presentations.
So, to provide a little more direction to my colleagues in the
"40G now or the HSSG stalls" crowd, I am asking you to include relative cost
trajectories in your analysis of 40G vs 10G cost models, and to include
technology enhancements to LAG (TRILL, I/O Virtualization, QSFP, MPO) in your
performance analysis.
If you feel that this is unnecessary, I am requesting that you
communicate this position to me as soon as possible so that I can prepare a
presentation on these areas of concern for the July meeting.
Respectfully,
Dan Dove
Dove Networking Solutions - Serving ProCurve Networking by
HP