10 Gbit PHY, HARI
Joel,
While your response to Ed is well written and
to the point, I believe you have missed one of
Ed's points which I agree with. In his own way,
Mr. Chang is stating a principle which is quite
logical.
If we have a 10 Gbit serial line rate, the further
we extend this serial line rate back towards the
switching plane, the better. Fewer lines being switched
generally translates into fewer gates, less power, tighter
layouts, etc. For example, I would prefer to tackle the
problem of how to route and switch a single 10Gbit line
than four 2.5Gbit lines. It is a microwave problem in
either case which doesn't yield to a traditional
backplane/edge connector approach. We are certainly not
able to use any "buss" or "star" connections. The line will
most definitely be point to point, and must have a properly
designed individual termination, even at 2.5Gbit/s.
Perhaps we ought to consider approaches which utilize RF/
microwave hardware such as pluggable style edge connectors
like the SMB style and mini coax for the routing of high
speed 12.5GBaud lines across the backplane. This hardware
has been developed for the purposes we now have. I would be
interested to hear your thoughts, since my personal experience
with high frequency backplanes is limited to radar processors
and other traditional RF/microwave applications. I have never
had the responsibilities of a LAN or MAN switch designer, and
therefore I am not sure this is a reasonable alternative.
Best Regards,
Pat Gilliland
patgil@xxxxxxxxxxx
-------------------------------------------------------------
I am not sure Ed got the right name in here, because I don't agree with
most of
it. :)
> However, it violates a lot of high-frequency circuit design rules; as a
> result, it may make HARI a unreliable block in a system.
I don't see it violating any rules.
> Furthermore, additional circuit, HARI, is working against our successful,
> Ethernet practice of keeping it simple, and low-cost -- unless we prove HARI
> is a "MUST" for 10GbE product, it should be an optional block.
>
> For an extremely high frequency PC layout, the path-length should be kept as
> short as possible. The PCB characteristic impedance has about +/- 20%
> tolerance which will cause waveform distortion being severe enough at 2.5
> Gbps data rate to cause excessive errors. Even the skew is minimized by
> deskew circuit, the waveform distortion by reflection will cause excessive
> JITTER by altering the bit-timing information of each bit (bit-cell timing
> near 300 ps), which provides the deskew circuit a wrong data. This is one
> of the reasons that high frequency transceiver and PLL are preferred to be
> in one chip.
I really think what this boils down to if you agree with the above
statement is
that you don't have a current understanding of circuit board geometry, how
it is
developed, and what kind of control can be used to over come these issues in
'the circuit board of today'. From my point of view, I can remove all the
issues
in this email through careful understanding of the pcb construction. ie, Ed's
points are mostly true if you through the gerber over the wall and take
what the
board shop sends back.
> Over 2.5 Gbps, and at 20" PC run length, the signal amplitude will be
> drastically reduced for each inch the signal travels to cause the
> destination data without sufficient Signal-to-Noise ratio -- inviting for
> excessive errors. In addition, the rise time will be drastically increased
> to add further jitter to cause wrong data into deskew circuit.
>
> The higher the data rate, the capacitive and inductive coupling noise are
> higher which are linearly proportional to data rate, and the parallel
> length (20" parallel is excessive at 2.5 Gbps). A ground plan between two
> adjacent signals may reduce crosswalk but not necessarily eliminate it -- no
> absolute assurance of eliminating the crosswalk effect. Furthermore, the
> radiation issues will much tougher to resolve.
>
> The PC runs through the backplane have to go through connectors to create
> waveform distortion caused by impedance mis-match of connectors. At 300 pc
> cell time, any glitch could be the sauce of errors.
>
> Unfortunately, circuit problems are very difficult to debug, which show up
> as excessive random errors. Some time, it takes over six months to find
> it, then there is no simple cure.
Not sure about this .... all of the issues I have seen in high speed
differential resulting in the failures listed above have been the result of
poor
power design at the source and destination. The only high frequency desing
rules
I have followed are: modeling, good power design, test cards, good power
design,
trace and via geometry testing/verification, and good power design.
> I further agree with Joel, that HARI will unnecessarily use up the most
> valuable area of a PC board; namely, high frequency area. I also agree that
> HARI will add more power consumption to what we are struggling to reduce.
> All of these are counter productive, unless HAIR is "MUST" for the product.
> Perhaps, HARI can be an option feature.
Not sure I agree with this, either. I don't see the power consumption
being an
issue because I don't know what it is yet, but I have not seen what the SiGe
technology is going to consume, either. As far as the 'high frequency
area', I
stand by this statement -> The lower the speed between connection points, the
less board fabrication design I have to develop. So, I would rather deal
with a
few more longer pairs of 3.125 then one long run of 10 because the board
fab is
much easier to implement. The cost for me will be the same, either way,
but one
is a lot less work.
> The system architecture are flexible. There are so many ways to achieve the
> same result. I am not sure that to integrate all channels which are
> inherently distributed in one big chip is the most cost-effective way to do
> it, considering all the potential problems. I would think a modular
> approach with scalability may prove to be more cost-effective and more
> flexible to use from architecture point of view.
>
> I would hope some one will present test data to assure the performance,
> before we have to vote with some reservations.
>
> Regards,
>
> Ed Chang
> NetWorth Technologies, Inc.
> EChang@xxxxxxxxxxxxxxxx
>
> -----Original Message-----
> From: owner-stds-802-3-hssg@xxxxxxxx
> [mailto:owner-stds-802-3-hssg@xxxxxxxx]On Behalf Of Joel Goergen
> Sent: Tuesday, November 23, 1999 8:59 AM
> To: HSSG
> Subject: HARI Systems Design
>
> Hello all and Happy Holidays.
>
> I am very perplexed about a few issues that really bother me. First,
> and for most, the comment: "Don't you think that this is either a
> little early, or does someone have a hidden agenda?" from Roy's email
> this morning in one of the many HARI threads. Unless I am mistaken, I
> have not viewed this, nor any proposal by anyone as a hidden agenda. If
> you did call HARI a hidden agenda, then you could call two phys a hidden
> agenda, SONET a hidden agenda, etc, etc. Correct me if I am wrong,
> people, but I thought all the presentations were from people who believe
> they have a good idea to offer to the standard, might benifit them a
> little, but still a good presentation - or have I just stayed on the
> farm a little too long?
>
> In terms of systems design, "As for real estate on the PC board.
> Vendors need to think about reducing the size
> of their boards and systems. More and more floor space is being taken
> by these systems as well as power and cooling. Reducing the size of the
> boards, reducing the amount of electronics, reducing power requirements,
> and increasing the density of the connections is becoming an issue in
> large installations, like those that will use P802.3ae. Hari tends to
> take exactly the opposite direction in system design. Hari makes it
> easy for the system designer to become sloppy, not requiring them to
> become tighter and better." I think I would like to take this line by
> line.
>
> Vendors need to think about reducing the size of their boards and
> systems : Well, how about customers should require less features. Then
> I wouldn't have to go to extraordinary means to get all the components
> shoved into a small bucket. If you want less power, less space, less
> connections, then drop some features.
>
> Board Size: We fit more onto boards today on a gate per sq in level
> then we ever have at a lower price. We are beginning to abandon fr-4
> for newer materials at less cost per route length and more routing
> density.
>
> Thermal and power: Reducing voltages to 2.5v and 3.3v have helped.
> Power bricks are getting much more efficient. Using CMOS over GaAs and
> Bipolar have helped. We are all required to meet Telcordia
> requirements, so there is only so much heat per sq foot we are allowed
> to produce. Space and thermal and weight ARE already a standard.
>
> Hari makes it easy for the system designer to become sloppy, not
> requiring them to become tighter and better: Wow! Hari or something
> similar does no such thing. It allows the designer the ability to
> re-partition the problem. But, on the other hand, maybe I should be
> against Hari because then that would force most people to think about
> 10gig serial streams for long distances on copper traces ( which most
> companies can not aford to develop ) and go with very wide 622mhz data
> paths so my boards get thicker and more expensive - this allows the
> designer the greater headache of routing and board fab issues. I don't
> know, if sloppy were true, all of us would be out of business. I have
> seen most of the systems available today and we all pretty much design
> the same, have the same issues, and make the same trade-offs to supply
> the customer all the features they require to make the sale.
>
> So, in summary, Hari is a starting place, as I have mentioned before.
> Even the GMII and MII, etc have issues. But we have a proposal(s) of a
> start .... how about we try to constructively look at what hari solves
> or doesn't solve. So how do we design Hari to be 'phy independent'?
> Because at the moment, Hari solves most of my SI issues. Oh yeah, the
> job description for the SI guy is as follows : Comes up with last minute
> desperate solutions to impossible problems caused by the System
> Architect.
>
> Joel
--
Joel Goergen
Lucent Technologies
High Performance Networking Division
10250 Valley View, Suite 113
Eden Prairie, MN, 55344
Email: goergen@xxxxxxxxxx http://www.lucent.com
Phone: (612) 943-8990 Cell: (612) 670-5930
Direct: (612) 996-6932 Pager: (800) 200-0586
Fax: (612) 996-6695