Re: HARI Latency
Dan:
I don't think anybody has agreed on objectives for Hari. Certainly the
committee
has no agreement on objectives for Hari.
From my point of view Hari's purpose to connect between chips on PCBs or
over backplanes. It is a transmission link not an interface and provides a
complete PHY/PMD over PCB traces. It can be used to connect a MAC to a
LAN-PMD or to connect two MACs over a backplane. I believe we should have
some objectives for Hari like: operates on up to 1 meter for PCB trace with
2 connectors at an error rate under 1 in 10-12, provides a certain degree
of skew correction (pick a number), can be interfaced to an LAN-PHY using a
jitter elimination buffer (this is a 2 port repeater of sorts), limits E&M
emission, etc.
Cheers,
Paul
At 10:48 AM 12/16/99 -0700, DOVE,DANIEL J (HP-Roseville,ex1) wrote:
>
>Hi Rich,
>
>Would you respond to my earlier point about HARI byte vs word striping
>and the negligible latency impact when applied to a PMD interface?
>
>I believe that word striping offers simplicity and preserves the 8B10B
>coding used for GigE at the expense of additional latency, but I also
>believe that this latency has virtually ZERO impact when applied to the
>PMD implementations which are going to swamp it.
>
>On the other hand, I believe that word striping latency would seriously
>impact backplane implementations, however, this is not the stated objective
>for HARI, is it? If not, why should we not go to word striping?
>
>Dan Dove
>
>___________ _________________________________________________________
>_________ _/ ___________ Daniel Dove Principal Engineer __
>_______ _/ ________ dan_dove@xxxxxx LAN PHY Technology __
>_____ _/ ______ Hewlett-Packard Company __
>____ _/_/_/ _/_/_/ _____ Workgroup Networks Division __
>____ _/ _/ _/ _/ _____ 8000 Foothills Blvd. MS 5555 __
>_____ _/ _/ _/_/_/ ______ Roseville, CA 95747-5555 __
>______ _/ ________ Phone: 916 785 4187 __
>_______ _/ _________ Fax : 916 785 1815 __
>__________ _/ __________________________________________________________
>
>
>> -----Original Message-----
>> From: Rich Taborek [mailto:rtaborek@xxxxxxxxxxxxx]
>> Sent: Wednesday, December 15, 1999 10:52 PM
>> To: HSSG
>> Subject: Re: Hari KrishMAS
>>
>>
>>
>> Patrick Gilliland wrote:
>> >
>> > Glad to hear you are still the champion of MAS. I am
>> > looking forward to the demonstration. I hope you will
>> > consider the 1Gbit/PAM10 option on 1300nm I outlined
>> > earlier. I believe there are serious problems of laser
>> > safety for a 850nm MAS transmitter.
>>
>> Pat,
>>
>> My outstanding MAS proposal is PAM5x4 @ 5 Gbaud (2.5 GHz) to
>> transport 10 Gbps
>> of Ethernet data. I don't suggest going to 10 levels (I
>> assume this is what you
>> mean by PAM10) due to the large SNR degradation while at the
>> same time not
>> pushing the technology enough (I assume that by 1 Gbit you
>> mean 1 Gbaud).
>> Besides, 10 bits gives you 3+ bits per baud, and assuming
>> that you use the + for
>> Forward Error Correction, the 3 bits left over times 1 Gbaud
>> yields a data
>> transfer rate of 3 Gbps. This falls far short of the
>> required P802.3ae 10 Gbps
>> data transfer rate requirement.
>>
>> MAS is laser wavelength independent. There are no problems
>> with laser safety at
>> 850 nm that I know that are MAS specific. I don't understand
>> what your concern
>> is on this issue?
>>
>> > -------------------------------------------------------
>> > >> I am not sure what the choices are here interms of the MAC/XMII
>> > >> interface. Can the XMII/MAC interface benefit from Hari
>> > >> encoding? I do not see why a HARI or other interface might
>> > >> not be used between SERDES and MAC. Or maybe a HARI would fit
>> > >> between the SERDES and the optical transceiver.
>> > >
>> > >Hari IS proposed as the interface between the SerDes
>> (PMA) and the optical
>> > >transceiver (PMD).
>> >
>> > ------------------------------------------------------------
>> >
>> > I think you have misunderstood me here. I am suggesting a
>> > HARI interface might be useful chip-to-chip or as an interface
>> > between the SERDES and the MAC.
>>
>> Hari IS a chip-to-chip interface. It makes no sense to use
>> Hari between the
>> SerDes and the MAC unless the SerDes you're talking about is
>> part of the
>> transceiver and the other side of the SerDes is in the MAC
>> (integrated MAC/PHY).
>> Pictures always speak 1000 words. Please allow me to
>> illustrate the potential
>> elements (chips and such) in a 10 GbE LAN PHY:
>>
>> +-----------+ XGMII +--------------+
>> | +-------> | +------------------+
>> | | . | E S| Hari |S (E) |
>> | 10 GbE | . | 10 GbE n e+------------->e (n)
>> Transceiver | Medium
>> | | 36 | D r+------------->r (D) Module
>> +------------>
>> | MAC | . | PCS/PMA e D+------------->D (e)
>> (PMD) | 1-4 fibers
>> | | . | c e+------------->e (c) |
>> | | . | s| FR-4 PCB |s |
>> | +-------> | Traces +------------------+
>> +-----------+ short +--------------+ <=20"
>>
>> Figure 1 - Location of Hari, MAC and discrete PCS/PMA Chip
>> within a 10 GbE
>> Device
>>
>> +--------------+
>> | | +------------------+
>> | E S| Hari |S (E) |
>> | 10 GbE n e+--------------->e (n) Transceiver | Medium
>> | D r+--------------->r (D) Module +------------>
>> | MAC/PHY e D+--------------->D (e) (PMD) | 1-4 fibers
>> | c e+--------------->e (c) |
>> | s| FR-4 PCB |s |
>> | | Traces +------------------+
>> +--------------+ <=20"
>>
>> Figure 2 - Location of Hari and Integrated MAC/PHY Chip
>> within a 10 GbE Device
>>
>> Note that one of the primary purposes of Hari are to reset
>> the link jitter
>> budget within the Transceiver Module (PMD) allowing the
>> PMD-medium-PMD jitter
>> budget to be independent to that of all components from each
>> PMD back to it's
>> respective PHY.
>> >
>> > -------------------------------------------------------------
>> >
>> > >I don't believe that you've answered my question about
>> making 10-12.5 Gbps
>> > >traces work for any practical distance between the SerDes
>> and the optical
>> > >transceiver.
>> > >
>> > --------------------------------------------------------------
>> >
>> > I have answered this question adequately. I believe you are
>> > making a semantic distinction. My answer is to move whatever
>> > parallel/serial conversion chipset one wants to entertain such
>> > as a HARI outside of the optical PMD device. Simple. Distance
>> > is not an issue in this case, because as I have pointed out this
>> > circuit element can be backed up directly against the optical
>> > transceiver.
>>
>> Given your answer, what interface do you then propose
>> between the SerDes back to
>> the protocol chip (e.g. MAC, PCS, etc.)? The general
>> requirements for this
>> interface are:
>>
>> - Interface reliability;
>> - 10 Gbps data transport;
>> - Protocol and PMD independence;
>> - Low pin count to enable high port density protocol ASICs, removable
>> Transceivers, reasonable routing of high-speed traces through FR-4;
>> - Jitter budget independence from the link medium jitter budget;
>> - CMOS friendly, at least to 0.25 micron;
>> - FR-4 friendly;
>> - Reasonable distances (to 20");
>> - Scalability to higher speeds.
>>
>> > The real question I would like you to focus on is the flip side.
>> > Why complicate the optical transceiver any further in light of
>> > the arguments I have already made? What compelling reasons are
>> > there which dictate the inclusion of a HARI chip inside the
>> > optical transceiver which can not be solved any other way by
>> > external parallel/serial conversion?
>>
>> - Interface reliability due to 8B/10B error control;
>> - 10 Gbps data transport;
>> - Protocol and PMD independence;
>> - Low pin count to enable high port density protocol ASICs, removable
>> Transceivers, reasonable routing of high-speed traces through FR-4;
>> - Jitter budget independence from the link medium jitter budget;
>> - CMOS friendly, at least to 0.25 micron;
>> - FR-4 friendly;
>> - Reasonable distances (to 20");
>> - Scalability to higher speeds.
>>
>> As an architect, I always try to ensure that the whole
>> system works. I believe
>> that you have only addressed a small and portion of my
>> question adequately. That
>> is, you have addressed the 10 Gbps SerDes to PMD issue by
>> backing the "SerDes up
>> directly against the optical transceiver". I believe that
>> this is already the
>> way it is. Now please consider the path all the way back to
>> the protocol ASIC.
>> If your answer is that the latter interface is Hari, then I
>> believe that we
>> agree, since this is what I show in figures 1 and 2 above,
>> and all your logic
>> plus Hari logic is part of the PMD.
>>
>> > ---------------------------------------------------------------
>> >
>> > >> Also, cost effective TIA and laser driver solutions abound
>> > >> in GaAs at the 10Gb/s rate. I would point you to the web
>> > >> sites of Anadigics, Triquint, and Vitesse. In my role
>> > >> in developing advanced transceiver designs, I see an ever
>> > >> growing variety of product offerings. I am also seeing
>> > >> drastic price reductions in mature GaAs building blocks
>> > >> such as TIA, post amplifers, laser drivers. The SiGe
>> > >> products will have to hit these targets, and the increased
>> > >> availability will prove decisive.
>> > >
>> > >Once again, the point we were discussion here was the
>> path from the MAC to
>> > the
>> > >transceiver in typical 10 GbE switch/routers, not TIAs
>> and laser drivers.
>> > >
>> >
>> > ------------------------------------------------------------
>> >
>> > I forgot to mention GIGA, Phillips, Arizona Microtek, Microcosm,
>> > Cognet, and a few others. See my comment above in response to
>> > this. I believe what we are going back and forth and to and fro
>> > about here is simply a question of partitioning. I believe some
>> > of the parallel/serial conversion duties you have assigned to the
>> > optical PMD belong outside the optical transceiver, however in
>> > close proximity.
>>
>> Yes. It is clearly a question of partitioning. However, the
>> partitioning being
>> proposed is not indiscriminate and is intended to provide a
>> workable and
>> interoperable architecture. The principal purpose of Hari
>> interfacing the PMD to
>> the MAC/PHY is to separate the PMD-to-PMD jitter budget from
>> the rest of the
>> system. I consider this Hari attribute to be of paramount
>> importance to the
>> development of a successful and cost-effective 10 GbE standard.
>>
>> > ---------------------------------------------------------------
>> > >
>> > >16X622Mbps carries a clock and can't come close to making
>> Hari distances. The
>> > >other problems are clock distribution and way too many
>> pins to consider making a
>> > >pluggable module (i.e. >100 pins vs. today's 20 pin GBICs)
>> > >
>> > >10X1.25Gbps serial clearly goes farther than Hari, but
>> Hari distances are
>> > >sufficient for most applications and Hari distances can
>> be significantly
>> > >extended with careful layout, equalization, etc. 10X will
>> add approximately 48
>> > >signal and ground pins to a pluggable module and doesn't
>> push CMOS technology
>> > >enough. Therefore, it is not a cost effective alternative to Hari.
>> > >
>> >
>> > ----------------------------------------------------------------
>> >
>> > You are making my point about I/O reductions to the optical PMD
>> > even more emphatically than I was prepared to myself. The best
>> > definition of a pluggable transceiver has only one high speed,
>> > impedance controlled line at the full line rate. Otherwise, one
>> > creates a nightmare in terms of cross-channel isolation in the
>> > connector.
>>
>> Hari is 4 x 3.125 Gbps. You propose 1 x 10+ Gbps with many
>> components behind the
>> transceiver to add jitter and make a reliable system
>> solution very difficult to
>> specify, and interoperability difficult to achieve.
>> >
>> > ---------------------------------------------------------------
>> > >> Please refer to an article in EE Times of November 29, 1999
>> > >> entitled "Lucent to show SiGe process for 10-Gbit Sonet"
>> > >> (Shakespeare never wrote a 10-Gbit Sonnet).
>> > >> It describes a process whereby the SiGe bipolar transistors
>> > >> can be easily added to the existing CMOS process as a
>> > >> selective epitaxial growth. A mixed technology ASIC with
>> > >> portions capable of 10Gb/s is just what is needed to respond
>> > >> to your concerns above.
>> > >>
>> > >> And, you get your CMOS right where it belongs.
>> > >
>> > >But it's not going to compare favorably with a Hari CMOS
>> solution in terms of
>> > >power and cost.
>> > >
>> > ----------------------------------------------------------------
>> >
>> > I believe it will compare very favorably in terms of power
>> > consumption. Of course, the initial cost will be higher, but
>> > 2.5Gb/s CMOS is not so inexpensive yet.
>>
>> This is what competition is all about! I'm game!
>>
>> > -------------------------------------------------------------
>> >
>> > >And how much jitter budget are you leaving for the
>> PMD-to-PMD (medium)
>> > >interface? Hari provides the medium interface with a
>> jitter budget which is
>> > >independent of either of the two SerDes to optical
>> transceiver interfaces.
>> > This
>> > >is one of the most important, if not THE most important
>> attributes of Hari.
>> > >
>> >
>> > --------------------------------------------------------------
>> >
>> > Jitter independence is nice, but not when the expense is a higher
>> > overall jitter total.
>>
>> I don't understand your answer. Is there any significance to
>> adding up
>> independent jitter budgets?
>>
>> > -------------------------------------------------------------
>> >
>> > The use of MAS might be better accomplished by separating out
>> > any HARI decode chip and integrating it with a multi-level
>> > copper driver which would enable short distance pluggable
>> > copper connections over coax or other suitable copper media.
>> >
>> > A linearized pluggable laser transmitter could then be used
>> > interchangeably with the linear copper pluggable transceiver.
>>
>> A single integrated Hari-MAS chip is capable of driving a
>> single coax (twin-ax
>> for full-duplex). This option has always been part of my
>> media independent MAS
>> proposal.
>>
>> > --------------------------------------------------------------
>> >
>> > >> I do not see any value to adding such a CMOS HARI chip
>> > >> inside the optical PMD definition.
>> > >
>> > >Here are a couple:
>> > >- Lowering the line rate requirements
>> > >- Interfacing directly with existing high-speed Mux/Demux chips
>> > >- Removing as much logic out of the latter elements for
>> future generation
>> > parts
>> > >- Providing the medium interface with the greatest jitter budget
>> > >- Reducing the pin count to the PMD and MAC/PHY enabling
>> configurations
>> > such as
>> > >pluggable SFF transceivers
>> > >- Allowing 20" or longer traces between the PMD and MAC/PHY
>> > >- Enabling low-cost power integrated MAC/PHYs
>> > >- Allowing the continued use of cost effective and common FR-4 PCB
>> > >
>> >
>> > ------------------------------------------------------
>> >
>> > I am on record as favoring bandwidth reduction techniques
>> > on the level of chip-chip-interfaces within the cabinet at
>> > a minimum. When scalability is an issue, I do not think
>> > MAS or multi-channel techniques offer the advantages of true
>> > serial.
>>
>> Your thoughts are not borne out by the datacom and
>> communications industry in
>> general. For example, multi-level signaling AND multiple
>> channels have been used
>> is scaling Ethernet from 10 Mbits to 100 and then 1000. As a
>> matter of fact,
>> 1000BASE-T uses PAM5, 4 pairs of wire and supports
>> simultaneous full-duplex
>> traffic over each wire pair. The same is true in most
>> wireless and wired
>> systems. Kestrel Solutions is offering an 0C-192 optical
>> solution employing
>> Frequency Shift Keying.
>>
>> > ------------------------------------------------------------
>> > >>
>> > >> Low cost transceiver designs are often multi-platform
>> > >> and multi-protocol. The best way to achieve this goal
>> > >> is what the industry has already decided. Produce optical
>> > >> transceivers which can handle traffic at a given bit rate
>> > >> or range of bit rates. Make this same transceiver serve
>> > >> multiple markets as we do today.
>> > >
>> > >What I've discussed above is multi-protocol. It is usable
>> for Ethernet, Fibre
>> > >Channel, InfiniBand, etc.
>> >
>> > -------------------------------------------------------------------
>> >
>> > Are you implying a solution for chip-chip and backplane connections
>> > is also ideal for a network?
>>
>> Absolutely. Since there are many, many chip-to-chip
>> connections in all networks.
>>
>> I don't really understand your question. Hari is a
>> chip-to-chip connection which
>> enables the optimization of network link architectures. I
>> don't view Hari as a
>> network.
>>
>> > ---------------------------------------------------------------
>> >
>> > >> Additionally, your desire to include HARI in the optical
>> > >> PMD does not address the needs of hub and other traffic
>> > >> forwarding equipment designs. For this application where
>> > >> the design needs to take in 10-12 GbE lines and trunk them
>> > >> up to a 10GbE serial rate, you would require them to first
>> > >> convert each line to the parallel HARI format and then
>> > >> reconvert to serial 10GbE in order to accomodate your
>> > >> wishes to include HARI in the optical PMD. The trunking of
>> > >> ten 1.25Gbaud lines need not be sucha cumbersome task.
>> > >
>> > >GbE to 10 GbE translation goes through at least the MAC
>> layer. MAC frame formats
>> > >are common for these Ethernet speeds. I may be missing
>> your issue, but we don't
>> > >do add-drop multiplexing in Ethernet. Please clarify.
>> >
>> > --------------------------------------------------------------
>> >
>> > I am not suggesting we support add-drop. I am suggesting HARI
>> > may be a lot of extra gates in these instances where everything
>> > can be done on a single chip.
>>
>> CMOS gates are cheap, especially in light of the benefits
>> provided by Hari. Hari
>> is a great example of the power of CMOS in addressing high
>> speed network
>> equipment design requirements.
>>
>> > Best Wishes for the Holidays,
>> >
>> > Pat Gilliland
>> > patgil@xxxxxxxxxxx
>>
>> --
>>
>> Merry Christmas,
>> Best Regards,
>> Rich
>>
>> -------------------------------------------------------------
>> Richard Taborek Sr. Tel: 408-330-0488 or 408-370-9233
>> Chief Technology Officer Cell: 408-832-3957
>> nSerial Corporation Email: rtaborek@xxxxxxxxxxxxx
>> 2500-5 Augustine Dr. Alt email: rtaborek@xxxxxxxxxx
>> Santa Clara, CA 95054
>>
>
Paul A. Bottorff, Director Switching Architecture
Enterprise Solutions Technology Center
Nortel Networks, Inc.
4401 Great America Parkway
Santa Clara, CA 95052-8185
Tel: 408 495 3365 Fax: 408 495 1299 ESN: 265 3365
email: pbottorf@xxxxxxxxxxxxxxxxxx