Re: Link Status thoughts
Rich,
I thought I will fill in a few gaps in the protocol you presented to
hopefully complete it,
especially in steps 1 - 4. And restate it in my words.
All we need is for the RS layer (or wherever this protocol will
reside), to emit a RF code on its output, whenever it receives a LF code
at
its input. For this discussion, I will assume that it is the RS layer
which will
handle this.
On power-up, each layer, PCS, XGXS etc., starts emitting
local fault (LF), till they acquire sync at which point they start
passing through
whatever comes from below them.
So on power-up, the RS layer automatically starts sending a RF code
since it
starts receiving LF code. Once every layer from the remote end to
itself, gets
out of the LF state (i.e they become transparent to their previous
layer), the RS
layer starts receiving whatever the remote RS layer is sending, which is
the RF
code or idles.
Once the local RS layer gets the RF code or idles, we know that it can
hear the mating
calls of its partner, so it starts sending Idles. If it receives Idles
say a few of them
consecutively (actually one is enough if this protocol is done in the
RS layer) i.e. the RF
code from the remote end have stopped coming, then it knows
that the remote end can also hear it, (otherwise it wouldnt be getting
idles), so it
is now ready to start sending data.
There is no need for timers and the startup protocol is link length
independent.
The whole operation can be captured in a 3-state diagram.
The only potential difference compared to normal operation, is that
under normal
operation, when the RS layer receives the LF code, besides starting to
transmit RF
code at its output, it will also invoke the station management to
initiate debug
by probing MDIO bits etc. At power up, you could suppress that (or let
it be).
Under normal operation, when it receives a RF code, besides sending
Idles at its output,
it initiates debug.
regards,
Birdy
-----------------Included Message -------------------------------------
Pat,
Once again, I agree with your general architecture for Link Status
reporting. I agree that your choices of candidate signals for the PHY.
To reiterate from your note these consist of:
- Remote Fault
- Local Fault
- Idle = !(RF + LF)
You also list "yet another signal" and I strongly agree that there is no
need for any other signals. This includes Break Link, a signal for which
I cannot determine a valid purpose. The initialization process works
just fine without Break Link. Break Link seems to be an artifact of
Auto-Negotiation. Since AN is not a 10GE objective, there is no
requirement for Break Link.
Idle is issued during link initialization and continuously between
packets during normal operation in the absence of errors in the Local
and Remote DTE's as well as connecting link. The Idle stream alone, with
no handshakes, is sufficient for link initialization when both DTE's
power up/reset simultaneously. If one one DTE powers up before the
other, link initialization is a bit more complex and looks something
like this:
1) The Local Device powers up first, resets, and sends Idles;
2) Since the Link Partner is powered down, it transmits nothing;
3) The Local Device receives nothing and indicates Remote Fault to it's
Link Partner;
4) The Link Partner powers up, resets, and sends Idles;
5) The Local Device receives Idles and stops indicating Remote Fault to
it's Link Partner. The Local Device sets Link Status to OK;
6) The Link Partner may see Remote Fault or Idle first. Reset timers
should ensure that the residual Remote Fault indication doesn't cause a
problem. The Links Partner essentially sees Idle after reset. The Link
Partner sets Link Status to OK;
7) MAC frames can now flow over the link.
Note that Break Link is not needed during Link Initialization and only
serves to complicate the protocol.
However, Remote Fault serves a clear purpose during initialization.
Remote Fault is the report of absence of signal or sync at the DTE
receiver, over the transmitter of the same DTE. If you agree with the
initialization protocol above, then Remote Fault should be signaled
along with, or instead of, Idle whenever the opposite direction simplex
link is not operational.
Remote Fault is a signaling protocol. The protocol must be recognized at
the Link Partner and distinguished from the normal operational signal,
which is Idle (if you agree with the protocol above). I believe that
there is no disagreement that protocol synchronization at the receiver
is required in order to recognize any signal or message. If the Local
Device signals Remote Fault instead of Idle, the Link Partner will
either receive nothing or recognize the Remote Fault signal. "In
between" conditions, such as receiving a message on fewer than all lanes
(i.e. alignment not required), is not useful as an end-to-end signaling
protocol. The latter case denotes a non-operational link. I prefer that
Remote Fault signaling protocol be either LSS or an alternating
Idle/Sequence. The Sequence protocol I've proposed for use for 10
Gigabit Fibre Channel is an example of a an alternating Idle/Sequence
where the Sequence alternates with random AKR for random /A/ spacing
intervals.
On to Local Fault: Local Fault, as you suggest, is the condition and
signal associated with the Loss_of_Signal or Loss-of-Sync, etc.
conditions. Local Fault may be recognized and signaled up stream by any
link element including intermediate link elements such as an XGXS, PMD
retimer, WIS, etc. *** Note that this is a key decision point *** 10GE
architecture already specifies intermediate link elements. A 10GE link
may employ multiple such link elements. Each of these link elements
should be capable of both recognizing and forwarding Local Fault
conditions. If not, these elements become part of the problem rather
than the solution. The next question is whether the Local Fault signal
is in-band or out-of-band. Since pins=BAD, this seems like an
implementation issue, and Local Fault needs to be able to go over the
medium, it seems that Local Fault must at least be specified as an
in-band signal. The signaling requirements for Local Fault appear to be
identical to those of Remote Fault. Consequently, I propose that the
Local Fault signaling protocol be either LSS or an alternating
Idle/Sequence.
The most complex scenario involving Remote Fault and Local Fault that I
can envision is one where both Remote Fault and Local Fault conditions
exist for the same link, in the same direction. For the sake of
completeness, the Remote Fault and Local Fault transport protocol should
allow for the indication of all combinations of these two conditions.
Note that this scenario is not covered in the latest RF/BL proposal from
Shimon Muller.
I'd be honored to work on a common Link Initialization and Link Status
Reporting protocol presentation with you for Tampa.
Best Regards,
Rich
--
pat_thaler@xxxxxxxxxxx wrote:
>
> Ben,
>
> Link_status is not complete at the PCS. The XAUI link
> is long enough that it is possible that a marginal
> device has been attached and lock is not obtained.
> We can have the case where an XGXS connects to a
> daughter card or transceiver module and that device
> is not present. Also, as you point out, a DTE XGXS
> can function as a PCS.
>
> For indicating a local fault in the receive direction,
> this is pretty simple to cope with. For the sake of
> easing the discussion, lets say that the status line
> is a local fault line - asserted when there is loss
> of signal, loss of lock, or loss of sync. It may be
> an inband signal or an out of band signal.
>
> At each sublayer, the local fault out (going up the chain)
> should then be the OR of the local fault in (from
> the direction toward the MDI) and the internal fault
> detection (loss of signal, loss of lock, loss of sync)
> of the sublayer.
>
> If the signal is out of band, this is just an OR gate.
> If the signal is in band, then if I'm getting valid
> signal from below I send it on - if the layer below
> has detected a local fault the signal I forward will
> indicate the local fault. If I'm not getting valid
> signal from below, I send the local fault in band signal.
>
> I would suggest that we either keep local fault out of
> band all the way up the chain or we convert it from
> out of band to in band at the PCS. My preference would
> be the latter though the one concern I have is it uses
> more of our fairly limited code space to signal it simply.
>
> I have thought of one way around the code space problem.
> Right now in the Muller proposal, RF is signaled on
> all 4 lanes. One could send the RF code word on two lanes
> with K or R on the other two lanes and reuse the same
> code word for LF but put it on the other two lanes or
> across all 4 lanes (in case one is worried about lane
> mix-ups). This is a small complication of the Muller
> proposal since currently the same thing is sent on
> all 4 lanes, but perhaps no more difficult to implement
> then having two kinds of code and it preserves our
> unused code space.
>
> The other question you raise is what do we do when the
> receive link is fine but there is a problem on the
> transmit side between the MAC and the MDI. For instance
> a PHY XGXS can't get sync on the XAUI sighal. I think
> that that should also assert local fault toward the MAC.
> Identifying the exact nature of the local fault should
> be left to the MDIO registers. That still leaves the
> question of what that PHY XGXS should transmit. Candidates
> arme:
> Remote Fault
> Local Fault
> Idle
> yet another signal.
>
> If we send Remote Fault then Remote Fault's meaning becomes
> the other end of the line detects a problem. The problem
> might be in the receive or transmit side, but something
> isn't in lock over there. You have to go over to the remote
> node to narrow it down.
>
> If we send Local Fault, then Local Fault becomes more like
> Receive fault - there is a problem somewhere between the
> transmit of the other MAC and me - and Remote Fault is
> a Transmit fault - there is a problem between my MAC transmit
> and the other MAC's receive. To figure out if the "Local
> Fault" is from the other guy's Phy I have to query my
> sublayers via MDIO to see if any of them is out of lock
> and generating Local Fault. This seems better to me
> but it implies we should change the signal names.
>
> If we send Idle, then it seems a hole in our fault detection.
> Why do we bother to publish some faults to the remote node
> and leave out others?
>
> A new signal - I really want to keep this simple and our
> code space for simple signals is limited so I'd rather
> not do this one.
>
> I'll try to get a presentation ready for next week.
>
> Pat
>
> -----Original Message-----
> From: Ben Brown [mailto:bbrown@xxxxxxxx]
> Sent: Tuesday, October 31, 2000 5:46 PM
> To: 802.3ae
> Subject: Re: Link Status thoughts
>
> Pat,
>
> Would link_status be complete at the PCS? What if a
> XAUI link was used to extend the XGMII? Does this
> need any indication of the link from the PCS toward
> the MDI? What about the case of a DTE XGXS turned
> into a PCS because the PHY XGXS/PCS/PMA is really
> only a retimer? Does the DTE XGXS not care about
> signal_detect in one case but care about it in the
> other?
>
> All these cases show me a pretty confusing picture.
> I'm hoping someone can enlighten all of us next
> week :)
>
> Ben
>
> pat_thaler@xxxxxxxxxxx wrote:
> >
> > Ben,
> >
> > I've looked at your slides. I'm not sure if you intend all
> > the signals shown coming out of blocks (especially those on
> > slides 8 and 10) to be pins. If so, that is way too many
> > pins for link status.
> >
> > What I propose is one pin from the PMD to the PMA indicating
> > whether the link is good (that is, signal detect is okay on
> > all inputs) or not. One pin from the PMA to the PCS/WIS
> > indicating whether the receive link at the PMA and below
> > is okay. That would be an AND of the signal it gets from
> > the PMD and the PMA's lock signal. The WIS to PCS interface
> > is only defined as a logical interface, so it would have
> > a message defined to pass whether the link below was up -
> > the AND of the signal it gets from the PMA and its frame
> > sync acquired status.
> >
> > Pat
> >
> > -----Original Message-----
> > From: Ben Brown [mailto:bbrown@xxxxxxxx]
> > Sent: Thursday, October 26, 2000 8:44 AM
> > To: 802.3ae
> > Subject: Link Status thoughts
> >
> > Hey,
> >
> > After a discussion some of us had during the editorial
> > meeting yesterday, I thought I'd put some of our thoughts
> > down on paper. There are still quite a few questions
> > buried in these slides. Anyone with comments, please
> > feel free to share them. I'd like to have these comments
> > before the November meeting so we can put a proposal on
> > the table.
> >
> > Thanks,
> > Ben
>
> -----------------------------------------
> Benjamin Brown
> AMCC
> 2 Commerce Park West
> Suite 104
> Bedford NH 03110
> 603-641-9837 - Work
> 603-491-0296 - Cell
> 603-647-2291 - Fax
> 603-798-4115 - Home Office
> bbrown@xxxxxxxx
> -----------------------------------------
-------------------------------------------------------
Richard Taborek Sr. Phone: 408-845-6102
Chief Technology Officer Cell: 408-832-3957
nSerial Corporation Fax: 408-845-6114
2500-5 Augustine Dr. mailto:rtaborek@xxxxxxxxxxx
Santa Clara, CA 95054 http://www.nSerial.com