RE: Link Status thoughts
My comments are preceeded by my initials in brackets: <PAT>
Pat
-----Original Message-----
From: Osamu ISHIDA [mailto:ishida@xxxxxxxxxxxxxxxxxxx]
Sent: Thursday, November 02, 2000 6:05 AM
Pat,
... <stuff deleted> ...
Therefore, I think we could agree that the Local Fault signal (or Break
Link in my term) on the receive path will be responded by the Remote
Fault signal on the transmit path at the sublayer that the RF
mechanism is implemented.
<PAT> Yes, I could agree with that. However, you seem to use BL in
a more complex way than this implies. The proposal you sent me Wednesday
sends a BL whenever a reset is occurring. If I am interpreting correctly,
it also gives a sublayer that detects a fault on the input for a direction
the choice of sending either BL or idle without any LS columns. A receiver
interprets failure of the LS heartbeat as a BL condition. I have a couple
of problems with this:
- If a device is seeing BL conditions, it has no way to determine whether
the device at the other end is having intermittent transmitter problems
or if it is just resetting frequently. (Maybe someone is just doing one
of those annoying software installs that requires bunches of reboots :^(
or are cycling power frequently.)
- I have yet to hear any good argument for why one side of the link needs
to know the other is resetting. The only case where we have cared about
communicating a reset is to run auto-negotiation. 802.3 links that don't
auto-negotiate don't have a way to send a "reset". Even if we know the
other side is resetting, we don't have a use for that information. If
one tries to use it to initiate ones own reset, then one gets into
annoying
problems specifying timers to keep reset from being an endless loop.
So, my current position is that a device should not send a special signal
when it is resetting. If at some point I get convinced that there is a
reason
that a device at one end of the link needs to tell the other end of the link
that a reset is in progress, then that indication will need to be a
different
indication the the one used for "there is a disruption in this simplex
path".
Note that Shimon, et al uses BL in yet another way. Receiving BL initiates
a reset. I'm not convinced that it is necessary to be able to ask the other
end of the link to reset, though I'm also not against providing a mechanism
for it.
So one of the fundamental issues that needs to be resolved before we
are likely to get concensus is which of the following need to be supported
by link signalling:
A) The simplex link in this pair isn't working
B) This simplex link isn't working
C) There is a fault detected internal to this physical layer
D) There is a physical fault that can't be isolated to this physical layer
E) I'm resetting
F) You should reset
I believe we should do A and B rather than C and D. MDIO management can
be used to determine if the fault that triggered A or B can be isolated
to a particular sublayer. We seem to be getting dangerously close to
concensus on this part. I'm lukewarm about supporting F and against
supporting E. If we support E or F, each one should be distinctly
detectable.
<back to Osamu's text>
I think the remaining different preferences between us on this RF/LF(BL)
issue are:
Pat:
- Relay the RF&LF status, sublayer by sublayer, by defining each
(RS/)XGXS/PCS-specific RF/LF signals.
- In Local Fault condition, LF must be sent up the receive path.
(NoLF means 'link is fine')
- Use some variety of /Z/Z/Z/Z/ Column for signaling identification.
Osamu:
- Define RF&LF(BL) mechanism only in RS. RF&LF(BL) signals be
transparent in intermediate sublayers (XGXS/PCS) in the standard.
Leave the practical instantiation point (chip) for the implementation.
- In Local Fault condition, LF(BL) need not be sent up the receive path.
NoLF means another Local Fault 'Link Signaling is NOT fine'.
Instead, when 'link is fine', NoRF&NoLF should be passed at regular
intervals as a heartbeat.
- Use some variety of /Z/D/D/D/ Column for signaling identification.
<PAT> RE: "Leave the practical instation point (chip) for the
implementation."
Any time you have:
sublayer A
sublayer B
compatability interface
sublayer C
sublayer D
then the practical instantiation point for a function is up to the
implementer as long as the function occurs on the defined side of
the compatability interface. For instance, if layer A is an RS
and layer B is a DTE XGXS without an exposed XGMII interface between
them, then a function that is defined as part of the RS could be
performed by circuitry that is mostly doing XGXS tasks. In fact, the
functions of the RS and XGXS could be all mixed together and its
none of the standards business. The important thing is that when
we look at whats going on on the compatability interface we see all
the right behaviors. This is always true and if that is what you
meant by the statement, I absolutely agrre.
On the other hand, it wouldn't be correct to leave the function
out of sublayer A and put it into sublayer C in instance above.
If functions are moved across compatability interfaces, then the
interface isn't providing compatability - a BAD thing. If the
compatability interface is optional, one can of course implement
sublayers A through D in an implementation and reorganize the
funcitonality to ones heart's content.
<back to Osamu>
I would like to add my comment on a heartbeat. In New Orleans the
main objection to this LSS nature seemed to come from the argument 'we
already have Idles for a heartbeat', resulted in Y:5, N:29 ( A:>40)
(straw poll). However, I am not yet convinced that many of them
recognized the fact that 802.3ae will have multiple intermediate
links such as XGXS-to-XGXS, PCS-to-PCS, and XGXS-to-XGXS. I could
agree that Idles are best used as a heartbeat for each intermediate
link. My argument here is that we had better adopt another heartbeat
for overall link status since this minimizes the intermediate PCS/XGXS
requirement on the standard; just producing Idles if they do not have
input sync. Neither Idle Equivalent nor BL/RF relay at each PCS/XGXS
is required. Note that this might work well even without a pin from
PMD/PMA to PCS for out-of-band LF signaling, while I myself has no
preference whether or not use a Pin here.
<PAT> The reason one wants a pin from PMD/PMA to PCS for out-of-band
LF signalling is partly because of the crosstalk problem. When a
loss of signal condition occurs, then the sensitivity of the receiver
may allow it to pick up crosstalk from the transmitter well enough
that it gets good signal from it at least some of the time. A LF
pin prevents the receiver from locking to that.
I do not understand what you mean by "neither Idle Equivalent nor
BL/RF relay at each PCS/XGXS is required." I thought your proposal
was outputing idles from a sublayer when the input was not locked.
Isn't that "Idle Equivalent"? Each sublayer in you proposal that
is receiving BL/RF on an input sends BL/RF on its output. Isn't
that "BL/RF relay"? It seems that the difference between the
proposals is that in mine a sublayer that can't lock to its
input sends out idle with BL inserted while in yours it has
a choice of doing that or sending idle without BL at the cost
of requiring sending a link status indicating a good link and
a heartbeat checker for receiving link status. Either way should
work and either can be implemented reasonably simply.
<Osamu>
2) link status mechanism by the channel_reset
I have assumed that 802.3ae MAC requires a control register bit
for resetting the Layer-1 channel; i.e. clearing the link status bit in
the Link Partner by the Local Device's STA. I am
using the term link_reset for this control register bit. Furthermore,
I have also assumed another control bit power_down that notifies the Link
Partner that the Local Device is going to shut down. If either of
these control register bit is asserted, I have designed that the Local
Device sends Break Link on the transmit path to the Link Partner.
Looking from the Link Partner's side, this is completely the same
as Local Fault signaling since both come on the receive path from the
Local Device to the Link Partner. Both are Alarm Indication Signal
(AIS) received in the Link Partner. Both should clear the overall
link status bit in the Link Partner.
That's why I am mixing up these two mechanisms; RF/LF and BL.
<PAT> Here we come to an area of disagreement starting with your
opening assumption. The 802.3ae MAC doesn't have any reset function
defined in management. There is a repeater reset, but not a MAC reset.
There are individual sublayer resets defined for MDIO devices - Phy
devices, but none for the RS or MAC. Resetting MACs has generally
been left as a proprietary function invoked locally.
Other than for auto-negotiation, there has been no need for 802.3
devices to reset their link partner or for a device to go know that
its partner is resetting. There is no 802.3 action to take when a link
partner is going to shut down. If there was such an action, it wouldn't
be the same as what one does when a noise hit causes a momentary fault.
There is no need for a device to tell a link partner that it is resetting.
There is therefore no need for some kind of BL to RF handshake of
that information. If there is a need to send the knowledge that
a device is resetting to its partner or to cause the partner to reset,
then the actions that the partner takes on receiving this information are
probably not the same as upon learning that there is a fault. Therefore,
if we need reset signalling it should have its own signal.
Until there are satisfactory reasons explained for communicating reset
information, we should not add support for it.
<Osamu>
Furthermore, this Local Fault signal (or Break Link in my term) will
eventually be responded by Remote Fault signal at the RF mechanism
in the Link Partner. This implies that the channel_reset issued by
the Local Device's STA can be acknowledged by receiving this RF.
<PAT> I don't see a use for this acknowledge protocol. Furthermore,
if you are suggesting that the RF needs to be received before the
reset can complete, then I even more strongly disagree. It would
be adding a protocol without a purpose.
<Osamu>
So, if we design that the control register bit channel_reset is
latched high until RF_rcvd, we can perform the complete reset of
the duplex channel without employing Shimon's state synchronization
process between the Local Device and the Link Partner. I have not
yet convinced that we should employ such status synchronization that
requires longer waiting time (-300ms) or unnecessary link-distance
upper-bound.
I believe that BL responded by RF is better than BL responded by BL.
<PAT> Unfortunately with confliting schedules and meeting our editor
commitments there has not been enough time spent on refining RF/BL
proposals. I think Shimon's sync process could be simplified. I like
the signalling in that proposal but would be in favor of removing
the handshaking aspects.
Regards,
Pat
Best Regards,
Osamu
At 12:55 00/11/01 -0700, pat_thaler@xxxxxxxxxxx wrote:
http://www.ieee802.org/3/10G_study/email/msg03676.html
>
> Ben,
>
> Link_status is not complete at the PCS. The XAUI link
> is long enough that it is possible that a marginal
> device has been attached and lock is not obtained.
> We can have the case where an XGXS connects to a
> daughter card or transceiver module and that device
> is not present. Also, as you point out, a DTE XGXS
> can function as a PCS.
>
> For indicating a local fault in the receive direction,
> this is pretty simple to cope with. For the sake of
> easing the discussion, lets say that the status line
> is a local fault line - asserted when there is loss
> of signal, loss of lock, or loss of sync. It may be
> an inband signal or an out of band signal.
>
> At each sublayer, the local fault out (going up the chain)
> should then be the OR of the local fault in (from
> the direction toward the MDI) and the internal fault
> detection (loss of signal, loss of lock, loss of sync)
> of the sublayer.
>
> If the signal is out of band, this is just an OR gate.
> If the signal is in band, then if I'm getting valid
> signal from below I send it on - if the layer below
> has detected a local fault the signal I forward will
> indicate the local fault. If I'm not getting valid
> signal from below, I send the local fault in band signal.
>
> I would suggest that we either keep local fault out of
> band all the way up the chain or we convert it from
> out of band to in band at the PCS. My preference would
> be the latter though the one concern I have is it uses
> more of our fairly limited code space to signal it simply.
>
> I have thought of one way around the code space problem.
> Right now in the Muller proposal, RF is signaled on
> all 4 lanes. One could send the RF code word on two lanes
> with K or R on the other two lanes and reuse the same
> code word for LF but put it on the other two lanes or
> across all 4 lanes (in case one is worried about lane
> mix-ups). This is a small complication of the Muller
> proposal since currently the same thing is sent on
> all 4 lanes, but perhaps no more difficult to implement
> then having two kinds of code and it preserves our
> unused code space.
>
> The other question you raise is what do we do when the
> receive link is fine but there is a problem on the
> transmit side between the MAC and the MDI. For instance
> a PHY XGXS can't get sync on the XAUI sighal. I think
> that that should also assert local fault toward the MAC.
> Identifying the exact nature of the local fault should
> be left to the MDIO registers. That still leaves the
> question of what that PHY XGXS should transmit. Candidates
> arme:
> Remote Fault
> Local Fault
> Idle
> yet another signal.
>
> If we send Remote Fault then Remote Fault's meaning becomes
> the other end of the line detects a problem. The problem
> might be in the receive or transmit side, but something
> isn't in lock over there. You have to go over to the remote
> node to narrow it down.
>
> If we send Local Fault, then Local Fault becomes more like
> Receive fault - there is a problem somewhere between the
> transmit of the other MAC and me - and Remote Fault is
> a Transmit fault - there is a problem between my MAC transmit
> and the other MAC's receive. To figure out if the "Local
> Fault" is from the other guy's Phy I have to query my
> sublayers via MDIO to see if any of them is out of lock
> and generating Local Fault. This seems better to me
> but it implies we should change the signal names.
>
> If we send Idle, then it seems a hole in our fault detection.
> Why do we bother to publish some faults to the remote node
> and leave out others?
>
> A new signal - I really want to keep this simple and our
> code space for simple signals is limited so I'd rather
> not do this one.
>
> I'll try to get a presentation ready for next week.
>
> Pat
>
>
> -----Original Message-----
> From: Ben Brown [mailto:bbrown@xxxxxxxx]
> Sent: Tuesday, October 31, 2000 5:46 PM
> To: 802.3ae
> Subject: Re: Link Status thoughts
>
>
>
>
> Pat,
>
> Would link_status be complete at the PCS? What if a
> XAUI link was used to extend the XGMII? Does this
> need any indication of the link from the PCS toward
> the MDI? What about the case of a DTE XGXS turned
> into a PCS because the PHY XGXS/PCS/PMA is really
> only a retimer? Does the DTE XGXS not care about
> signal_detect in one case but care about it in the
> other?
>
> All these cases show me a pretty confusing picture.
> I'm hoping someone can enlighten all of us next
> week :)
>
> Ben
>
> pat_thaler@xxxxxxxxxxx wrote:
> >
> > Ben,
> >
> > I've looked at your slides. I'm not sure if you intend all
> > the signals shown coming out of blocks (especially those on
> > slides 8 and 10) to be pins. If so, that is way too many
> > pins for link status.
> >
> > What I propose is one pin from the PMD to the PMA indicating
> > whether the link is good (that is, signal detect is okay on
> > all inputs) or not. One pin from the PMA to the PCS/WIS
> > indicating whether the receive link at the PMA and below
> > is okay. That would be an AND of the signal it gets from
> > the PMD and the PMA's lock signal. The WIS to PCS interface
> > is only defined as a logical interface, so it would have
> > a message defined to pass whether the link below was up -
> > the AND of the signal it gets from the PMA and its frame
> > sync acquired status.
> >
> > Pat
> >
> > -----Original Message-----
> > From: Ben Brown [mailto:bbrown@xxxxxxxx]
> > Sent: Thursday, October 26, 2000 8:44 AM
> > To: 802.3ae
> > Subject: Link Status thoughts
> >
> > Hey,
> >
> > After a discussion some of us had during the editorial
> > meeting yesterday, I thought I'd put some of our thoughts
> > down on paper. There are still quite a few questions
> > buried in these slides. Anyone with comments, please
> > feel free to share them. I'd like to have these comments
> > before the November meeting so we can put a proposal on
> > the table.
> >
> > Thanks,
> > Ben
> >
> > --
> > -----------------------------------------
> > Benjamin Brown
> > AMCC
> > 2 Commerce Park West
> > Suite 104
> > Bedford NH 03110
> > 603-641-9837 - Work
> > 603-491-0296 - Cell
> > 603-647-2291 - Fax
> > 603-798-4115 - Home Office
> > bbrown@xxxxxxxx
> > -----------------------------------------
-----------------------------------------
Osamu ISHIDA
NTT Network Innovation Laboratories
TEL +81-468-59-3263 FAX +81-468-55-1282