Buffer underflow difficulties with WAN-PHY
Hi,
After reading the UNI-PHY proposal:
> http://grouper.ieee.org/groups/802/3/ae/public/may00/bottorff_1_0500.pdf
I see a buffer underflow problem with the implementation of a SONET
WAN-PHY because of the difference between the 10 Gbit data rate on
XGMII/XAUI, and the slower (~9.2Gbit) effective rate, after SONET and
64B/66B overhead. I have found no discussion of this point in the
presentations or mail archives, so I would like to raise it now. My
apologies if this point has already been resolved.
The difficulty I see arises in the transmission of data from the PHY
toward the MAC over the XGMII/XAUI interface.
________PHY_________
/ \
~9.2Gb/s ~9.5Gb/s ~9.95
----- ------- ----- -------
| MAC |<-----|64B/66B|<-----| WIS |<------|PMA/PMD|
----- ------- ----- -------
XGMII/XAUI
10Gb/sec
Since the effective data rate on the WAN is slower than on the XGMII
interface to the MAC, the PHY must buffer some packet data before
starting transmission over the faster XGMII or XAUI interface, or it
will run out of data, i.e. buffer underflow.
The following picture illustrates the situation by showing the timing
of data transferred over the SONET and XGMII interfaces for one packet:
__576 bytes___
/ \
SONET DDDDDDD_DDDDDDD_DDDDDDD_DAAAAAAAAAAAJJJJJDIIII_IIIIII
|<--buffered data-->|
XGMII IIIIIIIIIIIIIIIIIIIDDDDDDDDDDDDDDDDDDDDDDDIIIIIIIIIII
A,J = SONET overhead bytes
D = packet data
I = idle symbols
_ = gap due to 64B/66B overhead
In order for the last byte of the packet to be transmitted over XGMII
after it is received, the transmission of the first byte must be
delayed, i.e. buffered.
Assuming the PHY is designed for a 1500 byte maximum packet size, the
PHY must buffer approximately 630 bytes for the worst case:
52 bytes for rate mismatch (~3.5% x 1500 bytes)
576 bytes for A0/A1/JO SONET gap
---
628 bytes total
52 bytes are required because of the lost bandwidth due to 64B/66B
overhead. 576 bytes are required because of the gap in data during
the transmission of the SONET A0/A1/J0 bytes, which are all bunched
together. There are a total of 3x192=576 consecutive overhead bytes.
Thus the PHY is starved for 576 byte times, and so that many
additional bytes must be accumulated before transmission of the first
byte over XGMII/XAUI.
Note that this buffering requirement has nothing to do with buffering
that may be required for different rate control methods. Rate control
addresses the buffer overflow problem for data going in the opposite
direction, i.e. from MAC to WAN.
The buffering requirement gets even worse if the PHY is to work with
jumbo packets. For example, a 32Kbyte packet would require buffering
of approximately 3.1K bytes (8% x 32K + 576).
An alternative to buffering packet data in the PHY is to allow the PHY
to transmit some kind of NULL symbol over the XGMII/XAUI interface in
the absence of data. The nulls would be discarded by the MAC. Thus
the PHY could begin sending data over the XGMII/XAUI interface without
buffering. The nulls essentially allow the data rate on the
XGMII/XAUI interface to be slowed down to the instantaneous delivery
rate from the WAN/SONET.
The following picture illustrates how NULL insertion would work:
__576 bytes___
/ \
SONET DDDDDDD_DDDDDDD_DDDDDDD_DAAAAAAAAAAAJJJJJDIIII_IIIIII
XGMII DDDDDDDNDDDDDDDNDDDDDDDNDNNNNNNNNNNNNNNNNDIIIINIIIIII
A,J = SONET overhead bytes
D = packet data
I = idle symbols
_ = gap due to 64B/66B overhead
N = null symbols
Rather than delay the transmission of the first byte of the packet, it
is sent out immediately, and when gaps in the data occur due to SONET
or 64B/66B overhead, a NULL symbol is sent over XGMII. Thus no
additional buffering is needed in the PHY. In order to maintain byte
lane alignment, NULLs would have to be inserted in multiples of four.
So, this looks pretty attractive for the PHY designer, because he can
reduce the complexity of his design. But, this gain comes potentially
at the expense of the MAC designer who might need to introduce
additional buffering in his design if he needs to have the data in
uninterrupted packets. For example, if the application is a low
latency repeater that would need to transmit packets without NULL
symbols, the repeater would need to perform buffering to eliminate the
NULLs. On the other hand, in many applications, a whole packet would
need to be buffered anyway before being used, so the buffering would
come naturally without additional complexity.
After considering the alternatives, I think that I prefer the second
solution -- the use of NULL symbols -- to address buffer underflow.
It reduces the complexity of the PHY, allows jumbo packets to work,
and in many applications completely eliminates the need for buffering
due to the data rate difference. In low latency applications,
buffering is needed, and the burden of that buffering is shifted from
the PHY to the MAC (or higher layers). But that shift is only a
break-even, not a net increase in complexity.
Juan Pineda
Bravida Corporation
+1.650.688.6416
Email:juan@xxxxxxxxxxxxxxx