| Thread Links | Date Links | ||||
|---|---|---|---|---|---|
| Thread Prev | Thread Next | Thread Index | Date Prev | Date Next | Date Index |
|
(to: Pat Thaler <pthaler@xxxxxxxxxxxx>; STDS-802-3-DMLT@xxxxxxxxxxxxxxxxx; STDS-802-1-L@xxxxxxxxxxxxxxxxx) Folks, It seems like the IEEE 802 reflectors are removing “CCs”. I see the same email on both reflectors. Trying again so both 802.1 and 802.3 interested people can see Pat’s response. It seems to me that there probably isn’t a simple answer that’s best for everything. In the short term are we just going to document the impact of preemption on the
PFC transmission latency? Regards Peter _______________________________________________
Peter Jones Cisco Systems
Principal Engineer 3600 Cisco Way
ENG Switching Software San Jose, CA, 95134 USA
Tel: +1 408 525 6952 Fax: +1 408 527 4698
Email: petejone at cisco.com
Twitter: @petergjones _______________________________________________
From: Pat Thaler [mailto:pthaler@xxxxxxxxxxxx]
(Peter, you email started +Tony and 802.1, but I don’t see their addresses in the to or cc.) Yes Denis, I realize that in the general case having PFC in the non-Express path could mean an unbounded delay in sending the PFC MAC Control frame. But if PFC MAC Control frames are sent in the express path, they could delay express traffic by more than 84 bytes because PFC can be enabled for multiple priorities so one could have
·
Priority x passes its high water threshold (or was previously paused and drops below its low water threshold) which triggers a PFC MAC Control primitive
·
Transmission of a PFC MAC Control frame to pause (or unpause) priority x ·
Priority y passes its high water threshold (or was paused and drops below low water threshold) triggering another PFC MAC Control primitive ·
Transmission of a PFC MAC Control frame to pause (or unpause) priority y The delay of 139 octet times was too much for some TSN use cases. Even 84 octets which isn’t that much smaller is probably an issue for some. The hold primitive was created to get the delay waiting for a preempted frame to complete to
zero by preempting before scheduled traffic arrives. Here we have a case where PFC can cause a delay of 186 octets.
An unbounded delay in sending the pause would be unacceptable for most uses of PFC. E.g. when PFC is being used to provide no drop operation for a protocol like FCoE which was designed to run over a flow controlled network with no congestion
drop. A delay in transmitting scheduled traffic is unacceptable in many 802.1Qbu scheduled traffic use cases.
Therefore, there is a conflict between these two capabilities in some cases. There may be some TSN links where one cannot use PFC to get no drop behavior on some priorities while meeting scheduled traffic latency objectives on other priorities.
This is not necessarily a problem – PFC was generally designed to enable some protocols that are used in the data center. For example, one is unlikely to be running FCoE on an automotive network or on the manufacturing floor in an industrial machine. There may be other networks where the scheduled express traffic is designed with frequent enough breaks to allow PFC MAC Control frames on the premptable path to be sent in the breaks between express traffic. Such breaks in scheduled
express traffic would have to be greater than a max packet time plus inter packet time to allow a frame in progress to complete and the MAC Control frame to be sent. For no drop in that case, one would need additional buffer headroom equal to the scheduled
traffic schedule windows. There may also be networks where express traffic can tolerate 84 octets or even 84 octet times * the number of PFC priorities of delay so that the PFC frames can go on the express path. On the higher speed networks where protocols like
FCoE are generally used, 84 octets take a lot less time than on a 100 Mb/s or 1 Gb/s link.
Regards, Pat From: Peter Jones (petejone) [mailto:petejone@xxxxxxxxx]
(+Tony & 802.1) Hmmm, interesting. As an aside, taking one step back, TSN in general is addressing some of the same “no network drop” cases that PFC states it addresses (from 36.1.1 PFC Overview, “PFC
enables to not discard frames due to congestion for protocols that require this property.”). That being said, I assume that prior to IET, PFC messages (as a M_CONTROL.request) are considered for transmission as “more important” than anything in the normal queueing/transmission
selection process, and so the worst case delay in sending a message is an MTU work of transmission time.
I agree that we need to figure this out, and I’m wondering if this problem is only relevant to PFC, or are there other “control messages” that have expectation for only
have an MTU’s work of delay time. If we say that PFC uses the express MAC, then I think that makes the network scheduling very complex because you have to estimate how many PFCs need to be sent. I’m
not a PFC expert, but what does it say about the max rate/min time between PFC messages? If we can push PFC though the express channel, it probably improves PFC performance at the cost of scheduled express traffic. Regards Peter _______________________________________________
Peter Jones Cisco Systems
Principal Engineer 3600 Cisco Way
ENG Switching Software San Jose, CA, 95134 USA
Tel: +1 408 525 6952 Fax: +1 408 527 4698
Email: petejone at cisco.com
Twitter: @petergjones _______________________________________________
From: Beaudoin, Denis [mailto:dbeaudoin@xxxxxx]
The only concern for PFC frames on the non preempted path is that it now increases the receiver run out by the maximum preempted frame since a preempted frame can now delay the PFC frame, this delay allows the
remote to send more regular traffic. It is even worse if a single max frame is preempted multiple times before the PFC frame is sent. My coworker pointed out that since a preempted frame can be preempted multiple times and each preemption could be a burst
of preempted frames, than the run out was not fixed and could be huge! If the PFC frame was on the preempted channel, the preempted traffic would only be delayed by max of 84 byte times. Regards Denis Beaudoin DMTS Texas Instruments
dbeaudoin@xxxxxx
W: 214-480-3287/77 M: 214-475-9193 From: Pat Thaler [mailto:pthaler@xxxxxxxxxxxx]
I recall any discussion in the task group of that and I’m not sure there is a uniform answer.
My own view, there will be a lot of cases where PFC and express traffic are not used at the same time. (I’d prefer to use express to describe the traffic on the path that can preempt – TSN is a broader term that includes uses of scheduled
traffic or the credit based shaper without preemption.) If they are both being used, in the most time sensitive use cases (e.g. we created the hold primitive because some found the possible 139 octet worst case delay before preempting too long) , one wouldn’t want sending a PFC to delay sending
a scheduled frame. On the other hand, in less extreme time sensitive use cases, it may be desirable to put PFC frames on the express path. Something to discuss in January. Regards, Pat From: Beaudoin, Denis [mailto:dbeaudoin@xxxxxx]
Typo… Regards Denis Beaudoin DMTS Texas Instruments
dbeaudoin@xxxxxx
W: 214-480-3287/77 M: 214-475-9193 From: Beaudoin, Denis
Ludwig, looking at the IET did we decide where 802.1Qbb pause frames are sent?
That is are pause frames considered TSN frames or non-TSN frames? The reason I ask is that if pause frames are TSN frames, then Rx run out could be smaller than today, but assuming that TSN may not be supported by the link partner than what is required today is a minimum. But if pause frames are non-TSN frames, then the pause run out is increased by the largest TSN frame plus IPGs times two plus some slop! If TSN frames can be 2000bytes, then this increases the Rx runnout requirements
by 4000+ bytes. Regards Denis Beaudoin DMTS Texas Instruments
dbeaudoin@xxxxxx
W: 214-480-3287/77 M: 214-475-9193 From: Winkel, Ludwig [mailto:ludwig.winkel@xxxxxxxxxxx]
Dear Colleagues: This is the announcement of the review of the IEEE P802.3r Interspersing Express Traffic (IET) Task Force (TF) draft amendment D1.0. Instructions for accessing the draft and submitting comments follow this announcement. IEEE P802.3br is an amendment to IEEE Std 802.3-2012 which is available through the “Get 802” program at the URL: http://standards.ieee.org/getieee802/802.3.html
The Project Authorization Request for this amendment is posted at: http://www.ieee802.org/3/br/P802d3br_PAR.pdf The Five Criteria Responses for this amendment are posted at: http://www.ieee802.org/3/br/8023-DMLT-SG-1311-Winkel-5C-v2.3.pdf The Objectives for this amendment are posted at: http://www.ieee802.org/3/br/8023-DMLT-SG-1309-Winkel-Objectives-v2.3.pdf Other related documents are posted at: http://www.ieee802.org/3/br/index.html ___________________________________________ REVIEW SCOPE: The scope of the review is the entire IEEE P802.3br D1.0 Draft. REVIEW OPEN: Thursday, 27th, November 2014 REVIEW CLOSE: Monday, 1st, January 2015 11:59pm AoE ------------------------------------------------------------------------- HOW TO ACCESS THE DRAFT The draft is posted for Task Force review purposes only, and neither the draft nor access information should be copied or redistributed to others in violation of document copyrights. Draft 1.0 should be used for commenting and may be downloaded from: URL:
http://grouper.ieee.org/groups/802/3/br/private/01_TF_Drafts/P8023br-D1.00p0.pdf Username and password: distributed at meetings The document is posted in Adobe "pdf" document format and can be downloaded and printed if desired. If you do not have Adobe Acrobat Reader, it is available for free downloading from Adobe at: URL:
http://www.adobe.com/prodindex/acrobat/readstep.html _______________________________________ HOW TO SUBMIT COMMENTS Acceptable comment types: E = Editorial ER = Editorial Required (an Editorial comment that the commenter feels very strongly about) T = Technical TR = Technical Required (a Technical comment that the commenter feels very strongly about) For further information regarding comments, please see
http://www.ieee802.org/3/bj/public/may12/diab_01_0512.pdf 1. Prepare your comments in one of the following two ways: a. You are STRONGLY REQUESTED to use the comment tool in the TF private area at the URL: URL:
http://ieee802.org/3/WG_tools/filemaker/index.html Please download and read the Comment Tool Tutorial: http://www.ieee802.org/3/bj/public/may12/diab_01_0512.pdf To use the tool, extract all components into a folder and do not move any component from that folder as this will cause the executable to not work correctly. The tool is: Comment_Tool_Generic_Solution.exe The comment tool runs under Microsoft Windows. It has also been run in VM environments on other platforms. b. To submit comments in text (ASCII) form, please use the form below. This will make it possible for the editor to properly record and track all submissions. Make as many copies of the template as necessary and submit as an ASCII text
file or as part of your e-mail. PLEASE NOTE: The ASCII method involves a manual transcription, hence you are recommended to check your comments to ensure accurate transcription when the comment file is made available. Please use the comment tool above if possible. ------CUT AND PASTE TEXT BELOW THIS LINE ONLY PLEASE-------- CommentID: (Leave Blank) CommenterName: CommenterEmail: CommenterPhone: CommenterCellPhone: CommenterCompany: Clause: Subclause: Page: Line: CommentType: (E, ER, T, or TR) Comment: CommentEnd: SuggestedRemedy: RemedyEnd: --------------------end comment template------------------ 2. Send your comments to:
pthaler@xxxxxxxxxxxx, CC Ludwig.Winkel@xxxxxxxxxxx ,
mike@xxxxxxxxxxxxxxx and paste into the subject line: IEEE P802.3br IET Task Force D1.0 comments _____________________________________________________________ Thank you for your participation in this review and careful review of the draft. Pat Thaler, Broadcom Editor, IEEE P802.3br Task Force Ludwig Winkel, Siemens Chair, IEEE P802.3br interspersing Express Traffic Task Force
http://www.ieee802.org/3/br/index.html Mit freundlichen Gruessen / With best regards,
Chair of the IEEE P802.3br Task Force Vice-chair of IEC SG8 Convenor of IEC SC65B/WG16 Convenor of IEC SC65C/WG17 Convenor of IEC SC65C/MT9 Convenor of CENELEC TC65X/WG1 Mail: Siemens AG , PD TI FC
|