Re: [10GMMF] Agenda (draft) for March LRM
Steve,
The committee must write a response to each comment. The proposal going into next weeks meeting is that the "PROPOSED REJECT. Suggested remedy not complete" comments have that response and that that response is accepted by the committee with a technical vote. I expect the committee to pre-reviewed the response and the comments and to accept the proposed global response for those comments or propose another global response or choose to include the comments in comment review and then to propose an individual response to each one. At the opening session when we agree the agenda I will ask for clear guidance from the committee on how to use our time efficiently and for a consensus view on how we should treat the proposed reject bucket. If the group wishes to go to working group Ballot we have an obligation to the editor to plan our work so that he has the maximum time to get the edits done for the Thursday IEEE 802.3 meeting.
Also, I was attended the Fast Ethernet, Gigabit Ethernet and 10Gigabit Ethernet committees at the pint when they asked to go WG Ballot. The process these committees followed is effectively the same as the LRM process. Specifically, the editors in chief marked many comments, both technical and editorial, as Proposed Reject due to being incomplete in one form or another and asked that they be resubmitted in a more complete form or with more information at the next cycle - which would then be the first WG Ballot.
Let's take the example of 10Gigabit Ethernet. The following review cycles are recorded in the archive:
Comments received during Task Force review
D2.0
D2.1
Comments received during Working Group Ballot
D3.0
D3.1
D3.2
D3.3
D3.4
Comments received during Sponsor Ballot
D4.0
D4.1
D4.2
D4.3
D2.1 was the last draft before WG Ballot.
The advice of the Editor in Chief at the beginning of the meeting to resolve comments on D2.1 was (see: http://grouper.ieee.org/groups/802/3/ae/public/mar01/bbooth_1_0301.pdf):
This Meeting
* Make skeleton complete
- Fill technical holes
- Flesh out in Working Group (Ballot)
* Technically complete
- Fill the technical holes first
- Everything else is a minor point
- There is more ballot cycles to come...
MINIMIZE THE CHANGES!!
In a quick visual scan of the comments report at the opening of the D2.1 review meeting I found the following:
At least 10 Editorial comments Proposed reject due to incompleteness in one form or another.
Comment numbers: 33, 627, 631, 210, 220, 491, 646, 617, 701, 424,702, 283...
At least 37 Technical comments Proposed reject due to incompleteness in one form or another.
Comment numbers: 289,383,625,170,561,214,215,219,594, 634,216,218,211,217,327,562,221,481,628,595,222,506,636,304,564,480,365,186,185,451,194,198,366,459,653....
As opposed to our wording of "PROPOSED REJECT. Suggested remedy not complete" the 10Gigabit editor did a good job of wording the proposed rejects some examples are:
383: Suggested remedy is incomplete. The commenter is requested to re-submit the comment at the
next ballot with a method for indicating when the PMD is ready for MAC frame transmission.
"Suggested remedy is incomplete" with a short clarifying statement is a common response to incomplete comments but other variations I found were:
289: Editor respectfully requests that commenter re-submit this comment in the next ballot cycle, and
that the commenter indicate the incorrect cross-references.
625: This comment does not add to the technical completeness of this draft. The editor
humbly requests that the commenter re-submit this comment in the next ballot cycle as editorial
comment
366: The committee would like to see more analysis and more experimental evidence. Please
resubmit your comment against D3.0.
653: Editor will resubmit comment on next ballot.
365: Please resubmit as necessary with appropriate proposal for transmitter.
The receive sensitivity does not include Tx distortions, cable distortions and so on. It is
measured with a perfect square wave with the correct OMA.
These are all examples of issues that by your definition would be open technical issues that are being proposed as reject with resubmission at the WG Ballot.
LRM review cycles have been:
Task Force Draft Reviews
D0.1
D1.0
D1.1
WG Ballot (we hope)
D2.0
This mirrors the 10Gigabit Ethernet cycles. 10Gigabit Ethernet was a much larger body of work than LRM. So I would hope we don't have as many WG Ballots as 10Gigabit Ethernet.
As I have said we are doing the same as other projects.
RE: TR comments at TF review. There are no TR's at TF review. Only Technical and Editorial. TR's are enforced at WG ballot. But at TF review we do acknowledge that a TR means that the commenter feels very strongly and such comments should be taken seriously.
See you in Atlanta,
David
-----Original Message-----
From: owner-stds-802-3-10gmmf@IEEE.ORG
[mailto:owner-stds-802-3-10gmmf@IEEE.ORG]On Behalf Of Swanson, Steven E
Sent: 11 March 2005 21:05
To: STDS-802-3-10GMMF@LISTSERV.IEEE.ORG
Subject: Re: [10GMMF] Agenda (draft) for March LRM
David,
Re: I don't understand? This is perfectly normal process for IEEE
802.3. Please give me examples of projects where it was done was
differently? Every project I have been part of has done it this way???
I have been part of Gigabit Ethernet, 10G Ethernet and EFM and I don't
recall any of those projects where this was done. EFM went to 6 TF
ballots before going to WG ballot - there were tons of comments at every
stage (many of them with unclear remedies) but every comment had to be
addressed and resolved PRIOR to going to the next ballot, even TF
ballots and certainly before WG ballot. Here is an excerpt from
Frazier's slides at each stage - there is a common theme - "we must
craft and adopt a response to each comment." There were 66 TRs on D1.2,
81 TRs on D1.3, 59 TRs on D1.414, and 38 TRs on D1.732. I don't recall
any unresolved TRs, at least on the optics side.
Reviewing draft D1.2
* Our editors have produced P802.3ah/D1.2 (and 1.1a), using
D1.1 plus the comments that we resolved in Kauai
* The number of comments dropped off substantially
between D1.1 and D1.2
* We must craft and adopt a response to each comment
* Our editors will produce P802.3ah/D1.3 after this meeting,
based on the comment responses
* We will continue the Task Force review process
Reviewing draft D1.3
* Our editors have produced P802.3ah/D1.3, using D1.2 plus the
comments that we resolved in Vancouver
* We have received ~1053 comments on D1.3 (Thank you!)
* We must craft and adopt a response to each comment
* Our editors will produce P802.3ah/D 2 after this meeting,
based on the comment responses
* We will continue the Task Force review process
Reviewing draft D1.414
* Our editors have produced IEEE P802.3ah/D1.414 using
D1.3 plus the comments that we resolved in Dallas
* The number of comments increased substantially between
D1.3 and D1.414 (a good thing)
* We must craft and adopt a response to each comment
* Our editors will produce IEEE P802.3ah/D1.732 after this
meeting, based on the comment responses
* We will continue the Task Force review process
Let's discuss in Atlanta.
Best regards,
Steve
-----Original Message-----
From: owner-stds-802-3-10gmmf@IEEE.ORG
[mailto:owner-stds-802-3-10gmmf@IEEE.ORG] On Behalf Of David Cunningham
Sent: Friday, March 11, 2005 2:24 PM
To: STDS-802-3-10GMMF@LISTSERV.IEEE.ORG
Subject: Re: [10GMMF] Agenda (draft) for March LRM
Steve,
Your questions and concerns are good ones. I'm sure there are many
other people in the TF that need these questions answered too.
RE:
I remain puzzled by your explanation of "no open technical issues."
ANS:
At the threshold of WG Ballot the normal IEEE 802.3 definition of no
open technical issues is that there are no new functions or completely
test procedures to be added to the specification to make it complete.
As far as I can tell there are no new functions or tests mentioned in
any of the comments. All the comments are directed towards improving,
technically or editorially the functions, tests and specifications
already in the standard.
RE:
I firmly believe that there are several open technical issues on this
draft.
ANS:
By open technical issues I assume you mean that you think that some of
the specification values or test procedures within draft D1.1 may not be
correct or will need to changed. For example you may believe we cannot
support the operating range on some fiber types or you may believe that
the parameters for some of the conformance tests are not correct etc.,
To start Working Group Ballot these specifications do not have to be
absolutely correct. However, the TF must have decided, by concensus,
reasonable initial values for each parameter. It is expected that the TF
will continue to work on the specifications and that the draft will be
reviewed and improved during the Ballot process.
RE:
"Given the unanimous vote to forward D1.1 to IEEE 802.3 for preview I
must manage the meeting to maximize the likelihood of achieving Working
Group Ballot." - it appears that we have already made a decision to go
to WG ballot.
ANS:
LRM voted to send draft D1.1 to IEEE 802.3 for preview in ANTICIPATION
of asking IEEE 802.3 permission to go to Working Group Ballot. In the
closing LRM plenary session at the March meeting LRM must take a
technical vote to direct me to ask IEEE 802.3 permission to go to
working group ballot. The decision is not made until that vote is taken.
So the group still has to decide whether it wants to go to Working Group
Ballot out of the March meeting.
However, as chair, I believe I have stated my responsibility to the
group correctly. When the group decided to send D1.1 to IEEE 802.3 for
pre-review this signalled to me that the group strongly desires to go to
WG Ballot out of the March Meeting. In addition, I checked at the
January meeting that our timeline was still supported by the group, it
was. The LRM timeline states that we plan to go to WG ballot in March05.
I believe the consensus of the LRM group is as follows:
1) D1.1 is complete, no TBD's.
2) All specification values have reasonable initial values in terms of
starting to ballot the draft.
3) There are no open technical issues in terms of requiring new features
or tests.
My job as chair is to ensure that in the refining of D1.1 into D1.2 we
do nothing to reduce consensus or break the draft in terms of
completeness and lack of open technical issues (of the type I have
described). As I have said the group needs to take seriously the issue
of only making changes that undeniably IMPROVE D1.2 compared to D1.1.
RE: If it was a matter of waiting until the March meeting for the vote,
was there any need to generate a whole bunch of comments to reject?
ANS:
The comments are marked "Rejected by the Editor". At the beginning of
our comment review the LRM group will have to take a vote to accept
these as rejected comments. If the LRM group does not accept them as
rejected comments then we will have rearrange the agenda to attempt to
resolve them at this meeting.
I directed the editor to mark these comments as "Rejected by the Editor"
in the belief that, given our time constraints, it is not efficient to
attempt to resolve these incomplete comments. I also assumed that the
consensus of LRM was that draft D1.1 is complete with no open technical
issues per my previous definitions.
However, all feedback on the draft is valuable and should be welcomed.
It should not be ignored. These comments point to particular issues with
the specification that obviously concern some people and will likely
need to be addressed in WG Ballot. This is why, although I have assumed
the group will accept these comments as rejected comments, I have
include a session on the agenda to review the comments and to understand
how we can work to resolve any issues the commenter may have between
March and May.
RE:
Why didn't we just go to WG ballot out of the January meeting? It
doesn't appear that we have made any significant changes to the previous
draft. There were no TBDs in D1.1 nor did we add any new functions that
I am aware of.
ANS:
A TF must request WG Ballot at a Plenary meeting. Therefore the best the
TF could do is what we have in fact done. We have made IEEE 802.3 aware
that we are ready for WG Ballot and sent them a complete pre-view draft.
We are allowed to attempt to improve the preview draft but must show
improvements to IEEE 802.3 before we ask for WG Ballot.
We can also get ready for WG Ballot for example by understanding the
next level of issues that need to be address. The "Rejected by the
Editor" comments are a very good place to start planning for WG Ballot.
This all very normal process within IEEE 802.3.
RE:
It seems like all comments attempting to address the
"open technical issues" have been rejected.
ANS:
As I have explained there are no open technical issues with D1.1. in a
form that by past precedent should impede LRM going to WG Ballot. Also,
in my opinion none have been raised in the review of D1.1.
The comments marked "Rejected by the Editor" suggest concern that some
of the initial specifications are incorrect. However, these comments
made no specific requests for new features, functions or tests. Rather
they tended to request studies, improvements, extrapolations,
guess's.... be made to resolve suggested concerns in various
specifications, functions and tests already within the draft.
Furthermore, I would suggest that many of the requests for study to
improve particular specification values can not be completed at the
Plenary meeting. These comments are unlikely to improve D1.1 next week.
By understanding the issues we may be able to do the studies and improve
D1.2 for the May interim. These comments are not being ignored. They
are being used to develop a plan to address issues that may be important
at WG Ballot.
Unless of course the group decides to attempt to resolve them at the
meeting - which is its right.
RE:
Is the need to go to WG ballot a timeline issue? Does it make sense to
put in any comments prior to WG ballot?
ANS:
IEEE 802.3 has a tradition of driving its project to timelines. This is
generally a good thing as it ensures the work gets done in a reasonable
time and does not extend indefinitely. Obviously there are market and
customers timing needs too. We must take those seriously. This is not
an academic project or process.
As chair I am the project manager. But my master is really the
consensus of the group. The group consensus is that the timeline
matters and that it wants to go to WG Ballot in March. I have an
obligation to the group to drive a process that is most likely to result
in the outcome it has by consensus requested. Until the group provides
me with different marching orders I have no choice but to drive towards
WG Ballot. I have no problem with this as I agree with the group, in my
opinion D1.2 will be more than ready for WG Ballot.
And yes, it does make sense to give a project feedback and comments
before Working Group Ballot. How else would a group develop its complete
draft for it first WG Ballot? I would also hope that the TF members
realize they have an obligation to produce a complete draft and to
improve it by providing feedback on the preceding drafts.
RE:
I am sorry to hassle you about process but for some reason, something
just doesn't feel right to me.
ANS:
I don't understand? This is perfectly normal process for IEEE 802.3.
Please give me examples of projects where it was done was differently?
Every project I have been part of has done it this way???
Regards,
David
-----Original Message-----
From: owner-stds-802-3-10gmmf@IEEE.ORG
[mailto:owner-stds-802-3-10gmmf@IEEE.ORG]On Behalf Of Swanson, Steven E
Sent: 11 March 2005 16:13
To: STDS-802-3-10GMMF@LISTSERV.IEEE.ORG
Subject: Re: [10GMMF] Agenda (draft) for March LRM
David,
Thanks for the clarification on the requirements that we must meet
before going to WG ballot - I was planning to ask this question at the
meeting so that I could make an informed vote on whether to go to WG
ballot. However, I remain puzzled by your explanation of "no open
technical issues." I firmly believe that there are several open
technical issues on this draft. To me, it almost seems like the ballot
conducted after the January meeting was not needed because you state
that "Given the unanimous vote to forward D1.1 to IEEE 802.3 for preview
I must manage the meeting to maximize the likelihood of achieving
Working Group Ballot." - it appears that we have already made a decision
to go to WG ballot. If it was a matter of waiting until the March
meeting for the vote, was there any need to generate a whole bunch of
comments to reject? I am not trying to be an obstructionist here but I
would like to better understand the process.
Why didn't we just go to WG ballot out of the January meeting? It
doesn't appear that we have made any significant changes to the previous
draft. There were no TBDs in D1.1 nor did we add any new functions that
I am aware of. It seems like all comments attempting to address the
"open technical issues" have been rejected. Maybe I am overreacting here
since the same comments will come in against the WG ballot but again, I
am confused by the process. Is the need to go to WG ballot a timeline
issue? Does it make sense to put in any comments prior to WG ballot?
I am sorry to hassle you about process but for some reason, something
just doesn't feel right to me.
Best regards,
Steve
-----Original Message-----
From: owner-stds-802-3-10gmmf@IEEE.ORG
[mailto:owner-stds-802-3-10gmmf@IEEE.ORG] On Behalf Of David Cunningham
Sent: Thursday, March 10, 2005 9:01 PM
To: STDS-802-3-10GMMF@LISTSERV.IEEE.ORG
Subject: [10GMMF] Agenda (draft) for March LRM
Dear IEEE 802.3aq,
Attached is the draft agenda for our D1.1 review meeting.
Given the project timeline the primary goal of next weeks meeting is to
gain permission from IEEE 802.3 to go to Working Group ballot. According
to the IEEE802.3 operating rules the requirements we must meet are the
following:
"2.8.2 Draft Standard Balloting Requirements
Before a draft is submitted to WG letter ballot it shall in addition
have met the following requirements:
a) It must be complete with no open technical issues.
b) It must be made available for pre-view by the membership by the
Monday prior to the plenary week. If any changes are made to the draft
after the draft was made available for pre-view the textual changes
shall be presented for review during the closing plenary immediately
prior to the vote for approval to go to WG ballot.
c) It must be formatted according to the IEEE style selected by the WG
Chair. This style will be selected to minimize the editorial work
required for publication of the draft.
d) It must be approved for submittal to WG ballot at the 802.3 WG
closing plenary."
Given the unanimous vote to forward D1.1 to IEEE 802.3 for preview I
must manage the meeting to maximize the likelihood of achieving Working
Group Ballot. Therefore, I have arranged the agenda so that we complete
comment review on the non-rejected technical comments and as many
editorial comments as possible on the first day of the IEEE 802.3aq
meeting. This will provide our editor with the maximum amount of time to
make the edits to the document for the Thursday review of it by IEEE
802.3.
I have reviewed the comments and submitted presentations to confirm that
we do not need to hear the submitted presentations to be able to resolve
the non-rejected comments. We do however need to hear them before we
discuss the rejected comments. Therefore, most of Wednesday is devoted
to presentations and to reviewing the rejected comments. The
presentations have a lot in common with the issues raised in the
rejected comments and so I suggest we go through them first. The
rejected comments will likely re-emerge in our first Working Group
Ballot. We must review them, understand them, and agree plans to
resolve them for the May interim meeting.
To achieve permission for Working Group Ballot in March we must be very
disciplined and make only precise changes to the draft. For IEEE 802.3
to allow us to go to WG ballot rough rules-of-thumb would be as follows:
1) D1.2 should have many pages of the draft per D1.1.
2) Pages with edits should have only a few lines or table entries
changed.
3) We should give priority to feedback from the IEEE 802.3 pre-viewers
not in our group; after all they are our masters.
4) We should do nothing that might reduce the consensus on D1.1 within
LRM.
5) Complex changes should be kept for WG Ballot.
To go to Working Group ballot we need a complete draft not an absolutely
perfect or correct draft. The draft is complete, some specifications
and tests need fine-tuning, some more simulations need to be done. This
is normal for this stage of the process.
Regarding the IEEE 802.3 requirement of "no open technical issues": if
we have no TBD's, no new functions to add, and our comments are aimed at
improving specifications or tests already within the draft, then we have
no open technical issues. As far as I can see all the comments we
received, even the rejected ones, could not be portrayed as open
technical issues.
I will end the meeting at 10:30 AM on Thursday. This is to allow Nick
and myself some preparation time to finalize the report we have to give
to IEEE 802.3 on Thursday afternoon.
See you in Atlanta,
David