Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Confusing inconsistencies between stages of the REC track for documents and for amendments #938

Open
frivoal opened this issue Oct 25, 2024 · 8 comments

Comments

@frivoal
Copy link
Collaborator

frivoal commented Oct 25, 2024

Foreword:
This issue is an attempt at simplifying the Process of REC maintenance using amendments. This is orthogonal to, and doesn't address:

These will still need to be looked into separately. But I think the proposal I have here would still be a meaningful step forward in addressing the general confusion surrounding this part of the process, as expressed in #700 and in most discussions we have about this part of the Process.


During Process 2020, when the notion of Amendments was introduced, we tried to make working with Amendment more lightweight than the full REC track, and instead of having the equivalent of the 4 stages ((FP)WD->CR->PR->REC) for the amendments, we collapsed in into 3: Candidate Amendments->Proposed Amendments->REC

However, having a different number of stages also meant that the conditions to be fulfilled along the way couldn't be attached to the exact same stages. So we re-giggled them so that in order to reach REC, all the same requirements would be fulfilled, but in a somewhat different order, which we attempted to make simpler / more lightweight.

Even though we successfully did preserve all the requirements of a REC before amendments could be folded into the REC, the result ended up being confusing for several reasons:

  • the entry/exit criteria and associated verification for a Candidate Amendment are not the same as those for a Candidate Recommendation.
  • the entry/exit criteria and associated verification for a Proposed Amendment are not the same as those for a Proposed Recommendation.
  • For the purposes of the Patent Policy, unlike a Candidate Recommendation, a Candidate Amendment is not a Patent Review Draft, but rather a Working Draft
  • The AC review isn't the last stage before folding into the REC, so it cannot serve as a verification that everything else has been done correctly.

Since then, we have retired Proposed Recommendation from the REC track. So now, both the document Process and the current Amendment process have the same number of stages, but they're still not the same stages, and the criticism above still applies.

However, the progression that we now have for Documents is a lot simpler than the one for Amendments, and in line with the traditional way of doing things. I think we should align the Amendment process with it:

  • Amendments should be introduced as Draft Amendments (not Candidate Amendments)
  • the combination of the REC with the Draft Amendments included should count as a Working Draft for the purposes of the Patent Policy (just as Candidate Amendments currently do)
  • Promoting a Draft Amendment to Candidate Amendment should require the same criteria as going to CR (wide review, issues addressed, verified by a transition request), and when successful, should trigger a patent exclusion opportunity (just like CRs do)
  • the combination of the REC with the Candidate Amendments included should count as a Patent Review Draft for the purposes of the Patent Policy (like a CR does)
  • Normatively folding a Candidate Amendment into a REC should require the same criteria as taking a CR to REC: Implementation experience and issues addressed, verified by a transition request, followed by an AC-Review

The graphic below describes, in its middle column, the REC track for documents. The left column is the current amendment Process, which is visibly very different. The right column is the alternative amendment process I am suggesting here. I hope this makes it clear that anyone familiar with the REC track progression will have a much more intuitive time with the proposal than with the current approach. (This graph omits some details, including the fact that you can iterate at some stages before moving forward; this would make the graph more busy without adding information relevant to the question at hand.)

graph TB;

t1[Current Amendment Process]@{shape: doc} ~~~ CA-l
CA-l(["**Candidate** Amendment"])--->|Group Decision| PA([**Proposed** Amendment]) -->|Automatic| excl-A-l[[Call for Exclusion]]
PA -->|Automatic| AC-rev-A-l
subgraph lc[Last Call for Review of Proposed Amendments]
  excl-A-l 
  AC-rev-A-l
end
excl-A-l --> merge-2
AC-rev-A-l[[AC-Review]] -->|W3C Decision| merge-2@{shape: f-circ}
merge-2-->|Group Decision| CA-check-l[[
Transition request:
wide review
issues formally addressed
implementation experience
]]-->|Team Verification| REC-A-l([Fold into **W3C Recommendation**])


t2[Document Process]@{shape: doc} ~~~ WD
WD([Working **Draft**])-->|Group Decision| WD-check[[
Transition request:
wide review
issues formally addressed
]]
WD-check-->|Team Verification| CR([**Candidate** Recommendation]) -->|Automatic| excl[[Call for Exclusion]]
excl  -->|Group Decision| CR-check[[
Transition request:
implementation experience
issues formally addressed
]]-->|Team Verification| AC-rev[[AC-Review]]-->|W3C Decision| REC([**W3C Recommendation**])

t3[Suggested Amendment Process]@{shape: doc} ~~~ DA
DA(["**Draft** Amendment"])-->|Group Decision| DA-check[[
Transition request:
wide review
issues formally addressed
]]
DA-check-->|Team Verification| CA([**Candidate** Amendment]) -->|Automatic| excl-A[[Call for Exclusion]]
excl-A -->|Group Decision| CA-check[[
Transition request:
implementation experience
issues formally addressed
]]-->|Team Verification| AC-rev-A[[AC-Review]]-->|W3C Decision| REC-A([Fold into **W3C Recommendation**])

CR:::PP-PRD
CA:::PP-PRD
CA-l:::PP-WD
PA:::PP-PRD
REC:::PP-PRD
REC-A:::PP-PRD
REC-A:::PP-REC
REC-A-l:::PP-REC
REC-A-l:::PP-PRD
REC:::PP-REC
WD:::PP-WD
DA:::PP-WD
classDef PP-WD fill:#7cf,stroke:#050
classDef PP-PRD fill:#7fc,stroke:#050
classDef PP-REC stroke-width:4px;

REC ~~~ Legend
subgraph Legend [Patent Policy States]
    direction TB
    PWD([Working Draft]):::PP-WD
    PRD([Patent Review Draft]):::PP-PRD
end
Loading
For the curious: same graphic, with iterations at the same stage and regressions included
graph TB;

t1[Current Amendment Process]@{shape: doc} ~~~ CA-l
CA-l(["**Candidate** Amendment"])--->|Group Decision| PA([**Proposed** Amendment]) -->|Automatic| excl-A-l[[Call for Exclusion]]
PA -->|Automatic| AC-rev-A-l
subgraph lc[Last Call for Review of Proposed Amendments]
  excl-A-l
  AC-rev-A-l
end
excl-A-l --> merge-2
AC-rev-A-l[[AC-Review]] -->|W3C Decision| merge-2@{shape: f-circ}
merge-2-->|Group Decision| CA-check-l[[
Transition request:
wide review
issues formally addressed
implementation experience
]]--->|Team Verification| REC-A-l([Fold into **W3C Recommendation**])
merge-2 .->|Revise| PA & CA-l

t2[Document Process]@{shape: doc} ~~~ WD
WD([Working **Draft**])-->|Group Decision| WD-check[[
Transition request:
wide review
issues formally addressed
]]
WD-check-->|Team Verification| CR([**Candidate** Recommendation]) -->|Automatic| excl[[Call for Exclusion]]
excl -->merge-3@{shape: f-circ} -->|Group Decision| CR-check[[
Transition request:
implementation experience
issues formally addressed
]]-->|Team Verification| AC-rev[[AC-Review]]-->|W3C Decision| REC([**W3C Recommendation**])
merge-3 .->|Revise| WD-check & WD

t3[Suggested Amendment Process]@{shape: doc} ~~~ DA
DA(["**Draft** Amendment"])-->|Group Decision| DA-check[[
Transition request:
wide review
issues formally addressed
]]
DA-check-->|Team Verification| CA([**Candidate** Amendment]) -->|Automatic| excl-A[[Call for Exclusion]]
excl-A -->merge-4@{shape: f-circ} -->|Group Decision| CA-check[[
Transition request:
implementation experience
issues formally addressed
]]-->|Team Verification| AC-rev-A[[AC-Review]]-->|W3C Decision| REC-A([Fold into **W3C Recommendation**])
merge-4 .->|Revise| DA-check & DA

CR:::PP-PRD
CA:::PP-PRD
CA-l:::PP-WD
PA:::PP-PRD
REC:::PP-PRD
REC-A:::PP-PRD
REC-A:::PP-REC
REC-A-l:::PP-REC
REC-A-l:::PP-PRD
REC:::PP-REC
WD:::PP-WD
DA:::PP-WD
classDef PP-WD fill:#7cf,stroke:#050
classDef PP-PRD fill:#7fc,stroke:#050
classDef PP-REC stroke-width:4px;

REC ~~~ Legend
subgraph Legend [Patent Policy States]
    direction TB
    PWD([Working Draft]):::PP-WD
    PRD([Patent Review Draft]):::PP-PRD
end
Loading
@fantasai
Copy link
Collaborator

fantasai commented Oct 25, 2024

I think aligning the terminology between the two makes sense, and making it possible to work an amendment through the same steps as REC track also makes sense...

But I think this is too heavyweight for simple corrections. There needs to be some way to batch things into fewer stages and fewer transitions.

@frivoal
Copy link
Collaborator Author

frivoal commented Oct 28, 2024

I'm not sure where you draw that conclusion from. Compared to the current amendment process, assuming linear progression:

  • The ability to batch changes together is unchanged
  • the number of calls for exclusions is unchanged
  • the number of AC reviews / W3C decisions is unchanged
  • the number of Group decision is unchanged
  • the number of things the Team has to verify is unchanged

The only thing that increases is that the Team verification is split into two, like on the REC track, where wide review needs and issues having been addressed to be verified before entering the Candidate stage (currently called Proposed) rather than after. Aside from that (which I'd argue is the correct thing to do anyway), assuming linear progression, the workload is identical.

If we assume that an amendment will be iterated upon a few times before reaching its final maturity, then the story is arguably even better. In either case, iterating at the level of maturity (currently called Candidate, would be Draft) has no administrative cost, and that isn't changing. But under the current process, iterating on an amendment at the second level of maturity (currently called Proposed, would be Candidate) triggers an AC Review every time, which is rather heavy, while under the proposed approach, it would instead trigger a Team verification every time, which is much lighter, and keep the AC Review as the final step, intended to happen only once, after implementation experience is demonstrated.

So I think the assertion that this is too heavy is unfounded. It it approximately the same as the current approach, and arguably lighter if you iterate. And much less confusing since the sames steps are called the same thing. And with AC-Review as the last step, as it should be.

If we really want to cut steps, I suppose we could keep almost what I suggested, yet remove the team verification between Draft Amendment and Candidate Amendment (and move the requirement to verify wide review to the later transition call).

Click to see what this looks like as a graph
graph TB;

t3[Yet another possible Amendment Process]@{shape: doc} ~~~ DA
DA(["**Draft** Amendment"])-->|Group Decision| CA([**Candidate** Amendment]) -->|Automatic| excl-A[[Call for Exclusion]]
excl-A -->|Group Decision| CA-check[[
Transition request:
wide review
implementation experience
issues formally addressed
]]-->|Team Verification| AC-rev-A[[AC-Review]]-->|W3C Decision| REC-A([Fold into **W3C Recommendation**])

CA:::PP-PRD
REC-A:::PP-PRD
REC-A:::PP-REC
DA:::PP-WD
classDef PP-WD fill:#7cf,stroke:#050
classDef PP-PRD fill:#7fc,stroke:#050
classDef PP-REC stroke-width:4px;

REC-A ~~~ Legend
subgraph Legend [Patent Policy States]
    direction TB
    PWD([Working Draft]):::PP-WD
    PRD([Patent Review Draft]):::PP-PRD
end
Loading

The downside (which is already true of amendments in the current Process, but I think it's bad) is that this means that things can be called Candidate or Proposed without any entry criteria other than group consensus, which is not the case on the normal REC track. I think that's a bug to fix, not a feature to keep.

@fantasai
Copy link
Collaborator

The downside (which is already true of amendments in the current Process, but I think it's bad) is that this means that things can be called Candidate or Proposed without any entry criteria other than group consensus, which is not the case on the normal REC track. I think that's a bug to fix, not a feature to keep.

I think that's appropriate for corrections. Wide review needs to be appropriately wide, and invoking all of the machinery that's needed for a new feature or a new spec for a handful of corrections is unnecessary work for everyone.

@frivoal
Copy link
Collaborator Author

frivoal commented Oct 29, 2024

I can see that for corrections. What about additions?

@fantasai
Copy link
Collaborator

fantasai commented Nov 1, 2024

I'm fine with requiring the full stack for additions. It seems appropriate.

@frivoal
Copy link
Collaborator Author

frivoal commented Nov 2, 2024

I that case, I think we should keep the Team verification between Draft amendments and Candidate amendments, but indicate that the expectations for wide review depend on the nature of the changes, and may be waived for corrections, especially when they're not extensively modifying large swaths of the spec

@frivoal
Copy link
Collaborator Author

frivoal commented Nov 12, 2024

After off-line discussion with @fantasai, here's where we stand:

Goals:

  • AC Review of an amendment should be the last thing before it's folded into the REC (otherwise it cannot catch mistakes).
  • Words like "Candidate" "Draft" or "Proposed" should not have different meanings when applied to an amendment vs a spec.
  • Amending a REC must not break any requirements of being a REC.

Proposal:

  • Define "Draft" and "Candidate" amendment statuses as per the first comment in this issue.
  • Draft amendments just need WG approval to publish.
  • Candidate amendments need wide review and Team verification to publish, and must have addressed any issues raised against the amendment. (Just like WD → CR.) WGs should batch these to avoid excessive load on the Team and Member legal teams.
  • AC Review is called on candidate amendments, which must demonstrate implementation experience, after Team verification. (Just like CR → REC.)
  • For simple corrections and adjustments, Team verifications before and after the Candidate phase can be collapsed: if all criteria are already fulfilled, the WG can simultaneously request both Candidate status and AC Review to progress to REC.
  • "Wide review" for an amendment is proportional to its impact: fixing an error in an algorithm can be reviewed by just the WG and the Team verifier; restructuring a network request protocol algorithm probably needs security and implementer review but not accessibility review; etc. The Team verifier is responsible for judging this, just as for CRS update requests.

@frivoal
Copy link
Collaborator Author

frivoal commented Dec 23, 2024

removing "Agenda+" for now. This is absolutely something I want to pursue, but not in the narrow time window that remains for Process 2025.

@frivoal frivoal removed the Agenda+ Marks issues that are ready for discussion on the call label Dec 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants