Skip to content

Commit 3d46724

Browse files
k8s-ci-robotk8s-ci-robot
authored
Merge pull request #5242 from deads2k/production-quality
KEP-5241: Beta Feature Gate Promotion Requirements
2 parents 2dca874 + 40e5daf commit 3d46724

File tree

4 files changed

+276
-1
lines changed

4 files changed

+276
-1
lines changed

‎keps/NNNN-kep-template/README.md

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -385,13 +385,23 @@ Below are some examples to consider, in addition to the aforementioned [maturity
385385
- Gather feedback from developers and surveys
386386
- Complete features A, B, C
387387
- Additional tests are in Testgrid and linked in KEP
388+
- More rigorous forms of testing—e.g., downgrade tests and scalability tests
389+
- All functionality completed
390+
- All security enforcement completed
391+
- All monitoring requirements completed
392+
- All testing requirements completed
393+
- All known pre-release issues and gaps resolved
394+
395+
**Note:** Beta criteria must include all functional, security, monitoring, and testing requirements along with resolving all issues and gaps identified
388396
389397
#### GA
390398
391399
- N examples of real-world usage
392400
- N installs
393-
- More rigorous forms of testing—e.g., downgrade tests and scalability tests
394401
- Allowing time for feedback
402+
- All issues and gaps identified as feedback during beta are resolved
403+
404+
**Note:** GA criteria must not include any functional, security, monitoring, or testing requirements. Those must be beta requirements.
395405
396406
**Note:** Generally we also wait at least two releases between beta and
397407
GA/stable, because there's no opportunity for user feedback, or even bug reports,

‎keps/prod-readiness/sig-architecture/5241.yaml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
kep-number: 5241
2+
stable:
3+
approver: "@johnbelamaric"

‎keps/sig-architecture/5241-beta-featuregate-promotion-requirements/README.md

Lines changed: 231 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,231 @@
1+
<!--
2+
**Note:** When your KEP is complete, all of these comment blocks should be removed.
3+
4+
To get started with this template:
5+
6+
- [ ] **Pick a hosting SIG.**
7+
Make sure that the problem space is something the SIG is interested in taking
8+
up. KEPs should not be checked in without a sponsoring SIG.
9+
- [ ] **Create an issue in kubernetes/enhancements**
10+
When filing an enhancement tracking issue, please make sure to complete all
11+
fields in that template. One of the fields asks for a link to the KEP. You
12+
can leave that blank until this KEP is filed, and then go back to the
13+
enhancement and add the link.
14+
- [ ] **Make a copy of this template directory.**
15+
Copy this template into the owning SIG's directory and name it
16+
`NNNN-short-descriptive-title`, where `NNNN` is the issue number (with no
17+
leading-zero padding) assigned to your enhancement above.
18+
- [ ] **Fill out as much of the kep.yaml file as you can.**
19+
At minimum, you should fill in the "Title", "Authors", "Owning-sig",
20+
"Status", and date-related fields.
21+
- [ ] **Fill out this file as best you can.**
22+
At minimum, you should fill in the "Summary" and "Motivation" sections.
23+
These should be easy if you've preflighted the idea of the KEP with the
24+
appropriate SIG(s).
25+
- [ ] **Create a PR for this KEP.**
26+
Assign it to people in the SIG who are sponsoring this process.
27+
- [ ] **Merge early and iterate.**
28+
Avoid getting hung up on specific details and instead aim to get the goals of
29+
the KEP clarified and merged quickly. The best way to do this is to just
30+
start with the high-level sections and fill out details incrementally in
31+
subsequent PRs.
32+
33+
Just because a KEP is merged does not mean it is complete or approved. Any KEP
34+
marked as `provisional` is a working document and subject to change. You can
35+
denote sections that are under active debate as follows:
36+
37+
```
38+
<<[UNRESOLVED optional short context or usernames ]>>
39+
Stuff that is being argued.
40+
<<[/UNRESOLVED]>>
41+
```
42+
43+
When editing KEPS, aim for tightly-scoped, single-topic PRs to keep discussions
44+
focused. If you disagree with what is already in a document, open a new PR
45+
with suggested changes.
46+
47+
One KEP corresponds to one "feature" or "enhancement" for its whole lifecycle.
48+
You do not need a new KEP to move from beta to GA, for example. If
49+
new details emerge that belong in the KEP, edit the KEP. Once a feature has become
50+
"implemented", major changes should get new KEPs.
51+
52+
The canonical place for the latest set of instructions (and the likely source
53+
of this file) is [here](/keps/NNNN-kep-template/README.md).
54+
55+
**Note:** Any PRs to move a KEP to `implementable`, or significant changes once
56+
it is marked `implementable`, must be approved by each of the KEP approvers.
57+
If none of those approvers are still appropriate, then changes to that list
58+
should be approved by the remaining approvers and/or the owning SIG (or
59+
SIG Architecture for cross-cutting KEPs).
60+
-->
61+
# KEP-5241: Beta Feature Gate Promotion Requirements
62+
63+
<!--
64+
This is the title of your KEP. Keep it short, simple, and descriptive. A good
65+
title can help communicate what the KEP is and should be considered as part of
66+
any review.
67+
-->
68+
69+
<!--
70+
A table of contents is helpful for quickly jumping to sections of a KEP and for
71+
highlighting any additional information provided beyond the standard KEP
72+
template.
73+
74+
Ensure the TOC is wrapped with
75+
<code>&lt;!-- toc --&rt;&lt;!-- /toc --&rt;</code>
76+
tags, and then generate with `hack/update-toc.sh`.
77+
-->
78+
79+
<!-- toc -->
80+
- [Release Signoff Checklist](#release-signoff-checklist)
81+
- [Summary](#summary)
82+
- [Motivation](#motivation)
83+
- [Goals](#goals)
84+
- [Non-Goals](#non-goals)
85+
- [Proposal](#proposal)
86+
- [Risks and Mitigations](#risks-and-mitigations)
87+
- [What if I need to add capability to my feature?](#what-if-i-need-to-add-capability-to-my-feature)
88+
- [Who will make sure that new KEPs follow the promotion rules?](#who-will-make-sure-that-new-keps-follow-the-promotion-rules)
89+
- [Graduation Criteria](#graduation-criteria)
90+
- [Drawbacks](#drawbacks)
91+
- [This may slow the rate that new features are promoted.](#this-may-slow-the-rate-that-new-features-are-promoted)
92+
- [Alternatives](#alternatives)
93+
<!-- /toc -->
94+
95+
## Release Signoff Checklist
96+
97+
<!--
98+
**ACTION REQUIRED:** In order to merge code into a release, there must be an
99+
issue in [kubernetes/enhancements] referencing this KEP and targeting a release
100+
milestone **before the [Enhancement Freeze](https://git.k8s.io/sig-release/releases)
101+
of the targeted release**.
102+
103+
For enhancements that make changes to code or processes/procedures in core
104+
Kubernetes—i.e., [kubernetes/kubernetes], we require the following Release
105+
Signoff checklist to be completed.
106+
107+
Check these off as they are completed for the Release Team to track. These
108+
checklist items _must_ be updated for the enhancement to be released.
109+
-->
110+
111+
Items marked with (R) are required *prior to targeting to a milestone / release*.
112+
113+
- [ ] (R) Enhancement issue in release milestone, which links to KEP dir in [kubernetes/enhancements] (not the initial KEP PR)
114+
- [ ] (R) KEP approvers have approved the KEP status as `implementable`
115+
- [ ] (R) Design details are appropriately documented
116+
- [ ] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input (including test refactors)
117+
- [ ] e2e Tests for all Beta API Operations (endpoints)
118+
- [ ] (R) Ensure GA e2e tests meet requirements for [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md)
119+
- [ ] (R) Minimum Two Week Window for GA e2e tests to prove flake free
120+
- [ ] (R) Graduation criteria is in place
121+
- [ ] (R) [all GA Endpoints](https://github.com/kubernetes/community/pull/1806) must be hit by [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md)
122+
- [ ] (R) Production readiness review completed
123+
- [ ] (R) Production readiness review approved
124+
- [ ] "Implementation History" section is up-to-date for milestone
125+
- [ ] User-facing documentation has been created in [kubernetes/website], for publication to [kubernetes.io]
126+
- [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes
127+
128+
<!--
129+
**Note:** This checklist is iterative and should be reviewed and updated every time this enhancement is being considered for a milestone.
130+
-->
131+
132+
[kubernetes.io]: https://kubernetes.io/
133+
[kubernetes/enhancements]: https://git.k8s.io/enhancements
134+
[kubernetes/kubernetes]: https://git.k8s.io/kubernetes
135+
[kubernetes/website]: https://git.k8s.io/website
136+
137+
## Summary
138+
139+
Features gates must include all functional, security, monitoring, and testing requirements along with
140+
resolving all issues and gaps identified prior to being enabled by default.
141+
The only valid GA criteria are “all issues and gaps identified as feedback during beta are resolved”.
142+
143+
## Motivation
144+
145+
Features gates that are enabled by default are enabled in every production Kubernetes cluster in the world.
146+
We must avoid making every production cluster into unstable or incomplete feature testing clusters.
147+
Even feature gates that make flags accessible, but require a secondary configuration to use must be
148+
stable, because it is unrealistic to expect everyone to understand the graduation stages of various flags
149+
for each release: the only stages that really matter are "takes enabling an explicit alpha feature gate"
150+
and "my production cluster accepts this as valid by default".
151+
152+
### Goals
153+
154+
* Features gates must include all functional, security, monitoring, and testing requirements along with
155+
resolving all issues and gaps identified prior to being enabled by default.
156+
* The only valid GA criteria are “all issues and gaps identified as feedback during beta are resolved”.
157+
158+
### Non-Goals
159+
160+
* Changing beta APIs off by default rules.
161+
* Change the imperfect mechanisms we have for API evolution.
162+
163+
## Proposal
164+
165+
Kubernetes feature gates have three levels: GA (locked on), GA (disable-able), Beta, and Alpha.
166+
1. GA (locked-on) means that a feature gate is unconditionally enabled in all production kubernetes clusters and
167+
that feature cannot be disabled.
168+
2. GA (disable-able) is only for features gates that include a new API serialization that cannot be enabled by default
169+
until the API reaches stable. This means that the first time the API is enabled in production, the feature will
170+
be GA, but also can be disabled. This is a less common state and does not apply to most features.
171+
3. Beta means that a feature gate is usually enabled in all production Kubernetes clusters by default
172+
and that feature can be disabled.
173+
Exceptions exist for entirely new APIs and some node features, but this broadly the case.
174+
4. Alpha means that a feature gate is disabled in all production Kubernetes clusters by default and
175+
can be optionally enabled by setting a `--feature-gate` command line argument.
176+
177+
Making the jump to GA (cannot be disabled), without actual field experience is irresponsible.
178+
The first time we take a feature gate enabled by default in production Kubernetes clusters, we must
179+
have a way to disable the feature in case of unexpected stability, performance, or security issues.
180+
181+
Enabling incomplete features in production Kubernetes clusters by default is irresponsible.
182+
Features that are known to be incomplete naturally bring with them additional stability, performance, and security issues.
183+
Once a feature has been enabled in a production Kubernetes cluster by default, adding to it carries
184+
greater risk to upgrading clusters and the ecosystem.
185+
The feature can easily have become relied upon by workloads and other platform extensions.
186+
If an accident happens in adding those capabilities with stability, performance, and security the
187+
cost to disable those features in a cluster becomes significantly greater and breaks existing
188+
clusters, workloads and use-cases.
189+
This posture makes upgrades higher risk than necessary.
190+
191+
To balance these concerns, we are changing how we evaluate Beta and GA stability criteria.
192+
The only valid GA criteria are “all issues and gaps identified as feedback during beta are resolved”.
193+
Promotion from Beta to GA must have no significant change for the release.
194+
This means that Beta criteria must include all functional, security, monitoring, and testing requirements along
195+
with resolving all issues and gaps identified prior to beta.
196+
197+
Phasing in larger features over time can be done by bringing separate feature gates through alpha, beta, and GA.
198+
Each feature gate needs to meet the beta and GA criteria for completeness, functional, security, monitoring, and testing.
199+
After meeting the criteria for enabled by default, and at the SIG's discretion, the new feature gate could be
200+
set to enabled by default in the release it is introduced.
201+
Importantly, the features need to behave in a way that allows old and new clients to interoperate and new additions
202+
to larger features able to be independently disablable with their own path for GA.
203+
204+
### Risks and Mitigations
205+
206+
#### What if I need to add capability to my feature?
207+
To handle this situation, we described above how to add second feature gate for the new behavior.
208+
This provides a mechanism for adding needed capability, but ensures that
209+
cluster-admins never end up stuck after upgrade because they rely on v1.Y-1 behavior that new capability
210+
in v1.Y broke under the same feature gate.
211+
212+
#### Who will make sure that new KEPs follow the promotion rules?
213+
We'll adjust the KEP template to indicate the allowed criteria, so authors should notice.
214+
SIG approvers should enforce those standards.
215+
PRR approvers can be a final backstop.
216+
217+
### Graduation Criteria
218+
219+
This document is our new position once merged until it is superceded by another position statement.
220+
221+
## Drawbacks
222+
223+
### This may slow the rate that new features are promoted.
224+
For this to be true, that would mean that we previously enabled feature gates in production that were knowingly
225+
incomplete for functional, security, monitoring, testing, or known bugs.
226+
We hope this was not the common case, but if it was the common enough to have an impact, we're pleased that
227+
the result is preventing incomplete feature gates from being enabled in production clusters.
228+
229+
## Alternatives
230+
231+
None proposed so far.

‎keps/sig-architecture/5241-beta-featuregate-promotion-requirements/kep.yaml

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
title: Beta Feature Gate Promotion Requirements
2+
kep-number: 5241
3+
authors:
4+
- "@deads2k"
5+
owning-sig: sig-architecture
6+
participating-sigs:
7+
status: implemented
8+
creation-date: 2025-04-02
9+
reviewers:
10+
- "@liggitt"
11+
- "@thockin"
12+
approvers:
13+
- "@johnbelamaric"
14+
- "@dims"
15+
- "@derekwaynecarr"
16+
17+
see-also:
18+
- "/keps/sig-architecture/3136-beta-apis-off-by-default"
19+
replaces:
20+
21+
stage: stable
22+
23+
latest-milestone: "v1.34"
24+
25+
milestone:
26+
stable: "v1.34"
27+
28+
feature-gates:
29+
disable-supported: false
30+
31+
metrics:

0 commit comments

Comments
 (0)