Skip to content

NO-JIRA: [RHCOS10] Migrate base images from UBI9 to UBI10 #453

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
PillaiManish wants to merge 3 commits into
base: main
Choose a base branch
from
Open

NO-JIRA: [RHCOS10] Migrate base images from UBI9 to UBI10 #453

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2752c18
UPSTREAM: <carry>: Migrate base images from UBI9 to UBI10 for RHCOS10
PillaiManish Mar 26, 2026
495e344
Merge branch 'main' into rhcos10-ubi10-migration
PillaiManish Mar 28, 2026
7bc606b
Change base image to ubi-minimal:10.1
PillaiManish Apr 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 87 additions & 0 deletions .work/compliance/rhcos10/PR2-ubi10-migration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# PR2: RHCOS10 — Migrate Base Images from UBI9 to UBI10

## Purpose

Migrate all container base images from UBI9 to UBI10 to align with the RHCOS10 host OS.
This is the follow-up to PR1 (`rhcos10-ubi9-compat-test`), which validated that UBI9 images
run on RHCOS10 nodes. This PR adopts UBI10 as the native base for RHCOS10 deployments.

## Changes

### Registry change

All images move from the unauthenticated public registry to the authenticated Red Hat registry:

```
registry.access.redhat.com → registry.redhat.io
```
Comment on lines +15 to +17
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Add a language hint to the fenced code block

The fenced block at Line 15 has no language specifier (MD040). Add one (for example, text) to clear lint.

Suggested patch 
-```
+```text
 registry.access.redhat.com  →  registry.redhat.io
</details>

<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

```suggestion
🧰 Tools
🪛 markdownlint-cli2 (0.22.0)

[warning] 15-15: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In @.work/compliance/rhcos10/PR2-ubi10-migration.md around lines 15 - 17, The
fenced code block containing "registry.access.redhat.com  →  registry.redhat.io"
is missing a language hint (MD040); add a language specifier (e.g., "text") to
the opening ``` fence so the block becomes ```text and resolves the lint error
while preserving the arrow line content.


### UBI minimal images (pinned version)

`ubi9/ubi-minimal:9.6` → `ubi10/ubi-minimal:10.1`

| Dockerfile | Before | After |
|---|---|---|
| `images/helm-operator/Dockerfile` | `registry.access.redhat.com/ubi9/ubi-minimal:9.6` | `registry.redhat.io/ubi10/ubi-minimal:10.1` |
| `images/operator-sdk/Dockerfile` | `registry.access.redhat.com/ubi9/ubi-minimal:9.6` | `registry.redhat.io/ubi10/ubi-minimal:10.1` |
| `images/scorecard-test/Dockerfile` | `registry.access.redhat.com/ubi9/ubi-minimal:9.6` | `registry.redhat.io/ubi10/ubi-minimal:10.1` |
| `images/custom-scorecard-tests/Dockerfile` | `registry.access.redhat.com/ubi9/ubi-minimal:9.6` | `registry.redhat.io/ubi10/ubi-minimal:10.1` |

### Full UBI image (pinned version)

`ubi9/ubi:9.5` → `ubi10/ubi:10.1`

| Dockerfile | Before | After |
|---|---|---|
| `images/scorecard-untar/Dockerfile` | `registry.access.redhat.com/ubi9/ubi:9.5` | `registry.redhat.io/ubi10/ubi:10.1` |

### UBI minimal images (floating latest tag)

`ubi9/ubi-minimal:latest` → `ubi10/ubi-minimal:latest`

| Dockerfile | Before | After |
|---|---|---|
| `ci/dockerfiles/go-e2e.Dockerfile` | `registry.access.redhat.com/ubi9/ubi-minimal:latest` | `registry.redhat.io/ubi10/ubi-minimal:latest` |
| `ci/dockerfiles/scorecard-proxy.Dockerfile` | `registry.access.redhat.com/ubi9/ubi-minimal:latest` | `registry.redhat.io/ubi10/ubi-minimal:latest` |

### OCP product image (release/helm/Dockerfile)

Previously used OCP CI registry images pinned to RHEL9. Replaced with publicly available Red Hat registry images:

| Stage | Before | After |
|---|---|---|
| Builder | `registry.ci.openshift.org/ocp/builder:rhel-9-golang-1.24-openshift-4.22` | `registry.redhat.io/ubi10/go-toolset:10.1` |
| Runtime | `registry.ci.openshift.org/ocp/4.22:base-rhel9` | `registry.redhat.io/ubi10:10.1` |

### E2E test curl pod (ci/tests/e2e-helm.sh)

The metrics verification step spins up a temporary `kubectl run` pod using a UBI image to curl the metrics endpoint. Updated from UBI9 to UBI10:

```
registry.access.redhat.com/ubi9/ubi-minimal:latest
→
registry.redhat.io/ubi10/ubi-minimal:latest
```
Comment on lines +60 to +64
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Add a language hint to this fenced block (MD040).

Line 60 has the same lint issue; add text to the opening fence.

Suggested patch 
-```
+```text
 registry.access.redhat.com/ubi9/ubi-minimal:latest
 →
 registry.redhat.io/ubi10/ubi-minimal:latest
</details>

<!-- suggestion_start -->

<details>
<summary>📝 Committable suggestion</summary>

> ‼️ **IMPORTANT**
> Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

```suggestion
🧰 Tools
🪛 markdownlint-cli2 (0.22.0)

[warning] 60-60: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In @.work/compliance/rhcos10/PR2-ubi10-migration.md around lines 60 - 64, The
fenced code block showing the registry lines (the block containing
"registry.access.redhat.com/ubi9/ubi-minimal:latest →
registry.redhat.io/ubi10/ubi-minimal:latest") lacks a language hint and triggers
MD040; update the opening fence from ``` to ```text so the block becomes a text
fenced block, preserving the existing lines and arrows exactly.


## Files NOT Changed

| File | Reason |
|---|---|
| `release/helm/upstream.Dockerfile` | Uses `ubi8/ubi-minimal` — separate RHEL8 lineage, unrelated to this migration |
| `ci/dockerfiles/builder.Dockerfile` | Uses `openshift/origin-release:golang-1.13` — legacy, not RHEL9-specific |
| `.ci-operator.yaml` | Build root (`rhel-9-release-golang-1.24-openshift-4.22`) is managed by OCP CI team in `openshift/release` |

## Test Plan

- [ ] All images build successfully against `ubi10` base
- [ ] `release/helm/Dockerfile` builds successfully with `go-toolset:10.1` as builder
- [ ] CI jobs pass on RHCOS10 cluster nodes with UBI10 base images
- [ ] `microdnf` commands in `images/operator-sdk/Dockerfile` work under UBI10
- [ ] E2e metrics check passes with UBI10 curl pod (`ci/tests/e2e-helm.sh`)
- [ ] No regressions observed compared to UBI9 baseline (PR1)

## References

- [Red Hat UBI10 Container Catalog](https://catalog.redhat.com/en/software/containers/ubi10/ubi/66f2b46b122803e4937d11ae)
- [Red Hat UBI10 Minimal Container Catalog](https://catalog.redhat.com/en/software/containers/ubi10/ubi-minimal)
- PR1 baseline: `.work/compliance/rhcos10/PR1-ubi9-compat-test.md`
2 changes: 1 addition & 1 deletion ci/dockerfiles/go-e2e.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ FROM osdk-builder as builder

RUN ci/tests/scaffolding/e2e-go-scaffold.sh

FROM registry.access.redhat.com/ubi9/ubi-minimal:latest
FROM registry.redhat.io/ubi10/ubi-minimal:latest

ENV OPERATOR=/usr/local/bin/memcached-operator \
USER_UID=1001 \
Expand Down
2 changes: 1 addition & 1 deletion ci/dockerfiles/scorecard-proxy.Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ FROM osdk-builder as builder

RUN ci/tests/scaffolding/scorecard-proxy-scaffold.sh

FROM registry.access.redhat.com/ubi9/ubi-minimal:latest
FROM registry.redhat.io/ubi10/ubi-minimal:latest

ENV PROXY=/usr/local/bin/scorecard-proxy \
USER_UID=1001 \
Expand Down
2 changes: 1 addition & 1 deletion ci/tests/e2e-helm.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ EOF
token=$(kubectl get secret service-account-secret -o jsonpath={.data.token} | base64 -d)

# verify that the metrics endpoint exists
if ! timeout 1m bash -c -- "until kubectl run --attach --rm --restart=Never test-metrics --image=registry.access.redhat.com/ubi9/ubi-minimal:latest -n memcached-operator-system --overrides='{\"spec\":{\"securityContext\":{\"runAsNonRoot\": true, \"capabilities\": {\"drop\": [\"ALL\"]}, \"allowPrivelegeEscalation\": false, \"seccompProfile\": {\"type\": \"RuntimeDefault\"}}}}' -- curl -sfkH \"Authorization: Bearer ${token}\" https://memcached-operator-controller-manager-metrics-service:8443/metrics; do sleep 1; done";
if ! timeout 1m bash -c -- "until kubectl run --attach --rm --restart=Never test-metrics --image=registry.redhat.io/ubi10/ubi-minimal:latest -n memcached-operator-system --overrides='{\"spec\":{\"securityContext\":{\"runAsNonRoot\": true, \"capabilities\": {\"drop\": [\"ALL\"]}, \"allowPrivelegeEscalation\": false, \"seccompProfile\": {\"type\": \"RuntimeDefault\"}}}}' -- curl -sfkH \"Authorization: Bearer ${token}\" https://memcached-operator-controller-manager-metrics-service:8443/metrics; do sleep 1; done";
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🌐 Web query:

Kubernetes PodSecurityContext vs Container securityContext fields capabilities allowPrivilegeEscalation

💡 Result:

PodSecurityContext (at spec.securityContext) and Container securityContext (at spec.containers[].securityContext) serve different purposes in Kubernetes, with distinct fields for capabilities and allowPrivilegeEscalation. PodSecurityContext applies pod-wide settings like runAsUser, runAsGroup, fsGroup, supplementalGroups, seLinuxOptions, seccompProfile, appArmorProfile, sysctls, but does NOT include capabilities or allowPrivilegeEscalation fields. These are pod-level defaults for user/group IDs, volume ownership, and shared security profiles, but capabilities and privilege escalation are handled per-container. Container securityContext applies to individual containers (and initContainers/ephemeralContainers) and includes capabilities (add/drop lists for Linux capabilities) and allowPrivilegeEscalation (boolean controlling no_new_privs flag to prevent privilege escalation via setuid etc.). Container settings override pod-level where applicable (e.g., runAsUser). allowPrivilegeEscalation defaults to true if unset. It is forced true if privileged: true or capabilities add CAP_SYS_ADMIN, and cannot be set false when privileged: true (pod rejected). Set to false for security to block escalation. Capabilities allow fine-grained privileges: drop: ["ALL"] then add: ["NET_BIND_SERVICE"] for least privilege. Pod Security Standards (Restricted) require no add capabilities except limited list, and allowPrivilegeEscalation: false. Examples: Pod-level (no capabilities/allowPrivilegeEscalation): spec: securityContext: runAsUser: 1000 fsGroup: 2000 Container-level: spec: containers: - securityContext: capabilities: drop: ["ALL"] add: ["NET_BIND_SERVICE"] allowPrivilegeEscalation: false runAsNonRoot: true

Citations:

Move the container-only security fields out of spec.securityContext.

capabilities and allowPrivilegeEscalation belong on spec.containers[0].securityContext, not the pod-level spec.securityContext. Additionally, allowPrivelegeEscalation is misspelled (should be allowPrivilegeEscalation). As written, the temp metrics pod does not receive the intended hardening, and stricter admission controllers will cause this loop to timeout.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@ci/tests/e2e-helm.sh` at line 74, The pod override in the kubectl run command
places container-only fields inside spec.securityContext and misspells
allowPrivilegeEscalation; update the JSON passed to --overrides in the
test-metrics invocation so that spec.securityContext only contains pod-level
settings (e.g., runAsNonRoot), and move capabilities and
allowPrivilegeEscalation (spelled exactly as "allowPrivilegeEscalation") into
spec.containers[0].securityContext for the test-metrics pod started by kubectl
run --attach --rm --restart=Never test-metrics --image=... so the temporary
metrics pod receives the intended container-level hardening and does not get
rejected by admission controllers.

then
echo "Failed to verify that metrics endpoint exists"
kubectl describe pods
Expand Down
2 changes: 1 addition & 1 deletion images/custom-scorecard-tests/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ COPY . .
RUN GOOS=linux GOARCH=$TARGETARCH make build/custom-scorecard-tests

# Final image.
FROM registry.access.redhat.com/ubi9/ubi-minimal:9.7
FROM registry.redhat.io/ubi10/ubi-minimal:10.1

ENV HOME=/opt/custom-scorecard-tests \
USER_NAME=custom-scorecard-tests \
Expand Down
2 changes: 1 addition & 1 deletion images/helm-operator/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ COPY . .
RUN GOOS=linux GOARCH=$TARGETARCH make build/helm-operator

# Final image.
FROM registry.access.redhat.com/ubi9/ubi-minimal:9.7
FROM registry.redhat.io/ubi10/ubi-minimal:10.1

ENV HOME=/opt/helm \
USER_NAME=helm \
Expand Down
2 changes: 1 addition & 1 deletion images/operator-sdk/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ COPY . .
RUN GOOS=linux GOARCH=$TARGETARCH make build/operator-sdk

# Final image.
FROM registry.access.redhat.com/ubi9/ubi-minimal:9.7
FROM registry.redhat.io/ubi10/ubi-minimal:10.1

ARG TARGETARCH
RUN microdnf install -y make gcc which tar gzip
Expand Down
2 changes: 1 addition & 1 deletion images/scorecard-test/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ COPY . .
RUN GOOS=linux GOARCH=$TARGETARCH make build/scorecard-test

# Final image.
FROM registry.access.redhat.com/ubi9/ubi-minimal:9.7
FROM registry.redhat.io/ubi10/ubi-minimal:10.1

ENV HOME=/opt/scorecard-test \
USER_NAME=scorecard-test \
Expand Down
2 changes: 1 addition & 1 deletion images/scorecard-untar/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM registry.access.redhat.com/ubi9/ubi:9.7
FROM registry.redhat.io/ubi10/ubi:10.1

## Create a new non-root user to run as
ENV HOME=/opt/scorecard-untar \
Expand Down
5 changes: 3 additions & 2 deletions release/helm/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
FROM registry.ci.openshift.org/ocp/builder:rhel-9-golang-1.25-openshift-4.22 AS builder
FROM registry.redhat.io/ubi10/go-toolset:10.1 AS builder

ENV GO111MODULE=on \
GOFLAGS=-mod=vendor

# we need to patch the Makefile prior to building
USER root
COPY . /go/src/github.com/operator-framework/operator-sdk
RUN cd /go/src/github.com/operator-framework/operator-sdk \
&& make -f ci/prow.Makefile patch build

FROM registry.ci.openshift.org/ocp/4.22:base-rhel9
FROM registry.access.redhat.com/ubi10/ubi-minimal:10.1

ENV HOME=/opt/helm \
USER_NAME=helm \
Expand Down