Something with my profile is preventing UID switch in rootless containers

[jimmy-lt]

Hi,

I have a permission issue with my profile which is preventing me to run any container which internal user is not UID 0.

Take this Containerfile:

FROM registry.access.redhat.com/ubi9/ubi-init:latest as base

RUN useradd --create-home --shell /bin/bash user
RUN useradd --create-home --shell /bin/bash --system system

CMD [ "/bin/bash" ]

Running it as "root" works fine:

$ podman run --rm -it 31101e91b9db whoami          
root

Trying to run as one of the created user fails:

$ podman run --rm -it --user user 31101e91b9db whoami
{"msg":"exec container process `/usr/bin/whoami`: Permission denied","level":"error","time":"2025-11-26T15:49:54.324227Z"}
$ podman run --rm -it --user system 31101e91b9db whoami
{"msg":"exec container process `/usr/bin/whoami`: Permission denied","level":"error","time":"2025-11-26T15:50:00.076843Z"}

The issue is really tied to my profile since other users on the system can run podman just fine. I can't seem to pinpoint the origin of the issue.

Sub{u,g}id setup seems to be fine:

$ grep me /etc/sub{u,g}id
/etc/subuid:me:524288:65536
/etc/subgid:me:524288:65536
$ podman unshare cat /proc/self/uid_map
         0       1000          1
         1     524288      65536

I'm running Fedora 43.

# podman info
host:
  arch: amd64
  buildahVersion: 1.42.0
  cgroupControllers:
  - cpu
  - io
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.13-2.fc43.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.13, commit: '
  cpuUtilization:
    idlePercent: 98.6
    systemPercent: 0.35
    userPercent: 1.05
  cpus: 20
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: workstation
    version: "43"
  emulatedArchitectures:
  - linux/arm
  - linux/arm64
  - linux/arm64be
  - linux/loong64
  - linux/mips
  - linux/mips64
  - linux/ppc
  - linux/ppc64
  - linux/ppc64le
  - linux/riscv32
  - linux/riscv64
  - linux/s390x
  eventLogger: journald
  freeLocks: 2048
  hostname: host
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 524288
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 524288
      size: 65536
  kernel: 6.17.8-300.fc43.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 47213182976
  memTotal: 67081904128
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.17.0-1.fc43.x86_64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.17.0
    package: netavark-1.17.0-1.fc43.x86_64
    path: /usr/libexec/podman/netavark
    version: netavark 1.17.0
  ociRuntime:
    name: crun
    package: crun-1.25-1.fc43.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.25
      commit: d9a0adce065c7747ab88ea6ccc42b15a626e08e1
      rundir: /run/user/1000/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20250919.g623dbf6-1.fc43.x86_64
    version: |
      pasta 0^20250919.g623dbf6-1.fc43.x86_64
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  rootlessNetworkCmd: pasta
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 42932887552
  swapTotal: 42932887552
  uptime: 5h 54m 45.00s (Approximately 0.21 days)
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
store:
  configFile: /home/me/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/me/.local/share/containers/storage
  graphRootAllocated: 171798691840
  graphRootUsed: 99622297600
  graphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 5
  runRoot: /run/user/1000/containers
  transientStore: false
  volumePath: /home/me/.local/share/containers/storage/volumes
version:
  APIVersion: 5.7.0
  BuildOrigin: Fedora Project
  Built: 1762819200
  BuiltTime: Tue Nov 11 01:00:00 2025
  GitCommit: 0370128fc8dcae93533334324ef838db8f8da8cb
  GoVersion: go1.25.4 X:nodwarf5
  Os: linux
  OsArch: linux/amd64
  Version: 5.7.0

[jimmy-lt]

Well I had set too restrictive permissions on my profile directory, what was preventing the subids to access the storage. I had it fixed with the following:

$ chmod o=rX ~/.local/share/containers/storage