Skip to content

zloop.sh reports every iteration as a crash when run unprivileged with a piped core_pattern #18966

Description

@mkhllr

System information

Type Version/Name
Distribution Name Ubuntu
Distribution Version 24.04.4 LTS
Kernel Version 6.8.0-136-generic
Architecture x86_64
OpenZFS Version master (4e00623), scripts/zloop.sh unmodified

Describe the problem you're observing

On a system whose kernel.core_pattern is a pipe, zloop.sh counts a crash for
every iteration when it is not run as root, however well ztest does.

The pattern is read at startup:

	coreglob="$(grep -E -o '^([^|%[:space:]]*)' /proc/sys/kernel/core_pattern)*"

	if [[ $coreglob = "*" ]]; then
		echo "Setting core file pattern..."
		echo "core" > /proc/sys/kernel/core_pattern
		coreglob="$(grep -E -o '^([^|%[:space:]]*)' \
		    /proc/sys/kernel/core_pattern)*"
	fi

Ubuntu's pattern starts with |, so the extraction yields the empty string and
coreglob becomes *. Writing core_pattern needs privileges, so for an
ordinary user that write fails, the re-read returns * again, and core_file()

	ls -tr1 $coreglob 2>/dev/null | head -1

returns the first file in the directory zloop was started from. store_core()
then treats it as a core:

	if [[ $ztrc -ne 0 ]] || [[ -f "$core" ]]; then

The loop writes ztest.history and ztest.out into that directory before it
invokes ztest, so the check is already true on the first iteration. Every
iteration is archived as a crash and zloop.sh exits 1.

The zloop workflow invokes the script under sudo, so CI is unaffected.

Describe how to reproduce the problem

Using a stub in place of ztest, so that every iteration certainly succeeds:

$ cat > /tmp/fakeztest.sh <<'EOF'
#!/bin/sh
d=""
while [ $# -gt 0 ]; do
	case "$1" in -f) d="$2" ;; esac
	shift
done
[ -n "$d" ] && : > "$d/ztest.0a"
exit 0
EOF
$ chmod +x /tmp/fakeztest.sh

$ cat /proc/sys/kernel/core_pattern
|/usr/share/apport/apport -p%p -s%s -c%c -d%d -P%P -u%u -g%g -F%F -- %E

$ rm -rf /tmp/zloop-run /tmp/zloop-cores
$ mkdir -p /tmp/zloop-work /tmp/zloop-run
$ cd /tmp/zloop-run     # zloop bails at startup if it finds a "core" here
$ ZTEST=/tmp/fakeztest.sh ZDB=/bin/true \
    /path/to/zfs/scripts/zloop.sh -I 5 -c /tmp/zloop-cores -f /tmp/zloop-work -- -T 1 -P 1

Include any warning/errors/backtraces from the system logs

Setting core file pattern...
/home/mike/zfs-mmp/scripts/zloop.sh: line 83: /proc/sys/kernel/core_pattern: Permission denied
core dump directory (/tmp/zloop-cores) does not exist, creating it.
08/21 01:24:19 /tmp/fakeztest.sh -G -VVVVV -K raidz -m 2 -r 0 -D 0 -S 0 -R 1 -v 2 -a 9 -C special=random -s 512m -f /tmp/zloop-work/zloop-run -T 1 -P 1
*** ztest crash found - moving logs to /tmp/zloop-cores/zloop-260821-012419
continuing...
08/21 01:24:19 /tmp/fakeztest.sh -G -VVVVV -K eraidz -m 0 -r 7 -D 0 -S 0 -R 1 -v 1 -a 9 -C special=random -s 512m -f /tmp/zloop-work/zloop-run -T 1 -P 1
*** ztest crash found - moving logs to /tmp/zloop-cores/zloop-260821-012419
continuing...
08/21 01:24:19 /tmp/fakeztest.sh -G -VVVVV -K eraidz -m 0 -r 9 -D 0 -S 0 -R 3 -v 1 -a 12 -C special=random -s 512m -f /tmp/zloop-work/zloop-run -T 1 -P 1
*** ztest crash found - moving logs to /tmp/zloop-cores/zloop-260821-012419
continuing...
08/21 01:24:19 /tmp/fakeztest.sh -G -VVVVV -K raidz -m 2 -r 0 -D 0 -S 0 -R 1 -v 2 -a 9 -C special=random -s 512m -f /tmp/zloop-work/zloop-run -T 1 -P 1
*** ztest crash found - moving logs to /tmp/zloop-cores/zloop-260821-012419
continuing...
08/21 01:24:19 /tmp/fakeztest.sh -G -VVVVV -K raidz -m 2 -r 0 -D 0 -S 0 -R 1 -v 5 -a 9 -C special=random -s 512m -f /tmp/zloop-work/zloop-run -T 1 -P 1
*** ztest crash found - moving logs to /tmp/zloop-cores/zloop-260821-012419
continuing...
zloop finished, 5 crashes found
/home/mike/zfs-mmp/scripts/zloop.sh: line 344: /proc/sys/kernel/core_pattern: Permission denied

I met this with a real ztest first, which is what makes it awkward to spot: six
iterations came back as crashes, and re-running those command lines by hand
showed each of them exiting 0.

Unprivileged operation came up before in #9613, which proposed switching to
coredumpctl so the script would not need to set core_pattern at all; it was
closed unmerged. Short of that, the script could stop when it cannot establish a
usable pattern, since it cannot collect cores in that state anyway, or
core_file() could be given a glob that will not match ordinary files, so a
missing core stays missing. Happy to put a patch together if there is a
preference.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions