runtime: segfaults in cross compiled binaries for solaris 11.3

[karlik8899]

What version of Go are you using (go version)?

$ go version
go version go1.21.0 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

linux centos 8

go env Output

$ go env
GO111MODULE='auto'
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/x/.cache/go-build'
GOENV='/home/x/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/x/go/pkg/mod'
GOOS='linux'
GOPATH='/home/x/go'
GOPROXY='https://proxy.golang.org'
GOROOT='/opt/go1.21.0'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/opt/go1.21.0/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.21.0'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/home/x/myrepo/test2/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build1105527576=/tmp/go-build -gno-record-gcc-switches'

What did you do?

func main() {
fmt.Println("Abcd")
}

env GOOS=solaris GOARCH=amd64 CGO_ENABLED=0 go build -o test-sol-1.21.0

What did you expect to see?

Abcd

What did you see instead?

Segmentation Fault (core dumped)

sotruss ~/test-sol-1.21.0
test-sol-1.21.0 -> libc.so:*sysconf(0xf, 0xffff80ffbffff6a0, 0x42ce0f)
test-sol-1.21.0 -> libc.so:*mmap(0x0, 0x40000, 0x3)
test-sol-1.21.0 -> libc.so:*mmap(0x0, 0x20000, 0x0)
test-sol-1.21.0 -> libc.so:*mmap(0x0, 0x100000, 0x0)
test-sol-1.21.0 -> libc.so:*mmap(0x0, 0x800000, 0x0)
test-sol-1.21.0 -> libc.so:*mmap(0x0, 0x4000000, 0x0)
test-sol-1.21.0 -> libc.so:*mmap(0x0, 0x20000000, 0x0)
test-sol-1.21.0 -> libc.so:*mmap(0x0, 0x20000000, 0x0)
test-sol-1.21.0 -> libc.so:*mmap(0xc000000000, 0x4000000, 0x0)
test-sol-1.21.0 -> libc.so:*mmap(0x0, 0x2000000, 0x3)
test-sol-1.21.0 -> libc.so:*mmap(0x0, 0x114c10, 0x3)
test-sol-1.21.0 -> libc.so:*mmap(0xc000000000, 0x400000, 0x3)
test-sol-1.21.0 -> libc.so:*mmap(0xffff80ffbf625000, 0x20000, 0x3)
test-sol-1.21.0 -> libc.so:*mmap(0xffff80ffbe97f000, 0x1000, 0x3)
test-sol-1.21.0 -> libc.so:*mmap(0xffff80ffbde06000, 0x1000, 0x3)
test-sol-1.21.0 -> libc.so:*mmap(0xffff80ffbb830000, 0x1000, 0x3)
test-sol-1.21.0 -> libc.so:*mmap(0xffff80ffa9780000, 0x1000, 0x3)
test-sol-1.21.0 -> libc.so:*mmap(0xffff80ff89580000, 0x1000, 0x3)
test-sol-1.21.0 -> libc.so:*mmap(0x0, 0x100000, 0x3)
test-sol-1.21.0 -> libc.so:*mmap(0x0, 0x10000, 0x3)
test-sol-1.21.0 -> libc.so:*mmap(0x0, 0x10000, 0x3)
test-sol-1.21.0 -> libc.so:*open(0x52d4b0, 0x0, 0x0)
/usr/bin/sotruss: line 78: 277: Memory fault(coredump)

ldd ~/test-sol-1.21.0
libsendfile.so => /lib/64/libsendfile.so
libsocket.so => /lib/64/libsocket.so
libc.so => /lib/64/libc.so
libnsl.so.1 => /lib/64/libnsl.so.1
libmp.so.2 => /lib/64/libmp.so.2
libucrypto.so.1 => /lib/64/libucrypto.so.1
libelf.so.1 => /lib/64/libelf.so.1
libcryptoutil.so.1 => /lib/64/libcryptoutil.so.1
libz.so.1 => /lib/64/libz.so.1

[mknyszek]

This seems like a failure early in the runtime. Do you have gdb on your system or something similar? Where does it actually fail?

Also, do you have a way to reproduce? Thanks.

CC @golang/solaris

[karlik8899]

gdb Output

My binary from issue 59359
gdb ~/goconntest.1.18

GNU gdb (GDB) 7.6

Copyright (C) 2013 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law.  Type "show copying"

and "show warranty" for details.

This GDB was configured as "i386-pc-solaris2.11".

For bug reporting instructions, please see:

http://www.gnu.org/software/gdb/bugs/...

Reading symbols from /export/home/pa033460/goconntest.1.18...done.

warning: Missing auto-load scripts referenced in section .debug_gdb_scripts

of file /export/home/x/goconntest.1.18

Use `info auto-load python [REGEXP]' to list them.

(gdb)

(gdb)

(gdb) r

Starting program: /export/home/x/goconntest.1.18

[Thread debugging using libthread_db enabled]

[New Thread 1 (LWP 1)]

[New LWP    2        ]

[New LWP    3        ]

[New LWP    4        ]

[New LWP    5        ]

Jenkins bad status: 401 Unauthorized

[Inferior 1 (process 24530    ) exited with code 01]

(gdb)

(gdb)

(gdb) quit
gdb ~/test3-go1.21.0

GNU gdb (GDB) 7.6

Copyright (C) 2013 Free Software Foundation, Inc.

License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law.  Type "show copying"

and "show warranty" for details.

This GDB was configured as "i386-pc-solaris2.11".

For bug reporting instructions, please see:

http://www.gnu.org/software/gdb/bugs/...

Reading symbols from /export/home/x/test3-go1.21.0...done.

warning: Missing auto-load scripts referenced in section .debug_gdb_scripts

of file /export/home/x/test3-go1.21.0

Use `info auto-load python [REGEXP]' to list them.

(gdb)

(gdb)

(gdb) r

Starting program: /export/home/x/test3-go1.21.0

[Thread debugging using libthread_db enabled]

[New Thread 1 (LWP 1)]
Program received signal SIGSEGV, Segmentation fault.

[Switching to Thread 1 (LWP 1)]

0xffff80ffbef7ab2d in open () from /lib/64/libc.so

(gdb)

[ianlancetaylor]

For the record, this worked for me.

> GOOS=solaris GOARCH=amd64 CGO_ENABLED=0 go build ~/hello.go
> copy hello executable to x86 Solaris 11.4 system
> ./hello
Hello, world

Can you run the where command in gdb after receiving the signal? Thanks.

[karlik8899]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1 (LWP 1)]
0xffff80ffbef7ab2d in open () from /lib/64/libc.so
(gdb) where
#0 0xffff80ffbef7ab2d in open () from /lib/64/libc.so
#1 0x0000000000466a3e in runtime.asmsysvicall6 () at /home/x/sdk/go1.21.0/src/runtime/sys_solaris_amd64.s:72
#2 0x0000000000465178 in runtime.asmcgocall () at /home/x/sdk/go1.21.0/src/runtime/asm_amd64.s:902
#3 0xffff80ffbf667f38 in ?? ()
#4 0x0000000000000000 in ?? ()
(gdb)

[mknyszek]

@karlik8899 I appreciate your engagement and cooperation, but I think to really make progress on this one we need to be able to reproduce it. Just sending instructions and output back and forth is going to be fairly slow.

Though, in triage, we're noticing that this seems to be a segfault inside the open function in libc, and we're thinking it probably shouldn't segfault. For example, libc on a bunch of systems is specified to return an EFAULT even when an invalid memory location (e.g. filepath string) is passed to it. There may be something completely wrong with how we're cross-compiling.

Does this only happen if you cross-compile? Have you tried building the program on your Solaris machine? Thanks.

[karlik8899]

I downloaded go1.21.0.solaris-amd64.tar.gz and ran go
Segmentation Fault (core dumped)

gdb go/bin/go
GNU gdb (GDB) 7.6
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i386-pc-solaris2.11".
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/...
Reading symbols from /export/home/x/sdk/go/bin/go...(no debugging symbols found)...done.
(gdb) r
Starting program: /export/home/x/sdk/go/bin/go
[Thread debugging using libthread_db enabled]
[New Thread 1 (LWP 1)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1 (LWP 1)]
0xffff80ffbef7ab2d in open () from /lib/64/libc.so
(gdb) where
#0 0xffff80ffbef7ab2d in open () from /lib/64/libc.so
#1 0x000000000046fbde in runtime.asmsysvicall6.abi0 ()
#2 0x000000000046e278 in runtime.asmcgocall.abi0 ()
#3 0x0000000000436ac5 in runtime.sysvicall3Err.abi0 ()
#4 0x000000000043622e in runtime.open ()
#5 0x0000000000435aba in runtime.getRandomData ()
#6 0x0000000000407da9 in runtime.alginit ()
#7 0x0000000000f1fea0 in runtime.debug ()
#8 0x0000000000000080 in ?? ()
#9 0x0000000000000080 in ?? ()
#10 0xffff80ffbffff890 in ?? ()
#11 0x000000000043cc65 in runtime.schedinit ()
#12 0x0000000000000000 in ?? ()
(gdb)

I couldn't find an older version of golang for Solaris.