cmd/compile: failures with "wait: no child processes" on solaris-amd64-oraclerel since 2021-04-02

[bcmills]

#!watchflakes
post <- goos == "solaris" && log ~ `wait: no child processes`

##### ../test
go tool compile: wait: no child processes

go tool dist: FAILED: go list -gcflags=all= -ldflags=all= -f={{if .Stale}}	STALE {{.ImportPath}}: {{.StaleReason}}{{end}} std: exit status 1

2021-05-12T14:43:27-485474d/solaris-amd64-oraclerel
2021-05-07T21:35:41-f24eac4/solaris-amd64-oraclerel
2021-05-07T20:56:39-f05e912/solaris-amd64-oraclerel
2021-05-06T02:20:28-43c390a/solaris-amd64-oraclerel
2021-05-04T03:22:47-138d2c9/solaris-amd64-oraclerel
2021-05-03T17:46:12-90ec257/solaris-amd64-oraclerel
2021-05-02T18:22:19-0d32d9e/solaris-amd64-oraclerel
2021-04-30T16:53:35-89bf297/solaris-amd64-oraclerel
2021-04-30T16:27:50-3366556/solaris-amd64-oraclerel
2021-04-29T14:15:04-862ddb3/solaris-amd64-oraclerel
2021-04-28T20:36:50-fa6ed6e/solaris-amd64-oraclerel
2021-04-28T19:13:50-ad989c7/solaris-amd64-oraclerel
2021-04-27T19:40:24-b9dfaf7/solaris-amd64-oraclerel
2021-04-22T01:40:02-5daefc5/solaris-amd64-oraclerel
2021-04-22T01:29:53-0636d88/solaris-amd64-oraclerel
2021-04-21T14:16:01-e5bc4f2/solaris-amd64-oraclerel
2021-04-20T22:47:50-48e3d92/solaris-amd64-oraclerel
2021-04-15T00:40:43-566a87c/solaris-amd64-oraclerel
2021-04-14T19:50:58-d1362d7/solaris-amd64-oraclerel
2021-04-12T16:44:49-70ed28e/solaris-amd64-oraclerel
2021-04-08T20:55:59-0243799/solaris-amd64-oraclerel
2021-04-06T14:15:29-b345a30/solaris-amd64-oraclerel
2021-04-06T13:01:37-5cd8a34/solaris-amd64-oraclerel
2021-04-02T19:27:06-34b87b4/solaris-amd64-oraclerel
2021-04-02T13:30:18-b165085/solaris-amd64-oraclerel

Milestoning to Go 1.17 since this seems to be a regression starting in April, but it isn't release-blocker because it somehow appears to affect only Solaris(‽).

CC @golang/release in case this can be explained by any changes to the builder around that time.

[dmitshur]

I'm not aware of builder changes on our side, but solaris-amd64-oraclerel is a reverse builder whose owner is @rorth, and perhaps they may be aware of some change on the side of the reverse builder.

[bardo]

Currently hitting this on SmartOS:

$ go build
# github.com/twpayne/chezmoi/v2
/opt/local/go117/pkg/tool/solaris_amd64/link: running gcc failed: wait: no child processes

$ go version
go version go1.17.2 solaris/amd64

Is there a known workaround? I'm afraid I do not know Go enough to investigate, but I can help out in testing whatever is needed.

[rorth]

Hi Corrado,

Currently hitting this on SmartOS: ``` $ go build # github.com/twpayne/chezmoi/v2

what's with the v2 part? While I can checkout git://github.com/twpayne/chezmoi, there's no v2 tag or branch!?

/opt/local/go117/pkg/tool/solaris_amd64/link: running gcc failed: wait: no child processes $ go version go version go1.17.2 solaris/amd64

First of all, why is this solaris/amd64? It should be illumos/amd64 instead, I believe. Solaris and Illumos (derivatives) have diverged considerably over time. That said, I tried a build on Solaris 11.4 with Go master, but failed: # github.com/google/gops/agent ***@***.***/agent/sockopt_unix.go:32:58: undefined: unix.SO_REUSEPORT AFAICT there's no definition of SO_REUSEPORT for either Solaris or Illumos, so I expect you will hit this issue, too.

[twpayne]

what's with the v2 part?

The v2 part is part of the Go module, see for example this post on go.dev.

While I can checkout git://github.com/twpayne/chezmoi, there's no v2 tag or branch!?

There are many tags, the latest is v2.7.2. Note that you should use the latest master branch of chezmoi, however.

That said, I tried a build on Solaris 11.4 with Go master, but failed:
# github.com/google/gops/agent @.***/agent/sockopt_unix.go:32:58: undefined: unix.SO_REUSEPORT
AFAICT there's no definition of SO_REUSEPORT for either Solaris or Illumos, so I expect you will hit this issue, too.

This was resolved in twpayne/chezmoi#1541 which removed the dependency on gops. You'll need to use the latest master branch to include this change.

[rorth]

Tom Payne ***@***.***> writes:

> what's with the v2 part? The `v2` part is part of the Go module, see for example [this post on go.dev](https://go.dev/blog/v2-go-modules).

I see, thanks.

> While I can checkout git://github.com/twpayne/chezmoi, there's no v2 tag > or branch!? There are [many tags](https://github.com/twpayne/chezmoi/tags), the latest is `v2.7.2`. Note that you should use the latest `master` branch of chezmoi, however.

I believe I did so before, but I'm certainly on master now.

> That said, I tried a build on Solaris 11.4 with Go master, but failed: > # github.com/google/gops/agent > ***@***.***/agent/sockopt_unix.go:32:58: undefined: unix.SO_REUSEPORT > AFAICT there's no definition of SO_REUSEPORT for either Solaris or > Illumos, so I expect you will hit this issue, too. This was resolved in twpayne/chezmoi#1541 which removed the dependency on gops. You'll need to use the latest `master` branch to include this change.

Indeed: this gets me beyond that issue (as expected), only to hit # github.com/twpayne/chezmoi/v2 /var/gcc/go/master/pkg/tool/solaris_amd64/link: running gcc failed: exit status 1 Undefined first referenced symbol in file flock /tmp/go-link-1624481765/go.o ld: fatal: symbol referencing errors collect2: error: ld returned 1 exit status which is expected on Solaris at least, which lacks flock(2). Illumos may be different, though.

[twpayne]

Thanks for the further investigation!

Re flock, this is likely because chezmoi uses https://go.etcd.io/bbolt which uses file locking. If flock is not available on Solaris, then it looks like chezmoi will have to use a different database backend on Solaris.

For what it's worth, I'm able to compile chezmoi on Illumos, although there are some failing tests at the moment. See twpayne/chezmoi#1544 for details.

[twpayne]

Let's move the chezmoi discussion to twpayne/chezmoi#1548 as I don't think that this is a Go issue.

[bardo]

First of all, why is this solaris/amd64? It should be illumos/amd64
instead, I believe. Solaris and Illumos (derivatives) have diverged
considerably over time.

I'm not sure why SmartOS doesn't use Illumos, especially considering that it has been specifically called out in #20603 but here it is:

$ go env | egrep 'GO(ARCH|OS)'
GOARCH="amd64"
GOOS="solaris"

I see that in the build files GOARCH is set, but not GOOS: https://github.com/joyent/pkgsrc/blob/trunk/lang/go/version.mk
I'm not really sure about how this works, but there may be some kind of autodetection that defaults to Solaris.

[bardo]

Interestingly, enforcing the Illumos platform with GOOS=illumos go build results in a successful build. I'll surely open a bug about this on the SmartOS package, but I wonder if it could help shed some light on the issue here.