Tmp directory tests are failing on macos

[jfischoff]

[dhess]

Hi,

We've successfully deployed tmp-postgres in our test suite now, and it's working great, except on our x86-64 Macs, where our tests will occasionally fail for strange reasons. If we re-run the failed job, it nearly always succeeds the second time.

The failure modes differ. Sometimes the failure occurs because the server refuses a connection, sometimes the connection times out, and sometimes an insert will fail because PostgreSQL complains that the database doesn't exist, despite our migration script having run and set up the database prior to the insert failing.

For compatibility with Sqitch, which we're using for migrations, we run the server on localhost:5423 and connect via TCP loopback. We're making use of tmp-postgress database caching per your helpful blog post, though we're not yet using every trick in your arsenal, mainly just the initial approach you describe in that post. We also disable parallelism in our tests in order to achieve maximum isolation, and to keep the test harness relatively simple.

We've yet to see any of these issues on our aarch64 Macs, nor our x86-64 Linux machines.

When I searched the tracker for "macos", I saw this issue. Can you provide some detail on what sorts of failures you're seeing? I'm curious if it has anything to do with our issues.

[jfischoff]

Sorry, this was a note to myself to update a test after a change was made to support the TMP environment variable (#256), I'm pretty sure. I don't think there was anything actually wrong.

The main issue that I've seen, that I have struggled to understand, is this one: #251.

The failure modes differ. Sometimes the failure occurs because the server refuses a connection, sometimes the connection times out, and sometimes an insert will fail because PostgreSQL complains that the database doesn't exist, despite our migration script having run and set up the database prior to the insert failing.

Huh this actually quite confusing.

Something to look out for is the postgres might not be shutting down correctly. This will result in it running out of semaphores when it starts. However typically, the error message will say this specifically.

[dhess]

I've just had a failure on an Apple Silicon Mac, as well, so this isn't Intel-specific, which is actually comforting. :)

[jfischoff]

Can you paste the error message?

…

On Thu, Mar 3, 2022 at 4:57 PM Drew Hess ***@***.***> wrote: I've just had a failure on an Apple Silicon Mac, as well, so this isn't Intel-specific, which is actually comforting. :) — Reply to this email directly, view it on GitHub <#259 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABJEO6HMTUAIIUVH5AH37DU6FNY5ANCNFSM4N2UEPFQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[dhess]

I have a guess at what might be going on. Let me look into that and if I can't get to the bottom of it, I'll post the gists of the errors. Thanks for your help!

[jfischoff]

Sounds good to me!

…

On Thu, Mar 3, 2022 at 5:01 PM Drew Hess ***@***.***> wrote: I have a guess at what might be going on. Let me look into that and if I can't get to the bottom of it, I'll post the gists of the errors. Thanks for your help! — Reply to this email directly, view it on GitHub <#259 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABJEO3ZYNQJB6UE763VMXDU6FOGNANCNFSM4N2UEPFQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.Message ID: ***@***.***>