Set thread counts for client & server using env variables with better defaults

[pomo-mondreganto]

No description provided.

[pomo-mondreganto]

A bit of a backstory: we've encountered two issues using sccache in our cluster.

First, when running in a cpu-limited container environment, tokio erroneously detects the total number of CPUs instead of the container limit, which is correct (this behaviour exists in all schedulers known to me, e.g. the Goland one). Second, it spawns a gigantic tokio scheduler with num_cpus worker threads for each compilation process (client invocation). In our setup (128 logical CPUs, 64 cpu container limit, 64 concurrent compilation jobs) sccache spawned 128 * 64 threads on compilation, which is completely unnecessary and overflows the thread limit for the container.

[glandium]

First, when running in a cpu-limited container environment, tokio erroneously detects the total number of CPUs instead of the container limit, which is correct (this behaviour exists in all schedulers known to me, e.g. the Goland one).

num_cpus uses sched_getaffinity, and should return the right number. What does your container do to limit CPU?

[pomo-mondreganto]

First, when running in a cpu-limited container environment, tokio erroneously detects the total number of CPUs instead of the container limit, which is correct (this behaviour exists in all schedulers known to me, e.g. the Goland one).

num_cpus uses sched_getaffinity, and should return the right number. What does your container do to limit CPU?

It's a regular Kubernetes installation with containerd runtime, containers using cpu_request/cpu_limit by k8s.

[pomo-mondreganto]

I've just checked that in a container with the following settings:

│     Limits:                                                                                                            │
│       cpu:                3                                                                                            │
│       ephemeral-storage:  100Gi                                                                                        │
│       memory:             4Gi                                                                                          │
│     Requests:                                                                                                          │
│       cpu:                3                                                                                            │
│       ephemeral-storage:  100Gi                                                                                        │
│       memory:             4Gi

sched_getaffinity returns all cores:

>>> import os
>>> os.sched_getaffinity(0)
{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127}

[glandium]

I guess k8s allocates less scheduling slots rather than less CPUs.

[codecov-commenter]

Codecov Report

Attention: 7 lines in your changes are missing coverage. Please review.

Comparison is base (fb0ab0c) 31.11% compared to head (ba4e92b) 30.89%.

Files	Patch %	Lines
src/commands.rs	0.00%	1 Missing and 3 partials ⚠️
src/server.rs	0.00%	1 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1993      +/-   ##
==========================================
- Coverage   31.11%   30.89%   -0.23%     
==========================================
  Files          51       51              
  Lines       19419    19425       +6     
  Branches     9341     9356      +15     
==========================================
- Hits         6043     6001      -42     
- Misses       7785     7797      +12     
- Partials     5591     5627      +36     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pomo-mondreganto]

Can someone take a look at the "cancelled" test? No output and no reason for cancellation.

[sylvestre]

Don't bother, it is fine :)

[pomo-mondreganto]

So can I consider my work finished here? :)

[pomo-mondreganto]

Any plans on merging this? I might be adding some more features in the near future, and I'd prefer implementing them over upstream, not my fork.

[sylvestre]

please explain in which contexts someone might need this

[sylvestre]

it would be nice to add tests which verifies that it works correctly

[sylvestre]

ping ?

[sylvestre]

please reopen when ready