Support read-only mode to prevent unwanted cache update

[dvcolomban]

🚀 Feature Proposal

Support a read-only mode that allows cache read from clients but doesn't contribute new cache entries.

Motivation

In the context of a mono-repo ci integration, we would like to share ci cache artefacts with local machines, but we don't want users to be able to contribute new values.

The purpose is to speed up local work without risking contamination of the ci and cd artefacts from potentially broken or unsafe local clients.

Example

With such a feature, we would be able to generate cache read/write from the ci, where we can guarantee state and monitoring with a normal instance.

At the same time, another instance of the remote repo could allow local users to consume the same cache entry (backed by s3 storage in our case) without risking broken builds.

I've looked at the official turbo documentation, and it seems they don't natively offer such a feature.
The closest to it, are some cache management flags that allow to prevent cache uploads, but those are easily circumvented and necessite to split ci and dev scripts.

An open issue on turbo addresses similar needs but seems that nothing much came of it yet.

Do you have any plan to implement this ? And how it could be done ?

We're ready to contribute to a PR implementing this if you're open to it :)

[fox1t]

I love this idea! We can add a deployment flag that will return a 401 to clients on PUT but will work on HEAD and GET! What do you think?

@matteovivona FYI.

[dvcolomban]

I read that an issue with returning non 2xx return codes to clients can lead turbo to abandon remote caching and only relying on local cache (see this comment).

If we want to keep this working we would need to return an ok response.
Maybe a 202 to distinguish from non intercepted calls ?

[dvcolomban]

If I'm not mistaken, any PUT filtering would have to be done here right?

Could we also use a URL whitelist instead of a separate read-only instance if we want only to allow PUT from trusted ci workflow ?

[fox1t]

Hey, that is the only part of the codebase that touches PUTs.
I think that having a whitelist is a great idea! You can add an onRequest hook to that route definition.

[anthonyshew]

Going to close this as a duplicate of #1188 since it has more reactions. If you'd like to signal boost with reactions/continue discussion, please do so there!

[NullVoxPopuli]

How do you configure it tho? do all devs still share the same secret-key for access? (this is dangerous)

[fox1t]

How do you configure it tho? do all devs still share the same secret-key for access? (this is dangerous)

Hey @NullVoxPopuli, can you elaborate? I am not sure I get it.

[NullVoxPopuli]

so, in order to contact a cache server, turbo needs a TURBO_TOKEN to authenticate with the server.

Exposing that to all employees gives folks the chance to try to poison the cache, or manually upload invalid caches for a particular cache-hash.

But if we keep the TURBO_TOKEN secret (only for CI), and then somehow have employees authenticate in some other way (in read-only mode), the risk is mitigated - disgruntled employees can no longer try to poison the cache(s).

[fox1t]

We might add SQLite as a persistence layer and save it to the same storage where the cache is saved. At that point, we can start exposing an "admin" interface.