Extend Cache Policy to support more configuration parameters

[thresheek]

Is your feature request related to a problem? Please describe.
Currently, on 5.2.0, Cache Policy doesnt allow users to specify inactivity and use_temp_path parameters as available for proxy_cache_path directive. This means that users cant set the inactive cache lifetime and are forced to spend disk IO on moving the cache files (e.g. when using a StatefulSet and a separate volume).

Those are the options that were specifically asked by the customers.

Additionally, it's not possible to configure:

• anything cache manager related in proxy_cache_path - max_size, min_free, manager_* which are crucial to maintain the filesystem usage for caches

• loader_* is less widely used, but sometimes tuning can be beneficial in the scenarios where NGINX is restarted and needs to prime the cache info

• proxy_cache_key which would allow users to redefine the caching keys for the cache items

• proxy_cache_use_stale and proxy_cache_revalidate are useful to minimize the impact on backends when updating the expired cached items

• proxy_no_cache and proxy_cache_bypass would allow to bypass the caching for specific requests

• proxy_cache_lock* directives are useful for highly loaded services, eliminating the pressure on backends

Describe the solution you'd like
A support for aforementioned items in the Cache Policy

Describe alternatives you've considered
Snippets can still be used to implement the desired functionality.

[thresheek]

I've got a confirmation from user that they use the following directives:

proxy_cache
proxy_cache_background_update
proxy_cache_key
proxy_cache_lock
proxy_cache_min_uses
proxy_cache_path
proxy_cache_revalidate
proxy_cache_use_stale
proxy_ignore_headers

They confirmed they use inactive and use_temp_path in proxy_cache_path, and redefine proxy_cache_key using a mix of nginx-provided built-in variables and variables set up by map, too.

[haywoodsh]

Original design

#7773 (comment)

[haywoodsh]

Possible updated policy included all fields mentioned above:

apiVersion: k8s.nginx.org/v1
kind: Policy
metadata:
  name: my-cache-policy
spec:
  cache:
    cacheZoneName: "mycache"                    # Required - proxy_cache (zone name)
    cacheZoneSize: "10m"                        # Required - proxy_cache_path (keys_zone size)
    allowedCodes: [200, 301, 302]              # Optional - proxy_cache_valid (which HTTP codes to cache)
    allowedMethods: ["GET", "POST"]            # Optional - proxy_cache_methods (which methods to cache)
    time: "10m"                                 # Optional - proxy_cache_valid (default time)
    cachePurgeAllow: ["10.0.0.1","127.0.0.0/24"] # Optional - proxy_cache_purge (NGINX Plus only)
    overrideUpstreamCache: true                 # Optional, false by default - proxy_ignore_headers (Cache-Control, Expires, Set-Cookie, Vary)
    levels: "1:2"                               # Optional - proxy_cache_path (directory levels)

    #new fields required
    inactive: "60m"                             # Optional - proxy_cache_path (inactive parameter)
    useTempPath: false                          # Optional, false by default - proxy_cache_path (use_temp_path=off)
    maxSize: "10g"                              # Optional - proxy_cache_path (max_size)
    minFree: "1g"                               # Optional - proxy_cache_path (min_free)
    managerFiles: "100"                         # Optional - proxy_cache_path (manager_files)
    managerSleep: "50ms"                        # Optional - proxy_cache_path (manager_sleep)
    managerThreshold: "200ms"                   # Optional - proxy_cache_path (manager_threshold)
    cacheKey: "$scheme$request_method$host$request_uri" # Optional - proxy_cache_key (custom cache key)
    cacheUseStale: ["error", "timeout", "updating", "http_500"] # Optional - proxy_cache_use_stale (serve stale conditions)
    cacheRevalidate: true                       # Optional, false by default - proxy_cache_revalidate
    cacheBackgroundUpdate: true                 # Optional, false by default - proxy_cache_background_update
    cacheMinUses: 3                             # Optional - proxy_cache_min_uses (requests before caching)
    cacheLock: true                             # Optional, false by default - proxy_cache_lock
    cacheLockTimeout: "5s"                      # Optional - proxy_cache_lock_timeout
    cacheLockAge: "30s"                         # Optional - proxy_cache_lock_age
    noCacheConditions: ["$cookie_nocache", "$arg_nocache"] # Optional - proxy_no_cache (skip caching conditions)
    cacheBypassConditions: ["$http_authorization"] # Optional - proxy_cache_bypass (bypass cache conditions)
    ignoreHeaders: ["Set-Cookie", "Vary"]       # Optional - proxy_ignore_headers (additional headers to ignore) potentially overlap with overrideUpstreamCache