move scripts into something that tracks script dependencies

trying just here as well because we are already doing cool stuff with sd
and gron.

Signed-off-by: clux <[email protected]>

Author: clux

README.md

@@ -1,44 +1,34 @@
 Experimenting with Kubernetes protobufs.
 
-## Protobufs
+## Build Dependencies
 
-### Download
+- [gron](https://github.com/tomnomnom/gron)
+- [just](https://github.com/casey/just)
+- [sd](https://github.com/chmln/sd)
+- [jq](https://stedolan.github.io/jq/)
 
-Get protos by extracting them from Kubernetes releases:
+## Protobufs
+We get protos by extracting them from pinned Kubernetes releases:
 
 - https://github.com/kubernetes/api/releases
 - https://github.com/kubernetes/apimachinery/releases
 - https://github.com/kubernetes/apiextensions-apiserver/releases
 - https://github.com/kubernetes/kube-aggregator/releases
 - https://github.com/kubernetes/metrics/releases
 
-```bash
-# In `protos/`
-VERSION=1.22.0
-for x in api apimachinery apiextensions-apiserver kube-aggregator metrics; do
-  mkdir ./$x -p;
-  curl -sSL https://github.com/kubernetes/$x/archive/refs/tags/kubernetes-$VERSION.tar.gz | tar xzf - -C ./$x/ --strip-components=1;
-  fd -e proto -x sh -c "mkdir -p k8s.io/'{//}'; mv '{}' k8s.io/'{}'" ';' . ./$x;
-  rm -rf ./$x;
-done
-```
+We then do minor transforms on top of that to prepare for building.
+Results of this step is committed already. But to run, invoke `just protos`
 
-### Patch
+## Openapi
+To complement the protos with generic information, we also download the swagger schema, patch it, and transform it as described below.
 
-Removing `k8s.io.`:
+Results of this step is committed already. But to run, invoke `just swagger`.
 
-```bash
-fd -e proto -x sd 'k8s\.io\.(.+);' '$1;' {}
-fd -e proto -x sd 'import "k8s\.io/(.+)";' 'import "$1";' {}
-mv protos/k8s.io/* protos/
-rmdir protos/k8s.io/
-```
 
-### Generate
-Refresh input paths to generate, then build:
+## Building
+With all dependent `swagger` and `protos` files built, run:
 
 ```bash
-fd -e proto | sort > protos.list
 cargo build
 ```
 
@@ -68,7 +58,7 @@ See [`prost_build`](https://github.com/tokio-rs/prost/blob/32bc87cd0b7301f6af1a3
 [`FileDescriptorSet`]: https://github.com/tokio-rs/prost/blob/32bc87cd0b7301f6af1a338e9afd7717d0f42ca9/prost-types/src/protobuf.rs#L1-L7
 
 
-## OpenAPI
+## OpenAPI Strategy
 
 We need to use `swagger.json` to fill in some information.
 
@@ -91,28 +81,6 @@ We should be able to find the following:
     - May also have paths for all namespaces for some verbs (e.g., `list` all pods)
   - Subresource if path contains `/{name}/` (`/` after `{name}`)
 
-### Download
-
-In `openapi/`
-
-```bash
-VERSION=1.22.0
-curl -sSL -o swagger.json \
-    https://raw.githubusercontent.com/kubernetes/kubernetes/v$VERSION/api/openapi-spec/swagger.json
-```
-
-### Bug Fix
-
-Fix path operation annotated with a `x-kubernetes-group-version-kind` that references a type that doesn't exist in the schema. (See [`k8s-openapi`](https://github.com/Arnavion/k8s-openapi/blob/445e89ec444ebb1c68e61361e64eec4c4a3f4785/k8s-openapi-codegen/src/fixups/upstream_bugs.rs#L9)).
-
-```bash
-gron swagger.json \
-| perl -pe 's/(?<=kind = ")(Pod|Node|Service)(?:Attach|Exec|PortForward|Proxy)Options(?=")/$1/' \
-| gron -u \
-> swagger-patched.json
-mv swagger-patched.json swagger.json
-```
-
 ### Transforming
 
 Transform `swagger.json` to something easier to explore.

justfile

@@ -0,0 +1,62 @@
+VERSION := "1.22.0"
+
+default:
+  @just --list
+
+# Download protos schemas from upstream
+protos-dl:
+    #!/usr/bin/env bash
+    set -exuo pipefail
+    rm -rf protos && mkdir protos && cd protos
+    for x in api apimachinery apiextensions-apiserver kube-aggregator metrics; do
+        mkdir ./$x -p
+        curl -sSL https://github.com/kubernetes/$x/archive/refs/tags/kubernetes-{{VERSION}}.tar.gz | tar xzf - -C ./$x/ --strip-components=1
+        fd -e proto -x sh -c "mkdir -p k8s.io/'{//}'; mv '{}' k8s.io/'{}'" ';' . ./$x
+        rm -rf ./$x
+    done
+
+# Patch protos schemas to fix import paths
+protos-patch:
+    #!/usr/bin/env bash
+    set -exuo pipefail
+    fd -e proto -x sd 'k8s\.io\.(.+);' '$1;' {}
+    fd -e proto -x sd 'import "k8s\.io/(.+)";' 'import "$1";' {}
+    mv protos/k8s.io/* protos/
+    rmdir protos/k8s.io/
+
+# Generate protos path list for prost
+protos-list:
+    fd -e proto | sort > protos.list
+
+# Download and generate all protos dependent files
+protos: protos-dl protos-patch protos-list
+
+# Download swagger
+swagger-dl:
+    #!/usr/bin/env bash
+    set -exuo pipefail
+    curl -sSL -o openapi/swagger.json \
+        https://raw.githubusercontent.com/kubernetes/kubernetes/v{{VERSION}}/api/openapi-spec/swagger.json
+
+# Patch swagger schema for upstream bugs
+swagger-patch:
+    #!/usr/bin/env bash
+    set -exuo pipefail
+    cd openapi
+    # Fix path operation annotated with a `x-kubernetes-group-version-kind` that references a type that doesn't exist in the schema.
+    # See https://github.com/Arnavion/k8s-openapi/blob/445e89ec444ebb1c68e61361e64eec4c4a3f4785/k8s-openapi-codegen/src/fixups/upstream_bugs.rs#L9
+    gron swagger.json \
+        | perl -pe 's/(?<=kind = ")(Pod|Node|Service)(?:Attach|Exec|PortForward|Proxy)Options(?=")/$1/' \
+        | gron -u \
+        > swagger-patched.json
+    mv swagger-patched.json swagger.json
+
+# Transform swagger schema into api-resources.json
+swagger-transform:
+    #!/usr/bin/env bash
+    set -exuo pipefail
+    cd openapi
+    jq -f list-resources.jq < swagger.json > api-resources.json
+
+# Download and generate all swagger dependent files
+swagger: swagger-dl swagger-patch swagger-transform