k8 secrets RESTful support

[s-alad]

What does this PR do?

adds support for kubernetes secrets

Motivation

supporting container secrets

Additional Notes

RBAC: the customer using k8s secrets needs to have a proper RBAC authorization setup in order to allow the secret-backend to access the correct namespaces.

The ServiceAccount of the pod running the Agent/SGC must have automountServiceAccountToken: true (default) in order for our system to be able to interact with the k8 REST API.

The ServiceAccount must be granted permission to read secrets from each namespace where secrets are needed via RBAC:

• A Role in each secrets namespace defining the 'get' permission on secrets
• A RoleBinding in each secrets namespace linking your Agent's ServiceAccount to that Role

Sources:

• https://kubernetes.io/docs/tasks/run-application/access-api-from-pod/#directly-accessing-the-rest-api
• https://kubernetes.io/docs/reference/access-authn-authz/authentication/
• https://kubernetes.io/docs/concepts/security/service-accounts/
• https://kubernetes.io/docs/concepts/configuration/secret/
• https://kubernetes.io/docs/reference/kubernetes-api/config-and-storage-resources/secret-v1/#Secret
• https://kubernetes.io/docs/reference/kubernetes-api/common-definitions/status/
• https://github.com/kubernetes/client-go/blob/master/rest/config.go#L543
• https://thegreycorner.com/2025/01/22/accessing-the-kubernetes-api-using-captured.html#accessing-namespaced-resources

Possible Drawbacks / Trade-offs

• size increase: 288 bytes

Describe how to test/QA your changes

Setup a test cluster and create a k8 secret to test on. This script allows for an easy setup and test:

• requires kind
• touch test-k8.sh
• chmod +x ./test-k8.sh
• expected: {"datadog-sgc-test/dd-api-keys;api_key":{"value":"test-123","error":null},"datadog-sgc-test/dd-api-keys;app_key":{"value":"test-456","error":null}}

#!/bin/bash
set -e

kind delete cluster --name sgc-test 2>/dev/null || true
kind create cluster --name sgc-test

kubectl create namespace datadog-sgc-test

kubectl create secret generic dd-api-keys \
  --namespace datadog-sgc-test \
  --from-literal=api_key=test-123 \
  --from-literal=app_key=test-456

kubectl apply -f - <<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
  name: secret-backend
  namespace: datadog-sgc-test
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: secret-reader
  namespace: datadog-sgc-test
rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: secret-backend
  namespace: datadog-sgc-test
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: secret-reader
subjects:
- kind: ServiceAccount
  name: secret-backend
EOF

GOOS=linux GOARCH=amd64 go build -o datadog-secret-backend

docker build -t secret-backend:test -f - . <<'DOCKERFILE'
FROM alpine:latest
COPY datadog-secret-backend /datadog-secret-backend
ENTRYPOINT ["/datadog-secret-backend"]
DOCKERFILE

kind load docker-image secret-backend:test --name sgc-test

kubectl run secret-backend-test \
  --namespace datadog-sgc-test \
  --image=secret-backend:test \
  --restart=Never \
  --overrides='{"spec":{"serviceAccountName":"secret-backend"}}' \
  --command -- sleep 3600

kubectl wait --for=condition=Ready pod/secret-backend-test -n datadog-sgc-test --timeout=60s

echo ""
echo "testing in-cluster config:"
kubectl exec -n datadog-sgc-test secret-backend-test -- sh -c \
  'echo "{\"secrets\":[\"datadog-sgc-test/dd-api-keys;api_key\",\"datadog-sgc-test/dd-api-keys;app_key\"],\"version\":\"1.0\",\"type\":\"k8s.secrets\",\"config\":{}}" | /datadog-secret-backend'

echo ""
kind delete cluster --name sgc-test