hardening: Use `network_mode: service:gateway` to avoid sudo and elevated capabilities in agent container

[jakub-bochenski]

Related to SANDBOX-23

See here: https://github.com/VirtusLab/sandcat/blob/4b92dc4/compose-all.yml#L14
points to wg-client service here: https://github.com/VirtusLab/sandcat/blob/4b92dc4d94074282c0214438dd6a375fef183280/compose-proxy.yml#L4-L22

Wireguard is no needed for this, we can just have an alpine etc. container configure the routing and have the agent container use the same network namespace.

This will remove the need for sudo and NET capabilities on the agent container.
It will also make the agent image smaller which is also a good thing

[mattolson]

The capability reduction argument is valid. The agent container currently retains NET_ADMIN and NET_RAW for its entire lifetime, even though they're only used once during init-firewall.sh. A compromised or misbehaving agent could potentially use NET_ADMIN to modify iptables rules and disable the firewall. Unlikely, but theoretically possible.

I'd like to pursue this, but decoupled from WireGuard and transparent proxying. The general shape:

• Add a lightweight gateway sidecar (alpine + iptables) that owns the firewall configuration
• Agent container uses network_mode: "service:gateway" and inherits the locked-down network namespace
• Drop NET_ADMIN and NET_RAW from the agent container entirely
• Startup ordering: proxy (healthy) -> gateway (configures firewall, healthy) -> agent

This also addresses the image size concern you raised. The agent image no longer needs iptables or iproute2.

One complication: our agent entrypoint currently runs three sudo scripts, not just the firewall:

1. init-firewall.sh - configures iptables (moves to gateway)
2. install-proxy-ca.sh - copies the mitmproxy CA cert into the system trust store
3. configure-apt-proxy.sh - writes apt proxy config

The CA and apt scripts modify the agent container's filesystem, so they can't move to the gateway. To fully eliminate SETUID/SETGID from the agent, we'd also need to solve CA trust and apt proxy setup without sudo. Some options:

• Use SSL_CERT_FILE / NODE_EXTRA_CA_CERTS / REQUESTS_CA_BUNDLE env vars pointing at the mounted CA cert, instead of installing into the system trust store. This avoids needing root to run update-ca-certificates.
• Mount apt proxy config as a read-only file from the host or generate it in an init container, rather than writing it at runtime.
• Pre-configure both at image build time where possible.

I don't want to ship the gateway sidecar gated on solving all of this at once. A reasonable phased approach:

1. Move firewall to gateway sidecar, drop NET_ADMIN/NET_RAW from agent
2. Keep SETUID/SETGID temporarily for CA and apt proxy setup
3. Follow up by replacing sudo-based CA/apt setup with env vars and mounted config
4. Drop SETUID/SETGID once those are done, reaching cap_drop: ALL on the agent

I'll put together a plan doc with the compose layout and migration path.

[jakub-bochenski]

LGTM

The only aspect I'm worried about is:

Use SSL_CERT_FILE / NODE_EXTRA_CA_CERTS / REQUESTS_CA_BUNDLE env vars

Setting tool specific envs seem brittle, there's always more and more programs to cover (e.g. Java :))
Personally I'd accept sudo for this if it's the only way to install the certificate.

Some minor points:

This also addresses the image size concern you raised.

Just to clarify: I'm not so much worried about the image size in bytes. Rather I think the image should be as simple as possible. So the less it does the better. Easier to understand, and harder to break when customizing

I don't want to ship the gateway sidecar gated on solving all of this at once. A reasonable phased approach:

I agree, in the OP i meant that it removes one need for sudo, not that it should get rid of all sudo cases.

[jakub-bochenski]

Created #132 for the certificate problem