Changing the proxy host

[notnek]

Has anyone changing their proxy host with success? I have a staging server that is running 3 separate rails apps.

• app1 => staging.app1.com
• app2 => app2-staging.example.com
• app3 => staging.app3.com

This all works fine. But I wanted to move app2 to staging.app2.com. Deploys fine, I can ls kamal-proxy's mappings and I see it mapping staging.app2.com to the correct and running docker container.

However, when I visit the domain, I get a too many redirects error.

When I view the logs, a request to app2 matches app1 exactly except the target parameter is an empty string where app1 has the container ID/port as expected.

I've tried removing the app with the old config, and then deploying the new config. I've rebooted kamal-proxy. I've restarted the server. I'm not sure what else to try. The mapping looks correctly. staging.app2.com maps to the right container ID and port.

I can also revert the proxy config change and re-deploy and it works on the old host again.

Has anyone ran into this or have suggestions on how to resolve? Thanks!

[kevinmcconnell]

@notnek I've not seen this before. What does the redirect response look like, is it literally redirecting back to the same location, or is there a loop involved?

Also can you see these requests in your app logs as well?

Kamal Proxy doesn't really issue redirects itself (other than to go from HTTP->HTTPS), so I'm wondering if these requests are reaching the app but somehow causing a redirect from there. The empty target does seem strange though!

[notnek]

Here's a request log before the change which is when the app loads fine.

2025-02-27T19:22:50.361892544Z {"time":"2025-02-27T19:22:50.361742423Z","level":"INFO","msg":"Request","host":"app2-staging.example.com","port":443,"path":"/","request_id":"e89e824e-642d-4a3a-b008-ebc273c632c0","status":200,"service":"app2-staging","target":"ad6a5cb8a9af:80","duration":147923718,"method":"GET","req_content_length":0,"req_content_type":"","resp_content_length":6103,"resp_content_type":"text/html; charset=utf-8","client_addr":"***","client_port":"***","remote_addr":"***","user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.3 Safari/605.1.15","proto":"HTTP/2.0","scheme":"https","query":"","req_cache_control":"","req_last_modified":"","req_user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.3 Safari/605.1.15"}

And then, after only changing proxy.host, running kamal app remove and then kamal deploy. It errors out after 20 of these. I also had other devs on my team try loading the site and they get the same errors.

2025-02-27T19:28:11.572734172Z {"time":"2025-02-27T19:28:11.572554865Z","level":"INFO","msg":"Request","host":"staging.app2.com","port":80,"path":"/","request_id":"480f3e2e-c9cf-456c-8a82-cd061fc3654a","status":301,"service":"app2-web-staging","target":"","duration":18896,"method":"GET","req_content_length":0,"req_content_type":"","resp_content_length":70,"resp_content_type":"text/html; charset=utf-8","client_addr":"***","client_port":"***","remote_addr":"***","user_agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.3 Safari/605.1.15","proto":"HTTP/1.1","scheme":"http","query":""}

The app never gets the request so nothing in the app logs. When I remove the app, I start getting the blue 404 page. The empty target is definitely the weird part. It clearly knows the target mapping:

docker exec kamal-proxy kamal-proxy ls

Service           Host              Target           State    TLS
app2-web-staging  staging.app2.com  ae85f78d8e8a:80  running  yes

I am using Cloudflare for the DNS but both staging.app2.com and app2-staging.example.com are setup the same.

Thanks for any insight!

[kevinmcconnell]

Ah, I suspect Cloudflare is connecting to the proxy over plain HTTP, so the proxy is returning a redirect to HTTPS. But that HTTPS request again goes first to Cloudflare, which connects to the proxy over HTTP again, and so on.

If you want Cloudflare to do only DNS, you could turn off its proxy option to connect directly to Kamal Proxy instead.

But if you want to connect through Cloudflare, can you make sure your SSL mode is set to “Full (strict)”? It should work that way, as it'll use SSL end-to-end.

Let me know if that helps!

[notnek]

🤦🏻 of course. Thanks for pointing that out. I had just setup that domain on CF and didn't realized it had defaulted to Flexible. Changed it and now it works. Thanks so much!

[kevinmcconnell]

Sure thing. Glad to hear it's working now!