HTTP server missing ReadTimeout / ReadHeaderTimeout — vulnerable to slowloris-style connection exhaustion #196
-
Issue: HTTP server missing ReadTimeout / ReadHeaderTimeout — vulnerable to slowloris-style connection exhaustion
ProblemThe Go s.httpServer = &http.Server{
Handler: handler,
}Go's own documentation recommends setting these timeouts to prevent slow-client attacks:
ImpactWithout these timeouts, any client can hold a kamal-proxy connection open indefinitely by slowly dripping request data. In production, we're seeing ~1000 requests/day from a botnet (mostly
These requests return 404 correctly, but each one ties up a connection slot for minutes. The requests show up in kamal-proxy's access logs as slow 404s with empty Example log entry: {
"host": "cancerckmecircle.com",
"service": "",
"target": "",
"status": 404,
"duration": 153461846083,
"method": "POST",
"req_content_type": "application/octet-stream",
"client_addr": "5.127.41.175"
}Suggested fixSet sensible defaults on both HTTP servers in s.httpServer = &http.Server{
Handler: handler,
ReadHeaderTimeout: 10 * time.Second,
ReadTimeout: 30 * time.Second,
IdleTimeout: 120 * time.Second,
}These should also be configurable through the proxy section of the Kamal deploy config, similar to how proxy:
response_timeout: 30 # already exists
read_timeout: 30 # proposed
idle_timeout: 120 # proposedSensible defaults alone would close the vulnerability, but configurability would let users tune these for their specific use case (e.g. large file uploads may need a longer This is distinct from the per-service WorkaroundCurrently the only workaround is firewall-level IP blocking, which doesn't help against distributed sources. Related
|
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
@SanjinDedic agreed, we should add timeouts for this case. I'd rather not enforce proxy-level timeouts on the deployed services. But I think what we could do is have default server-level timeouts, and then per-service, configurable timeouts that override them. That also means each service can vary its timeout settings between deploys without requiring a proxy restart. I'll put together a fix for this. Thanks for bringing it up. |
Beta Was this translation helpful? Give feedback.
@SanjinDedic agreed, we should add timeouts for this case.
I'd rather not enforce proxy-level timeouts on the deployed services. But I think what we could do is have default server-level timeouts, and then per-service, configurable timeouts that override them. That also means each service can vary its timeout settings between deploys without requiring a proxy restart.
I'll put together a fix for this. Thanks for bringing it up.