fix(hetznercloud): poll zone action endpoint with adequate timeout#1149
fix(hetznercloud): poll zone action endpoint with adequate timeout#1149buddhaCode wants to merge 5 commits into
Conversation
Record updates submitted via Zone RRSet operations (e.g. set_records) return
an asynchronous action that belongs to the zone resource. Two problems made
these updates fail even though the record was changed correctly server-side:
1. waitAction polled the server actions endpoint (/v1/servers/actions/{id})
instead of the zone actions endpoint (/v1/zones/actions/{id}), so the
lookup always returned 404 (action not found). The action path scheme
matches Hetzner's official hcloud-go library (ResourceActionClient with
resource "zones").
2. Even against the correct endpoint, zone actions take ~20s to reach the
success status, but waitAction only polled 3 times with a 1s sleep (~3s),
giving up with "did not complete after 3 tries". Poll every 5s for up to
60s instead.
Both were verified against the live Hetzner Cloud API: the old endpoint
returns 404 for a zone action id while the new one returns 200, and the real
ddns-updater binary now completes the update without error.
Add unit tests covering the queried URL, the running->success polling loop,
the timeout, and the success/404/action-error cases.
Fixes qdm12#1136
Add a regression test for the wildcard rapid-update loop reported in issue qdm12#1039: a wildcard owner "*" must keep the literal "*" in the rrset path (the API returns 404 for "%2A") and poll the zone actions endpoint. Covers both the create and set_records paths via a recording transport. Extract the shared dummy token into a testToken constant.
|
Additional note: this also fixes the wildcard rapid-update loop reported in #1039 (comment by @Daxolion). That report looked like a separate, wildcard-specific bug, but it is the same Zone-action polling issue fixed here. A wildcard owner I verified this against the live Hetzner Cloud API with a real wildcard owner, on both the create and the I'm intentionally not adding a |
The Hetzner Cloud API returns an action error as an object {code, message}, not a bare string. Mock that shape so the test reflects the real API response.
Use the canonical reference anchor (#tag/zone-actions/get_zone_action) so the link points at the right operation.
|
Pushed two small follow-up commits while cross-checking the provider against the official API reference and
While doing this I noticed a related, pre-existing point that is outside this PR's scope: Would you prefer I fold that fix into this PR, or keep it separate so this one stays focused on the zone-action endpoint? Happy either way. |
What
The
hetznercloudprovider marks successful record updates as failed,leaving the container permanently unhealthy after an IP change. This fixes two
distinct bugs in the asynchronous action handling.
Fixes #1136
Background: server actions vs zone actions
The Hetzner Cloud API returns an asynchronous action (a background job with
an id and a status) for many operations, and you poll that action until it
reports
success. Crucially, actions are scoped to the resource type theybelong to, and each type has its own lookup path
/{resource}/actions/{id}:/v1/servers/actions/{id}— actions for Cloud Servers (VPS instances),a completely separate Hetzner Cloud product.
/v1/zones/actions/{id}— actions for DNS zones, which is what thisprovider works with.
A DNS record update produces a zone action, so it must be queried on the
zone path. Querying it on the server path asks the VPS/server subsystem for an
id it has never seen →
404 not_found.Root cause
Record updates via Zone RRSet operations (e.g.
set_records) return a zoneaction that
waitActionthen polls until it reachessuccess. Two thingswere wrong:
Wrong endpoint. It polled the server actions endpoint
https://api.hetzner.cloud/v1/servers/actions/{id}instead of the zoneactions endpoint
https://api.hetzner.cloud/v1/zones/actions/{id}, so thelookup always returned
404 not_found. The record itself was alreadyupdated server-side, but the 404 was treated as an update failure.
(waitaction.go#L20)
Too short a timeout. Even against the correct endpoint, zone actions
take ~20s to reach
success, butwaitActiononly polled 3 times with a1s sleep (~3s total) and then gave up with "did not complete after 3 tries".
Fixing only the URL would just swap the 404 for a timeout error — the
container would stay unhealthy.
The action path scheme (
/{resource}/actions/{id}withresource = "zones")matches Hetzner's official hcloud-go
library.
Fix
duration, while keeping the existing success / error / completion checks.
Verified against the live Hetzner Cloud API
…/v1/servers/actions/{id}→404 not_found; new endpoint…/v1/zones/actions/{id}→200with"command": "set_rrset_records"and"resources": [{ "type": "zone" }].ddns-updaterbinary, before:ERROR … 404: not_found; after:the update completes with no error (verified by reading the record back).
Alternatives considered
treat the write as synchronous and consider the update successful as soon
as
set_records/add_recordsreturns201, without waiting for theaction to reach
success. The record value is in fact applied server-sidebefore the action completes (confirmed by reading it back while the action
is still
running). This could be offered as an opt-in provider setting(e.g.
"skip_action_check": true) for users who prefer speed/robustnessover confirmation. It is not the default here because it gives up genuine
confirmation that the zone change was accepted.
further API quirks, but masks real failures.
This PR keeps the (now working) status check as the default. Happy to add the
opt-in
skip_action_checkoption as well if you'd like it in the same PR.Open question on the timing values
I went with a fixed 5s interval / 60s total, which comfortably covers the
~20s I measured. Happy to change this if you'd prefer:
settings (the provider already documents optional params like
ttlindocs/hetznercloud.md).
Let me know your preference and I'll adjust.
Tests
waitaction_test.gocovers the queried URL (the regression), therunning → successpolling loop, the timeout path, and the success / 404 /action-error cases.
go test ./...andgolangci-lint runpass.