Problem
When sandbox creation fails because no node can host it, the client receives a generic 500 error:
This tells the user nothing about why placement failed. The actual cause is buried in server logs as an internal error and is never surfaced. Common root causes include:
- CPU model mismatch: the template was built on a newer CPU generation (e.g. Emerald Rapids, model 207) but all cluster nodes run an older generation (e.g. Ice Lake, model 106). Cross-generation placement is asymmetric — an n2 build can resume on an n4 node, but not the reverse.
- Label mismatch: the team or template requires scheduling labels (e.g.
gpu, fast-disk) that no available node carries.
- All nodes not-accepting: nodes are draining, unhealthy, or saturated.
Without seeing which filter eliminated all nodes it is impossible to self-diagnose or file a useful bug report.
Root cause
BestOfK.chooseNode returns a FailedToPlaceSandboxError whose Error() string includes machine=... and labels=... constraints, but create_instance.go stores that in the internal Err field and returns a hardcoded ClientMsg: "Failed to place sandbox" that never reaches the client.
Proposed fix
- Track per-filter rejection counts inside
BestOfK.sample() (not-accepting, cpu-incompatible, label-filtered, excluded).
- Embed those counts plus the build CPU constraints in
FailedToPlaceSandboxError.Error().
- Propagate
err.Error() to ClientMsg in create_instance.go.
Example new error message:
Failed to place sandbox: no compatible node found (38 nodes checked: 0 not-accepting, 38 cpu-incompatible, 0 label-filtered, 0 excluded); build cpu: arch=x86_64 family=6 model=207
This lets the caller immediately identify a CPU-generation mismatch or label gap without digging through server logs.
Problem
When sandbox creation fails because no node can host it, the client receives a generic 500 error:
This tells the user nothing about why placement failed. The actual cause is buried in server logs as an internal error and is never surfaced. Common root causes include:
gpu,fast-disk) that no available node carries.Without seeing which filter eliminated all nodes it is impossible to self-diagnose or file a useful bug report.
Root cause
BestOfK.chooseNodereturns aFailedToPlaceSandboxErrorwhoseError()string includesmachine=...andlabels=...constraints, butcreate_instance.gostores that in the internalErrfield and returns a hardcodedClientMsg: "Failed to place sandbox"that never reaches the client.Proposed fix
BestOfK.sample()(not-accepting, cpu-incompatible, label-filtered, excluded).FailedToPlaceSandboxError.Error().err.Error()toClientMsgincreate_instance.go.Example new error message:
This lets the caller immediately identify a CPU-generation mismatch or label gap without digging through server logs.