One question, answered in your vocabulary rather than this project's: what can I reasonably test against this emulator?
Every row carries an anchor. A yes names the suite or the assertion that
proves it, a conditional row names the capability it keys on, and a no links
the section of limits.md that says why. A row with nothing to point
at does not belong here, and a test refuses one
(TestEveryConfidenceRowCarriesItsProof).
This is the coarse view, for a human deciding whether to try. The fine view is
per operation and generated: routes.md prints what every mounted
operation has earned — the generated block below carries the count — and
coverage/evidence.json is the record it prints.
409 operations mounted, 384 driven by a real client in the recorded run. The 25 that are not each state why at their route, and routes.md prints the reason under the pack that owns it.
| What you want to validate | Verdict | Proof, or the limit |
|---|---|---|
| Terraform and OpenTofu syntax, plan and lifecycle | yes | mise run conformance: apply, an empty second plan, destroy — conformance.md |
A CLI workflow end to end (scw, octl, exo) |
yes | the same run drives all three against one emulator — conformance.md |
| Request and response shapes, field by field | yes, with a stated ceiling | every answer is checked against the provider's own API description, and what a schema cannot catch is named in conformance.md |
| That your client reads a field the cloud really returns | yes, where a recording exists | recorded shapes arbitrate; the blind spot is counted in the run's own output — conformance.md |
| Resource dependency order: what refuses to be deleted under what | yes, while the relation is synchronous | the refusal assertions in each suite; a relation the real cloud holds for a while AFTER a delete is not reproduced, and that is stated with the measurement in limits.md |
| Error handling: that your code decodes a refusal | yes | each suite demands refusals and the emulator refuses to close the bracket when none happened — conformance.md |
| Cloud-init, an SSH login, a security group that filters | with a runtime | FEINT_VM=incus mise run conformance and mise run conformance:ssh; keys on capabilities.firewall, whose bounds are stated in limits.md |
| Two VPCs that cannot reach each other | OVN only | keys on capabilities.isolation, never on a mode name — limits.md |
| A security group with a stateless rule | OVN only | keys on capabilities.stateless_rules: Incus applies allow-stateless only on OVN networks, so a bridge mode refuses such a group by name and logs a WARN; the refused version is not written, and a stateful version written earlier under the same name stays attached as it was — limits.md |
| A load balancer that distributes real connections | OVN, and per pack | keys on the conjunction of capabilities.balancing (the runtime can) and enforced.balancing (this pack hands its balancers over) — each half has been true while the other was false — limits.md |
That what the API claims is really on the host: a running machine exists, a claimed firewall left rule sets, a claimed balancer distributes |
OVN, per provider | mise run conformance:witness reads the claims off each pack's API and the witnesses off the runtime with incus, and a pack that claims no dataplane is skipped by name — the gate #475, #481, #483 and #484 share (#486); what a green control-plane run does not prove is stated in conformance.md |
| That a stack's machines actually do something: a service that listens, answers over the published address and survives a restart; a port a rule opens answering while a port no rule opens refuses, in one pass | OVN, per stack | mise run conformance:functional reads what each stack declares in its own proof.json and probes it from another machine or from the station, never through the host: a harness reaching the machine with incus exec would have found none of the four defects it was built for (#503). The bounds it skips by name are the routed NIC's and the same-subnet one, both in limits.md |
| Reaching a private address from the host that runs the emulator | depends on the mode | keys on capabilities.private_from_host — limits.md |
| A kernel-level test: sysctl, modules, the boot path | with a VM runtime | FEINT_VM=incus-vm, keying on capabilities.own_kernel: a container shares the host kernel and cannot carry any of it — limits.md |
| That a bad image or instance-type id is rejected | with serve --strict-catalog, against the catalogue you declared |
off by default, and the default is the decision; each pack refuses in its own shape, and which of those shapes were recorded is in the table — limits.md |
| Provider quotas, real capacity, prices | no | limits.md |
| Authentication, IAM permissions, a signature that must fail | no | limits.md |
Waiting behaviour: a resource that stays provisioning for a while |
partly, with serve --consistency eventual |
a Scaleway server walks the states fr-par walks, one per read (#637); an Outscale volume is creating then available and a snapshot in-queue then completed, and a snapshot taken during creating meets the measured 409 InvalidVolumeState (#124); immediate is still the default — limits.md |
| Retry and backoff against 429s, 5xx or a control-plane outage | yes, on the operations you arm | PUT /_feint/faults names an operation and a status, off by default, and deterministic under a client that does not parallelise — the schedule counts arrivals, limits.md; what an injected refusal does NOT count as is stated in conformance.md |
| S3 bytes through declared explicit doors | opt-in, AWS CLI 2.32.17, scw 2.62.0, exo 1.95.1 upload/list/download/delete | mise run conformance:s3:mounted drives private Scaleway buckets, Exo multipart/range downloads (6,340,608 bytes), AWS restart/resume/completion, and Exo dangling-upload abort/bucket removal; actual bytes, dates and inventories are checked. Normal Exoscale bucket creation is not certified — object-storage-routing.md |
| Hardcoded Scaleway Terraform Object Storage endpoints | opt-in, Terraform with the published provider 2.82.0 through feint run |
TestScalewayTerraformStorageThroughFeintRunOnTheHostNetwork, on the host network with no sudo and no namespace: apply complete, 139,264 identical bytes read back from the local byte store by sha256, an empty second plan, a complete teardown. Not run on a pull request — it needs a provider mirror, so it is a measurement you reproduce rather than a gate, and OpenTofu is not covered — object-storage-routing.md |
| A Terraform run against Exoscale | yes, from provider v0.71.0 | the published provider drives this pack since #644; an older one, which splits between this emulator and a paying account, is refused by user agent naming the floor — limits.md |
| Control-plane multi-zone or multi-region behaviour on Exoscale | no | one zone per process, selectable with FEINT_EXOSCALE_ZONE, and the reason is the client — limits.md |
| Traffic actually flowing through a gateway or a NAT | no | records move, packets do not — limits.md |
| That a run stays on your machine | yes, for what Feint serves | the APIs it serves answer locally; a client that composes its own endpoint for a product outside that scope leaves, and the four measured paths are in limits.md — feint run closes the first of them since 0.15.0 |
yes means a real client drove it in the run that produced
coverage/evidence.json, and that the suite
asserted something about the answer. It does not mean every field of every
response is right: two defects sat behind a driven axis for months, which is
why the record publishes seven independent axes and never adds them into a score.
with a runtime means the control plane answers without one, and the claim only holds when machines are real. Ask the emulator rather than guessing:
curl -s localhost:4599/_feint/health | jq .capabilitiesA capability nobody declares reads as absent, deliberately, so a check skips itself instead of asserting what no driver promised.
no is a decision with a reason, not a gap waiting to be filled by accident. Each one links the section that states the cost. Several would change with work somebody could do — fault injection, an opt-in identifier check, observable transient states — and those are issues, not omissions.
It is not a coverage percentage, and it deliberately carries no score. A single number lets a weak claim ride on a strong one; that is the arithmetic version of the overstatement this whole project exists to avoid. If you want the fine view, it is per operation and it is generated.