A patch about the denominator. Nothing a client can observe moved: an unserved
route answered 501 before this release and answers 501 after it.
Fixed
-
The coverage inventory had a second witness and never consulted it (#622).
Two inventories are committed here — the contract, extracted from each
provider's published document, and the coverage report, produced by scanning
their Go SDK — and for Scaleway they disagreed. The portal publishes fivePUT
whole-object routes ininstance/v1(SetImage,SetSecurityGroup,
SetSecurityGroupRule,SetSnapshot,SetVolume) and
iam/v1alpha1.CheckPermissions;scaleway-sdk-gonever wrapped any of them.
So all six were outsidetotal, andunknown: 0was exact over a set that did
not contain them.Measured on all three packs: six for Scaleway, zero for Outscale and Exoscale,
whose SDKs are generated from the same documents their contracts come from. The
gap exists exactly where a document and an SDK have different provenance.The six are triaged now, in
coverage/contract-only.json, each with a status —
declinedorbacklog— and a reason. They could not go inDeclined(),
because a decline the scan does not know is reported as an orphan, and orphans
are the one signal the drift gate exists to keep reliable.Reported by a consumer generating an Ansible collection from the Scaleway
OpenAPI documents and using feint as its integration backend — which is exactly
the client that reaches those five routes, and the reason they arebacklog
rather thandeclined.
Added
-
feint coverage --document <file> --contract-only <file>cross-checks the
scanned surface against the provider's own document and exits 2 on an operation
nobody decided about, the way an upstream addition already does. It is a
separate flag from--contracton purpose:--contractalso regroups the
report onto the document's tags, and reusing it would have moved every
published table as a side effect of adding a check.CLI surface version 22. Two additions to one existing verb; nothing was
removed and no exit code moved, so a pipeline keyed on 21 keeps working.
Proved against
The clients
| Client | Version | Pinned in | Drives, in CI |
|---|---|---|---|
scw
| 2.56.3 | .github/workflows/conformance.yml (SCW_VERSION)
| Scaleway |
| Terraform | 1.13.3 | .github/workflows/conformance.yml (TERRAFORM_VERSION)
| Outscale, Scaleway |
| OpenTofu | 1.12.5 | .github/workflows/conformance.yml (TOFU_VERSION)
| Outscale, Scaleway |
octl
| 0.0.31 | .github/workflows/conformance.yml (OCTL_VERSION)
| Outscale |
exo
| 1.95.6 | .github/workflows/conformance.yml (EXO_VERSION)
| Exoscale |
The Terraform providers
The client above is the engine; what answers this emulator is the provider.
Each row is one required_providers entry, read where it is written.
| Fixture or stack | Provider | Constraint | What that is worth | Applied in CI |
|---|---|---|---|---|
tools/conformance/faults
| outscale/outscale
| ~> 1.7
| constraint: resolved fresh on each run, so the version that answered is not knowable here | yes |
tools/conformance/outscale/terraform
| outscale/outscale
| ~> 1.7
| constraint: resolved fresh on each run, so the version that answered is not knowable here | yes |
tools/conformance/outscale/terraform-doorway
| outscale/outscale
| ~> 1.7
| constraint: resolved fresh on each run, so the version that answered is not knowable here | yes |
tools/conformance/scaleway/terraform
| scaleway/scaleway
| 2.81.0
| exact: the version that answered | yes |
examples/stacks/exoscale
| exoscale/exoscale
| — | not pinned: whatever the registry served that day | no |
examples/stacks/outscale
| outscale/outscale
| ~> 1.7
| constraint: resolved fresh on each run, so the version that answered is not knowable here | yes |
examples/stacks/outscale/modules/net
| outscale/outscale
| ~> 1.7
| constraint: resolved fresh on each run, so the version that answered is not knowable here | yes |
examples/stacks/scaleway
| scaleway/scaleway
| 2.81.0
| exact: the version that answered | yes |
examples/quickstart/outscale
| outscale/outscale
| ~> 1.7
| constraint: resolved fresh on each run, so the version that answered is not knowable here | yes |
examples/quickstart/scaleway
| scaleway/scaleway
| 2.81.0
| exact: the version that answered | yes |
Read the third column narrowly, because it is the one a consumer pins against.
An exact constraint names the version that answered. A constraint is
resolved by terraform init -upgrade on the runner, every run, so the version
that answered is not knowable from this repository — it is a floor, not a
proof. Not pinned is not pinned: nothing here says which version ran, and
no number is invented to fill the cell.
The stacks the last column says CI does not apply are declared rather than
merely absent, so a no is a decision somebody wrote down and not a stack
nobody wired up:
examples/stacks/exoscale— suspended — no Terraform for Exoscale
until upstream exoscale/terraform-provider-exoscale#573 is fixed: the
published provider honoursEXOSCALE_API_ENDPOINTfor one of the two
clients it builds, so an apply splits between the emulator and a paying
account, and #525 measured five signed requests leaving for the real cloud
from afeint downon this stack (2026-08-26).feint uprefuses the
engine at the doorstep; the exo CLI drives the pack instead. What the
pinned fork had proved stays dated in docs/limits.md: 16 resources, empty
second plan, clean destroy, 2026-08-24.
Generated from the conformance workflow and the fixtures at this tag.
A newer client is not known to work until the pin moves and the suite
passes on it: docs/clients.md says what each cell is worth.