schemathesis 4.31.0

Release 4.31.0

on Python PyPI

🚀 Added

• Python 3.15 support.
• Tip to check the base URL when the API refuses connections.
• ctx.add_report() for CLI extensions to list their reports in the terminal and HTML report.
• auth.auto-signup config option to disable automatic sign-up.
• Warning when most entries of a dictionary fail the schema of their parameter.
• Warning when --request-timeout or max_response_time is 1000 seconds or more.
• object_level_authorization check: one WFC peer reading an object another peer created.
• Error feedback stops sending unknown body fields after a Jackson Unrecognized field rejection.
• --report html and [reports.html] write an offline HTML report with failures and reproduction commands.

🔧 Changed

• Readable report names in the terminal Reports list, e.g. JUnit instead of JUNIT.
• Summary uses the same warning titles and wording as the warnings block.
• Reject phases.coverage.generation.max-examples; the coverage phase has no case cap.

🐛 Fixed

st fuzz

• Skipping checks for requests sent after the first failure in a scenario.
• Not naming filters that matched no operations.
• Showing "Stop reason: Completed" when no operation was tested.
• Ignoring warnings.fail-on for unmatched filters.
• Summaries not accounting for operations they did not test.
• Summary missing the "Baseline" and "Filtered" counts.
• Suggesting --wait-for-schema on connection errors when it is already set.
• Crashing with Inconsistent data generation when the server stops responding.
• Success icon for an interrupted run.
• Hanging and fuzzing forever when the event consumer raises mid-run.

Output and exit codes

• Double blank lines and trailing spaces in output piped to a file or CI log.
• "No checks ran" given as the skip reason for operations whose requests got no response.
• Phase line showing only passes when the server stopped responding during the phase.
• Phase shown as passed when no checks ran on any of its operations.
• Phase line claiming passed scenarios when no checks ran.
• Final line reporting "No issues found" when no checks ran.
• Final line not naming why the run failed: unsaved baseline or warnings with fail-on.
• Exit code 1 from a fail-on missing-auth warning withdrawn after a later successful response.
• Exit code 1 from a fail-on missing-test-data warning withdrawn during stateful testing.
• API capabilities reported as unsupported when the server is unreachable.

Interrupting with Ctrl-C

• Baseline file written from partial results.
• Final line reporting "No issues found" or "Empty test suite".
• Phase line counting not-yet-run operations twice as skipped.

Missing inferred stateful links

• Paginated lists of wrapped items, e.g. {items: [{track}]}.
• Deeply nested body foreign keys, e.g. shipping.address.warehouse_id.
• Ids returned next to the created resource, e.g. {bookingid, booking}.
• Request body fields declared in referenced schemas, e.g. allOf: [$ref].
• Paths with a version segment after the collection, e.g. /books/v1.
• POST /books to GET /books/{title} via request body title.

Stateful testing

• Broken inferred links for paginated lists of wrapped items.
• Inferred update links filling nested body foreign keys with the updated item's id, e.g. recipient.clientId.
• Inferred links sending a numeric id to string path parameters when a string uuid exists.
• Traceback and early stateful phase exit with --mode negative for steps with nothing to negate.
• Extraction failures listed in a different order between runs with the same seed.
• Stateful-only runs repeatedly calling operations that always return 405 Method Not Allowed.
• Stateful phase repeating an operation's error, e.g. an unknown GraphQL scalar, as a run-level error.

Error feedback

• Attributing query and path parameter errors of POST/PUT requests to the body.
• Ignoring "required" errors on array item fields like items.0.name.
• Dropping nested-field errors like items[0].name or sending them as dotted body keys.
• Ignoring server errors on fields of $ref objects nested in the request body.
• Rounding large integer bounds, e.g. Java long maximum written as 2**63.
• Unsatisfiable allOf request bodies after the server rejects unknown fields.

Data generation

• Coverage phase generating no valid values for schemas with keywords but no type.
• Coverage phase dropping $ref target constraints when sibling keywords are looser in Open API 3.1.
• Coverage phase skipping the valid array when every example, examples and default is invalid.
• Coverage phase skipping the object-body negative for non-object type when required precedes properties.
• Application constants drawn in the examples phase.
• Crash in the examples phase when a response example field holds a list of scalars.
• Crash in negative mode on schemas with a boolean not, e.g. not: false.
• Negative request bodies for enum or const schemas changing after they were generated.
• Fuzzing negative mode never generating fractional numbers for integer values.
• Fuzzing sending examples of query parameters the server rejected as unknown.
• Generated strings failing pattern when a group holds a possessive quantifier, e.g. (a*+).
• Positive cases treated as negative when an auth hook adds an undeclared or differently-cased header.

negative_data_rejection

• Missed failures for integer parameters sent as 1_0, 5 or non-ASCII digits.
• False positive for header values invalid only through surrounding whitespace.
• False positive for a configured User-Agent the schema declares as a header.

Configuration

• st run ignoring checks and generation under [phases.<phase>] for examples, coverage and stateful phases.
• Top-level [[operations]] overriding [[project.operations]] for the same operation.
• [[operations]] continue-on-failure = false not overriding a project-level true.
• Root-level [servers.variables] ignored for the schema-wide servers URL.
• Header [parameters] overrides and configured headers dropping each other when both are set.
• Raw oneOf validation text in config errors for wrong-typed request-retries and operation filters.

pytest, Python API and hooks

• pytest examples phase ignoring [parameters] overrides in fill-missing cases.
• pytest negative testing skipped as impossible on operations with configured basic auth.
• Check failures shown as a bare Failure() in repr and pytest assertion messages.
• as_strategy(headers=...) modifying the passed headers dictionary.
• Wildcard response deserializers like @schemathesis.deserializer("application/*") never matching.
• Hooks loaded from a file path in SCHEMATHESIS_HOOKS running several times per call.
• Coverage cases dropped when a schema filter_case hook depends on a global map_case.

Reports

• Duplicated and unsorted run-level check failures in JUnit and Allure reports.
• Allure reports missing coverage failures for undeclared HTTP methods.

Others

• --auth-wfc discarding peers, refresh_interval and user set under [auth.wfc].
• --auth-wfc-user silently ignored when no Web Fuzzing Commons auth file is configured.
• Missing unresolvable-reference warning for responses whose $ref reaches it through a cycle.
• Crash on schemas with a non-object info or a non-string info.title.
• WFC auth abandoning the first user when a Swagger 2.0 basic case omits credentials.
• Misleading additional-properties hint when a Pydantic error names a declared field.
• st replay failing to load the schema for crashes recorded with --tls-verify=false.

🔥 Removed

• Python 3.10 support.