schemathesis 3.16.0 on Python PyPI

Hello friends!

Schemathesis 3.16.0 brings a brand new GitHub integration and many readability improvements to the test results. You will have much less visual clutter reading Schemathesis reports, and the failures will be much more concise from now.

GitHub Integration

Now you can use a native GitHub application to collect & display API fuzzing results directly in your PR!
It makes Schemathesis much simpler to use in your GitHub Actions workflows and brings more Schemathesis.io features closer to you:

API Issues Dashboard: Debug issues faster with all the report details in a UI;
Additional checks: Get your API checked more thoroughly by extra conformance checks;

Grab your installation now!

After installation, you'll be redirected to SaaS to grab your access token, which you need to add to your workflow.

The simplest way to integrate it is:

jobs:
  api-tests:
    runs-on: ubuntu-20.04
    container: schemathesis/schemathesis:stable

    env:
      # Your API schema location
      API_SCHEMA: 'https://example.schemathesis.io/openapi.json'
      # Explicitly enable Schemathesis reporting integration
      SCHEMATHESIS_REPORT: 'true'
      # Set your token from secrets
      SCHEMATHESIS_TOKEN: ${{ secrets.SCHEMATHESIS_TOKEN }}
      # OPTIONAL. A name to collect multiple reports under the same roof
      # Grouping reports will allow Schemathesis to show you how your API changes over time
      SCHEMATHESIS_API_NAME: my-api

    steps:
      # Runs positive Schemathesis tests with all checks enabled
      - run: st run $API_SCHEMA --checks=all

Then you'll get your report:

Read the full integration guide in our docs.

A few notes:

Report upload size has some limitations now. My guess is that 250kb should be enough for most cases - let me know if I am wrong;
Using reporting with public repositories is free. Private ones are available on the paid plan;

What is next?

This release doesn't support GitHub organizations. The support will be added soon as well as more features like:

Precise API Coverage: Per-keyword coverage achieved by the test suite;
Stateful Flows: See how stateful tests are exercising your API endpoints in flow diagrams;
Schema fixes: Fix your API schema immediately by applying an autogenerated patch.

Your feedback is important ❤️

Please, let us know what you think about the integration in this discussion.

🐛 Fixed

Compatibility with hypothesis >= 6.49 in #1538.
Support for Hypothesis' skipping tests when the generate phase is disabled.
Generating headers with schemas of array or object types.
Do not report optional headers missing.

🔧 Changed

Pytest: Turn off schema validation by default. Pass validate_schema=True to the schema loader of your choice to keep the old behavior.
Code samples: Do not display irrelevant HTTP headers. It affects ones automatically set by requests when test cases go to the application under test.
CLI: Display the test environment metadata only if the -v option is present.

🔥 Removed

SaaS: Implicit reporting when the proper credentials were specified. From now on, every report upload requires the explicit --report CLI option.
CLI: Text representation of HTTP requests in the CLI output.

schemathesis 3.16.0 Release 3.16.0 on Python PyPI