prefect 2.19.0

Release 2.19.0

on Python PyPI

✨ This release includes a number of enhancements and fixes!

Support for major infrastructure and distributed task integrations

As prefect-dask and other integrations have been added to the prefect codebase, this release adds these integrations as extra requirements of the prefect package, making it easier to install support for everything in your Prefect stack:

pip install 'prefect[dask]'

We loved this community contribution so much, we did it for all our first-party integrations:

pip install 'prefect[aws,kubernetes,dask,dbt,sqlalchemy,slack]'

You can see the full list of Prefect's extra requirements in our setup.py.

Support for timeout seconds in global concurrency context manager

You may want to fail immediately if a global concurrency slot is unavailable. Rather than block and wait, you can now specify a timeout_seconds argument in the global concurrency context manager and catch a TimeoutError if a slot is not available within the specified time.

@flow
def fail_immediately_flow():
    try:
        with concurrency("there-can-be-only-one", occupy=1, timeout_seconds=0.1):
            do_something_resource_intensive()
    except TimeoutError:
        return Cancelled(message="Another flow run is already running")

Manage global concurrency limits via the CLI

Global concurrency limits let you control how many operations can run simultaneously-- now you can create, read, edit, and delete global concurrency limits via the Prefect CLI!

To create a new concurrency limit, use the prefect gcl create command. You must specify a --limit argument, and can optionally specify a --slot-decay-per-second and --disable argument.

prefect gcl create my-concurrency-limit --limit 5 --slot-decay-per-second 1.0

You can inspect the details of a concurrency limit using the prefect gcl inspect command:

prefect gcl inspect my-concurrency-limit

To update a concurrency limit, use the prefect gcl update command. You can update the --limit, --slot-decay-per-second, --enable, and --disable arguments:

prefect gcl update my-concurrency-limit --limit 10

We also have many more bug fixes and in-flight work! See the release notes for details!