PrefectHQ/prefect 3.0.0rc1

Release 3.0.0rc1

on GitHub

Release 3.0.0rc1

We're excited to announce the release candidate of Prefect 3.0. It's the most flexible, powerful, fastest version of Prefect yet. Prefect 3.0 includes several exciting new features. Install it by running pip install prefect==3.0.0rc1 and check out the docs here.

Run tasks independently of flows

You can now run and serve tasks outside of flows and inside of other tasks.

from prefect import task

@task
def my_background_task(name: str):
    print(f"Hello, {name}!")

if __name__ == "__main__":
    my_background_task.delay("ford")

Transactional semantics

Use rollback and commit hooks to facilitate idempotent python code.

from prefect import flow, task
from prefect.transactions import transaction
@task
def first_task():
    print('first')

@first_task.on_rollback
def roll(txn):
    print('rolling back')

@task
def second_task():
    raise RuntimeError("oopsie")

@flow
def txn_flow():
    with transaction():
        first_task()
        second_task()
if __name__ == "__main__":
    txn_flow()

Open source Events and Automations

Trigger actions, such as sending notifications, pausing schedules, starting flow runs and more in response to Prefect events.

More flexible variables and new artifact types

Variables can now be any JSON compatible type including dicts, lists, and integers. Progress and Image artifacts make it easy to add visual annotations to your flow run graph.

Faster and richer CLI

Improved CLI speed and several added commands and conveniences.

Updated navigation, styling, and interaction design

The new Runs page displays both flow and task run information, and an improved sidebar and switcher makes navigating Prefect simpler than ever.

Enhancements

• Create artifact for unsuccessful dbt task runs — #13348
• Add filter on task_run.expected_start_time — #13491
• Add utilities to serialize context to a dictionary and hydrate context from a dictionary — #13529
• Add API endpoints for deployment count and next flow run — #13544
• Allow flow parameter schema generation when dependencies are missing — #13315
• Change the default value for enforce_parameter_schema from False to True — #13594
• Migrate schemas to pydantic v2 — #13574
• Removes block auto-instrumentation — #13407
• Migrate all uses of the banned characters validation to a self-validator — #13370
• Ignore and warn on unrecognized settings - #13624

Fixes

• Remove unnecessary flow run infrastructure override access checks — #13401
• Enforce False case when flow run id is null — #13464
• Fix workspace variable hydration to allow for JSON data — #13548
• Remove unused settings/experimental work pool flags: PREFECT_EXPERIMENTAL_ENABLE_WORK_POOLS and PREFECT_EXPERIMENTAL_WARN_WORK_POOLS — #13144
• Pin pydantic>=2.7 for Secret — #13613
• Skip on cancellation hooks if runner can't load flow — #13660
• Refactor lazy imports to avoid accidental eager imports — #13296
• Allow block registration to use client schemas for server model creation — #13602
• Replace our customized Duration types with plain timedeltas — #13603

Experimental

• Add prefect.yaml and cli support for new schedule fields — #13318

Documentation

• Transition documentation hosting from Netlify to Mintlify — #13634
• Add Python 3.12 to list of Docker images — #13321
• Update index.md — #13353
• Improve tutorial section — #13297
• Fix jinja template in automations doc — #13422
• Update development section docs — #13247
• Update Ray integration docs — #13467
• Update Variables docs to include JSON types — #13493
• Update quickstart guide for usability — #13562
• Remove deployments-block-based concept page and refs for 3.0 — #13626
• Remove infrastructure concept page and refs for 3.0 — #13629
• Update docs image paths and remove outdated images — #13666
• Remove references to prefect.software from docs — #13382
• Update host.md — #13351
• Simplify rate limits page — #13689
• Removing references to deprecated block types and add disclaimer — #13651
• Update guides — #13253
• Remove storage concept page and refs - #13630

Integrations

• Migrate prefect-dbt to pydantic v2 - #13718
• Migrate prefect-email to pydantic v2 — #13654
• Migrate prefect-slack to pydantic v2 — #13673
• Migrate prefect-shell to pydantic v2 — #13675
• Migrate prefect-gcp to pydantic v2 — #13650
• Migrate prefect-github to pydantic v2 — #13655
• Migrate prefect-gitlab to pydantic v2 — #13656
• Migrate prefect-docker to pydantic v2 - #13697
• Migrate prefect-sqlalchemy to pydantic v2 - #13700
• Add PrefectDistributedClient to prefect-dask — #13537
• Update RayTaskRunner for compatibility with new engine — #13575
• Update DaskTaskRunner for compatibility with the updated engine — #13555
• prefect-dbt artifact consolidation and markdown fixes — #13379
• prefect-dbt - Cause unsuccessful dbt tasks to fail — #13405
• DBT Tasks extra_command_args Fix — #13308
• Update dbt-core dependency — #13394

Breaking Changes

• Remove prefect deployment build CLI from main — #13366
• Remove prefect agent CLI from main — #13365
• Remove prefect deployment apply CLI from main — #13367
• Remove PrefectAgent class — #13374
• Remove prefect.software — #13375
• Remove deployments module — #13373
• Remove EcsTask from main — #13417
• Remove AzureContainerInstanceJob from main — #13418
• Remove VertexAICustomTrainingJob from main — #13419
• Remove CloudRunJob from main — #13420
• Remove infrastructure blocks from main — #13424
• Remove Infrastructure, BlockWorker from main — #13430
• Remove deprecated storage blocks from main — #13410
• Remove prefect-agent as a possible work pool type — #13444
• Remove old engine — #13542
• Remove Python 3.8 support — #13331
• Remove deprecated module and its references — #13345
• Remove old task runners and futures modules — #13593
• Remove is_state — #13569
• Remove deprecated options from prefect work-queue and refs to agents - #13638

Contributors

• @bsignoret

• @jaraics made their first contribution in #13144

All changes: 2.19.0...3.0.0rc1

See release notes for details.