prefect-aws 0.4.0

Version 0.4.0

on Python PyPI

Major Features

• Add support for Prefect Cloud - #374, #406, #473, #491
• Add versioned serialization schemas for Flow, Task, Parameter, Edge, State, Schedule, and Environment objects - #310, #318, #319, #340
• Add ability to provide ResultHandlers for storing private result data - #391, #394, #430
• Support depth-first execution of mapped tasks and tracking of both the static "parent" and dynamic "children" via Mapped states - #485

Minor Features

• Add new TimedOut state for task execution timeouts - #255
• Use timezone-aware dates throughout Prefect - #325
• Add description and tags arguments to Parameters - #318
• Allow edge key checks to be skipped in order to create "dummy" flows from metadata - #319
• Add new names_only keyword to flow.parameters - #337
• Add utility for building GraphQL queries and simple schemas from Python objects - #342
• Add links to downloadable Jupyter notebooks for all tutorials - #212
• Add to_dict convenience method for DotDict class - #341
• Refactor requirements to a custom ini file specification - #347
• Refactor API documentation specification to toml file - #361
• Add new SQLite tasks for basic SQL scripting and querying - #291
• Executors now pass map_index into the TaskRunners - #373
• All schedules support start_date and end_date parameters - #375
• Add DateTime marshmallow field for timezone-aware serialization - #378
• Adds ability to put variables into context via the config - #381
• Adds new client.deploy method for adding new flows to the Prefect Cloud - #388
• Add id attribute to Task class - #416
• Add new Resume state for resuming from Paused tasks - #435
• Add support for heartbeats - #436
• Add new Submitted state for signaling that Scheduled tasks have been handled - #445
• Add ability to add custom environment variables and copy local files into ContainerEnvironments - #453
• Add set_secret method to Client for creating and setting the values of user secrets - #452
• Refactor runners into CloudTaskRunner and CloudFlowRunner classes - #431
• Added functions for loading default engine classes from config - #477

Fixes

• Fixed issue with GraphQLResult reprs - #374
• CronSchedule produces expected results across daylight savings time transitions - #375
• utilities.serialization.Nested properly respects marshmallow.missing values - #398
• Fixed issue in capturing unexpected mapping errors during task runs - #409
• Fixed issue in flow.visualize() so that mapped flow states can be passed and colored - #387
• Fixed issue where IntervalSchedule was serialized at "second" resolution, not lower - #427
• Fixed issue where SKIP signals were preventing multiple layers of mapping - #455
• Fixed issue with multi-layer mapping in flow.visualize() - #454
• Fixed issue where Prefect Cloud cached_inputs weren't being used locally - #434
• Fixed issue where Config.set_nested would have an error if the provided key was nested deeper than an existing terminal key - #479
• Fixed issue where state_handlers were not called for certain signals - #494

Breaking Changes

• Remove NoSchedule and DateSchedule schedule classes - #324
• Change serialize() method to use schemas rather than custom dict - #318
• Remove timestamp property from State classes - #305
• Remove the custom JSON encoder library at prefect.utilities.json - #336
• flow.parameters now returns a set of parameters instead of a dictionary - #337
• Renamed to_dotdict -> as_nested_dict - #339
• Moved prefect.utilities.collections.GraphQLResult to prefect.utilities.graphql.GraphQLResult - #371
• SynchronousExecutor now does not do depth first execution for mapped tasks - #373
• Renamed prefect.utilities.serialization.JSONField -> JSONCompatible, removed its max_size feature, and no longer automatically serialize payloads as strings - #376
• Renamed prefect.utilities.serialization.NestedField -> Nested - #376
• Renamed prefect.utilities.serialization.NestedField.dump_fn -> NestedField.value_selection_fn for clarity - #377
• Local secrets are now pulled from secrets in context instead of _secrets - #382
• Remove Task and Flow descriptions, Flow project & version attributes - #383
• Changed Schedule parameter from on_or_after to after - #396
• Environments are immutable and return dict keys instead of str; some arguments for ContainerEnvironment are removed - #398
• environment.run() and environment.build(); removed the flows CLI and replaced it with a top-level CLI command, prefect run - #400
• The set_temporary_config utility now accepts a single dict of multiple config values, instead of just a key/value pair, and is located in utilities.configuration - #401
• Bump click requirement to 7.0, which changes underscores to hyphens at CLI - #409
• IntervalSchedule rejects intervals of less than one minute - #427
• FlowRunner returns a Running state, not a Pending state, when flows do not finish - #433
• Remove the task_contexts argument from FlowRunner.run() - #440
• Remove the leading underscore from Prefect-set context keys - #446
• Removed throttling tasks within the local cluster - #470
• Even start_tasks will not run before their state's start_time (if the state is Scheduled) - #474
• DaskExecutor's "processes" keyword argument was renamed "local_processes" - #477
• Removed the mapped and map_index kwargs from TaskRunner.run(). These values are now inferred automatically - #485
• The upstream_states dictionary used by the Runners only includes State values, not lists of States. The use case that required lists of States is now covered by the Mapped state. - #485