- [BREAKING-EXPERIMENTAL] The
batch_data
attribute ofBatchRequest
has been removed. To pass in in-memory dataframes at runtime, the newRuntimeDataConnector
should be used - [BREAKING-EXPERIMENTAL]
RuntimeDataConnector
must now be passed Batch Requests of typeRuntimeBatchRequest
- [BREAKING-EXPERIMENTAL] The
PartitionDefinitionSubset
class has been removed - the parent classIDDict
is used in its place - [BREAKING-EXPERIMENTAL]
partition_request
was renameddata_connector_query
. The relatedPartitionRequest
class has been removed - the parent classIDDict
is used in its place - [BREAKING-EXPERIMENTAL]
partition_definition
was renamedbatch_identifiers
. The relatedPartitionDefinition
class has been removed - the parent classIDDict
is used in its place - [BREAKING-EXPERIMENTAL] The
PartitionQuery
class has been renamed toBatchFilter
- [BREAKING-EXPERIMENTAL] The
batch_identifiers
key onDataConnectorQuery
(formerlyPartitionRequest
) has been changed tobatch_filter_parameters
- [ENHANCEMENT] Added a new
RuntimeBatchRequest
class, which can be used alongsideRuntimeDataConnector
to specify batches at runtime with either an in-memory dataframe, path (filesystem or s3), or sql query - [ENHANCEMENT] Added a new
RuntimeQueryBatchSpec
class - [BUGFIX] Fixed issue where Sorters were not being applied correctly when
data_connector_query
contained limit or index #2617 - [DOCS] Updated docs to reflect above class name changes
- [DOCS] Added the following docs: "How to configure sorting in Data Connectors", "How to configure a Runtime Data Connector", "How to create a Batch Request using an Active Data Connector", "How to load a database table, view, or query result as a Batch"
- [DOCS] Updated the V3 API section of the following docs: "How to load a Pandas DataFrame as a Batch", "How to load a Spark DataFrame as a Batch"