- BREAKING CHANGE: move all reader options from the top-level batch_kwargs object to a sub-dictionary called
"reader_options" for SparkDFDatasource and PandasDatasource. This means it is no longer possible to specify
supplemental reader-specific options at the top-level ofget_batch
,yield_batch_kwargs
orbuild_batch_kwargs
calls, and instead, you must explicitly specify that they are reader_options, e.g. by a call such as:
context.yield_batch_kwargs(data_asset_name, reader_options={'encoding': 'utf-8'})
. - BREAKING CHANGE: move all query_params from the top-level batch_kwargs object to a sub-dictionary called
"query_params" for SqlAlchemyDatasource. This means it is no longer possible to specify supplemental query_params at
the top-level ofget_batch
,yield_batch_kwargs
orbuild_batch_kwargs
calls, and instead, you must explicitly specify that they are query_params, e.g. by a call such as:
context.yield_batch_kwargs(data_asset_name, query_params={'schema': 'foo'})
. - Add support for filtering validation result suites and validation result pages to show only failed expectations in
generated documentation - Add support for limit parameter to batch_kwargs for all datasources: Pandas, SqlAlchemy, and SparkDF; add support
to generators to support building batch_kwargs with limits specified. - Include raw_query and query_params in query_generator batch_kwargs
- Rename generator keyword arguments from data_asset_name to generator_asset to avoid ambiguity with normalized names
- Consistently migrate timestamp from batch_kwargs to batch_id
- Include batch_id in validation results
- Fix issue where batch_id was not included in some generated datasets
- Fix rendering issue with expect_table_columns_to_match_ordered_list expectation
- Add support for GCP, including BigQuery and GCS
- Add support to S3 generator for retrieving directories by specifying the
directory_assets
configuration - Fix warning regarding implicit class_name during init flow
- Expose build_generator API publicly on datasources
- Allow configuration of known extensions and return more informative message when SubdirReaderGenerator cannot find
relevant files. - Add support for allow_relative_error on internal dataset quantile functions, and add support for
build_continuous_partition_objec in Redshift - Fix truncated scroll bars in value_counts graphs