dyvenia/viadot v0.3.0

Viadot 0.3.0

on GitHub

Added

• new source SAPRFC for connecting with SAP using the pyRFC library (requires pyrfc as well as the SAP NW RFC library that can be downloaded here
• new source DuckDB for connecting with the DuckDB database
• new task SAPRFCToDF for loading data from SAP to a pandas DataFrame
• new tasks, DuckDBQuery and DuckDBCreateTableFromParquet, for interacting with DuckDB
• new flow SAPToDuckDB for moving data from SAP to DuckDB
• Added CheckColumnOrder task
• C4C connection with url and report_url documentation
  -SQLIteInsert check if DataFrame is empty or object is not a DataFrame
• KeyVault support in SharepointToDF task
• KeyVault support in CloudForCustomers tasks

Changed

• pinned Prefect version to 0.15.11
• df_to_csv now creates dirs if they don't exist
• ADLSToAzureSQL - when data in csv coulmns has unnecessary "\t" then removes them

Fixed

• fixed an issue with duckdb calls seeing initial db snapshot instead of the updated state (#282)
• C4C connection with url and report_url optimization
• column mapper in C4C source