dyvenia/viadot v2.1.0 on GitHub

What's Changed

✨ Add complete proxy settings in SAPRFC example by @trymzet in #403
Update tests by @winiar93 in #396
SQLServer To DuckDB by @angelika233 in #404
✨ Added databricks-connect support by @afraijat in #409
✨Added databricks source to viadot by @afraijat in #434
Decrease Docker image size by @trymzet in #458
Added rollback feature, improved comments formatting by @afraijat in #452
🔊 Replaced Prefect logging with Python logging by @afraijat in #459
Add databricks cleanup task by @trymzet in #502
Fix orion prefect deployment by @trymzet in #515
Fix import error from datahub_cleanup_task by @trymzet in #516
✨ Added ExchangeRates source to the library by @djagoda881 in #535
✨ Add Sharepoint 2.0 source by @trymzet in #534
♻️ Update ExchangeRates to use new config by @trymzet in #536
🔥 Remove unused requirements after having removed tasks by @trymzet in #537
♻️ Readd Databricks to init by @trymzet in #538
🔥 Databricks - remove env parameter by @trymzet in #539
♻️ Readding the AzureDataLake source by @trymzet in #540
✨ Add a default cluster port to Databricks source by @trymzet in #541
🐛 Add missing info about the required org_id credential by @trymzet in #542
✨ Add TableDoesNotExist exception by @trymzet in #543
Add from_df() method to Azure Data Lake source by @trymzet in #546
✨ Added parameters to functions in config.py by @djagoda881 in #548
♻️ Update Databricks to work with 2.0 configs by @trymzet in #553
✨ Adding a column to the to_df function by @djagoda881 in #550
✨ Created decorator to include viadot source to df by @fgoiriz in #567
✨ Added migration of cloud_for_customers.py source to 2.0 by @fgoiriz in #552
🐛 Modified Sources init.py by @fgoiriz in #569
🐛 Databricks bug fix by @afraijat in #571
Add SAPRFC source by @AnnaGerlich in #582
Add s3 source by @trymzet in #587
🐛Fix credential handling to handle AWS region by @trymzet in #588
🐛 Fix logger being called before initialization by @trymzet in #589
🐛 Fix credential handling when not specified by @AnnaGerlich in #590
🐛 Fixed and Extended S3 Source by @TillPickha in #603
🐛 Fix handling of empty viadot config by @trymzet in #611
📝 Update docs by @trymzet in #612
Add VSCode setup by @trymzet in #613
Remove prefect as base image by @trymzet in #617
📌 Fix boto dependency hell by @trymzet in #618
📌 Fix boto to particular version by @trymzet in #619
Remove ARM architecture by @trymzet in #620
Remove old dependencies by @trymzet in #622
➕ Add missing pyyaml dependency by @trymzet in #624
➕ Add pydantic dependency by @trymzet in #625
⬆️ Bump viadot package version by @trymzet in #626
⬆️ Upgrade dependencies to fix conflicts by @trymzet in #628
✨ Add test_download_file() test for Sharepoint source by @AnnaGerlich in #608
📝 Minor docs change by @trymzet in #635
Added tests for the RedshiftSpectrum source by @AnnaGerlich in #636
Fixed bug in exchange rates unit tests by @djagoda881 in #638
Databricks replace bug by @djagoda881 in #658
Utils response function by @djagoda881 in #661
Genesys source migration by @djagoda881 in #665
Genesys bug fix by @djagoda881 in #668
Databricks snakecase column bug by @djagoda881 in #673
📝 Added howto migrate from viadot 1 to viadot 2 by @afraijat in #680
⚡️Enhanced S3() and RedshiftSpectrum() sources by @AnnaGerlich in #678
📝 Refined viadot migration docs by @afraijat in #684
Automatically create table folder in RedshiftSpectrum.from_df() by @trymzet in #693
Cleanup compose by @trymzet in #696
Upgrade Databricks connector for 11.3+ runtimes by @trymzet in #697
♻️ Standardize credentials validation by @trymzet in #699
Databricks pandas types casting by @djagoda881 in #701
✨ Added close_connection() to sap_rfc by @AnnaGerlich in #709
🚑 Fixed AWS credentials handling by @AnnaGerlich in #713
🚑 Fixed typo in chunksize parameter name by @AnnaGerlich in #718
✨ Add MSSQL ODBC driver to image by @trymzet in #719
Add Trino source by @trymzet in #726
Source MinIO by @trymzet in #729
✨ Added SAPRFCV2 source class by @AnnaGerlich in #835
Update PyPI pipeline to use Trusted Publishing by @trymzet in #837
✨ Add partition_cols param to Minio.from_df(). by @trymzet in #838
Update version to 2.0a15 by @trymzet in #839
✨ Optimize MinIO and Trino sources by @trymzet in #845
⬆️ Bump version by @trymzet in #859
Implement recursive param in MinIO.rm() by @trymzet in #861
Add Decimal type support to Trino by @trymzet in #862
🔖 Bump version by @trymzet in #864
Trino connection context manager by @trymzet in #865
⚡️ Performance upgrade of SAPRFCV2 and Dockerfile update by @marcinpurtak in #863
➕ Add missing dependencies in setup.py by @djagoda881 in #871
🔖 Bumped viadot2 version to 2.0a19 by @djagoda881 in #872
🐛 Added dependencies directly to setup.py by @djagoda881 in #875
⬆️ Upgraded dependencies versions by @djagoda881 in #876
✨ Added validate() util for dataframe validation by @burzekj in #869
🚀 Bumped viadot2 version to 2.0a20 by @djagoda881 in #877
🧱 Added new dependence management in the project by @djagoda881 in #878
Bump azure-identity by @trymzet in #879
♻️ Apply downstream improvements to the Dockerfile by @trymzet in #881
👷 Add CI to 2.0 by @trymzet in #882
🐛 Fix pip not installing to viadot user's .local dir by @trymzet in #883
🐛 Fix Dockerfile build failing due to file permissions by @trymzet in #884
📝 Improve contributing docs by @trymzet in #889
🐛 Fixing crash when no data returned in SAPRFCV2 by @marcinpurtak in #887
♻️ Added Databircks as optional dependency and source by @djagoda881 in #880
🚀 Added new functionality to saprfc source regarding where statement by @adrian-wojcik in #899
Modified reading the file and inspect provided URL by @Rafalz13 in #914
Bugfix/saprfcv2 column shift by @marcinpurtak in #916
📝 Make docs great again by @trymzet in #924
🐛 Fix MkDocs build failing due to incorrect YAML parsing by @trymzet in #925
🐛 Update lockfiles by @trymzet in #927
Sharepoint handle null values by @Rafalz13 in #931
🔖 Bump version by @trymzet in #933
Sharepoint convert data types to string by @Rafalz13 in #937
🔖 Bump version by @Rafalz13 in #938
⬆️ Relax sql-metadata version requirement by @trymzet in #940
✨ Add orchestration module by @djagoda881 in #917

New Contributors

@afraijat made their first contribution in #409
@fgoiriz made their first contribution in #567
@TillPickha made their first contribution in #603

Full Changelog: v0.4.3...v2.1.0