Big points:
- Added a new algorithm for in-db diffing that uses OUTER JOIN, called "joindiff".
- Much faster than the original "hashdiff" algorithm!
- Automatically chosen if both dbs are the same
- Validates that the key column is unique and contains no NULLs (joindiff only)
- Explicitly switch between algorithms using the
--algorithmparameter.
- New feature to materialize joindiff results to DB
- New feature that diffs the schemas when both dbs are the same
- Added DuckDB support (thanks @jardayn!)
- Better support for alphanumerics
- Better support for boolean types
- Added
--versionswitch - New and improved database and query interface, named "sqeleton"
- Tons of bugfixes and improvements!
What's Changed
- Join-diff (in-db) + new query builder by @erezsh in #242
- Bugfix: Joindiff crashed when no numeric columns were used. by @erezsh in #255
- Deprecate use of FixedAlphanum by @erezsh in #254
- Refactor tests oct2022 by @erezsh in #253
- General tests now include Presto, Trino & Vertica; Includes small fixes by @erezsh in #256
- Added --materialize-all-rows switch + tests by @erezsh in #258
- Various small fixes and refactors by @erezsh in #260
- Downgrade mysql-connector-python to 8.0.29 by @erezsh in #262
- Update documentation link by @williebsweet in #263
- Small changes by @erezsh in #264
- Added link on how to get a slack invite by @jardayn in #265
- link to docs and incorporate roman/gerard feedback by @leoebfolsom in #266
- Tiny Cleanup by @erezsh in #267
- tests for unique key constraints (if possible) instead of always actively validating (+ tests) by @erezsh in #257
- Attempt to fix PR #269 by @erezsh in #272
- Contrib improvements + Fixed Test by @jardayn in #269
- Refactor dialect by @erezsh in #271
- Tests: Improvements to CI flow + fixes by @erezsh in #274
- Bugfix in alphanums (reported by Guarav Singh) by @erezsh in #277
- Fix databricks by @pik94 in #273
- Added support for Boolean types by @erezsh in #282
- Fixed broken "How To Use" links in README. by @daniel-leicht in #290
- Fix for issue #286 by @erezsh in #291
- Materialize: rename and reorder columns by @erezsh in #287
- Revised CLI output to be more understandable and detailed by @erezsh in #292
- New DB Driver guide update by @jardayn in #288
- Duckdb driver for Issue #176 by @jardayn in #276
- Update typing of TableSegment().count() by @MattDelac in #293
- Refactor common database interface into Sqeleton (databases, queries) by @erezsh in #285
- Added DDB as an extra by @jardayn in #296
- More Sqeleton refactoring by @erezsh in #295
- Added InfoTree as a more descriptive alternative to .stats by @erezsh in #297
- Refactor tests to use insert_rows_in_batches(), instead of internally… by @erezsh in #299
- CLI: Better errors + tiny bugfix by @erezsh in #303
- Rudderstack poc by @kylemcnair in #298
- add databases we support to readme by @leoebfolsom in #309
- Nov22 sqeleton refactor by @erezsh in #308
- Fix readme link by @dlawin in #310
- List tables from schema by @erezsh in #311
- Tests: Set bisection_factor=2 for much faster tests; Fix random failures in test_string_keys by @erezsh in #312
- Nov24 - Small fixes to tests by @erezsh in #313
- Adjustments for PR #314 by @erezsh in #315
- return all duplicated rows by @pik94 in #314
- Cleanup by @erezsh in #320
- Added version and --version switch (issue #318) by @erezsh in #319
- data-diff now uses database A's now instead of cli's now. by @erezsh in #306
- extract methods for stats by @dlawin in #300
- connect(): Added support for shared connection; Database.is_closed property by @erezsh in #323
- Better error messages in databases; Default database in clickhouse is now 'default'. by @erezsh in #325
- diff_tables() now accepts all JoinDiffer params by @erezsh in #326
- CLI: Automatically choose joindiff is dbs are the same (don't rely just on syntax) by @erezsh in #328
- Add version module and add version to tracking by @kylemcnair in #327
- Dec2 cleanup by @erezsh in #329
- fix link to docs by @leoebfolsom in #330
- Fix _normalize_table_path to always return a pair by @erezsh in #333
New Contributors
- @williebsweet made their first contribution in #263
- @jardayn made their first contribution in #265
- @daniel-leicht made their first contribution in #290
- @MattDelac made their first contribution in #293
- @kylemcnair made their first contribution in #298
- @dlawin made their first contribution in #310
Full Changelog: v0.2.8...v0.3.0
Let us know what you think in Discussions!