github ClickHouse/clickhouse-connect v1.13.0

one hour ago

clickhouse-connect v1.13.0

This release adds declarative skip indexes and projections to the SQLAlchemy dialect and Alembic autogenerate. It also adds Alembic view and SQL UDF operations and compares table engines and settings during autogenerate. It fixes a broad set of insert validation, query classification, binding and pandas/NumPy decoding bugs.

Installation

pip install clickhouse-connect

Upgrade notes

Some fixes change behavior that existing code may depend on. Check these before you upgrade:

  • Alembic autogenerate compares more by default. Explicit indexes, projections and table options are now compared with the server. Indexes or projections created only by earlier imperative migrations can generate drops when they are not declared in the metadata. Declare them with ClickHouseIndex or ClickHouseProjection, exclude them with Alembic filters, or pass clickhousedb_compare_indexes=False, clickhousedb_compare_projections=False or clickhousedb_compare_table_options=False to context.configure(). A change to the table engine, ORDER BY, PARTITION BY, TTL or a read-only setting now raises UnsupportedTableChangeError instead of writing an empty revision.
  • op.alter_column(..., server_default=None) now removes the default. The server rejects REMOVE DEFAULT when the column has no default, so a hand-written server_default=None that was a silent no-op now fails.
  • Inspector.get_columns() returns plain strings for server_default, clickhouse_materialized, clickhouse_alias and clickhouse_ttl. Wrap them in sqlalchemy.text() if you build Column objects from them directly.
  • Stricter Python codec inserts. Unknown Enum labels, None in non-nullable Enum and integer columns, and NaN or infinity in integer columns now raise DataError instead of writing 0. insert_df still writes 0 or NULL for missing values.
  • Placeholder-like text in strings and comments no longer selects server-side binding. These queries use client-side percent formatting, so a bare % must be escaped as %% when you pass parameters.
  • query_df with Nullable(Int128/UInt128/Int256/UInt256) now returns object dtype with Python int values. Before, values above 2**53 could be rounded through float64.

Highlights

Features

  • Declarative data skipping indexes with ClickHouseIndex and Inspector.get_clickhouse_indexes(), with Alembic autogenerate support.
  • Declarative projections with ClickHouseProjection and Inspector.get_clickhouse_projections(), with Alembic autogenerate support.
  • Alembic autogenerate compares the table engine, keys, TTL and settings. Changed engine settings render as op.modify_clickhouse_table_settings(...) with a reversible downgrade.
  • Alembic create, drop and CREATE OR REPLACE helpers for ordinary views and SQL UDFs. Inspector.get_clickhouse_function_names() and get_clickhouse_function_definition() reflect SQL UDFs.
  • All 21 ClickHouse Alembic operation classes are now public, with typed fields for policy hooks and Rewriters.
  • Inspector.get_view_definition() returns the stored SELECT of views and materialized views.
  • SQLAlchemy Tuple accepts an elements mapping of field names to types. Closes #1144.

Performance

  • String and bytes parameter escaping uses lookup tables. Large IN lists and Array(String) parameters bind about 3-4x faster, binary String parameters about 6x.
  • Python codec IPv6 inserts of address objects are about 10-35x faster.
  • The Rust codec reads scalar nullable date and datetime columns from typed buffers, which speeds up NumPy and pandas results.
  • SQL comment detection uses a compiled scanner on CPython with the C extensions.

Bug fixes

  • Query classification follows the ClickHouse lexer for comments, backtick identifiers, heredocs and escapes, so query_limit and command detection work as expected. Closes #925.
  • Enum type names without element numbers, such as Enum8('a', 'b'), now parse. Closes #1189.
  • Unknown Enum labels raise DataError instead of inserting 0. Closes #1156.
  • NaN and infinity in direct integer inserts raise DataError. Closes #1172.
  • query_df keeps large Nullable(Int128) and Nullable(Int256) family values exact. Fixes #1181.
  • query_df and query_np decode JSON values stored in shared data, and no longer drop shared 0 and false values.
  • client.insert converts NumPy datetime64 arrays to the unit of the target date or time column.
  • Python codec Decimal inserts truncate extra digits like the server and the Rust codec.
  • Unencodable FixedString values raise UnicodeEncodeError instead of writing zero bytes.
  • Top-level enum.Enum members in server-side parameters bind like their value.
  • Naive datetime inserts near and before the epoch no longer raise OSError on Windows. Addresses the Python codec part of #1162.
  • Placeholder-like text in strings, identifiers, comments and heredocs no longer selects server-side binding. Closes #1146.
  • Dictionaries bound to named Tuple parameters use the declared field order. Closes #1148.
  • Inspector.get_columns() returns JSON-serializable strings for column expressions. Closes #1141.
  • With the Rust codec, nullable date and datetime columns in NumPy and pandas results match the Python codec.
  • Alembic version-table inserts bypass insert deduplication, so a downgrade then upgrade no longer replays applied migrations.
  • Alembic autogenerate no longer proposes drop_table for views missing from the metadata.
  • op.alter_column(..., server_default=None) emits REMOVE DEFAULT, and None for codec, TTL, alias or materialized removes that property.
  • Reflection handles engine settings that start with a clause keyword, such as ttl_only_drop_parts, and quoted engine names such as Null.
  • Typed SQLAlchemy Map parameters keep DateTime64 fractional seconds with dict_parameter_format="map".
  • Integer insert errors say whether a value had the wrong type or was out of range.

See the CHANGELOG for full details.

Don't miss a new clickhouse-connect release

NewReleases is sending notifications on new releases.