pola-rs/polars py-1.42.0

Python Polars 1.42.0

on GitHub

⚠️ Deprecations

• Deprecate casts from string to temporal dtypes (#28056)

🚀 Performance improvements

• Add bytes-based concurrency control for cloud IO (#27924)
• Stop flushing multiplexer on phase change (#28013)
• Prevent empty chunks from triggering copy in rechunk (#27953)
• Improve performance of struct unnest inside list eval on streaming (#27905)
• Eliminate filters with contradictory predicates (#27775)

✨ Enhancements

• Deprecate casts from string to temporal dtypes (#28056)
• Add naive out-of-core spilling to Polars (#27998)
• Add experimental strict mode (#28023)
• Optimize non-negative len and null_count comparisons (#28002)
• Optimize len and null_count comparisons (#27727)
• Support SQL's implicit JOIN syntax (#27890)
• Add Expr.is_sorted (#26708)
• Add suggestions in wrong api usage or mismatched column names (#27954)
• Add external object_store support for non-native schemes (#27871)
• Skip Delta files and parquet row groups using per-field struct statistics (#27887)
• Improve IR sortedness detection for with_row_index and explode (#27875)
• Add DataFrame.is_sorted() (#27870)
• Expose fixed-size rolling window expressions in Python visitor (#27108)
• Fix SchemaError using lazy HConcat->Sink (#27770)
• Add tiered multi-file parquet metadata resolver (#27720)
• Cache and shuffle DNS for cloud object_store (#27659)

🐞 Bug fixes

• Add a deprecation warning for .explode() without empty_as_null argument (#28040)
• Fix select(len()) incorrectly returned 0 when using scan_iceberg with pyiceberg as reader override (#28044)
• Ensure slice expr always observes input order (#28041)
• Fix projection pushdown with row index out of order (#28043)
• Fix cross-join predicates not applied when using sink_* functions (#28042)
• Grammar corrections (recommend to → recommend, allows to → allows using) (#28038)
• Ensure optimisations aren't omitted by cached IR nodes (#28004)
• Computed GROUP keys in SQL projections (#27989)
• Handle empty DataFrames in dt.truncate and dt.round (#26120)
• Address join alias state leakage in SQL CTEs (#27985)
• Support scanning file:// URIs with percent-encoded paths (#27876)
• Lower SQL [NOT] IN (subquery) to semi/anti join (#27888)
• Check for continuity across morsels in AsOf join node (#27461)
• Add test for projection pushdown with hconcat regression (#27937)
• Traverse caches only once in explain (#27939)
• Preserve null placement when collapsing sort.reverse() into single sort (#27918)
• Incorrect schema order for hstack in projection pushdown optimizer (#27920)
• Fix duplicated map_elements warning on streaming engine (#27919)
• Use block_in_place_on for calls which can come from executor thread (#27855)
• Mismatch in max_threads -> pipeline configuration (#27854)
• Keep maintain_order on sliced unique (#27852)
• Fix SchemaError using lazy HConcat->Sink (#27770)
• Fix incorrect projection height when selecting only literals (#27825)
• Select with expr slice and len gave incorrect len (#27824)
• Prevent import panic when environment variable set to unexpected value (#27831)

📖 Documentation

• New On-Prem release (#27979)
• Add note on OOM behavior on Kubernetes (#27972)
• Properly render backslashes in CSV eol_char docstrings (#27962)
• Add entry for DataFrame.is_sorted() (#27913)
• Clarify compat_level export docs (#27715)
• Add robots.txt disallowing stale docs and added sitemap for better SEO (#27901)
• Update Polars <> Spark comparison (#27873)
• Bump to patched version (#27851)
• Replace Typeform sign-up URL with new enterprise link (#27838)
• Correct wrong head call (#27848)
• Add Polars On-Prem 0.5.0 release (#27849)
• Correct onprem license helm values (#27847)
• Update connecting Polars Cloud to AWS documentation (#27823)
• Correct cost/connectivity sections (#27813)
• Add sections about egress (#27805)
• Tragic typo in On-Prem docs (#27804)

🛠️ Other improvements

• Add AI agent disclsoure (#28064)
• Bump memmap2 version (#28050)
• Update sqlparser dependency (#28021)
• Update pyo3 and rust-numpy to 0.29.0 (#27970)
• Add 2.0 branch as primary branch (#27978)
• Patch jemalloc-sys for Makefile bug (#27945)
• Restore Cargo.lock (#27964)
• Add test for projection pushdown with hconcat regression (#27937)
• Update deny.toml configuration (#27949)
• Doctest skip refactor and add hugging face rate limited tests (#27885)
• Run Pyrefly type-checker on source code (#27722)
• Bump Rust dependencies (#27874)
• Changes needed for Rust 0.54.x (#27853)
• Use Vec instead of PlHashMap for ProjectionInfo.map (#27856)
• Reduce codegen-units (#27835)
• Refactor to avoid unbound-name Pyright/Pyrefly errors (#27827)

Thank you to all our contributors for making this release possible!
@0guban0v, @April-Sonnet, @BitWeaverDev, @EndPositive, @JakubValtar, @Kevin-Patyk, @Liyixin95, @MarcoGorelli, @Matt711, @TNieuwdorp, @alexander-beedie, @ankane, @azimafroozeh, @cBournhonesque, @carnarez, @dsprenkels, @gautamvarmadatla, @kdn36, @nameexhaustion, @orlp, @ritchie46, @tolleybot, @toreerdmann, @uurl and @xixixao