pola-rs/polars py-2.0.0-rc.1

Python Polars 2.0.0-rc.1

on GitHub

💥 Breaking changes

• Set the default engine for SQL to the streaming engine (#28973)

🚀 Performance improvements

• Reuse native metadata for Iceberg sinks (#29062)
• SQL exists predicate pushdown before subqueries (#29078)
• Collect primitive group-by aggregations into single chunk (#28825)
• Use cost information to determine whether to keep cached nodes (#29042)
• Use scan cardinality in build side if sampling is exhausted (#29040)

✨ Enhancements

• Defer SQL resolving until we convert to IR (#29077)
• Expose Iceberg sink writer options (#29065)
• Support partitioned Iceberg sinks (#28823)
• Expose fixed-array inner product in SQL (#29033)
• Add strip_prefix and strip_suffix to expression names (#29041)
• Use scan information in join reordering (#29038)
• Add cardinality information (#29024)
• Allow customizing explain output for debugging custom io sources (#23978)
• Add more expression properties to meta namespace (#28091)
• Improve SQL coverage (#29010)
• Improve SQL coverage (#29006)
• Add Join reordering. (#28985)
• Merge 2.0 into main (#28982)
• Set the default engine for SQL to the streaming engine (#28973)

🐞 Bug fixes

• Don't keep incrementing morsel id in MergeSortedNode pass-through (#29055)
• Ensure show_graph shows physical plan for auto engine (#29011)
• Assign SQL JOIN literal comparisons to the correct input (#28701)
• Make distributor linearizer capacity consistent (#29057)
• Correct peak visitors, dictionary bounds, and Date feature gating (#29031)
• Keep extension type schema consistent (#29039)
• Fix Series.sample order when with_replacement=True and shuffle=False (#28990)
• Reduce memory in multi_scan pipeline PostApplyExtraOps (#29049)
• Fix SO in rayon caused by join_where (#29051)
• Clarify GPU installation guidance (#29037)
• Remove POLARS_AUTO_STRUCTIFY env var backdoor (#29032)
• Ensure metrics are flushed between phases and before logging (#29014)
• SQL parse a typed DATE literal instead of casting it (#29007)
• Fix parquet decode of plain pages in dictionary column (#29001)
• Reserve fixed-size-list child capacity (#28980)
• Fix SQL casts (#28986)
• Fix incorrect condition in combine_validities_and_many and add test (#28909)
• Give sink_batches lazy: Literal[False] overload a default (#28951)

📖 Documentation

• Update 2.0.0-rc1 migration guide with latest changes (#28983)
• Fix invalid output in DataFrame.sample example (#28998)

🛠️ Other improvements

• Bump rust toolchain (#29080)
• Increase rustc stack size (#29076)
• Log pytest worker crashes and disable restarting (#29058)
• Replace FileInfo row_estimation tuple with typed ScanStats (#29023)
• Use an if for cfg(debug_assertions) to fix the benchmark job (#29002)
• Remove 2.0 branch from ci-tested branches (#28981)
• Skip tests that are currently failing on windows (#28962)

Thank you to all our contributors for making this release possible!
@0guban0v, @0xC61, @Aniket-a14, @LarryHu0217, @MatteoPossamai, @NeejWeej, @TNieuwdorp, @WaterWhisperer, @adamreeve, @dsprenkels, @jonasdedden, @kdn36, @math-hiyoko, @matthewbayer, @nameexhaustion, @orlp, @ritchie46, @severinh and @sivakumar-mahalingam