Kotlin/dataframe v1.0.0-rc02 on GitHub

Kotlin DataFrame 1.0.0-rc02 is released!

What’s new?

Simplified dataframe-geo setup in user projects: no need to manually add geotools dependencies, required geometry and CRS types are now exposed as transitive API.
Arrow/Parquet: timestamps with a time zone are read as Instant; nullable struct columns are read correctly.
Large documentation push: KDocs, tests and website pages for dozens of operations (valueCounts, reverse, transpose, xs, single, indices, unfold, concat, forEach, duplicate, associate, sum, mean, min/max, and more).
Besides that, every operation KDoc now has a link to the website.
Improved groupBy performance and memory allocations.
Compiler plugin now evaluates schema changes in String API overload calls
Kotlin Toolchain now supports DataFrame compiler plugin out of the box. Project setup to enable both core dependency and compiler plugin is as simple as this:

settings:
  kotlin:
    dataframe: enabled

Check Setup Kotlin DataFrame in Kotlin Toolchain.

And many other improvements and fixes! See below for a complete list of changes in this release grouped by category.

We invite you to try this release:

Use the 1.0.0-rc02 version in your Gradle, Maven and Kotlin Toolchain projects:
- See setup instructions for Gradle, Maven, Android, and Kotlin Toolchain projects!
- We also recommend trying the compiler plugin!
Use the special 1.0.0-rc02n version in Kotlin Notebook. The n version includes a patch that resolves notebook specific issue with statistical functions.
- You can specify the version in the descriptor or use %useLatestDescriptors.
  If you want to include kandy, list it after dataframe:
```
%useLatestDescriptors
%use dataframe, kandy
// or
%use dataframe(1.0.0-rc02n), kandy(0.8.6, 0.8.6)
```
- Use kernel version 0.16.0-736 or newer.

Features

Exposed GeoDataFrame geometry and CRS types to consumers by @zaleslaw in #2055
Read Arrow/Parquet timestamps with a time zone as Instant by @zaleslaw in #2063
Added support for Reorder plus site and tests updates by @zaleslaw in #2130
Improved diagnostics for missing POI service descriptors in fat JARs by @Allex-Nik in #2084
Defined append behavior on some corner cases and added tests by @Allex-Nik in #2061

Removals and Deprecations

Removed TraversePropertiesDsl from top-level toDataFrame {} operation by @zaleslaw with @Copilot in #2004
Inlined ColumnFilter typealias in public API function signatures by @zaleslaw with @Copilot in #2003
Migrated NamedValue.copy visibility according to future Kotlin 2.5 requirement by @koperagen in #2014
Documented deprecation of BigDecimal and BigInteger statistics by @AndreiKingsley in #2077

Compiler plugin

Annotated DataColumn.valueCounts for compiler plugin support by @koperagen in #2033
Enabled plugin evaluation of String API overloads by @koperagen in #2057
Supported get as alias to cols in CS DSL by @koperagen in #2071

Fixes and other changes

Fixed reading nullable nested (struct) columns from Parquet/Arrow by @zaleslaw in #2043
Fixed JSON serialization for all-null value column by @chimission in #2037
JSON IO fix for null by @AndreiKingsley in #2056
Fixed dataframe schemas equality check by @AndreiKingsley in #2042
Sqlite types fix by @AndreiKingsley in #2001
Verified JDBC test schemas and fixed column naming in DbType by @zaleslaw in #2135
Added SQL type audit tests to ensure driver compatibility by @zaleslaw in #2101
Fixed DataColumn.unfold by @zaleslaw in #2117
Fixed toList/toSequence bugs by @zaleslaw in #2128
Investigated GroupBy performance by @koperagen in #2059
Updated Kotlin to 2.4.20 by @Jolanrensen in #2085
Updated versions after 1.0.0-rc01 release by @koperagen in #1937
Migrated delegated property usages in build files for Gradle 10 by @koperagen in #2009
Configured Gradle daemon JVM toolchain by @koperagen in #2017
Replaced dependency updates Gradle plugin with Caupain by @Jolanrensen in #1919
Bumped KoDEx to 0.5.5 / 0.6.2 / 0.6.3 by @Jolanrensen in #1929, #2028, #2044
Enabled KoDEx cross module caches by @Jolanrensen in #2008
Reworked plugin-core shadow jar publishing to completely avoid dependencies in pom; fixed source dependency on :core in :samples by @koperagen in #2070
Simplified enabling compiler plugin in :core tests by @koperagen in #2058
Fixed flaky KoDEx build error by @Jolanrensen in #2019
Added the :samples test module to all contextual sources in KoDEx by @Jolanrensen in #2049
Used testImplementation for dataframe-jupyter in openapi-generator by @koperagen in #2013
Removed unused simplegit that required JDK 21+ by @koperagen in #2015
Added a module for shared test utils by @koperagen in #2010
Migrated MariaDB, MySQL, PostgreSQL tests to testcontainers by @koperagen in #2011
Updated outdated dependencies by @jetbrains-air[bot] in #2086, #2096, #2111, #2125

Documentation and Examples

Updated the Hibernate example by @Allex-Nik in #1923
Added Kotlin toolchain examples and documentation by @Jolanrensen in #2095 (bumped to 0.13.0 in #2138)
Restructured setup guides to have compiler plugin closer to library dependency setup by @koperagen in #1949
Updated setup instructions order, cleaned up mention of outdated Gradle plugin and example project by @koperagen in #1945
Updated documentation about removed plugins by @koperagen in #1948
Removed vague/outdated/duplicated paragraphs from README.md by @koperagen in #1939
Updated project logos and README with new Kotlin DataFrame branding by @AndreiKingsley in #2069
Added detailed AGENTS.md files for most of the modules by @zaleslaw in #2022
Hid TODO page by @koperagen in #1947
Updated print result in quickstart.md by @koperagen in #1942
Fixed broken convertTo link in require.md by @koperagen in #1943
Updated requireColumn KDocs, added col.cast() use case example by @koperagen in #2136
Added KDocs for DataSchema and DataRowSchema by @koperagen in #1775
Added KDocs for api/schema.kt by @zaleslaw in #2078
Added GroupBy transformation KDocs by @AndreiKingsley in #1936
Added KDocs for min/max, sum, mean by @Jolanrensen in #2012, #2032, #2102
Removed redundant {} braces around @include tags in min.kt/max.kt by @Jolanrensen in #2100
Added a small extra explanation to upperBound in Infer.Type by @Jolanrensen in #2092
Updated website docs, added KDocs and tests for valueCounts by @Allex-Nik in #2031
Added KDocs, tests and website docs for concat by @Allex-Nik in #2119; updated infix overload docs in #2141
Added KDocs for public reverse APIs by @zaleslaw in #2050
Documented transpose and transposeTo functions in DataRow API by @zaleslaw in #2047
Added KDocs for the four xs overloads by @zaleslaw in #2052
Added KDocs, tests, and a docs page for isEmpty / isNotEmpty by @zaleslaw in #2062
Added KDocs, tests, site updates for indices by @zaleslaw in #2068
Added KDocs for single plus site changes and tests by @zaleslaw in #2079
Added KDocs, tests and site updates for map.kt APIs by @zaleslaw in #2066
Added KDocs for forEach and extended the Iterating page by @zaleslaw in #2112
Added KDocs, tests and website examples for duplicate and duplicateRows by @zaleslaw in #2113
Added tests for associate and associateBy, documented key order by @zaleslaw in #2093
Added KDocs, tests and website examples for unfold, fixed DataColumn.unfold by @zaleslaw in #2117
Added KDocs and website docs for toList/toSequence, fixed their bugs by @zaleslaw in #2128
Added links from KDocs to website docs by @Allex-Nik in #2026
Added type mappings tests and docs for all supported DBs by @AndreiKingsley in #2002
Fixed drop and take docs by @Allex-Nik in #2027
Fixed fillNaNs samples by @Allex-Nik in #2054
Fixed Access APIs samples by @Allex-Nik in #2098
Removed outdated samples from :core by @AndreiKingsley in #2120
Synced datasets used in documentation with repository datasets by @AndreiKingsley in #2106
Fixed broken documentation URL nanAndNa.html → nanandna.html by @jetbrains-air[bot] in #2082
Fixed typos, grammar, broken links and misleading comments in KDocs and docs by @jetbrains-air[bot] in #2051, #2053, #2060, #2074, #2088, #2091, #2094, #2099, #2109, #2116, #2122, #2134, #2140

New Contributors

@zaleslaw with @Copilot made their first contribution in #2004
@jetbrains-air[bot] made their first contribution in #2051
@chimission made their first contribution in #2037

Full Changelog: 1.0.0-rc01...v1.0.0-rc02