Kotlin DataFrame 1.0.0-rc01 is released!
What's new?
- All our example projects now use the compiler plugin.
requireColumn- addition tocast,convertTofamily of functions. See requireColumn documentation for detailed example.- Documentation pages for pivot and groupBy provide code examples for all possible aggregations - 170 in total and all with rich visualization.
- Updated quickstart guide shows a few simple steps: loading data, transforming it, and visualizing it in IntelliJ IDEA project.
- Changes in
DataFrame.printto make it more comprehensive: rendering column types and total col x row by default; itsrowsLimitandvaluesLimitparameters now acceptnullvalue to disable truncation.
And many other improvements and fixes! See below for a complete list of changes in this release grouped by category.
We invite you to try this release:
- Use the
1.0.0-rc01version in your Gradle or Maven projects:- See setup instructions for Gradle, Maven and Android projects!
- We also recommend trying the compiler plugin!
- Use the special
1.0.0-rc01nversion in Kotlin Notebook. Thenversion includes a patch that resolves notebook specific issue with statistical functions.- You can specify the version in the descriptor or use
%useLatestDescriptors.
If you want to includekandy, list it afterdataframe:%useLatestDescriptors %use dataframe, kandy // or %use dataframe(1.0.0-rc01n), kandy(0.8.5, 0.8.5)
- Use kernel version
0.16.0-736or newer.
- You can specify the version in the descriptor or use
Features
- Added
requireColumnoperation by @koperagen in #1715 - Added
DataFrame.renderToMarkdown(): Stringfunction by @koperagen in #1760 - Made
emptyDataFramereified so resulting "null object" dataframes are suitable for aggregation and other column access operations like select by @koperagen in #1914 - Added
countDistinctoverloads onGroupByby @Allex-Nik in #1875 - Added Json instance overload to
readJson(inputStream, ...)by @Jolanrensen in #1869 - Improved rendering in
DataFrame.print()by @koperagen in #1760
rowsLimit,valuesLimitparameters now acceptnullto display all rows or complete column values.
NewborderStyleparameter to choose what ASCII symbols will be used for border
Types are now rendered on a separate row, and rendering is enabled by default - Aligned column name clashing handling across different IO by @AndreiKingsley in #1904
- Supported reading new
GEOMETRY,TIME_NS,VARIANTtypes from DuckDB by @Jolanrensen in #1924
Removals and Deprecations
- Removed remaining data modifiers from intermediate operations classes: Corr, SplitWithTransform for cleaner public API by @koperagen in #1826
renameintois deprecated in favor ofrenametoby @Jolanrensen in #1837- Deprecated format guessing
DataFrame.read()andSupportedDataFrameFormatin favor of specificDataFrame.readCsv/readJson/...functions by @Jolanrensen in #1916
Compiler plugin
- Annotated
DataColumn.map*for compiler plugin support by @koperagen in #1825 - Annotated
DataFrame.get(String columns)for compiler plugin support by @koperagen in #1853 - Annotated
allExceptCS DSL operations for compiler plugin support by @Jolanrensen in #1897 - Updated
DataFrame.concatsignature for better plugin support - Annotated DataFrameBuilder APIs for plugin support by @koperagen in #1828
Fixes and other changes
- Fixed double opt-in in notebooks by @Jolanrensen in #1822
- Set the dataframe-compiler-plugin-core module back to java 8 by @Jolanrensen in #1824
- Refactored the expressions-converter plugin for docs samples with easier API by @koperagen in #1816
- Created
dfbuild.dependencyUpdatesconvention plugin by @Jolanrensen in #1827 - Post Beta5 release project update by @AndreiKingsley in #1831
- Infrastructure version bumps by @Jolanrensen in #1834
- Added column kind and type checks in
explodeandungroupby @AndreiKingsley in #1847 - relaxing error for unexpected folder in examples/projects to warning by @Jolanrensen in #1849
- Disallowing
ValueColumn<AnyFrame>because FrameColumn should be used by @Jolanrensen in #1856 - Migrated to Korro 0.2.2 by @AndreiKingsley in #1846
- Fixed formatted frame toHtml methods by @AndreiKingsley in #1892
- Preserve resolved path from ColumnsSelector in
moveToImplso that it can be correctly used from compiler plugin by @koperagen in #1895 - Added a method for handling extension property exceptions by @AndreiKingsley in #1896
- Re-enabled incremental compilation by @Jolanrensen in #1899
- Fixed
formatfor column group by @AndreiKingsley in #1898 - Disabled renaming to camelcase for
generateInterfaces/DataClassesfunctions by @Jolanrensen in #1922 - Inlined
AnyFrame,AnyRow, andAnyBaseColtype aliases in public API by @Allex-Nik in #1863 - Ensured scoped table retrieval for MySQL/MariaDB connections by @zaleslaw in #1901
- Fixed SQL query validation: replace
strictValidationwithSqlValidationenum by @zaleslaw in #1881 - To type overloads with diff nullability by @Allex-Nik in #1839
- Enable Module dependencies in KoDEx context by @Jolanrensen in #1910
- Bumping project to Kotlin 2.4 by @Jolanrensen in #1891
dependencyUpdatesfor example projects by @Jolanrensen in #1893- Typesafe conventions bump and keywordsGenerator convention plugin by @Jolanrensen in #1915
- Updated dependencies and fixed compatibility issues by @Jolanrensen in #1924
Documentation and Examples
- Expanding and updating custom SQL DB docs by @Jolanrensen in #1710
- Parsing/convert docs update by @Jolanrensen in #1817
- Added parsing date-time migration docs for 1.0 by @Jolanrensen in #1830
- moving titanic example to examples/projects and enabling compiler plugin by @Jolanrensen in #1832
- Added kdocs for
DataFrame.concatby @koperagen in #1828 - fixed ExcludeFromSources in DslGrammarTemplateColumnsSelectionDsl.kt to exclude only the right elements by @Jolanrensen in #1838
- Docs warnings update by @Jolanrensen in #1843
- Improved error message in
readCsvwhencolTypescauses parsing exception by @Jolanrensen in #1848 - Update website documentation for
groupByandpivotby @Allex-Nik in #1800 - Moved JSON-OpenAPI example and updated docs by @Jolanrensen in #1845
- Added missing samples changes by @AndreiKingsley in #1861
- Added texts samples saving by @AndreiKingsley in #1862
- Apply
korroto samples in data sources by @Allex-Nik in #1833 - Updated data schema documentation by @AndreiKingsley in #1859
- Llms txt for website by @Jolanrensen in #1874
- Moving (and fixing) kotlin-spark example to /projects by @Jolanrensen in #1882
- Moving final examples to /projects by @Jolanrensen in #1886
- Clarify the descriptions of some functions in the
pivotdocs by @Allex-Nik in #1860 - Add
medianByandpercentileByto thegroupBygrammar by @Allex-Nik in #1878 - Configure
KorroGenerateTaskto run after test tasks by @AndreiKingsley in #1890 - Added troubleshooting guide for Data Schemas and Extension Properties by @AndreiKingsley in #1870
- Update the documentation of Row condition by @Allex-Nik in #1889
- Group by values exceptions by @Jolanrensen in #1905
- Updated quickstart guide by @AndreiKingsley in #1906
Full Changelog: v1.0.0-Beta5...1.0.0-rc01