Kotlin/dataframe v1.0.0-Beta4

on GitHub

Yes, this is another Beta release that brings us closer to the upcoming 1.0 version!
Many bugs have been fixed, a large amount of documentation has been added, and several quality-of-life improvements have been introduced.

We invite you to try this release:

• Use the 1.0.0-Beta4 version in your projects.
  • We also recommend trying the compiler plugin, which has received important updates.
• Use the special 1.0.0-Beta4n version in Kotlin Notebook.
  • The regular version still does not work with statistical functions. The n version includes a patch that resolves this issue.
  • You can specify the version in the descriptor or use %useLatestDescriptors.
    If you want to include kandy, list it after dataframe:

    %useLatestDescriptors
    %use dataframe, kandy
    // or
    %use dataframe(1.0.0-Beta4n), kandy(0.8.3)

  • New dataframe and kandy versions are not supported in older versions of Kotlin Jupyter kernel.
    Please use kernel version 0.16.0-736 or newer.

📚 See the Kotlin DataFrame documentation

See below for a complete list of changes in this release grouped by category.

Features

• Added Map<String, Any?>.toDataRow() function by @CarloMariaProietti in #1433
• Added independent FrameColumn formatting by @Jolanrensen in #1443
• Added Implicit Char to String v2 by @Jolanrensen in #1420
• Added .none {} to DataFrame by @CarloMariaProietti in #1462
• Added formatHeader by @AndreiKingsley in #1459
• Added move.before by @CarloMariaProietti in #1474
• Added dataframe creation from rows by @CarloMariaProietti in #1463
• Improved duplicateRows error message by @koperagen in #1494
• Improved rendering of ColumnGroup by @Allex-Nik in #1460
• Improved list.chunked() + List<List<T>>.toDataFrame for parsing .srt and similar text files by @koperagen in #1486
• Refactored DataFrame JDBC API plus DataSource handling by @zaleslaw in #1487
• Added move.toStart() keeping inside group by @CarloMariaProietti in #1489
• Added insert.before { } by @CarloMariaProietti in #1517
• Added Path overloads by @AndreiKingsley in #1563
• Migrated jvm-default, fix #1466 by @koperagen in #1583
• Added better H2 support by @zaleslaw in #1598

Removals and Deprecations

• Refactored rename, add overloads to addAll and deprecate old versions by @jetbrains-junie[bot] in #1418
• Candidates for removal deprecation by @AndreiKingsley in #1473
• Removed unused zero/zeroOrNull by @koperagen in #1550
• Merged KeyValueProperty and NameValuePair by @Jolanrensen in #1545

Fixes

• Fixed NPE in GeoDataFrame rendering by @ileasile in #1446
• Added loading dataframe-jupyter from core in notebooks by @Jolanrensen in #1415
• Fixed linter by @zaleslaw in #1453
• Fixed cumSum by @Jolanrensen in #1467
• Fixed quantile by @Jolanrensen in #1482
• Fixed .toDataFrame by @koperagen in #1475
• Fixed DataFrameSchema.equals() including order and CompareResult.Matches by @Jolanrensen in #1505
• Fixed Nothing cols by @Jolanrensen in #1551
• Removed redundant type parameter from transpose by @koperagen in #1553
• Fixed pivot shortcuts to use PivotDsl by @Jolanrensen in #1572
• Fixed bug with extracting metadata for many columns by @zaleslaw in #1573
• Added Optin experimental Instant in codegen if this type is used by @koperagen in #1601

Documentation and Examples

• Added join docs by @AndreiKingsley in #1437
• Updated dynamic XML badge in README.md by @Jolanrensen in #1438
• Added examples and docs for Beta3 by @Jolanrensen in #1441
• Added Parquet documentation by @zaleslaw in #1448
• Enabled Compiler plugin for :samples by @Jolanrensen in #1456
• Removed deprecated get from examples & update obsolete dataFrameOf example by @koperagen in #1481
• Updated documentation of append by @koperagen in #1483
• Fixed inline column/columnOf usages in split.md by @koperagen in #1503
• Added documentation for increasing the heap size for Kotlin Notebook by @tp-andygo in #1518
• Added config for https://context7.com/kotlin/dataframe by @koperagen in #1515
• Added groupBy Kdocs by @AndreiKingsley in #1533
• Updated createDataFrame.md with compiler-plugin friendly examples by @koperagen in #1519
• Added tests for the DataSource parameter by @zaleslaw in #1562
• Added Pivot kdocs by @AndreiKingsley in #1554
• Updated SetupGradle.md by @koperagen in #1577
• Updated info about arrow module with Parquet support by @koperagen in #1581
• Added join kdocs by @AndreiKingsley in #1574
• Updated Guide-for-backend-SQL-developers.md by @koperagen in #1580
• Updated Compiler-Plugin.md by @koperagen in #1576
• Added documentation and tests for the last and lastOrNull functions by @Allex-Nik in #1561
• Added documentation and tests for the first and firstOrNull functions by @Allex-Nik in #1547

Compiler Plugin

Development of the compiler plugin happens in Kotlin repository: https://github.com/JetBrains/kotlin/tree/master/plugins/kotlin-dataframe. Change log only includes relevant changes in the library.

• Annotated functions for df.select { extension.col("myCol") } and operation.into { "str" } syntax by @koperagen in #1444
• Annotated missed convert functions for compiler plugin support by @koperagen in #1447
• Added GroupBy statistics to compiler plugin by @Jolanrensen in #1485
• Annotated String.invoke, addAll(dataframes), AddDsl.unaryPlus by @koperagen in #1484
• Added annotation IntellijPluginApi to mark API used by KDF plugin by @Allex-Nik in #1440
• Added annotations for moveTo() to add compiler plugin support by @Jolanrensen in #1499
• Excluded libraries.json from compiler-plugin-core by @koperagen in #1521
• Excluded some unused classes to reduce plugin size by @koperagen in #1506

New Contributors

• @CarloMariaProietti made their first contribution in #1433
• @Allex-Nik made their first contribution in #1440
• @tp-andygo made their first contribution in #1518

Full Changelog: v1.0.0-Beta3...v1.0.0-Beta4