explorer 0.10.0

v0.10.0

on Hex

Added

• Add support for the decimals data type.

  Decimals dtypes are represented by the {:decimal, precision, scale} tuple,
  where precision can be a positive integer from 0 to 38, and is the maximum number
  of digits that can be represented by the decimal. The scale is the number of
  digits after the decimal point.

  With this addition, we also added the :decimal package as a new dependency.
  The Explorer.Series.from_list/2 function accepts decimal numbers from that
  package as values - %Decimal{}.

  This version has a small number of operations, but is a good foundation.

• Allow the usage of queries and lazy series outside callbacks and macros.
  This is an improvement to functions that were originally designed to accept callbacks.
  With this change you can now reuse lazy series across different "queries".
  See the Explorer.Query docs for details.

  The affected functions are:

  • Explorer.DataFrame.filter_with/2
  • Explorer.DataFrame.mutate_with/2
  • Explorer.DataFrame.sort_with/2
  • Explorer.DataFrame.summarise_with/2

• Allow accessing the dataframe inside query.

• Add "lazy read" support for Parquet and NDJSON from HTTP(s).

• Expose more options for Explorer.Series.cut/3 and Explorer.Series.qcut/3.
  These options were available in Polars, but not in our APIs.

Fixed

• Fix creation of series where a nil value inside a list - for a {:list, any()} dtype -
  could result in an incompatible dtype. This fix will prevent panics for list of lists with
  nil entries.

• Fix Explorer.DataFrame.dump_ndjson/2 when date time is in use.

• Fix Explorer.Series.product/1 for lazy series.

• Accept %FSS.HTTP.Entry{} structs in functions like Explorer.DataFrame.from_parquet/2.

• Fix encode of binaries to terms from series of the {:struct, any()} dtype.
  In case the inner fields of the struct had any binary (:binary dtype), it was
  causing a panic.

Changed

• Change the defaults of the functions Explorer.Series.cut/3 and Explorer.Series.qcut/3
  to not have "break points" column in the resultant dataframe.
  So the :include_breaks is now false by default.

Pull requests

• Fix DataFrame.filter_with spec by @mhanberg in #975
• fix typo cuontry ~> country by @audacioustux in #979
• Fix issue 976 by @billylanchantin in #980
• Fix "dump_ndjson" by activating "timezones" feature by @philss in #981
• Dependency updates - mix only by @philss in #983
• Fix typos by @preciz in #984
• Fix product for lazy series by @philss in #986
• Update Polars to v0.43.1 by @philss in #985
• Add initial support to Decimal by @philss in #987
• Add function clause for %FSS.HTTP.Entry{} by @billylanchantin in #992
• LazyFrame from http endpoint by @ceyhunkerti in #993
• Fix encode of binaries to terms inside struct series by @philss in #996
• Update dependencies by @philss in #997
• Add read support for lazy http ndjson by @ceyhunkerti in #998
• De-functionalize query internals by @billylanchantin in #989
• Fix return type in DataFrame.dtypes/1 typespec (#1002) by @mooreryan in #1003
• Add support for some operations with decimals by @philss in #988
• Expose options to Series' cut and qcut by @philss in #1007
• Fix cut/3 and qcut/3 when :include_breaks is false by @philss in #1009
• Release v0.10.0 by @philss in #1008

New Contributors

• @audacioustux made their first contribution in #979
• @preciz made their first contribution in #984
• @ceyhunkerti made their first contribution in #993
• @mooreryan made their first contribution in #1003

Full Changelog: v0.9.2...v0.10.0

SHA256 of the artifacts

28897fbb14f54a6a9d996cccf4d863dfdc27e589324bd22382c434402ddd258a  explorer-v0.10.0-nif-2.15-x86_64-pc-windows-gnu--legacy_cpu.dll.tar.gz
f102a65a788cd6cd5d9cecea33ff3e10f924ff90bf9b43b65a5acc86b38f1d40  explorer-v0.10.0-nif-2.15-x86_64-pc-windows-gnu.dll.tar.gz
d414eabfca363b1731f24917445b05a1b2c969c079e35d9cfb6aa59ef1d02af5  explorer-v0.10.0-nif-2.15-x86_64-pc-windows-msvc--legacy_cpu.dll.tar.gz
aa71834d756c39b02dd45e9474e3782758cf3d4617b61bc891a820ec8fb35981  explorer-v0.10.0-nif-2.15-x86_64-pc-windows-msvc.dll.tar.gz
ab0dc0df737611b73222072e865ba9a29fc4026ae5b33cd337c92c26137c835b  libexplorer-v0.10.0-nif-2.15-aarch64-apple-darwin.so.tar.gz
b837b3523b60e7804a0d38a8466ed8e0f3f68b2cea9ea10051811a0b20b08932  libexplorer-v0.10.0-nif-2.15-aarch64-unknown-linux-gnu.so.tar.gz
c8d6c07af4a6b76b4769931132f045c1eb02a7916089f4db13481d2f5f1db363  libexplorer-v0.10.0-nif-2.15-aarch64-unknown-linux-musl.so.tar.gz
51fa272758b6cf331054ba3f27c13a64347c68f193d0c337240f1c79217678d2  libexplorer-v0.10.0-nif-2.15-x86_64-apple-darwin.so.tar.gz
804602cab628dae931c5ab6c78d0c62b878da764ca5312b61e03b32c0cbb420d  libexplorer-v0.10.0-nif-2.15-x86_64-unknown-freebsd--legacy_cpu.so.tar.gz
ae8c48ff9b5d68a6587c423f71b5083a0e2e05dc79de7b136479527dd1508dec  libexplorer-v0.10.0-nif-2.15-x86_64-unknown-freebsd.so.tar.gz
493a49c6ff72382c5ea5c79b01430230cc606174559ee596effe1022d8b665ef  libexplorer-v0.10.0-nif-2.15-x86_64-unknown-linux-gnu--legacy_cpu.so.tar.gz
e7ead11dbca060e2f596e34507c304213bdda3ef0f26fc7c0f19db4edfd29b63  libexplorer-v0.10.0-nif-2.15-x86_64-unknown-linux-gnu.so.tar.gz
62e480e536ef9a132bbc542df4a043f52fa6512a2ed8c2c5165ff4cff47b0e07  libexplorer-v0.10.0-nif-2.15-x86_64-unknown-linux-musl.so.tar.gz