explorer 0.8.2

v0.8.2

on Hex

Added

• Add functions to work with strings and regexes.

  Some of the functions have the prefix "re_", because they accept a string that represents a regular expression.

  There is an important detail: we do not accept Elixir regexes, because we cannot guarantee that the backend supports it. Instead we accept a plain string that is "escaped". This means that you can use the ~S sigil to build that string.
  Example: ~S/(a|b)/.

  The added functions are the following:

  • Explorer.Series.split_into/3 - split a string series into a struct of string fields. This function accepts a string as a separator.

  • Explorer.Series.re_contains/2 - check is the string series matches the regex pattern. Like the "non regex" counterpart, it returns a boolean series.

  • Explorer.Series.re_replace/3 - replaces all occurences of a pattern with replacement in string series. The replacement can refer to groups captures by using the ${x}, where x is the group index (starts with 1) or name.

  • Explorer.Series.count_matches/2 - count how many times a substring appears in a string series.

  • Explorer.Series.re_count_matches/2 - count how many times a pattern matches in a string series.

  • Explorer.Series.re_scan/2 - scan for all matches for the given regex pattern.
    This is going to result in a series of lists of strings - {:list, :string}.

  • Explorer.Series.re_named_captures/2 - extract all capture groups as a struct for the given regex pattern. In case the groups are not named, their positions are used as names.

• Enable the usage of system certificates if OTP version 25 or above.

• Add support for the :streaming option in Explorer.DataFrame.to_csv/3.

• Support operations with groups in the Lazy Polars backend. This change makes the lazy frame implementation more useful, by supporting the usage of groups in following functions:

  • Explorer.DataFrame.slice/3

  • Explorer.DataFrame.head/2

  • Explorer.DataFrame.tail/2

  • Explorer.DataFrame.filter_with/2 and the macro version of it, filter/2.

  • Explorer.DataFrame.sort_with/3, although it ignores "maintain order" and "nulls last" options when used with groups.

  • Explorer.DataFrame.mutate_with/2 and its macro version, mutate/2.

Changed

• We now avoid raising an exception if a non existent column is used in Explorer.DataFrame.discard/2.

• Make the dependency of cacerts optional. This is because people using Erlang/OTP 25 or above can use the certificates provided by the system.
  So you may need to add the dependency of cacerts if your OTP version is older than that.

• Some precision differences in float operations may appear. This is due to an update in the Polars version to "v0.38.1". Polars is our default backend.

Fixed

• Fix Explorer.Series.split/2 inside the context of Explorer.Query.

• Add optional X-Amz-Security-Token header to S3 request. This is needed in case the user is passing down a token for authentication.

• Fix Explorer.DataFrame.sort_by/3 with groups to respect :nils option.
  This is considering only the eager implementation.

• Fix inspection of lazy frames in remote nodes.

Pull requests

• Bump Polars 0.37 by @lkarthee in #861
• DataFrame.discard/2 - don't raise for non existent column by @lkarthee in #872
• Add native expression for Series.split/2 by @H12 in #875
• Bump mio from 0.8.10 to 0.8.11 in /native/explorer by @dependabot in #876
• Implements Series.split_into/3 by @ryancurtin in #873
• Update Polars to v0.38 by @philss in #879
• Add optional x-amz-security-token header to S3 request by @jschniper in #881
• Rewrite LazyFrame by @philss in #882
• Update Rustler to v0.32.1 by @philss in #884
• Fix DF.sort_by/3 with groups to respect :nils option by @philss in #886
• Update Polars to v0.38.3 by @philss in #887
• Implements :streaming option for DataFrame.to_csv/3 by @ryancurtin in #889
• Support operations with groups in the Lazy Polars backend by @philss in #890
• Bump h2 from 0.3.25 to 0.3.26 in /native/explorer by @dependabot in #891
• Revert LazyFrame implementation with stack by @philss in #892
• Refactor eager DF implementation to make use of lazy backend by @philss in #893
• Add re_contains/2 and re_replace/3 to match with a regex by @philss in #894
• Add count_matches/2, re_count_matches/2, re_scan/2 and re_named_captures/2 to Series by @philss in #895
• Add changes to the change log for the upcoming version by @philss in #897
• Update dependencies by @philss in #899
• Pass down backend to lazy series and enable re_named_captures/2 usage by @philss in #896
• Release v0.8.2 by @philss in #900

New Contributors

• @H12 made their first contribution in #875
• @ryancurtin made their first contribution in #873
• @jschniper made their first contribution in #881

SHA 256 of Artifacts

fd4d7db73577544d1008827502461fbc82644b44879bf4d50b8c7c2f7a04ad1f  explorer-v0.8.2-nif-2.15-x86_64-pc-windows-gnu--legacy_cpu.dll.tar.gz
ba9f6afe86d37e52b7481a29e6011cdc834b2c0196ee6b4235497c4a405fe6e3  explorer-v0.8.2-nif-2.15-x86_64-pc-windows-gnu.dll.tar.gz
447e3150ebffa1712ed7b6d56e11dd2369126a92bf5466570e4f36ae46f200b9  explorer-v0.8.2-nif-2.15-x86_64-pc-windows-msvc--legacy_cpu.dll.tar.gz
2032955e04c6632fd4d6d1015f611b7b90a84a405710d49cce46b7b1e1f52b3d  explorer-v0.8.2-nif-2.15-x86_64-pc-windows-msvc.dll.tar.gz
9f10c1b25846de37ca2caf271c7728716d5d6783c82e823783ced53cc6a0b4b0  libexplorer-v0.8.2-nif-2.15-aarch64-apple-darwin.so.tar.gz
aceade08eab94230b8f9dc87a5850e5523a7cf7a4222495bf3fa012c4622cd54  libexplorer-v0.8.2-nif-2.15-aarch64-unknown-linux-gnu.so.tar.gz
0f0341d8a0928554ea2c083a653e599fdd406a0675bfc8e615465e6461726508  libexplorer-v0.8.2-nif-2.15-aarch64-unknown-linux-musl.so.tar.gz
63f9ffda8f9dbcacb12a3a522adbde370b78579a23a75183321fb0fa81f0a596  libexplorer-v0.8.2-nif-2.15-x86_64-apple-darwin.so.tar.gz
65793e232a26a91bcfb90867f6392e34228d8f3f23419f500bee47f08c3e8896  libexplorer-v0.8.2-nif-2.15-x86_64-unknown-freebsd--legacy_cpu.so.tar.gz
3b4d5b1d88cfe416a13e3f0f1fdc87dd54dbf78dff7ff40cb1f64aa7652d5b8a  libexplorer-v0.8.2-nif-2.15-x86_64-unknown-freebsd.so.tar.gz
3afb057bfecdf86199a9dc380be2f44c44f09ac428d806e7983232ba6f15601b  libexplorer-v0.8.2-nif-2.15-x86_64-unknown-linux-gnu--legacy_cpu.so.tar.gz
cfab4552f1f3791e38c6b6ce3d5099fbe88f686ec062ac181ef2b35f3432c1e3  libexplorer-v0.8.2-nif-2.15-x86_64-unknown-linux-gnu.so.tar.gz
7fa4961f08f9278f6b8585d2b4f89d5712ccf222de7a41edb054915d8ec7d50c  libexplorer-v0.8.2-nif-2.15-x86_64-unknown-linux-musl.so.tar.gz

Full Changelog: v0.8.1...v0.8.2