uwdata/arquero v7.0.0

on GitHub

Changes from v6.0.1:

• Breaking: Change the default handler of Apache Arrow data from the apache-arrow JS reference implementation to the smaller, faster, and more flexible @uwdata/flechette.

  This change improves performance and simplifies Arquero internals, as it obviates the need for internal custom Arrow data conversion steps and a variety of earlier performance and correctness workarounds. Arrow IPC binary decoding (in fromArrow) and building/encoding (in toArrow and toArrowIPC) now use Flechette.

  If passing an already instantiated Arrow table to fromArrow, Arquero supports both apache-arrow and @uwdata/flechette tables, including optimizations for groupby and filter operations over dictionary types. However, special extraction of decimals, dates, and nested types (lists, structs) is no longer provided by Arquero; instead we default to whatever the Arrow implementation provides. For Flechette, this includes customizable extraction options and efficient serialization to native JS array and object types. For Arrow-JS, decimals are not well-supported and lists are returned as Arrow Vector types and structs as slow Proxy row objects.

  If loading data directly from IPC bytes, Arquero results should be consistent. Moreover, as Flechette is small, it is bundled directly with Arquero, so (unlike before) no additional imports are needed to process Arrow data. However, when passing-in already-parsed apache-arrow tables, results will differ for some data types.

Full Changelog: v6.0.1...v7.0.0