numpy-groupies 0.16.4

Release 0.16.4

on Python PyPI

Release 0.16.4

Highlights

The suite measures every function in two sizes (#191)

The suite now measures the package against its group loop with one sweep, at two data sizes:

npg.aggregate(group_idx, a, func="sum")   # the sweep asks every function of the registry

• The sweep is driven by the registry: every function the package names is asked against the group loop of tests/reference.py, with and without nans, on every backend. The three copies of that sweep are merged into one (tests/test_sweep.py), and the second oracle of the complex tests is gone.
• It runs at 1,000 and at 200,000 values. The big size is above the floor of the parallel path of the numba backend, so the suite measures the sequential path and the parallel one with the very same test - the hand-written list of "the functions that go parallel" is gone. The pure python backend has no threads and walks the data in a python loop, so the big size is left out for it.
• New tests cover the validation in the words of the package, the aliases, the entry points, the out= contract, the fill value policy of every key, and the tools of groups.py - which had no test at all.
• The temporal tests are split by concern: the semantics, the tick arithmetic and the bare kernels. All 66 test functions of the old tests/test_temporal.py are still there.

The numba cache hits again (#191)

The identity of a dispatcher was not the same in every process: the fingerprint of its closure held the memory address of a python function, so the name of the on-disk cache file changed with every run and the cache never hit. Every process compiled every kernel again - the suite needed 5.2 GB and 73 seconds at four workers, and needs 3.0 GB and 30 seconds now.

The body of an overload implementation is part of the identity of the dispatchers that may reach it, and the walk follows the overload behind an overload: numba inlines those bodies into the caller, and a change of one of them used to leave the key of the cache untouched - a correctly typed, silently wrong kernel.

Bug fix

• The dispersion and the moments are centered on the first value of their group now: a group without spread has a variance of 0 and a skewness of 0 instead of the cancellation of the mean. The pure python implementation had kept the cancellation, and it showed on the python versions whose sum() over floats does not compensate the rounding of every step.
• A group without degrees of freedom (ddof at or above the size of the group) takes the fill value in every backend.
• The pure python implementation refuses the input it took silently: float and bool labels, labels beyond the size, a column of values without an axis, and the values that are no sequence of values (a dict, a set, a generator) - in the words of the package instead of an obscure TypeError from the list index.
• multi_arange([]) answers an empty array instead of raising.

Other changes

• The code that nothing called is gone: the group index check of the inputs, the gather kernels of the benchmark, the call overhead it measured and the stand-in for an implementation that is not there.
• The fixture that leaves the pure python backend out of the temporal tests lives in one place instead of two.