numpy-groupies 0.14.2

v0.14.2

on Python PyPI

The median got faster, and the numpy backend got faster with it. The numba backend selects the middle value with a Quickselect of its own, the numpy backend groups the values with a counting sort and sorts the group indices in the narrowest dtype that holds them. On the benchmark data (500,000 values in 1,000 groups) the numpy median comes down from 48 ms to 11 ms, the numba one from 11.5 ms to 9.3 ms.

Added

• The numba backend has the collecting function func="array" now (aliases list, split, splice, np.array, np.asarray), so every backend knows every function and the exceptions in the test suite are gone (#145). The result is an object array with one sequence per group, holding the values of that group in the order they appear in the input.
• The benchmark knows polars and can measure single functions (--funcs median,nanmedian) (#151). The polars participant asks the lazy engine that suits the function - the streaming one for most, the in-memory one for prod and median, which are faster there - and it gathers the result without an extra pass over the data.
• The group-loop column of the benchmark measures the reference implementation of the package, the one the tests sweep against (#152).

Changed

• The numba backend groups the values with the counting sort in the custom callable path and spares the median a copy of every block (#147): the custom callable comes down from 48 ms to 6.5 ms, the median from 13.9 ms to 10.7 ms.
• The operations that work on group-ordered data sort the group indices in the narrowest dtype that holds them (#148) - numpy's radix sort works per byte of its key, so the int64 indices this package is handed cost four times the int16 ones. The numpy sort comes down from 147 ms to 82 ms, the median from 58 ms to 24 ms.
• The known functions and the callables of the caller go through one registry per backend now (#150): the numpy backend loses its "generic" path, the numba backend its own custom-callable path. The lookup stays a plain dict subscript, 35 ns per call.
• The no-op building blocks of the numba kernels are shared between the operations (#146) - they are the same code in every operation, so one compiled artifact serves them all instead of one per operation and per cache file.
• The annotations of the public API are spelled out and the type of the group indices is the one that holds them (#144): group_idx is npt.NDArray[np.integer] everywhere, and the package type-checks under mypy in full.
• The comparison table in the README became a strength overview, and every example of the quickstart runs (#131).

Fixed

• The median of the numba and numpy backends is faster (#153): the numba backend selects the middle value with a Quickselect of its own instead of np.partition, the numpy backend asks for one kth and the maximum of the lower half instead of two kths. The numpy median comes down from 17.6 ms to 11.0 ms, the numba one from 10.6 ms to 9.3 ms. The numba backend answers the complex median with its own kernel now - it went to the numpy implementation before, which cost a copy of the data and an ordering of the values that the rest of the package does not use.
• An axis reduction with order="F" put the values into the wrong groups (#133), and the axis scans and the sorts lost the shape of the input (#134). The shape and the group of every value are preserved now, for both orders and both axes.
• The numpy backend returned flattened indices when an axis reduction named a numpy alias (np.argmin) instead of a string (#136) - the positions along the axis are right now.
• The pure python backend deviated from the other two in three places (#138): the arg reductions named a rank among the values they had skipped and pointed at the nan that poisoned the group, and the fill value went into the result as an object. All three answer like the other backends now.
• A group holding a nan answers the fill value of the arg reductions - a position cannot be a nan, so there is no position to name (#142 documents it, #138 fixes it).
• A median group holding a nan answers a nan in both parts of the result (#141). Before, the numpy and the numba backend said nan+0j - a claim about the imaginary part that the nan never reached - and the pure python one dropped the imaginary part of the result dtype.
• A fill_value=None where no result can hold it is refused with a clear message in all three backends (#140). Before, the numpy implementation died in a type check, the numba one failed to type a kernel, and the pure python one put the None into a column of floats.

Testing

• The backends are compared against the group loop of the reference implementation, never against each other (#137): 517 cases, where the pairwise comparison had 190. The suite is grouped by the contract a test enforces (#135).