github teaguesterling/plinking_duck v0.6.1
v0.6.1 — projection-aware parquet psam load

latest releases: v0.14.2, v0.14.1, v0.14.0...
3 months ago

Projection-aware parquet psam load

The .psam.parquet companion path now materializes and reads only the psam columns a query selects, instead of all columns. This matters for wide biobank psam files (many phenotype/covariate columns): a carrier SELECT IID query on a 10M-sample, 23-column psam drops from ~1.2s to 0.62s, and a counts-only query (no psam column) from ~0.5s to 0.05s. Row alignment and values are verified against the source across all rows.

Bug fixes (parquet psam path)

  • Typed covariate columns (INTEGER/DOUBLE) from a parquet psam are now read correctly — they previously triggered a type-mismatch error.
  • genotypes := 'columns' with a parquet psam now returns the psam metadata columns (was returning NULL).

See PR #44.

Don't miss a new plinking_duck release

NewReleases is sending notifications on new releases.