genvarloader 0.15.0

on Python PyPI

0.15.0 (2025-05-23)

⚠️ Breaking Changes ⚠️

• The .with_indices and .with_transform methods have been moved to be arguments to the .to_dataloader and/or .to_torch_dataset methods. Motivation: returning indices and transforms are not necessary outside of a dataloading context. Best practice to facilitate subsetting with GVL is to obtain a PyTorch Datset via gvl.Dataset.to_torch_dataset(), pass this to torch.utils.data.Subset, and then provide to this gvl._torch.get_dataloder. This is a very thin wrapper around the the native PyTorch DataLoader with better defaults for GVL to take advantage of multithreading (better than multiprocessing). Documentation updates are pending.

📦 Notable Features 📦

• The RefDataset class, an interface for lazily accessing and organizing subsequences from a reference genome
• The DatasetWithSites class, which combines a Dataset with a table of site-only variants e.g. from ClinVar or gnomAD and inserts the site-only variants into haplotypes. Currently only 1 variant at a time and only SNPs.
• Annotation tracks for Datasets, which take a BED file and broadcast its values to be treated as a track for all samples.
• Now includes SeqPro >= 0.4.2, which adds functionality for scanning GTFs as polars.LazyFrames and extracting GTF attributes via the scan and attr functions under the seqpro.gtf namespace.

Feat

• add reference property to Dataset and add path attribute to Reference
• add RefDataset to work with one or more reference genomes. Also change internal indexing to never materialize a full dataset index, dramatically reducing memory usage to support datasets with 1M+ regions. BREAKING CHANGE: move returning indices and transforms to torch API, since these features are generally unnecessary for non-dataloading contexts.
• sites-only changes for QoL. fix: consider output length < region length for sites overlap
• allow tracks to pass-through dataset with sites since SNPs have no affect on them
• wip: pad ragged annotated ref coords with max dtype value. pass sanity checks.
• wip: change ref_coord annotation so that right-pad values have position MAX_I32
• type-safe Dataset, passes all tests.
• refactor Dataset implementation to be (almost) fully type-safe.
• wip: sites-only variants
• wip: use sp.bed functions.
• wip: small updates
• apply sites-only SNPs, filtering non-SNPs out from VCFs.
• sites-only classes, intersecting them with Datasets, and obtaining information necessary to apply variants.
• add annot track to dummy dataset
• wip: initial implementation for read/write annotation tracks, incorporating them along the track dimension.
• deprecate unphased variants
• wip: dosages/CCFs on ragged variants
• prototype of returning ragged variants from Dataset

Fix

• shape of single item from RefDataset
• update init for seqpro bump
• bump seqpro to 0.4.0 which includes basic gtf ops
• update dummy dataset for changes to Reference. docs: add more docstrings
• jittering by folding it into data (re)construction
• contig offset mapping for in-memory reference and incrementing offset when writing cache
• bump genoray version to handle unsorted PVAR contigs
• bump genoray version for filtered PGEN fix
• ensure annot tracks match on-disk ordering
• update for internal breaking changes
• check for SNPs
• bump genoray so bioconda pgenlib is valid
• pass all tests
• internal breaking changes
• treat POS as 1-based to match VCF spec
• type annotations
• type annotation
• contig naming for reference fasta.
• contig normalization
• pass all tests.
• pass tests.
• pass tests.
• Dataset.open returns highest complexity ds by default (haps + all tracks, sorted).
• use pandera polars not pandas
• correct manipulation of active tracks
• dummy dataset
• wrong germline ccfs for 3rd germline variant and beyond
• wrong geno path
• parsing SVAR metadata, bump genoray

Refactor

• use genoray

[main c0fd8be] bump: version 0.14.4 → 0.15.0
2 files changed, 60 insertions(+), 1 deletion(-)