allelix 2.0.1

v2.0.1

on Python PyPI

Patch release. Focus: correctness fixes, plus a chain-of-trust ClinVar bug surfaced during real-data verification. Cache-format unchanged; existing v2.0.0 ClinVar caches refresh automatically on the next allelix db update.

Upgrade

pip install --upgrade allelix
allelix db update          # refreshes ClinVar via the new loader; everything else is signal-matched and skipped

The CLINVAR_INTERPRETER_VERSION bump (1 → 2) invalidates v2.0.0 ClinVar caches so the new ClinVar loader takes effect. No re-download is forced for gnomAD / AlphaMissense / CADD; their remote signals are unchanged.

Highlights

• Wrong-allele safety: three independent paths that could attach the wrong allele's frequency / pathogenicity / CADD score to your variants are now suppressed. When we can't confirm which alt the user carries, we show nothing instead of a number that might describe a different allele.
• Cache-integrity hardening: ClinVar .md5 validation, GWAS / ClinPGx row-count floor, gnomAD schema-tag enforcement, and db update's tagless-cache re-download close four ways the cache could silently end up wrong.
• Build detection: normalize_build_label no longer mis-tags files based on substring noise; 2038-01-01 does not parse as GRCh38, hg19/hg38 returns "ambiguous" instead of picking one.
• Terminal output: trimmed to a bare-min column set that fits typical 100–120-col terminals without Rich's auto-squeeze producing hairline-blank columns. Enrichment data still surfaces in HTML / JSON reports.

Known limitations

• ClinVar per-(classification, condition) pairing (tracked for v2.1). ClinVar variants with multiple SCV submissions display the primary aggregate classification and the union of all associated conditions (semicolon-joined). The exact 1:1 pairing between a specific classification and the specific condition it was submitted for is not preserved — the VCF source doesn't carry that mapping. Full per-(classification, condition) rows via ClinVar's submission_summary.txt.gz structured export are planned for v2.1. Affects ~0.01% of cached ClinVar records — exclusively multi-submission variants.

Changed

• scripts/fetch_testdata.sh URL. Now downloads from allelix/allelix/releases/download/v2.0.0/test_data.tar.gz (the ~1.27 GB asset including the v2.0.0 VCF/gVCF additions: real GIAB HG002 GRCh38 + GRCh37 benchmarks, a real GATK-HC gVCF, a 1000 Genomes chr22 multi-sample, plus the synthetic mocks). Previous URL pointed at the v1.1.1 asset, which lacked all the VCF samples.
• HuggingFace asset URLs moved to the allelix/ org. The gnomAD and AlphaMissense prebuilt-cache download URLs were previously hardcoded to huggingface.co/datasets/dial481/...; both datasets now live under the allelix/ HF org. HF serves redirects from the old paths, so existing pinned client installs continue to work; new installs pull from the canonical org URL. Commit SHAs in the resolve paths are unchanged — the asset bytes are identical and SHA256 integrity checks pass without modification.
• Terminal report: bare-min column set. The analyze table previously had 9–12 columns; on a typical 100–120 col terminal Rich auto-squeezed the middle columns to hairline-zero widths and cut rsIDs off mid-string. New layout: rsID | Gene? | Source | Significance | Mag | GT | Condition? (Gene and Condition dropped when no row carries data). Source uses compact display (GWAS Catalog → GWAS), significance drops the redundant source_ prefix (clinvar_pathogenic → pathogenic), Genotype renamed to GT. Terminal output only — every dropped field still surfaces in HTML and JSON reports unchanged.

Fixed — wrong-allele safety

• resolve_strand: drop complement-fallback. At multi-allelic sites, the complement of the user's true forward allele can coincidentally equal a different alt at the same position. The previous fallback (when the user allele didn't match {ref, alt} directly, try complement(user_allele) ∈ {ref, alt}) would return that coincidental complement, and CADD enrichment then stamped a wrong-allele score onto an annotation that read as describing the user's variant. Reproduction: resolve_strand('A', 'C', 'T') → 'T' (user's A is neither REF nor ALT but complement('A') = 'T' = ALT). The function now returns None whenever the user allele isn't directly in {ref, alt}. ADR-0010 already deferred proper strand handling; this makes the deferral explicit at the resolution layer.
• Enrichment: drop MAX-fallback for alt-less annotations (suppress-half). GWAS rows set alt="" because the catalog is rsID-keyed. The previous enrichment path took a MAX(af) / MAX(am_pathogenicity) / MAX(phred) GROUP BY rsid fallback for those rows, which at a multi-allelic site stamped the highest-frequency / highest-pathogenicity / highest-CADD alt's value next to the annotation as if it described the user's variant. Symmetric fix to the strand-resolve change above: skip the enrichment number rather than show one that may belong to a different allele. The position-fallback path (used for rsIDs resolved on the fly via ClinVar from the v2.0.0 rsID-less resolution work) is allele-specific and stays — it pairs the alt-less annotation's rsID with the user's actual carried alt. CADD enrichment grew the same position-fallback path for symmetry with gnomAD / AlphaMissense.
• ClinVar condition joins all CLNDN entries (suppress-half). The loader was treating | in CLNSIG and CLNDN as parallel arrays index-paired with the VCF's ALT, then picking [0] from each. That's wrong: the | separator is per-SCV (per submission), not per-ALT, and the two lists are not guaranteed to be in matching order. Symptomatic case was a "Frankenstein pairing": one SCV's classification shown next to another SCV's condition (real-world example: rs1063192 surfaced as Likely_pathogenic + Three_Vessel_Coronary_Disease when the Likely_pathogenic submission was actually for Breast cancer; the Three_Vessel_CD submission was Protective). The minimal v2.0.1 fix joins the full CLNDN list into a single condition string per record. clinical_significance keeps the primary aggregate (CLNSIG[0]).

Fixed — cache integrity

• GWAS / ClinPGx loaders: post-load row-count floor. Both upstream sources stream over chunked transfer with no Content-Length; a connection drop mid-stream produces a parseable-but-short TSV that previously committed to the cache and shipped fewer associations / clinical annotations than the user expected. Loaders now raise OSError before os.replace if the ingested count falls under GWAS_MIN_ROWS=100_000 / PHARMGKB_MIN_ROWS=5_000; on failure the existing cache is untouched and the user is told to allelix db update --force to retry.
• db update: tagless caches re-download instead of being stamped. A cache with no stored freshness signal almost always predates the signal mechanism, i.e. is old. The previous behavior was to stamp the live remote signal onto it and call it current, which permanently marked stale data as fresh — every subsequent run saw cached == remote and skipped, so only --force could escape. cached is None now triggers setup() like a remote-signal mismatch does, with a distinct console message so the user knows why.
• ClinVar .md5 body validated as 32 hex chars. When the ClinVar CDN returns a transient HTML error page, the previous parser treated the first whitespace-separated token (<!DOCTYPE) as the expected hash. That token then flowed into verify_file_hash, which on mismatch deletes the just-downloaded ~400 MB VCF and raises. A five-second 503 upstream wiped the freshly fetched cache. The signal parser now requires ^[0-9a-fA-F]{32}$ before treating the token as an md5.
• gnomAD is_ready(): tagless caches fail. The previous check was tag == f"sv:{GNOMAD_SCHEMA_VERSION}" or not tag, so any cache written before the schema-version mechanism existed was accepted as ready, defeating future invalidation. The escape is dropped — is_ready() now requires the exact current tag. Production caches built via install_prebuilt_cache already stamp the current tag.

Fixed — build detection

• normalize_build_label: anchored tokens, ambiguity rejection. The previous implementation was first-substring-wins over a bare "3X" in s chain. Bugs: 2038-01-01 registered as GRCh38, hg19/hg38 picked GRCh37 (the first substring hit), 38 (37 liftover) picked GRCh37. The normalizer is now driven by anchored regex tokens (\bgrch3[678]\b, \bhg(18|19|38)\b, \b(36|37|38)\b, etc.) and collects all distinct builds mentioned: exactly one → that build, zero or multiple → None. Direct callers (compare, export plink, header-vs-detected mismatch warning) now receive None instead of a silently-wrong build label.
• LivingDNA parser: only inspect build-marker comment lines. get_metadata previously fed every comment line into normalize_build_label and let the last match win, so a download-date comment could retag the file. It now skips comment lines that don't mention build / reference / genome, and breaks on the first valid build-marker line.
• Build detection: warn when inconclusive despite rsIDs. When position-detection inspected known-rsID positions but couldn't pick a build (votes tied across builds or no row matched any), the pipeline silently routed through header_build. A GRCh36 file with a GRCh37-mislabeled header used to silently get the GRCh37 ClinVar cache. Now fires a yellow warning naming the rsID-checks-attempted count, what the header claims, and how to force --build.

Fixed — reports

• diff: validate baseline entries, magnitude tolerance, no implicit zero. Two bugs in the report-diff path. (1) load_previous_report only validated the top-level shape; a baseline with "annotations": {} or entries missing source/rsid would flow into compute_diff and raise KeyError/TypeError rather than the documented ValueError. (2) "Changed" detection used exact float != (so JSON round-trip noise — 7.5 vs 7.499999999999999 — flagged as changed) and substituted 0.0 for a missing previous magnitude (so a baseline entry with no magnitude key looked like a change against any nonzero current value). Now uses math.isclose(abs_tol=1e-9) and carries previous_magnitude: float | None.
• GWAS _magnitude: <= at p-value boundaries. Strict < on the magnitude thresholds meant the canonical genome-wide-significance value p = 5e-8 landed in the suggestive bucket (4.0) rather than the significant bucket (6.0) — one full magnitude below a barely-significant 4.9e-8 hit. All boundary comparisons now use <=.
• GWAS rollup: consistent PheCode delimiter (partial). _gwas_base_trait was splitting on " (PheCode " (leading space) while _gwas_phecode_parent was searching for "(PheCode " (no leading space). Promoted the delimiter to a module-level _PHECODE_DELIM constant used by both. The deeper structural fix (carrying trait / p-value / PheCode as structured fields on Annotation) is deferred to v2.1.

Fixed — parsers and models

• Variant: alleles normalized to uppercase at construction (case half). Reference databases all ship uppercase alleles, and carrier matching is raw set membership against {variant.allele1, variant.allele2} — a lowercase user allele would silently miss every clinical annotator. Production parsers all emit uppercase today, but a user-supplied filter file or future format variant could leak lowercase through. Normalization now happens in Variant.__post_init__ so every emission point is covered at the single chokepoint. Strand-naive half of the same finding is deferred to v2.1 per ADR-0010.
• FTDNA can_parse rejects MyHeritage files. FTDNA and MyHeritage exports share the same RSID,CHROMOSOME,POSITION,RESULT header and a byte-identical data section. Both can_parse implementations were accepting MyHeritage files, with routing masked only by the registry's listing order. FTDNA now scans the first 50 lines for a MyHeritage substring and returns False if found; detection is mutually exclusive regardless of registry order.
• SNPedia parser: allowlist guard on raw_table interpolation. The raw_table name from detect_raw_table flows into three SQL queries via f-string interpolation (SQLite doesn't accept parameter binding for identifiers). Today the function only ever returns one of two literals or None — safe — but the signature is str | None with no constraint. Added a _VALID_RAW_TABLES = frozenset({"_raw_pages", "pages"}) guard at the top of _parse_raw_pages_inner; an unexpected name raises ValueError.
• split_csv_line docstring corrected. The previous docstring claimed it handles single, double, and double-double-quoted fields per real CSV quoting rules. The implementation is line.split(",") + strip('"'), which mishandles any quoted field containing a literal comma. Docstring now states the limitation and the intended consumer set (FTDNA / MyHeritage / Living DNA). No behavior change.

Verification

Full pytest: 1525 pass / 2 skip, 94.08% coverage. Ruff clean. Real-data battery on HG002:

• GIAB v4.2.1 GRCh38 benchmark VCF (NIST truth set)
• DeepVariant gVCF (GRCh38)
• GATK HaplotypeCaller VCF (GRCh37)

Strict-superset chain-of-trust at production filter: 0 benchmark annotations missing from the gVCF run. Wrong-allele safety invariants (strand-resolve complement, alt-less enrichment MAX-fallback, ClinVar CLNDN joining) verified end-to-end on the rebuilt cache.