allelix 2.2.0

v2.2.0

on Python PyPI

Minor release. Focus: the ClinVar source switch from VCF to per-SCV TSV — a coordinated three-stage redesign that structurally retires the CLNSIG|CLNDN Frankenstein-pair hazard, plus panel coverage states for custom panels, a new FTDNA file shape, two CLI subcommands, and two runtime-degradation nudges. One CLINVAR_INTERPRETER_VERSION bump (2 → 4) auto-invalidates existing ClinVar caches; everything else signal-skips.

Upgrade

pip install --upgrade allelix
allelix db update

The ClinVar cache rebuilds (CLINVAR_INTERPRETER_VERSION advanced 2 → 4). gnomAD / AlphaMissense / CADD / PharmGKB / GWAS / SNPedia signal-skip — no schema or interpreter changes touched their loaders. Roughly the same ~16 GB on-disk footprint after rebuild.

Highlights

• ClinVar source switched to per-SCV TSV pair. The loader now ingests variant_summary.txt.gz + submission_summary.txt.gz from NCBI's tab-delimited dump, emitting one cache row per (variant, SCV submission) instead of one combined row per variant. Structurally retires the CLNSIG|CLNDN Frankenstein-pair hazard: each row carries one submitter's exact (significance, condition) pair, so there is no cross-product pairing risk. Record count roughly doubles (~5.79M → ~9.09M GRCh38), expected from per-SCV granularity. The legacy VCF loader and its URL constants are removed.
• Panel coverage states for custom panels. When --filter-file supplies a rsID panel, the analyze command now distinguishes three states per requested rsID instead of silently dropping two of them. State 1 (genotyped + has annotations) renders normally in the main table; state 2 (genotyped + no annotations) surfaces in a "Panel coverage" section as "your chip has it, no findings"; state 3 (not in input) surfaces as "not on your chip." Critical distinction for wellness / pharma panels: "you don't carry the risk allele" and "this SNP wasn't on your chip" are different answers, and both were invisible before. Terminal output emits a one-line warning when state 3 is non-empty; JSON payload carries a panel_coverage object; HTML report shows a "Panel coverage" section.
• FTDNA FamFinder parser. Third FTDNA file shape supported — tab-delimited with separate ALLELE1 / ALLELE2 columns instead of a concatenated RESULT field (the FTDNA and FTDNA-Illumina-raw siblings).
• allelix db clean and allelix db path CLI subcommands. db clean removes downloaded reference database caches with size report, dry-run preview, scripting flag, and a safety guard that refuses when the target doesn't look like an allelix cache (typo'd --data-dir protection); --keep-cadd preserves the expensive opt-in cache. db path prints the resolved data directory for shell capture ($(allelix db path)); --check verifies writability. Replaces the manual rm -rf recipe in the README.
• Runtime degradation nudges. Two once-per-run info lines surface previously-silent post-pipeline conditions. When array input runs without a usable gnomAD ref-resolution path (--no-gnomad or no gnomAD cache), every variant ends up ref=None and the strand-aware path silently falls back to v2.0.x direct-match-only; analyze now emits a dim "strand-aware matching inactive" notice. When effective build is GRCh38 and >50% of input variants are rsID-less (the variant-caller ID=. case), analyze emits a yellow undercount warning recommending GRCh37 if available.
• Methylation panel + magnitude floor retune. Curated methylation gene panel with annotated citations; focused-subcommand magnitude default lowered 5.0 → 3.0 so ClinPGx LoE 3 hits (mag 4.0) on COMT / MTHFR / MTR / CBS actually surface to the methylation report. allelix analyze default unchanged at 5.0.
• JSON schema_version 5 → 6. Additive only — every v5 field is unchanged in v6; v5 consumers reading v6 output continue to succeed. The new field is the top-level panel_coverage object, which only appears when --filter-file is used. Diff loader accepts both v5 and v6 baselines.

Known limitations

Carried forward / introduced.

• GRCh38 rsID-less undercount. Variant-caller VCFs with ID=. get materially fewer annotations on GRCh38 than the same biology on GRCh37 because rsID resolution falls back to ClinVar position lookup — a smaller surface than dbSNP. The real fix is full dbSNP resolution, tracked for v2.5+; this release surfaces the condition as a runtime warning so users aren't silently undercounted.
• PharmGKB → ClinPGx rebrand. Internal annotator name, cache filename, and source string remain pharmgkb for backward compatibility. The full rename will bundle with the next schema-breaking change.

Added

• ClinVar per-SCV TSV loader (replaces the VCF loader); CLINVAR_VARIANT_SUMMARY_URL + CLINVAR_SUBMISSION_SUMMARY_URL are the new source URLs.
• AnalysisResult.panel_coverage() returns a typed dict {requested, found, missing, no_findings} or None when no panel was supplied.
• AnalysisResult fields: panel_rsids, genotyped_panel_rsids, panel_annotated_rsids, no_ref_variant_count, rsidless_variant_count. All populated only when relevant; default analyze runs see zero behavior change.
• _CLINVAR_PLACEHOLDER_CLNSIGS module constant in databases/manager.py. Named source-of-truth for the ClinVar placeholder skip set; the loader, the protocol §7b ship-gate, and the test suite all reference it.
• FTDNA FamFinder parser registered before FTDNAIlluminaParser in the parser registry so the more-specific 5-column header check fires first.
• allelix db clean and allelix db path subcommands.

Changed

• ClinVar source: VCF → per-SCV TSV pair. CLINVAR_INTERPRETER_VERSION advanced 2 → 4 so existing caches auto-reingest on the next db update. The legacy VCF code path (load_clinvar_vcf, iter_clinvar_records, parse_clinvar_version, the VCF URL constants, _vcf_filename_for_url, and the synthetic VCF fixture pair) is removed.
• JSON schema_version 5 → 6. Single bump for the additive panel_coverage object.
• Focused subcommand --min-magnitude default 5.0 → 3.0. allelix methylation and allelix pharmacogenomics only. allelix analyze default unchanged.
• Permission resolution consolidated. AllelixConfig.permission_for() is the single resolver returning Permission | None; both is_enabled() and the config show table-cell renderer delegate to it instead of inlining the three-state check.
• ClinVar reference-lookup streams the cursor. _load_ref_lookup switches from fetchall() to direct cursor iteration. Peak memory drops from "full result list + output dict" to "one row + output dict". iter_gwas_records intentionally retained as buffering (dedup-on-min-p-value requires full input visibility); docstring now acknowledges this instead of claiming streaming.

Fixed

• TSV loader filters ClinVar placeholder CLNSIGs. Values in _CLINVAR_PLACEHOLDER_CLNSIGS ("", "-", "not specified", "no classification provided") fell through to the _magnitude() 5.0 default — equal to the analyze display floor — and surfaced as bogus annotations on real rsIDs. Loader now drops them at ingest; case-insensitive membership testing mirrors _normalize_clnsig(). Values that legitimately map below the floor (not provided → 2.0, no_classification_for_the_single_variant → 2.0) are intentionally kept.
• TSV loader stamps a real version string. db status after the per-SCV rebuild previously showed "ClinVar version: None" because stamp_remote_signal inserts version=NULL by design and the TSV loader had no pre-existing row. Now stamps the build date YYYY-MM-DD UTC.
• Terminal output emits the regulatory disclaimer. HTML and JSON have carried the ADR-0003 "not medical advice / not a diagnosis" notice since v2.0.x; the terminal — the default analyze output — did not. Fires after the table on every analyze run.
• FULL_TEST_PROTOCOL.md §7 per-SCV condition contract refreshed. The flagship sanity block was still pinning the pre-TSV-loader VCF-era semicolon-joined condition shape. Rewritten to pin the per-SCV row shape (multi-row, never combined) and explain why the join-pin is structurally retired.
• FULL_TEST_PROTOCOL.md §15 stale exome-VCF expectation. unsupported_23andme_exome_vcf.txt is a valid VCFv4.1 file and is parsed by the VCF parser added after the fixture was named. Doc split into "still unsupported" (decodeme) and "previously-unsupported / now-supported regression pin."
• FULL_TEST_PROTOCOL.md §7b ClinVar significance-sentinel ship-gate. Post-db update SQL scan asserting zero rows match _CLINVAR_PLACEHOLDER_CLNSIGS in either build cache, with LOWER() for case-insensitive symmetry with the loader. Named ship-gate ("0 rows expected; any row → do not tag") and a forward-pointer to the drift guard as the durable allowlist backstop.

Verification

Full pytest fast tier: 1,676 pass / 0 skip / 0 warnings, 92.92% local coverage / ~92.77% CI matrix (both above the 92.5 floor). Ruff clean, ruff format clean, mypy clean, COLUMNS=80 narrow-width sweep green. plink2 v2.0.0-a.6.5 installed.

On a fresh clone + clean-room venv against a real db update:

• Cross-parser identity on user1190 held: 470/470 ×6 across the six DTC representations (down from v2.1.x's 483 because the ~13 bogus '-'-CLNSIG annotations stopped surfacing — correct behavior, not a regression).
• Real-data drift guard green against the rebuilt cache.
• Slow + integration suites green.
• Privacy invariant verified: no network outside the db-loader paths, no telemetry, --no-update lets the user run fully offline.

No data leaves your machine remains the offline-first promise.