agude/wayback-machine-archiver v3.5.0

v3.5.0: Code Quality & URL Validation

on GitHub

Release Notes:

This release focuses on code quality improvements, better error handling, and enhanced diagnostics. It consolidates duplicate code, standardizes patterns across the codebase, and adds URL validation to catch malformed input early.

There are no breaking changes in this release.

What's New

• URL Validation: URLs are now validated before archiving. Invalid URLs (missing scheme, non-HTTP protocols, malformed structure) are filtered out with a warning rather than being sent to the API where they would fail.

• Improved Exception Handling: Exception handling has been narrowed to catch specific error types (RequestException, ValueError, OSError) rather than broad Exception catches. This prevents masking unexpected errors while still handling known failure modes gracefully—including malformed API responses that return HTML instead of JSON.

• Debug Logging for API Troubleshooting: Added debug-level logging for status API responses and skipped job entries. Run with --log DEBUG to see raw API responses when diagnosing issues.

• Code Consolidation: Removed duplicate code and standardized patterns:

  • Consolidated LOCAL_PREFIX constant into sitemaps.py with proper export
  • Extracted _create_session_with_retries() helper for HTTP session setup
  • Removed unused check_status() method from SPN2Client
  • Standardized all string formatting to f-strings

• Test Improvements: Expanded test coverage from 39 to 64 tests. Standardized test fixtures to use monkeypatch for proper isolation and added tests for URL validation, sitemap processing, and workflow error handling.