github iipc/jwarc v0.31.0

latest releases: v0.37.0, v0.36.0, v0.35.0...
22 months ago

New features

  • Added optional support for brotli content encoding #88
  • Added HttpMessage.bodyDecoded() #88
  • WarcTool: Added dedupe subcommand
  • DedupeTool: Added --verbose option and silenced default logging

Bug fixes

  • GunzipChannel: Fixed incorrect record length calculation when gzip footer aligns with the end of the buffer
  • ValidateTool: Fixed digest validation #87
  • DedupeTool: Used matchType=exact to properly handle CDX queries for URLs ending with *
  • DedupeTool: Fixed record copying when transferTo copies fewer bytes than requested
  • DedupeTool: Prevented appending of an empty gzip member when no records were deduplicated
  • DedupeTool: Fixed exception when input files are in the current working directory

Don't miss a new jwarc release

NewReleases is sending notifications on new releases.