kopia/kopia v0.8.2

on GitHub

Security Notice

This is a re-release of v0.8.1 with newly generated signing keys to address the impact of the codecov.io breach on Kopia.

Release Notes

This version brings many performance, usability and stability improvements and several new and exciting features.

There were over 150 changes by 19 contributors which represents a big growth in community engagement.

New Features:

• Full Maintenance is now enabled by default with an interval of 24 hours.
• Added support for configurable Maintenance Safety to speed up cleanups in certain cases.
• Added support for --json flag to certain CLI commands which will produce machine-readable snasphot, policy, manifest, blob and content responses.
• Added support for scanning only one filesystem via files policy
• Major improvements to ignore logic to better follow .gitignore rules.
• Added support for Actions which allow running custom scripts before and after a directory is snapshotted with lots of useful applications.
• Added support for ACLs which allow defining custom access control rules for repository server clients.
• Docker Images are now available
• Support for write sessions, which ensure garbage collection does not prematurely delete blobs that are in the middle of being uploaded by intermittently-connected clients (e.g. laptops that go to sleep often and can't checkpoint frequently enough)
• Added support for stdin snapshots.
• Added support for user authentication using user profiles stored in the repository (#809)
• Brand new streaming GRPC protocol for talking to kopia repository server to improve performance and ensure session stickyness, which allows sharding and load-balancing.
• Reworked error handling when taking snapshots to not fail fast (except when explicitly requested).
• Switched to new FUSE implementation which supports MacFUSE 4.x
• Added WebDAV-based mounting alternative for platforms that don't have working FUSE mechanism.
• Streamlined flags for caching and improved cache sweep.
• Major performance improvements when performing lots of index lookups in heavily-fragmented indexes (speeds up snapshot verification by orders of magnitude)
• Passthrough options to allow other-users access and mounting in empty directory.
• Don't run maintenance as part of read-only actions.
• Implemented caching for contents downloaded from Kopia Repository Server
• Reduced latency for kopia repository server uploads by performing existence checks
• Improvements to snapshot verify to make output easier to understand.
• compress: upgrade klauspost/compress to v1.11.12 (#888)
• Terminal password prompt - switched to golang.org/x/term (#898)

New KopiaUI features:

• Added new tab called Tasks which shows the status, progress, logs, and statistics of current and past long-running tasks (restore, estimate, snapshot, maintenance).
• Streamlined snapshot creation flow to include estimation and initial policy definition.

Provider improvements:

• S3 client - upgraded minio to v7
• rclone - testing improvements
• S3, GCS, Azure, B2 - rewrote retry logic to retry on all but known fatal errors.
• S3 - support for buckets configured with object locking

Infrastructure:

• Build for Apple Silicon (M1) is now available. The CLI is now dual-architecture
  and runs natively on both amd64 and arm64.
• Switched CI/CD infrastructure to GitHub actions with two self-hosted runners for ARMHF and ARM64
• Makefile: various tricks to make PR builds faster
• S3: run tests against private minio deployment in docker (#894)
• SFTP: fixed retry logic
• RClone: improved error handling logic

Compatibility Information

• v0.8 release is compatible with repositories created with v0.7, v0.6 and v0.5.
• While older clients can read repositories created with v0.8 it's recommended to use the latest version at earliest convenience.

Breaking Changes

• When starting kopia server with --htpasswd-file, the UI access is now only granted to a user named kopia (if it exists in the password file). If such user does not exist in the password file, --server-username flag may be specified to indicate which of the users listed in the password file gets UI access (there can be only one). See #880 for more information.

• Added explicit --insecure flag to kopia server start to prevent accidentally starting non-secured
  servers for development. Starting the server without TLS or without authentication now requires this flag.

• Several low-level maintenance tools have been changed to support individual parameters with --safety=full|none flag instead of specific parameters which were unsafe.