We are very excited to announce the 0.6 release of Kopia! This is a big milestone on the way to an upcoming 1.0 release and there have not only been a large number of changes across the stack but also growth of the community.

Upgrade notes:

• Upgrading from 0.5.x is supported, make sure to follow instructions in #503 to enable maintenance tasks.
• Upgrading from 0.4.0 or earlier is not officially supported, it may work but use at your own risk. It's strongly recommended to create new repository using v0.6.0 and migrate existing snapshots as outlined in the documentation.

Changes in this release:

core

* big performance improvements when snapshotting large directories (#331)
* improvements for dealing with eventually-consistent stores (S3) (#437)
* GC safety improvements to resolve race condition when content is re-referenced when about to be deleted (#420)
* added OpenCensus (#339)
* introduced explicit maintenance operations that perform periodic repository cleanup/compaction (#411)
* disabled automatic compaction on repository opening - moved to maintenance tasks
* added AsyncWrites to ObjectWriter, which improves performance… (#369)
* object: ensure that all I objects have a content prefix which improves locality by putting them in q packs
* deduplicate multiple policies for the same source in policy manager, fixes #391
* fixed race condition during Open() where we may read incomplete file
* deprecated NONE algorithm, will not be available for new repositories (#395)
* server: automatically flush the repository after setting or deleting a policy (#489)
* snapshot checkpointing (#410)
* moved creating cache directory from connect to first use (#450)
* persist relative path to cache if possible, this allows config directory to be partially portable
* server: implemented 'flush' and 'refresh' API
* experimental support for remote repository (#427)
* repo: refactored public API (#318)

KopiaUI

 * support for multiple repositories + portability (#398)
 * highlight snapshot errors (#376)

Providers

* support for gather writes (#373)
* b2: added provider for backblaze b2
* sftp: add missing options for configuring sftp known_hosts
* s3: add CLI option for disabling tls verification while connecting to s3
* filesystem: added retry which addresses the macOS race condition

CLI Improvements:

* added flags to control progress output
* 'kopia server' made --ui default (#452)
* allow override of snapshot start time and end time
* improved 'snapshot delete' usage (#436)
* Remove legacy flags from snapshot create command (#441)
* support for zip, tar and tar.gz restore outputs (#482)

Infrastructure:

* macOS and Windows KopiaUI builds are now signed
* robustness testing framework
* testing: added blob.Storage wrapper that simulates eventual consistency (#434)
* switched back to using v-prefixed tag names.
* tests: added smoke test that exercises all combinations of encryption and hashing

Changelog

cfa30f2 switched back to using v-prefixed tag names.
ff244b5 go.mod: upgrade github.com/klauspost/compress to v1.10.10
8ead49b restore: support for zip, tar and tar.gz restore outputs (#482)
52e7631 kopia-ui: fixed missing 'log' when clicking 'Launch At Startup'
7e9ce61 server: automatically flush the repository after setting or deleting a policy (#489)
a544e0c kopia-ui: Added logging when waiting for notarization response from Apple.
2c21582 kopia-ui: added option to disable TLS and force HTTP
73ed5ad kopia-ui: cleaned up console logging
8c941cc Fix BUILD.md (#481)
0b2d12d kopia-ui: build deb and rpm packages
ce5e6dc [Robustness] Add first robustness tests
82a2fa0 [Robustness] Add test engine to manage snapshot verification testing (#468)
d44c01f content: do not short circuit compaction if there's one index but there are contents to delete
64a6cb4 parallelwork: fixed error handling, which caused parallel work to never finish on any error
1c25341 cli: fixed panic() during snapshot verification caused by refactoring to new Repository interface
3843ed6 cli: added content verify --include-deleted flag
ed9511b b2: fix GetMetadata() implementation
25934a5 content: fixed deletion of cleanup blobs (#462)
7975767 server: implemented 'flush' and 'refresh' API
293cb10 cosmetic changes to maintenance behavior (#461)
c13b5f8 Remove extra whitespaces and fix minor typos (#460)
85aa4ae Add 'repositories' page (#454)
deb1099 Trivial: remove unused logging.Option type
6bc47cb content: emit warning whenever the number of index blobs gets too high, which is an indication that maintenance has not run in a long time
960c334 maintenance: disabled automatic compaction on repository opening
6cb970f Revert "Fix #360: Leverage Go 1.14 testing.T.Cleanup() (#455)" (#458)
73b913c Fix #360: Leverage Go 1.14 testing.T.Cleanup() (#455)
b30da51 Wire up snapshot-store-compare behavior
d68273a Improvements for dealing with eventually-consistent stores (S3) (#437)
9123e3e cli: 'kopia server' made --ui default (#452)
130712a Remove unused content.Manager.closed channel (#451)
e445dc5 repo: moved creating cache directory from connect to first use (#450)
fff2b44 Upgrade AWS SDK to 1.31.3 from 1.31.1
0642b26 Upgrade github.com/klauspost/compress to 1.10.6
0afd1a5 Upgrade github.com/minio/minio
97e04d5 Upgrade golang.org/x dependencies
0875939 dep: upgrade protobuf dependents (#442)
23c1212 Build test/tools in Darwin as well (#447)
0c2ad99 Upgrade github.com/minio/minio-go/v6
53b82c9 dep: upgrade various dependencies (#445)
0a6a0d4 Upgrade AWS SDK to 1.31.1
ea681e4 deps: Upgrade compress package to 1.10.5 (#443)
e493177 Remove legacy flags from snapshot create command (#441)
8c4fb53 blob: support for GetMetadata() to get server-side timestamp and blob length (#440)
ca28469 cli: improved 'snapshot delete' usage (#436)
7e5fc52 removal of unnecessary extra )}
f3359d7 add sftp and webdav as repositories to "Getting started" documentation page, "Setting Up Repository" chapter
993c3bf linter: fixed how linter is installed on Darwin and Linux since it was getting wrong version
d657415 testing: added blob.Storage wrapper that simulates eventual consistency (#434)
200ef57 app: portability fixes (#431)
be4b897 Support for remote repository (#427)
1377d05 Maintenance changes (#423)
4f6dd94 Maintenance: report and record record maintenance run timings (#422)
a462798 content: added blob-level metadata cache (#421)
d707f36 GC content repair (#420)
2b8e882 filesystem: added retry which addresses the macOS race condition
4b4628a Repository maintenance support (#411)
573d104 object: ensure that all I objects have a content prefix
b671bda cli: add missing options for configuring sftp known_hosts
14fb49a content: asynchronous cache sweep
e50b5ec Trivial: Add package alias to silence IDE/gopls warning
1f1682b Snapshot checkpointing (#410)
70d4c87 cli: improvements to content selection for list/rewrite/stats/verify (#409)
057c278 Kopia UI: support for multiple repositories + portability (#398)
45195fa appveyor: moved WINDOWS_CERT_SHA1 to secure variable so that we don't sign PRs
c89b1a7 appveyor: added GH_TOKEN variable, disabled RDP
01f52f8 appveyor: fixed secure variable
24f6720 enable RDP on appveyor worker until it's stable
0017f59 Makefile: fix travis-setup on Windows
2055fe3 Makefile: support Windows Appveyor environment (#407)
16275a1 Makefile: changed how build number is calculated
37b69d8 site: tweaked layout and options for cookie consent popup
e098464 improved logging for sign.js
6ad9273 travis: setup tools to support signing Windows binaries
05ea35f testing: retry around B2 test
567bd33 added b2 to repository creation ui
4bed451 implemented b2 backend
3924fb2 Minor cleanup for snapshot time override (#392)
6cb9b8f repo: refactored public API (#318)
10bb492 repo: deprecated NONE algorithm, will not be available for new repositories (#395)
6097781 Support for gather writes (#373)
b08d394 policy: deduplicate multiple policies for the same source in policy manager, fixes #391
393d273 Policy set: fix nil pointer dereference #385 (#387)
ea510f3 repo: fixed race condition during Open() where we may read incomplete file
f2e6d51 repo: persist relative path to cache if possible, this allows config directory to be partially portable
9b68a63 Highlight snapshot errors in the UI and CLI (#376)
8687f1c object: added AsyncWrites to ObjectWriter, which improves performance… (#369)
2e7fe7c travis: switch osx to xcode11.3
e86e897 fix makefile for windows (#377)
6789f8e cli: allow override of snapshot start time and end time
a1f7a06 Add CLI option for disabling tls verification while connecting to s3 (#370)
1df5c81 correct a typo in Doc
73e21e0 correct a typo in kopia UI
239d809 performance: introduced buf.Pool which helps reuse memory buffers (#345)
becf08a testing: added missing condition variable broadcast after finishing flushing, fixes timeouts on Travis
4a444e6 content: reduce allocations in index scan
169f25c s3: pass blob length to minio in PutBlob()
b5d4421 upload: added missing writer.Close()
b102094 splitter: exposed MaxSegmentSize() for splitters
4564ed5 encryption: added MaxOverhead to encryptor
c9877bf performance: refactored content manager to avoid copying data
53a5926 content: wait for cache sweep goroutine to close (#361)
d930f06 cli: added flags to control progress output
06fbaba testing: ensure cleanup of content managers, fixes #356
272b657 upgrade all dependencies to latest versions
a8ef0b2 kopia-ui: fixed auto-update
4088eb5 site: updated CDN links for cookie consent
c150767 site: added cookie consent form
674543e site: added simple privacy policy
331ff07 cli: added number of files that are in the process of being hashed
78a921a Rename Simple mutex
60d1cc8 [robustness testing] Implement metadata store for test metadata using kopia
6b04b42 tests: added smoke test that exercises all combinations of encryption and hashing
e2a169d hotfix: kopia-0.5.0 did not properly support 256-bit hashes
8d452a8 performance: improvements to object manager (#336)
526445a CLI: pre-allocated buffers for crypto benchmark (#343)
e80f553 performance: plumbed through output buffer to encryption and hashing,… (#333)
0d1a627 Improved upload performance and memory usage (#331)
89c0c6b Refactor CLI stats (#341)
71e5563 Fix download link (#342)
6ce8410 Added OpenCensus (#339)
edc87fc Refactor content stats (#340)
9c3d419 Fixed social site logo - alternative #2 (#335)
514df69 performance: added wrapper around io.Copy()
ec56ed3 Makefile: fixed layering test
f07d164 Makefile: added layering-test, fixes #324
0585232 Add snapshotter interface and kopia implementation
d9ce3d0 Inject time in Kopia components (#314)
5f96b02 testing: added retry helper
bd3cace site: changed Community page to make it more obvious how to get to Slack and GitHub
88ce341 Trivial cleanup for internal diff (#316)
3ebef0a snapshot: removed content stats from snapshot.Stats, which are confusing and incorrect if snapshots are done in parallel
d862bc2 site: delete news section since it's empty
7638d4a site: clarified docs based on Reddit questions, removed stale link to user mailing list which is not in use