We're very pleased to present you restic 0.12.0! restic is distributed as a standalone binary: download the correct file for your operating system and architecture, extract the file and just run it. If you run into any issues, please report them at the GitHub issue tracker or visit the forum. If you already have restic >= 0.9.4, you can use restic self-update to get the latest version in a secure way.

The binaries released with each restic version are reproducible, which means that you can reproduce a byte identical version from the source code for that release. Instructions on how to do that in the Developer Documentation.

Changelog for restic 0.12.0 (2021-02-14)

The following sections list the changes in restic 0.12.0 relevant to restic users. The changes are ordered by importance.

Summary

• Fix #1681: Make mount not create missing mount point directory
• Fix #1800: Ignore no data available filesystem error during backup
• Fix #2563: Report the correct owner of directories in FUSE mounts
• Fix #2688: Make backup and tag commands separate tags by comma
• Fix #2739: Make the cat command respect the --no-lock option
• Fix #3087: The --use-fs-snapshot option now works on windows/386
• Fix #3100: Do not require gs bucket permissions when running init
• Fix #3111: Correctly detect output redirection for backup command on Windows
• Fix #3151: Don't create invalid snapshots when backup is interrupted
• Fix #3166: Improve error handling in the restore command
• Fix #3232: Correct statistics for overlapping targets
• Fix #3014: Fix sporadic stream reset between rclone and restic
• Fix #3152: Do not hang until foregrounded when completed in background
• Fix #3249: Improve error handling in gs backend
• Chg #3095: Deleting files on Google Drive now moves them to the trash
• Enh #2186: Allow specifying percentage in check --read-data-subset
• Enh #2453: Report permanent/fatal backend errors earlier
• Enh #2528: Add Alibaba/Aliyun OSS support in the s3 backend
• Enh #2706: Configurable progress reports for non-interactive terminals
• Enh #2944: Add backup options --files-from-{verbatim,raw}
• Enh #3083: Allow usage of deprecated S3 ListObjects API
• Enh #3147: Support additional environment variables for Swift authentication
• Enh #3191: Add release binaries for MIPS architectures
• Enh #909: Back up mountpoints as empty directories
• Enh #3250: Add several more error checks
• Enh #2718: Improve prune performance and make it more customizable
• Enh #2495: Add option to let backup trust mtime without checking ctime
• Enh #2941: Speed up the repacking step of the prune command
• Enh #3006: Speed up the rebuild-index command
• Enh #3048: Add more checks for index and pack files in the check command
• Enh #2433: Make the dump command support zip format
• Enh #3099: Reduce memory usage of check command
• Enh #3106: Parallelize scan of snapshot content in copy and prune
• Enh #3130: Parallelize reading of locks and snapshots
• Enh #3254: Enable HTTP/2 for backend connections

Details

• Bugfix #1681: Make mount not create missing mount point directory

  When specifying a non-existent directory as mount point for the mount command, restic used to create the specified directory automatically.

  This has now changed such that restic instead gives an error when the specified directory for the mount point does not exist.

  #1681 #3008

• Bugfix #1800: Ignore no data available filesystem error during backup

  Restic was unable to backup files on some filesystems, for example certain configurations of CIFS on Linux which return a no data available error when reading extended attributes. These errors are now ignored.

  #1800 #3034

• Bugfix #2563: Report the correct owner of directories in FUSE mounts

  Restic 0.10.0 changed the FUSE mount to always report the current user as the owner of directories within the FUSE mount, which is incorrect.

  This is now changed back to reporting the correct owner of a directory.

  #2563 #3141

• Bugfix #2688: Make backup and tag commands separate tags by comma

  Running restic backup --tag foo,bar previously created snapshots with one single tag containing a comma (foo,bar) instead of two tags (foo, bar).

  Similarly, the tag command's --set, --add and --remove options would treat foo,bar as one tag instead of two tags. This was inconsistent with other commands and often unexpected when one intended foo,bar to mean two tags.

  To be consistent in all commands, restic now interprets foo,bar to mean two separate tags (foo and bar) instead of one tag (foo,bar) everywhere, including in the backup and tag commands.

  NOTE: This change might result in unexpected behavior in cases where you use the forget command and filter on tags like foo,bar. Snapshots previously backed up with --tag foo,bar will still not match that filter, but snapshots saved from now on will match that filter.

  To replace foo,bar tags with foo and bar tags in old snapshots, you can first generate a list of the relevant snapshots using a command like:

  Restic snapshots --json --quiet | jq '.[] | select(contains({tags: ["foo,bar"]})) | .id'

  And then use restic tag --set foo --set bar snapshotID [...] to set the new tags. Please adjust the commands to include real tag names and any additional tags, as well as the list of snapshots to process.

  #2688 #2690 #3197

• Bugfix #2739: Make the cat command respect the --no-lock option

  The cat command would not respect the --no-lock flag. This is now fixed.

  #2739

• Bugfix #3087: The --use-fs-snapshot option now works on windows/386

  Restic failed to create VSS snapshots on windows/386 with the following error:

  GetSnapshotProperties() failed: E_INVALIDARG (0x80070057)

  This is now fixed.

  #3087 #3090

• Bugfix #3100: Do not require gs bucket permissions when running init

  Restic used to require bucket level permissions for the gs backend in order to initialize a restic repository.

  It now allows a gs service account to initialize a repository if the bucket does exist and the service account has permissions to write/read to that bucket.

  #3100

• Bugfix #3111: Correctly detect output redirection for backup command on Windows

  On Windows, since restic 0.10.0 the backup command did not properly detect when the output was redirected to a file. This caused restic to output terminal control characters. This has been fixed by correcting the terminal detection.

  #3111 #3150

• Bugfix #3151: Don't create invalid snapshots when backup is interrupted

  When canceling a backup run at a certain moment it was possible that restic created a snapshot with an invalid "null" tree. This caused check and other operations to fail. The backup command now properly handles interruptions and never saves a snapshot when interrupted.

  #3151 #3164

• Bugfix #3166: Improve error handling in the restore command

  The restore command used to not print errors while downloading file contents from the repository. It also incorrectly exited with a zero error code even when there were errors during the restore process. This has all been fixed and restore now returns with a non-zero exit code when there's an error.

  #3166 #3207

• Bugfix #3232: Correct statistics for overlapping targets

  A user reported that restic's statistics and progress information during backup was not correctly calculated when the backup targets (files/dirs to save) overlap. For example, consider a directory foo which contains (among others) a file foo/bar. When restic backup foo foo/bar was run, restic counted the size of the file foo/bar twice, so the completeness percentage as well as the number of files was wrong. This is now corrected.

  #3232 #3243

• Bugfix #3014: Fix sporadic stream reset between rclone and restic

  Sometimes when using restic with the rclone backend, an error message similar to the following would be printed:

  Didn't finish writing GET request (wrote 0/xxx): http2: stream closed

  It was found that this was caused by restic closing the connection to rclone to soon when downloading data. A workaround has been added which waits for the end of the download before closing the connection.

  #2598 #3014

• Bugfix #3152: Do not hang until foregrounded when completed in background

  On Linux, when running in the background restic failed to stop the terminal output of the backup command after it had completed. This caused restic to hang until moved to the foreground. This has now been fixed.

  #3152 https://forum.restic.net/t/restic-alpine-container-cron-hangs-epoll-pwait/3334

• Bugfix #3249: Improve error handling in gs backend

  The gs backend did not notice when the last step of completing a file upload failed. Under rare circumstances, this could cause missing files in the backup repository. This has now been fixed.

  #3249

• Change #3095: Deleting files on Google Drive now moves them to the trash

  When deleting files on Google Drive via the rclone backend, restic used to bypass the trash folder required that one used the -o rclone.args option to enable usage of the trash folder. This ensured that deleted files in Google Drive were not kept indefinitely in the trash folder. However, since Google Drive's trash retention policy changed to deleting trashed files after 30 days, this is no longer needed.

  Restic now leaves it up to rclone and its configuration to use or not use the trash folder when deleting files. The default is to use the trash folder, as of rclone 1.53.2. To re-enable the restic 0.11 behavior, set the RCLONE_DRIVE_USE_TRASH environment variable or change the rclone configuration. See the rclone documentation for more details.

  #3095 #3102

• Enhancement #2186: Allow specifying percentage in check --read-data-subset

  We've enhanced the check command's --read-data-subset option to also accept a percentage (e.g. 2.5% or 10%). This will check the given percentage of pack files (which are randomly selected on each run).

  #2186 #3038

• Enhancement #2453: Report permanent/fatal backend errors earlier

  When encountering errors in reading from or writing to storage backends, restic retries the failing operation up to nine times (for a total of ten attempts). It used to retry all backend operations, but now detects some permanent error conditions so that it can report fatal errors earlier.

  Permanent failures include local disks being full, SSH connections dropping and permission errors.

  #2453 #3180 #3170 #3181

• Enhancement #2528: Add Alibaba/Aliyun OSS support in the s3 backend

  A new extended option s3.bucket-lookup has been added to support Alibaba/Aliyun OSS in the s3 backend. The option can be set to one of the following values:

  • auto - Existing behaviour - dns - Use DNS style bucket access - path - Use path style bucket access

  To make the s3 backend work with Alibaba/Aliyun OSS you must set s3.bucket-lookup to dns and set the s3.region parameter. For example:

  Restic -o s3.bucket-lookup=dns -o s3.region=oss-eu-west-1 -r s3:https://oss-eu-west-1.aliyuncs.com/bucketname init

  Note that s3.region must be set, otherwise the MinIO SDK tries to look it up and it seems that Alibaba doesn't support that properly.

  #2528 #2535

• Enhancement #2706: Configurable progress reports for non-interactive terminals

  The backup, check and prune commands never printed any progress reports on non-interactive terminals. This behavior is now configurable using the RESTIC_PROGRESS_FPS environment variable. Use for example a value of 1 for an update every second, or 0.01666 for an update every minute.

  The backup command now also prints the current progress when restic receives a SIGUSR1 signal.

  Setting the RESTIC_PROGRESS_FPS environment variable or sending a SIGUSR1 signal prints a status report even when --quiet was specified.

  #2706 #3194 #3199

• Enhancement #2944: Add backup options --files-from-{verbatim,raw}

  The new backup options --files-from-verbatim and --files-from-raw read a list of files to back up from a file. Unlike the existing --files-from option, these options do not interpret the listed filenames as glob patterns; instead, whitespace in filenames is preserved as-is and no pattern expansion is done. Please see the documentation for specifics.

  These new options are highly recommended over --files-from, when using a script to generate the list of files to back up.

  #2944 #3013

• Enhancement #3083: Allow usage of deprecated S3 ListObjects API

  Some S3 API implementations, e.g. Ceph before version 14.2.5, have a broken ListObjectsV2 implementation which causes problems for restic when using their API endpoints. When a broken server implementation is used, restic prints errors similar to the following:

  List() returned error: Truncated response should have continuation token set

  As a temporary workaround, restic now allows using the older ListObjects endpoint by setting the s3.list-objects-v1 extended option, for instance:

  Restic -o s3.list-objects-v1=true snapshots

  Please note that this option may be removed in future versions of restic.

  #3083 #3085

• Enhancement #3147: Support additional environment variables for Swift authentication

  The swift backend now supports the following additional environment variables for passing authentication details to restic: OS_USER_ID, OS_USER_DOMAIN_ID, OS_PROJECT_DOMAIN_ID and OS_TRUST_ID

  Depending on the openrc configuration file these might be required when the user and project domains differ from one another.

  #3147 #3158

• Enhancement #3191: Add release binaries for MIPS architectures

  We've added a few new architectures for Linux to the release binaries: mips, mipsle, mips64, and mip64le. MIPS is mostly used for low-end embedded systems.

  #3191 #3208

• Enhancement #909: Back up mountpoints as empty directories

  When the --one-file-system option is specified to restic backup, it ignores all file systems mounted below one of the target directories. This means that when a snapshot is restored, users needed to manually recreate the mountpoint directories.

  Restic now backs up mountpoints as empty directories and therefore implements the same approach as tar.

  #909 #3119

• Enhancement #3250: Add several more error checks

  We've added a lot more error checks in places where errors were previously ignored (as hinted by the static analysis program errcheck via golangci-lint).

  #3250

• Enhancement #2718: Improve prune performance and make it more customizable

  The prune command is now much faster. This is especially the case for remote repositories or repositories with not much data to remove. Also the memory usage of the prune command is now reduced.

  Restic used to rebuild the index from scratch after pruning. This could lead to missing packs in the index in some cases for eventually consistent backends such as e.g. AWS S3. This behavior is now changed and the index rebuilding uses the information already known by prune.

  By default, the prune command no longer removes all unused data. This behavior can be fine-tuned by new options, like the acceptable amount of unused space or the maximum size of data to reorganize. For more details, please see https://restic.readthedocs.io/en/stable/060_forget.html .

  Moreover, prune now accepts the --dry-run option and also running forget --dry-run --prune will show what prune would do.

  This enhancement also fixes several open issues, e.g.: - #1140 - #1599 - #1985 - #2112 - #2227 - #2305

  #2718 #2842

• Enhancement #2495: Add option to let backup trust mtime without checking ctime

  The backup command used to require that both ctime and mtime of a file matched with a previously backed up version to determine that the file was unchanged. In other words, if either ctime or mtime of the file had changed, it would be considered changed and restic would read the file's content again to back up the relevant (changed) parts of it.

  The new option --ignore-ctime makes restic look at mtime only, such that ctime changes for a file does not cause restic to read the file's contents again.

  The check for both ctime and mtime was introduced in restic 0.9.6 to make backups more reliable in the face of programs that reset mtime (some Unix archivers do that), but it turned out to often be expensive because it made restic read file contents even if only the metadata (owner, permissions) of a file had changed. The new --ignore-ctime option lets the user restore the 0.9.5 behavior when needed. The existing --ignore-inode option already turned off this behavior, but also removed a different check.

  Please note that changes in files' metadata are still recorded, regardless of the command line options provided to the backup command.

  #2495 #2558 #2819 #2823

• Enhancement #2941: Speed up the repacking step of the prune command

  The repack step of the prune command, which moves still used file parts into new pack files such that the old ones can be garbage collected later on, now processes multiple pack files in parallel. This is especially beneficial for high latency backends or when using a fast network connection.

  #2941

• Enhancement #3006: Speed up the rebuild-index command

  We've optimized the rebuild-index command. Now, existing index entries are used to minimize the number of pack files that must be read. This speeds up the index rebuild a lot.

  Additionally, the option --read-all-packs has been added, implementing the previous behavior.

  #3006 https://github.com/restic/restic/issue/2547

• Enhancement #3048: Add more checks for index and pack files in the check command

  The check command run with the --read-data or --read-data-subset options used to only verify only the pack file content - it did not check if the blobs within the pack are correctly contained in the index.

  A check for the latter is now in place, which can print the following error:

  Blob ID is not contained in index or position is incorrect

  Another test is also added, which compares pack file sizes computed from the index and the pack header with the actual file size. This test is able to detect truncated pack files.

  If the index is not correct, it can be rebuilt by using the rebuild-index command.

  Having added these tests, restic check is now able to detect non-existing blobs which are wrongly referenced in the index. This situation could have lead to missing data.

  #3048 #3082

• Enhancement #2433: Make the dump command support zip format

  Previously, restic could dump the contents of a whole folder structure only in the tar format. The dump command now has a new flag to change output format to zip. Just pass --archive zip as an option to restic dump.

  #2433 #3081

• Enhancement #3099: Reduce memory usage of check command

  The check command now requires less memory if it is run without the --check-unused option.

  #3099

• Enhancement #3106: Parallelize scan of snapshot content in copy and prune

  The copy and prune commands used to traverse the directories of snapshots one by one to find used data. This snapshot traversal is now parallized which can speed up this step several times.

  In addition the check command now reports how many snapshots have already been processed.

  #3106

• Enhancement #3130: Parallelize reading of locks and snapshots

  Restic used to read snapshots sequentially. For repositories containing many snapshots this slowed down commands which have to read all snapshots.

  Now the reading of snapshots is parallelized. This speeds up for example prune, backup and other commands that search for snapshots with certain properties or which have to find the latest snapshot.

  The speed up also applies to locks stored in the backup repository.

  #3130 #3174

• Enhancement #3254: Enable HTTP/2 for backend connections

  Go's HTTP library usually automatically chooses between HTTP/1.x and HTTP/2 depending on what the server supports. But for compatibility this mechanism is disabled if DialContext is used (which is the case for restic). This change allows restic's HTTP client to negotiate HTTP/2 if supported by the server.

  #3254