Operation Details

This section describes the operations that Backrest can be configured to perform on your behalf in detail.

Overview

Backrest executes commands by forking the restic binary. Each Backrest version is validated against a specific version of restic. On startup Backrest searches for a versioned restic in it's data directory (typically ~/.local/share/backrest), followed by /bin/. The restic binary must be named restic-{VERSION}. You can overriderestic command by setting BACKREST_RESTIC_COMMAND env variable when starting Backrest. Otherwise, if no binary is found Backrest will download and install a recent version of restic from restic's github releases. When downloading a restic binary, the download is verified by checking the sha256sum of the downloaded binary against the sha256sum provided by restic and signed by the restic maintainers GPG key.

When running restic commands, Backrest injects the environment variables configured in the repo into the environment of the restic process and it appends the flags configured in the repo to the command line arguments of the restic process. Logs are collected for each command. In the case of an error, Backrest captures the last ~500 bytes of output and displays this directly in the error message (the first and last 250 bytes are shown if the output is longer than 500 bytes). Logs of the command are typically also available by clicking [View Logs] next to an operation, these logs are truncated to 32KB (with the first and last 16KB shown if the log is longer than 32KB).

Types of Operations

Backup

Restic docs on backup

A backup operation creates a snapshot of your data and sends it to a repository. The snapshot is created using the restic backup command.

As the backup runs Backrest will display the progress of the backup operation in the UI. The progress information is parsed from the JSON output of the restic backup command.

The backup flow is as follows

Hook trigger: CONDITION_SNAPSHOT_START, if any hooks are configured for this event they will run.
- If any hook exits with a non-zero status, the hook's failure policy will be applied (e.g. cancelling or failing the backup operation).
The restic backup command is run. The newly craeted snapshot is tagged with the ID of the plan creating it e.g. plan:{PLAN_ID}.
The summary event is parsed from the backup and is stored in the operation's metadata. This includes: files added, files changed, files unmodified, total files processed, bytes added, bytes processed, and most importantly the snapshot ID.
If an error occurred: hook trigger CONDITION_SNAPSHOT_ERROR, if any hooks are configured for this event they will run.
Hook trigger: CONDITION_SNAPSHOT_END, if any hooks are configured for this event they will run. This condition is always triggered even if the backup failed.

If the backup completed successfully, Backrest triggers followup operations

If a retention policy is set (e.g. not None) a forget operation is triggered for the backup plan.
If a prune policy is set (e.g. not None) a prune operation is triggered for the backup plan if it has been long enough since the last prune operation.

Forget

Restic docs on forget

A forget operation marks old snapshots for deletion but does not remove data from storage until a prune runs. The forget operation is run using the restic forget --tag plan:{PLAN_ID} command.

Retention policies are mapped to forget arguments:

By Count maps to --keep-last {COUNT}
By Time Period maps to the --keep-{hourly,daily,weekly,monthly,yearly} {COUNT} flags

Note that forget will never run for a plan if the forget policy is set to None.

Prune

Restic docs on prune

A prune operation removes data from storage that is no longer referenced by any snapshot. The prune operation is run using the restic prune command. Prune operations apply to the entire repo (e.g. are not scoped to data unreferenced by a specific plan).

Prunes are run in compliance with a prune policy which specifies:

Max Frequency Days - the minimum time in days between prune operations. A prune is skipped following a backup if the last prune was less than this many days ago. This is to prevent excessive pruning which can be slow and is bandwidth intensive as prunes may read and rewrite portions of the repository.
Max Unused Percent - the maximum percentage of the repository that may be left unused after a prune operation runs. Prune operations will try to repack blobs in the repository if more than this percentage of their is unused (e.g. formerly held data belonging to forgotten snapshots).

Prune operations do not support json output so the logs of a prune operation are not parsed for progress information, they are instead displayed as raw text in the Backrest UI as they are received from the restic process.

5.0 KiB Raw Blame History