5
0
mirror of git://git.proxmox.com/git/proxmox-backup.git synced 2025-01-20 14:03:53 +03:00

8007 Commits

Author SHA1 Message Date
Hannes Laimer
b17ebd5c2c datastore: handle deletion of removable datastore properly
Data deletion is only possible if the datastore is mounted, won't attempt
mounting it for the purpose of deleting data.

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2024-11-25 21:34:22 +01:00
Hannes Laimer
2f874935b5 add auto-mounting for removable datastores
If a device houses multiple datastore, none of them will be mounted
automatically. If a device only contains a single datastore it will be
mounted automatically. The reason for not mounting multiple datastore
automatically is that we don't know which is actually wanted, and since
mounting all means also all have to be unmounted manually, it made sense
to have the user choose which to mount.

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2024-11-25 21:34:22 +01:00
Hannes Laimer
919925519a bin: manager: add (un)mount command
We can't just directly delegate these commands to the API endpoints
since both mounting and unmounting are done in a worker, and that one
would be killed when the parent ends. In this case that would be the CLI
process, which basically ends right after spwaning the worker.

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2024-11-25 21:34:22 +01:00
Hannes Laimer
76609915d6 pbs-api-types: add mount_status field to DataStoreListItem
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2024-11-25 21:34:22 +01:00
Hannes Laimer
40a2b110bf api: add check for nested datastores on creation
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2024-11-25 21:34:22 +01:00
Hannes Laimer
91c67298f4 api: removable datastore creation
Devices can contains multiple datastores.
If the specified path already contains a datastore, `reuse datastore` has
to be set so it'll be added without creating a chunckstore.

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2024-11-25 21:34:22 +01:00
Hannes Laimer
2b31406a37 api: admin: add (un)mount endpoint for removable datastores
Removable datastores can be mounted unless
 - they are already
 - their device is not present
For unmounting the maintenance mode is set to `unmount`,
which prohibits the starting of any new tasks envolving any
IO, this mode is unset either
 - on completion of the unmount
 - on abort of the unmount tasks
If the unmounting itself should fail, the maintenance mode stays in
place and requires manual intervention by unsetting it in the config
file directly. This is intentional, as unmounting should not fail,
and if it should the situation should be looked at.

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2024-11-25 21:34:22 +01:00
Hannes Laimer
46d7e573a9 datastore: add helper for checking if a datastore is mounted
... at a specific location. Also adds two additional functions to
get the mount status, and ensuring a removable datastore is mounted.

Co-authored-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2024-11-25 21:34:22 +01:00
Dietmar Maurer
66389b2fd9 maintenance: add 'Unmount' maintenance type
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2024-11-25 21:34:22 +01:00
Hannes Laimer
652b774eb0 maintenance: make is_offline more generic
... and add MaintenanceType::Delete to it. We also want to clear any
cach entries if we are deleting the datastore, not just if it is marked
as offline.

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2024-11-25 21:34:22 +01:00
Hannes Laimer
89c650b83e pbs-api-types: add backing-device to DataStoreConfig
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2024-11-25 21:34:22 +01:00
Thomas Lamprecht
ffc8265e1f ui: login view: add missing trailing comma
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-11-25 21:34:22 +01:00
Thomas Lamprecht
4ef241a63b ui: update online help info reference-map
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-11-25 20:45:49 +01:00
Thomas Lamprecht
37440cd93a d/control: update versioned dependency for widget-toolkit
To ensure newly used components for the consent banner are available.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-11-25 18:51:35 +01:00
Thomas Lamprecht
d399fe50da update proxmox-rest-server dependency to 0.8.4
To ensure the adapted handlebars escaper that keeps '=' as is gets
used, required for the consent banner.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-11-25 18:50:28 +01:00
Gabriel Goller
ccf08921ee docs: add section about consent banner
Add short section on how to enable consent banner.

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2024-11-25 18:48:13 +01:00
Gabriel Goller
dea876fd5e ui: show consent banner before login
Before showing the LoginView, check if we got a non-empty consent text
from the template. If there is a non-empty text, display it in a modal.

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2024-11-25 18:48:13 +01:00
Gabriel Goller
28028b15b7 api: add consent api handler and config option
Add consent_text option to the node.cfg config. Embed the value into
index.html file using handlebars.

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2024-11-25 17:19:32 +01:00
Shannon Sterz
d3f2e69cad ui: set min length for new passwords to 8
Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
2024-11-25 15:51:47 +01:00
Shannon Sterz
fb5b6f3eab api: enforce minimum character limit of 8 on new passwords
we already have two different password schemas, `PBS_PASSWORD_SCHEMA`
being the stricter one, which ensures a minimum length of new
passwords. however, this wasn't used on the change password endpoint
before, so add it there too. this is also in-line with NIST's latest
recommendations [1].

[1]: https://pages.nist.gov/800-63-4/sp800-63b.html#passwordver

Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
2024-11-25 15:51:47 +01:00
Shannon Sterz
9f3733c5ed api: ignore password parameter in the update_user endpoint
currently if a password is provided, we check whether the user that is
going to be updated can authenticate with it. later on, the password
is then set as the same password. this means that the password here
can only be changed if it is the exact same one that is already used.

so in essence, the password cannot be changed through this endpoint
already. remove all of this logic here in favor of the
`PUT /access/password` endpoint.

to keep the api stable for now, just ignore the parameter and add a
description that explains what to use instead.

Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
2024-11-25 15:51:47 +01:00
Christian Ebner
6a619b2488 ui: sync job: fix source group filters based on sync direction
Fix switching the source for group filters based on the sync jobs
sync direction.

The helper to set the local namespace for the group filers was
introduced in commit 43a92c8c ("ui: group filter: allow to set
namespace for local datastore"), but never used because lost during
subsequent iterations of reworking the patch series.

The switching is corrected by:
- correctly initializing the local store and namespace for the group
  filer of sync jobs in push direction in the controller init, if a
  datastore is set.
- fixing an incorrect check for the sync direction in the remote
  datastore selector change listener.
- conditionally switching namespace to be set for the group filter in
  the remote and local namespace selector change listeners.
- conditionally switching datastore to be set for the group filter in
  the local datastore selector change listener.

Reported-by: Lukas Wagner <l.wagner@proxmox.com>
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>
2024-11-25 15:49:04 +01:00
Lukas Wagner
674ae4947b web ui: notification: remove matcher overiddes
These were put in place so that initial release of the new
notification system for Proxmox Backup Server can already include
improved notification matchers, which at that time have not been yet
merged into proxmox-widget-toolkit.

In the meanwhile, the changes have been merged an released in
proxmox-widget-toolkit 4.2.4, hence we can remove the override.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2024-11-25 15:46:47 +01:00
Lukas Wagner
b0448d0ad1 d/control: bump proxmox-widget-toolkit dependency
We need "notification: matcher: match-field: show known fields/values",
which was released in proxmox-widget-toolkit 4.2.4

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2024-11-25 15:46:47 +01:00
Christoph Heiss
eb126116ca docs: images: add installer guide screenshots
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
2024-11-25 15:45:15 +01:00
Christoph Heiss
5cacfe02da docs: add installation wizard guide
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
2024-11-25 15:44:27 +01:00
Christoph Heiss
d363818641 docs: add installation media preparation guide
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
2024-11-25 15:44:23 +01:00
Fabian Grünbichler
391822f9ce run cargo fmt
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-11-25 13:18:11 +01:00
Fabian Grünbichler
2009d8de41 api types: replace PathPatterns with Vec<PathPattern>
PathPatterns is hard to distinguish from PathPattern, so would need to be
renamed anyway.. but there isn't really a reason to define a separate API type
just for this.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-11-25 12:28:40 +01:00
Christian Ebner
1bb680017b fix #2996: client: allow optional match patterns for restore
When the user is only interested in a subset of the entries stored in
a file-level backup, it is convenient to be able to provide a list of
match patterns for the entries intended to be restored.

The required restore logic is already in place. Therefore, expose it
for the `proxmox-backup-client restore` command by adding the optional
array of patterns as command line argument and parse these before
passing them via the pxar restore options to the archive extractor.

Link to bugtracker issue:
https://bugzilla.proxmox.com/show_bug.cgi?id=2996

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-11-25 11:58:43 +01:00
Christian Ebner
70545af183 client: catalog shell: use dedicated api type for patterns
Use the common api type with schema based input validation for all
match pattern parameters exposed via the api macro.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-11-25 11:57:07 +01:00
Christian Ebner
33031f9835 pxar: bin: use dedicated api type for restore pattern
Instead of taking a plain string as input parameter, use the
corresponding api type performing additional input validation.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-11-25 11:57:07 +01:00
Christian Ebner
45b5556765 api-types: implement dedicated api type for match patterns
Introduces a dedicated api type `PathPattern` and the corresponding
format and input validation schema. Further, add a `PathPatterns`
type for collections of path patterns and implement required traits
to be able to replace currently defined api parameters.

In preparation for using this common api type for all api endpoints
exposing a match pattern parameter.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-11-25 11:57:07 +01:00
Christian Ebner
a6c3192233 docs: deduplicate background details for garbage collection
Currently, common details regarding garbage collection are documented
in the backup client and the maintenance task. Deduplicate this
information by moving the details to the background section of the
maintenance task and reference that section in the backup client
part.

Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-11-25 11:52:26 +01:00
Christian Ebner
75c695bea4 docs: add security implications of prune and change detection mode
Users should be made aware that the data stored in chunks outlives
the backup snapshots on pruning and that backups created using the
change-detection-mode set to metadata might reference chunks
containing files which have vanished since the previous backup, but
might still be accessible when access to the chunks raw data is
possible (client or server side).

Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-11-25 11:52:26 +01:00
Fabian Grünbichler
8e057c3874 sync config: forbid setting resync_corrupt for push jobs
they don't support it (yet), so don't allow setting it in the backend either.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-11-25 11:22:46 +01:00
Gabriel Goller
19818d1449 fix #3786: docs: add resync-corrupt option to sync-job
Add short section explaining the `resync-corrupt` option on the
sync-job.

Originally-by: Shannon Sterz <s.sterz@proxmox.com>
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-11-25 10:53:26 +01:00
Gabriel Goller
590187ff53 fix #3786: ui/cli: add resync-corrupt option on sync-jobs
Add the `resync-corrupt` option to the ui and the
`proxmox-backup-manager` cli. It is listed in the `Advanced` section,
because it slows the sync-job down and is useless if no verification
job was run beforehand.

Originally-by: Shannon Sterz <s.sterz@proxmox.com>
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-11-25 10:53:26 +01:00
Gabriel Goller
0974ddfa17 fix #3786: api: add resync-corrupt option to sync jobs
This option allows us to "fix" corrupt snapshots (and/or their chunks)
by pulling them from another remote. When traversing the remote
snapshots, we check if it exists locally, and if it is, we check if the
last verification of it failed. If the local snapshot is broken and the
`resync-corrupt` option is turned on, we pull in the remote snapshot,
overwriting the local one.

This is very useful and has been requested a lot, as there is currently
no way to "fix" corrupt chunks/snapshots even if the user has a healthy
version of it on their offsite instance.

Originally-by: Shannon Sterz <s.sterz@proxmox.com>
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-11-25 10:53:26 +01:00
Gabriel Goller
b5be65cf8a snapshot: add helper function to retrieve verify_state
Add helper functions to retrieve the verify_state from the manifest of a
snapshot. Replaced all the manual "verify_state" parsing with the helper
function.

Suggested-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2024-11-25 10:52:40 +01:00
Christian Ebner
3dc9d2de69 server: push: log encountered empty backup groups during sync
Log also empty backup groups with no snapshots encountered during the
sync so the log output contains this additional information as well,
reducing possible confusion.

Nevertheless, continue with the regular logic, so that pruning of
vanished snapshot is honored.

Examplary output in the sync jobs task log:
```
2024-11-22T18:32:40+01:00: Syncing datastore 'datastore', root namespace into datastore 'push-target-store', namespace 'test'
2024-11-22T18:32:40+01:00: Found 2 groups to sync (out of 2 total)
2024-11-22T18:32:40+01:00: skipped: 1 snapshot(s) (2024-11-22T13:40:18Z) - older than the newest snapshot present on sync target
2024-11-22T18:32:40+01:00: Group 'vm/200' contains no snapshots to sync to remote
2024-11-22T18:32:40+01:00: Finished syncing root namespace, current progress: 1 groups, 0 snapshots
```

Reported-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-11-25 10:10:37 +01:00
Christian Ebner
62228d39f2 server: push: add error context to api calls and priv checks
Add an anyhow context to errors and display the full error context
in the log output. Further, make it clear which errors stem from api
calls by explicitly mentioning this in the context message.

This also fixes incorrect error handling by placing the error context
on the api result instead of the serde deserialization error for
cases this was handled incorrectly.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>

FG: add missing format!
FG: run cargo fmt
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-11-22 14:08:15 +01:00
Fabian Grünbichler
83810759ee api types: extend backup archive name parsing tests
and also test the error triggered by a directory path being passed in.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-11-22 13:47:25 +01:00
Christian Ebner
db5bf33cfe api types: add unit tests for backup archive name parsing
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-11-22 13:47:25 +01:00
Christian Ebner
7ad5ad82e5 client: drop unused parse_archive_type helper
Parsing of the type based on the archive name extension is now
handled by `BackupArchiveName`.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>

FG: add removal of import

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-11-22 13:47:25 +01:00
Christian Ebner
6771869cc1 client/server: use dedicated api type for all archive names
Instead of using the plain String or slices of it for archive names,
use the dedicated api type and its methods to parse and check for
archive type based on archive filename extension.

Thereby, keeping the checks and mappings in the api type and
resticting function parameters by the narrower wrapper type to reduce
potential misuse.

Further, instead of declaring and using the archive name constants
throughout the codebase, use the `BackupArchiveName` helpers to
generate the archive names for manifest, client logs and encryption
keys.

This allows for easy archive name comparisons using the same
`BackupArchiveName` type, at the cost of some extra allocations and
avoids the currently present double constant declaration of
`CATALOG_NAME`.

A positive ergonomic side effect of this is that commands now also
accept the archive type extension optionally, when passing the archive
name.

E.g.
```
proxmox-backup-client restore <snapshot> <name>.pxar.didx <target>
```
is equal to
```
proxmox-backup-client restore <snapshot> <name>.pxar <target>
```

The previously default mapping of any archive name extension to a blob
has been dropped in favor of consistent mapping by the api type
helpers.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>

FG: use LazyLock for constant archive names
FG: add missing import

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-11-22 13:47:05 +01:00
Christian Ebner
addfae26cf api types: introduce BackupArchiveName type
Introduces a dedicated wrapper type to be used for backup archive
names instead of plain strings and associated helper methods for
archive type checks and archive name mappings.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>

FG: use LazyLock for constant archive names reduces churn, and saves some
allocations

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-11-22 13:46:35 +01:00
Christian Ebner
e932ec101e datastore: move ArchiveType to api types
Moving the `ArchiveType` to avoid crate dependencies on
`pbs-datastore`.

In preparation for introducing a dedicated `BackupArchiveName` api
type, allowing to set the corresponding archive type variant when
parsing the archive name based on it's filename.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-11-22 11:45:43 +01:00
Fabian Grünbichler
cbf7bbefb7 pxar: extract: make invalid ACLs non-fatal
these can occur in practice, and neither setting nor getting them throws an
error. if "invalid" ACLs are non-restorable, this means that creating a pxar
archive with such an ACL is possible, but restoring it isn't.

reported in our community forum:
https://forum.proxmox.com/threads/155477

Tested-by: Gabriel Goller <g.goller@proxmox.com>
Acked-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-11-22 10:38:49 +01:00
Fabian Grünbichler
4e37c678dc pxar: add file name to path_info when applying metadata
else, error messages using this path_info refer to the parent directory instead
of the actual file entry causing the problem. since this is just for
informational purposes, lossy conversion is acceptable.

Acked-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-11-22 10:38:42 +01:00