5
0
mirror of git://git.proxmox.com/git/proxmox-backup.git synced 2025-01-06 13:18:00 +03:00
Commit Graph

7702 Commits

Author SHA1 Message Date
Maximiliano Sandoval
2170d58b0b fs: update comment to reflect usage of C-string literals
Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-06-20 12:26:49 +02:00
Wolfgang Bumiller
603042d0a4 rename .cargo/config to .cargo/config.toml
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-06-20 12:24:27 +02:00
Maximiliano Sandoval
1a76efc616 cargo: use default-features
Fixes the compile-time warning:

warning: Cargo.toml: `default_features` is deprecated in favor of `default-features` and will not work in the 2024 edition
(in the `proxmox-router` dependency)

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-06-20 12:18:40 +02:00
Wolfgang Bumiller
da72994faf use XATTR_* constants instead of calling functions
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-06-20 11:08:58 +02:00
Wolfgang Bumiller
0c9f247cfd bump sys dependency to 0.5.7
for the new xattr constants

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-06-20 11:08:58 +02:00
Wolfgang Bumiller
6359e6d4d4 replace c_str! macro with c"literals"
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-06-20 11:07:11 +02:00
Fabian Grünbichler
fd3f72820e build: use cargo wrapper when building package
else we don't pick up the options set by the wrapper, which include generation
of debug symbols. until rustc 1.77, this was not needed because compiled
binaries always included a non-stripped libstd. now, without this change, the
binaries built with `cargo build --release` have no debug symbols at all
trigger a warning. fix this and include debug symbols when building a package,
like was originally intended for release package builds.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-06-20 08:50:41 +02:00
Fabian Grünbichler
a92a745fdc build: fix SUBCRATES for arbitrary working dirs
else this only works if the git working tree is in a dir called
'proxmox-backup'

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-06-19 17:49:35 +02:00
Fabian Grünbichler
376cd5897e build: adapt workspace member command
to work with cargo 1.77, which changed from

 pbs-api-types 0.1.0 (path+file:///home/fgruenbichler/Sources/proxmox-backup/pbs-api-types)

to

 path+file:///home/fgruenbichler/Sources/proxmox-backup/pbs-api-types#0.1.0

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-06-19 16:00:39 +02:00
Gabriel Goller
734c4601a5 close #4763: client: add command to forget backup group
Add the command `proxmox-backup-client group forget <group>` so
that we can forget (delete) whole groups with all the containing
snapshots.
To avoid printing full datastore paths (which are in the error messages)
we filter out the most common one (group not found) and rephrase it.

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
[WB: rebased & sorted import statements in client's main.rs]
[WB: replace extract_repository_from_value with
     remove_repository_from_value since the parameter is rejected on
     the remote side]
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-06-19 11:32:28 +02:00
Wolfgang Bumiller
8e924a7bc0 client: add 'remove_repository_from_value' helper
'extract_repository_from_value' takes an immutable reference and
doesn't remove the parsed parameter (whereas in contrast in our PVE
codebase, the 'extract_param' method does remove it).

This adds a variant that explicitly removes it called
'remove_repository_from_value'.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-06-19 11:31:46 +02:00
Gabriel Goller
00c88a42a2 pxar: use anyhow::Error in PxarBackupStream
Instead of storing the error as a string in the PxarBackupStream, we
store it as an anyhow::Error. As we can't clone an anyhow::Error, we take
it out from the mutex and return it. This won't change anything as
the consumation of the stream will stop if it gets a Some(Err(..)).

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2024-06-19 10:30:13 +02:00
Gabriel Goller
230527a360 pxar: add UniqueContext helper
To create a pxar archive, we recursively traverse the target folder.
If there is an error further down and we add a context using anyhow,
the context will be duplicated and we get an output like:

> Error: error at "xattr/xattr.txt": error at "xattr/xattr.txt": E2BIG [skip]

This is obviously not optimal, so in recursive contexts we can use the
UniqueContext, which quickly checks the context from the last item in
the error chain and only adds it if it is unique.

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2024-06-19 10:30:12 +02:00
Gabriel Goller
095ddcad48 pxar: remove ArchiveError
The sole purpose of the ArchiveError was to add the file-path to the
error. Using anyhow::Error we can add this information using the context
and don't need this struct anymore.

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2024-06-19 10:30:10 +02:00
Thomas Lamprecht
74d735eeed ui: gc job edit: fix i18n gettext usage
String concatenating a variable with some static text as gettext
parameter cannot really work, and it also does not make sense to do
most of the time, as even if we'd use some overly generic format
string like '{0} (disabled)', it would be not easy to translate
correctly in all languages in such a generic way.

So just use the actual full string, which is already contained in our
translation catalogue anyway…

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-06-18 16:15:27 +02:00
Thomas Lamprecht
5c15fb97b4 docs: drop blanket statement recommending against remote storage
This is basically semantic revert of e5c0d80c ("docs: add note for not
using remote storages") that, while well intended, has a few problems,
e.g.:
- This is the minimal/recommended requirements section, which should
  list the rough basic specs a setup must/should have. Listing
  everything that is not best to do would bloat this list
  significantly and it's just the wrong place for it, i.e., it isn't a
  recommended against list.
- while it's true that a remote storage will basically always have
  _some_ overhead over using the same HW with a (modern) local storage
  (file) system, that does **not** mean that the remote storage has
  insufficient performance characteristics. We know of lots of fast
  Ceph setups, even release benchmarks for them, or storages like
  BlockBridge, that provide high performance while being remote.

So avoid this X-Y-problem style argumentation and focus on what is
actually important, even though I naturally get that there are some
users that use slow NFS attached storages, but breaking style here
won't cure them and I'm sure that they are capable of setting up such
a slow local storage that it won't make a real difference compared to
the NFS one.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-06-17 17:52:03 +02:00
Wolfgang Bumiller
0d47038a0c bump proxmox-sys dep to 0.5.6
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-06-17 14:06:25 +02:00
Fabian Grünbichler
1d36b502f5 Merge branch '3.2.6'
branched off to avoid a breaking change on master
2024-06-17 10:38:02 +02:00
Fabian Grünbichler
472b52f54c bump version to 3.2.6-1
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-06-17 10:18:53 +02:00
Christian Ebner
dbfba9db89 client: pxar: fix fuse mount performance for split archives
Adapt to the decoder/accessor method changes introduced in the pxar
library, which were introduced in order to move the consistency check
for metadata and payload data archives.

The new location of the checks allows to access the pxar archive via
a `Split` variant reader instance, without penalization when just
accessing the metadata, not reading any payload data.

This greatly improves performance when accessing fuse mounted
archives.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>

bumped dependency after pxar version bump

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-06-17 10:17:42 +02:00
Dietmar Maurer
e1a506a0d0 config: acme: use latest proxmox_sys::fs::ensure_dir_exists
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-06-13 11:58:09 +02:00
Dominik Csapak
e5c0d80ca4 docs: add note for not using remote storages
such as NFS or SMB. They will not provide the expected performance
and it's better to recommend against them.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-06-11 11:41:58 +02:00
Dominik Csapak
a3e79113cf tape: handle PEWZ like regular early warning
as a safeguard, should the disabling not work for some reason.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-06-11 10:33:57 +02:00
Dominik Csapak
b7a6c5da06 tape: disable Programmable Early Warning Zone (PEWZ)
since that leads to errors that we don't currently catch before we
reach the regular early warning on tape.

This can be read/set by the Device Configuration Extension Mode Page.
ignore errors on reading or writing, since it may not be available on
LTO-4

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-06-11 10:33:39 +02:00
Dominik Csapak
192d70bbe2 tape: refactor setting the mode page
we'll reuse that code later for a different page/subpage

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-06-11 10:05:25 +02:00
Fabian Grünbichler
3a76fdc9e7 bump version to 3.2.5-1
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-06-10 13:45:49 +02:00
Fabian Grünbichler
4354cae7ba bump pxar to 0.11.1
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-06-10 13:39:33 +02:00
Fabian Grünbichler
481a9515b1 extract: don't interpret prelude as OsStr
that would drop the final byte, and the corresponding code has been removed
from pxar now as well.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-06-10 13:38:10 +02:00
Christian Ebner
afdc0f0e42 client: pxar: encode prelude based on writer variant
Currently, whether to encode the exlcude patterns passed via cli as
prelude or via the `.pxar-exclude-cli` is based on the presence of
a previous metadata accessor.
That leaves however to the encoding of the file entry instead of the
prelude for split archives in `data` mode and for the first snapshot
in a backup, creating undesired padding in the first payload chunk.

Therefore, use the pxar writer variant to make the decision instead.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-10 13:11:38 +02:00
Christian Ebner
903ab2e938 client: pxar: json encode cli exclude pattern in prelude
The current encoding is not extensible, so encode the cli exclude
patterns as json instead. By this, the prelude is easily seralized
and deserialized, while remaining human readable.

Originally-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-10 13:11:29 +02:00
Christian Ebner
3c9a29e5cf file-restore: list: improve pxar v2 performance
Do not attach the payload reader for split pxar archives, as only the
metadata has to be accessed for listing.
This avoids that the decoder performs consistency checks with the
payload stream, which require chunk download and decoding, making the
listing unusable slow.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-10 10:58:31 +02:00
Christian Ebner
8834153187 docs: add table listing possible change detection modes
Quick and concise listing of the available change detection modes for
reference.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-10 10:30:10 +02:00
Christian Ebner
04bb256abd client: backup spec: rename change detection mode default
The currently default variant is named `Default`, which is not future
prove since the default might change in the future. So rename it to
`Legacy` instead.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-10 10:30:05 +02:00
Fabian Grünbichler
fdcae4bd8a api: catalog: improve pxar v2 performance
by skipping the payloader reader entirely, it's not needed for listing contents
and would make accessing larger archives too expensive.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-10 10:10:33 +02:00
Fabian Grünbichler
8f9330b582 run cargo fmt
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-06-07 14:00:33 +02:00
Fabian Grünbichler
e653ace318 api: catalog/file-restore: use archive-name schema
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-06-07 14:00:15 +02:00
Christian Ebner
c0302805c4 client: backup: conditionally write catalog for file level backups
Only write the catalog when using the regular backup mode, do not write
it when using the split archive mode.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-07 13:56:24 +02:00
Christian Ebner
4dd816b343 www: content: lookup via metadata archive instead of catalog
In case of pxar archives with split metadata and payload data, the
metadata archive has to be used to lookup entries for navigation
before performing a single file restore.

Decide based on the archive filename extension whether to use the
`catalog` or the `pxar-lookup` api endpoint.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-07 13:56:24 +02:00
Christian Ebner
84981486cb file-restore: fallback to mpxar if catalog not present
The `proxmox-file-restore list` command will uses the provided path to
lookup and list directory entries via the catalog. Fallback to using
the metadata archive if the catalog is not present for fast lookups in
a backup snapshot.

This is in preparation for dropping encoding of the catalog for
snapshots using split archive encoding. Proxmox VE's storage plugin
uses this to allow single file restore for LXCs.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-07 13:56:24 +02:00
Christian Ebner
6cf75f2fe2 file-restore: never list ppxar as archive
Payload data archives cannot be used to navigate the content, so
exclude them from the archive listing, as this is used by
Proxmox VE to list in the file browser.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-07 13:56:24 +02:00
Christian Ebner
364e2d1133 api: datastore: add optional archive-name to file-restore
Allow to pass the archive name as optional api call parameter instead
of having it as prefix to the path.
If this parameter is given, instead of splitting of the archive name
from the path, the parameter itself is used, leaving the path
untouched.

This allows to restore single files from the archive, without having
to artificially construct the path in case of file restores for split
pxar archives, where the response path of the listing does not
include the archive, as opposed to the response provided by lookup
via the catalog.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-07 13:56:24 +02:00
Christian Ebner
cfb2632c9e api: datastore: conditional lookup for catalog endpoint
Add an optional `archive-name` parameter, indicating the metadata
archive to be used for directory content lookups instead of the
catalog. If provided, instead of the catalog reader, a pxar Accessor
instance is created to perform the lookup.

This is in preparation for dropping catalog encoding for snapshots
with split pxar archive encoding.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-07 13:56:24 +02:00
Christian Ebner
8d8142955f client: tools: add helper to lookup ArchiveEntrys via pxar
In preparation to lookup entries via the pxar metadata archive
instead of the catalog, in order to drop encoding the catalog
for snapshots using split pxar archives altogehter.

This helper allows to lookup the directory entries via the provided
accessor instance and formats them to be compatible with the output
as produced by lookups via the catalog.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-07 13:56:24 +02:00
Christian Ebner
3b95f09522 api: datastore: move reusable code out of thread
Move code that can be reused when having to  perform a lookup via the
pxar metadata archive instead of the catalog out of the thread.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-07 13:56:24 +02:00
Christian Ebner
30ea695518 api: datastore: factor out path decoding for catalog
The file path passed to the catalog is base64 encoded, with an exception
for the root.
Factor this check and decoding step out into a helper function to make
it reusable when doing the same for lookups via the metadata archive
instead of the catalog.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-07 13:56:24 +02:00
Christian Ebner
2baa9f8fb8 client: helper: fix minor formatting issue
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-07 12:09:21 +02:00
Christian Ebner
6b1110badf client: pxar: fix minor formatting issue
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-07 12:09:21 +02:00
Christian Ebner
1259234488 client: pxar: conditionally skip metadata reference test
The test will fail for all users not having euid/egid set to
1000/1000, as the reference test folder structure cannot be created
with the expected ownership.
Therefore, skip over the test if either euid or egid do not match
this condition.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-06 10:46:05 +02:00
Christian Ebner
bab0645cc6 client: pxar: do not attempt to set uid/gid in test
Setting the uid/gid for the files and folders of the test directory
structure will not work when lacking the permissions.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-06 10:46:05 +02:00
Fabian Grünbichler
766faeb04a bump pxar build-dep to 0.11
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-06-05 16:39:42 +02:00