IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
Helper method that takes an archive name as input and checks if the
given archive is present in the manifest, by also taking possible
split archive extensions into account.
Returns the pxar archive name if found or the split archive names if
the split archive variant is present in the manifest.
If neither is matched, an error is returned signaling that nothing
matched entries in the manifest.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
... and attach the split payload writer variant to the pxar archive
creation. By this, metadata and payload data will create different
dynamic indexes, allowing to lookup and reuse payload chunks without
the additional overhead of the pxar archive's metadata.
For now this functionality remains disabled and will be enabled in a
later patch once the logic for reusing the payload chunks is in
place.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Introduce a `PxarWriters` struct to bundle all writer instances
required for the pxar archive creation into a single object to limit
the number of function call parameters.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
... and adapt to the new reader/writer variant for encoder or
decoder/accessor to attach a dedicated payload input/output for split
pxar archives.
In preparation for look-ahead caching, where a passing around of
per-directory level encoder instances with internal references is
not feasible.
Previously, for each directory level a new encoder instance has been
generated, restricting possible implementation errors. These encoder
instances have been internally linked by references to keep track of
the state changes in a parent child relationship.
This is however not feasible when the encoder has to be passed by
mutable reference, as required by the look-ahead cache
implementation. The encoder has therefore been adapted to use a
single instance implementation with an internal stack keeping track
of the state.
Depends on the bumped pxar library version, including the patches to
attach the corresponding variant for the pxar reader/writer
instantiation.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Instead of taking ownership of the http client when starting a new
BackupWriter instance, only borrow the client.
This allows to reuse the http client to later reuse it to start also a
BackupReader instance as required for backup runs with metadata based
file change detection mode, where both must use the same http client.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
instead of rejecting any non-leaf certificate not pre-validated by OpenSSL,
treat them as valid but keep track of the fact that the pre-validation result
is no logner trustable.
certificate chains completely trusted by openssl are still accepted like
before, and leaf certificates without a chain are also handled the same (since
the verify callback is only ever called with depth == 0 in that case).
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
It is more readable than using match. We also inline variables in
eprintln!.
Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Move the exclude pattern matching further up to avoid unnecessary
instantiation of the metadata object, not needed if the entry was
matched.
No functional change intended.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Some filesystems (f.e. zfs) support xattrs bigger than 64kB, sadly we
can't get them because the kernel vfs limits us. The syscalls listxattr
and getxattr will return a E2BIG error in this case.
Added a flag --ignore-e2big-xattr to the client, this will ignore the
metadata (but still backup the file) if this error occurs.
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
... since the functions don't actually need to own the value.
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Tested-by: Gabriel Goller <g.goller@proxmox.com>
This commit adds the missing "Connection: upgrade" HTTP header [1]
when requesting an upgrade to HTTP 2.
Doing so is mandated in the HTTP Semantics RFC [2], and without this,
(reverse) proxies that strictly follow the standard could potentially
break.
[1]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Upgrade
[2]: RFC 9110, 7.8. Upgrade: “[...] sender of Upgrade MUST also send
an "Upgrade" connection option in the Connection header [...]”
Reported-By: McTwist <rajb89@hotmail.com>
Signed-off-by: Max Carrara <m.carrara@proxmox.com>
[ TL: added RFC reference and use case to commit message ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
the match-arms let one follow the different branches easier than the
relatively crowded if-condition it replaces.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Warnings in the task log/state normally means that the task actually
did its main job, but there was some detected (potential) issue that
the users should be made aware of. Exiting with an error code in that
case would be a bit odd.
While just exiting with success might not be the best solution either,
it's definitively more correct than a failure-exit-code, so go for
that for now as a stop-gap.
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
[ TL: rebased on current master (v3 was already applied) and rewrite
commit message accordingly ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Now we make an additional request on `api2/json/.../tasks/{upid}/status` to
get the `exitstatus` of the task. This allows us to `bail` and thus
get a non-zero exit code in the cli.
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
The match condition has gotten a bit large, and the error case is a
bit more concise with a pattern guard.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Passed a closure with the `stat()` function call to `matches()`. This
will traverse through all patterns and try to match using the path only, if a
`file_mode` is needed, it will run the closure. This means that if we exclude
a file with the `MatchType::ANY_FILE_TYPE`, we will skip it without running
`stat()` on it. As we updated the `matches()` function, we also updated all the
invocations of it.
Added `pathpatterns` crate to local overrides in cargo.toml.
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
Adds OverwriteFlags for granular control of which entry types should
overwrite entries present on the filesystem during a restore.
The original overwrite flag is refactored in order to cover all of the
other cases.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Creating symlinks or hardlinks might fail if a directory entry with the
same name is already present on the filesystem during restore.
When the overwrite flag is given, on failure unlink the existing entry
(except directories) and retry hard/symlink creation.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
this actually affected the matcher's ability to differentiate between directory
and file patterns, and the alternative would require matching patterns twice
for full coverage, so let's try a different approach altogether.
This reverts commit c8ed10095d.
When executing `proxmox-backup-client backup ...
--exclude "test/test.txt"` it still executed stat() on "test.txt",
which won't work when the current user doesn't have access to the
file or the parent folder. Now we check if the file is excluded,
and if it is not, then we execute stat().
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
We check if the manifest contains an index for the requested archive, if
it does not we avoid downloading it and report a more helpful error
message.
Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
This enum's purpose is to provide context to errors that occur during
the extraction of a pxar archive, making it possible to handle
extraction errors in a more granular manner.
For now, it's only implemented in `ExtractorIter::next()`, but may be
used in other places if necessary or desired.
Signed-off-by: Max Carrara <m.carrara@proxmox.com>
This change factors the body of `extract_archive()` into a separate
struct named `ExtractorIter` which implements the `Iterator` trait.
This refactor has two goals:
* Make it easier to provide and propagate errors and additional
information via `anyhow::Context`
* Introduce a means to handle errors that occur during extraction,
with the possibility to continue extraction if the handler decides
that the error is not fatal
The latter point benefits from the information provided by the former;
previously, errors could only be handled in certain locations
(e.g. application of metadata), but not on a "per-entry" basis.
Since `extract_archive()` was already using a "desugared" version of
the iterator pattern to begin with, wrapping its body up in an actual
`Iterator` made the most sense, as it didn't require changing the already
existing control flow that much.
Signed-off-by: Max Carrara <m.carrara@proxmox.com>
In order to preserve the source(s) of errors, `anyhow::Context` is
used instead of propagating errors via `Result::map_err()` and / or
`anyhow::format_err!()`.
This makes it possible to access e.g. an underlying `io::Error` or
`nix::Errno` etc. that caused an execution path to fail.
Certain usages of `anyhow::bail!()` are also changed / replaced
in order to preserve context.
Signed-off-by: Max Carrara <m.carrara@proxmox.com>
gives a higher (runtime) control via PBS_LOG, so that users can decide
themselves which messages, sources and levels are interesting for a particular
use case.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
since proxmox-backup-client is used in cron jobs and similar automated
fashions, PBS_LOG= should control the output..
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Instead of percent-encoding the UPID on every loop iteration plus on
abort, just encode it once before entering the loop.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
`proxmox-backup-client task log ..` and `proxmox-backup-manager task log ..`
are used to view the logs of tasks that have been started by another client, so
interrupting the task progress view should not forward the interrupt to the
running task. other call sites of the same helper(s) that spawn a task and then
print its progress should keep the old behaviour of interrupting the spawned
task on C^c.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
as a stop-gap measure. Otherwise, task logs for PVE backups started
via non-CLI will have the message
> storing login ticket failed: $XDG_RUNTIME_DIR must be set
show up when running a proxmox-backup-client command (e.g. setting
notes and when uploading the log). This is confusing to users[0].
[0]: https://forum.proxmox.com/threads/120492/
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Instead, report an error if storing the ticket info failed, so that the
user is informed that something went wrong and follow-up commands might
require authentication again.
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
these are only used by a single member at the moment, but we can move them to
the workspace to have a single location for version + base feature set
specification.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
besides harmonizing versions, the only global change is that the tokio-io
feature of pxar is now implied since its default anyway, instead of being
spelled out.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
pbs-buildcfg is the only one that needs to inherit the version as well, since
it stores it in the compiled crate.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
it's used by pxar-bin and proxmox-backup-client for
mounting, but pbs-client is used by more (eg. the
proxmox-backup-qemu library which really doesn't need to
pull in any fuse dependencies)
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
the dropped .into() is guarded by the bumped build-dependency on
proxmox-sys 0.4.1, the missing Eq is a new clippy lint.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Previously, autocompletion of archive names, for instance
in the case of
$ proxmox-backup-client restore <snapshot> <TAB>
did not work if no namespace was provided via the --ns option.
The fix is to fall back to the root namespace if the option is
not provided by the user.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
In order to be able to use a proxy with the proxmox-backup-client, use
ProxyConfig for parsing proxy server config from the environment. Also
added a section in the documentation that describes how to configure the
environment if a proxy server should be used.
Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
no real change for PBS usage - the ApiHandler enum is marked
non_exhaustive now because it has extra values if the new (enabled by
default) "server" feature is enabled.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
fuse_lowlevel.h says about read:
Read should send exactly the number of bytes requested except
on EOF or error, otherwise the rest of the data will be
substituted with zeroes.
but we simply forwarded the bytes we got from 'read_at'. The result was
that files were corrupt as soon as read_at returned not the exact number
of bytes requested. such short reads are easy to trigger with large
files (where reading a file has to cross many chunk boundaries).
To fix that, loop over 'read_at' until our buffer is full, or we read
0 bytes, indicating EOF.
reported in the forum:
https://forum.proxmox.com/threads/proxmox-backup-client-mounting-a-pxar-archive-gives-truncated-files.114447/
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Added comment, reworded slightly
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Also added WITH_OWNER and WITH_PERMISSION to Default-Flags,
because otherwise it would be needed to activly set these flags and most
filesystems that support XATTR and ACL also support
POSIX-Permissions & Ownership.
Signed-off-by: Markus Frank <m.frank@proxmox.com>
If overwrite is true, O_TRUNC is set (to clean the leftovers)
instead of O_EXCL and therefore overwrites the files and
does not error out.
Signed-off-by: Markus Frank <m.frank@proxmox.com>
the remaining ones are:
- type complexity
- fns with many arguments
- new() without default()
- false positives for redundant closures (where closure returns a static
value)
- expected vs actual length check without match/cmp
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
We decided to go this route because it'll most likely be
safer in the API as we need to explicitly add namespaces
support to the various API endpoints this way.
For example, 'pull' should have 2 namespaces: local and
remote, and the GroupFilter (which would otherwise contain
exactly *one* namespace parameter) needs to be applied for
both sides (to decide what to pull from the remote, and what
to *remove* locally as cleanup).
The *datastore* types still contain the namespace and have a
`.backup_ns()` getter.
Note that the datastore's `Display` implementations are no
longer safe to use as a deserializable string.
Additionally, some datastore based methods now have been
exposed via the BackupGroup/BackupDir types to avoid a
"round trip" in code.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Make it easier by adding an helper accepting either group or
directory
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
instead of an async recursive function. Not only is it less code,
recursive futures are not really nice and it should be faster too.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
And use the api-types for their contents.
These are supposed to be instances for a datastore, the pure
specifications are the ones in pbs_api_types which should be
preferred in crates like clients which do not need to deal
with the datastore directly.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
The type is a real enum.
All are API types and implement Display and FromStr. The
ordering is the same as it is in pbs-datastore.
Also, they are now flattened into a few structs instead of
being copied manually.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
