Commit Graph

155 Commits

Author SHA1 Message Date
Christian Ebner
207c0eb470 client: backup writer: fix minor formatting issue
no functional changes

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-04-21 13:28:38 +02:00
Gabriel Goller
52731339c2 pbs-client: fixed typo in error message
Fixed error message on the client: 'dynmamic' -> 'dynamic'.

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2024-04-10 18:08:26 +02:00
Christian Ebner
fcea0794c8 client: backup writer: only borrow http client
Instead of taking ownership of the http client when starting a new
BackupWriter instance, only borrow the client.

This allows to reuse the http client to later reuse it to start also a
BackupReader instance as required for backup runs with metadata based
file change detection mode, where both must use the same http client.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-04-04 10:51:45 +02:00
Fabian Grünbichler
df5854986c fix #5248: client: allow self-signed/untrusted certificate chains
instead of rejecting any non-leaf certificate not pre-validated by OpenSSL,
treat them as valid but keep track of the fact that the pre-validation result
is no logner trustable.

certificate chains completely trusted by openssl are still accepted like
before, and leaf certificates without a chain are also handled the same (since
the verify callback is only ever called with depth == 0 in that case).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-03-26 14:29:34 +01:00
Maximiliano Sandoval
6aff2de5d9 api: use if-let pattern for error-only handling
It is more readable than using match. We also inline variables in
eprintln!.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-25 17:17:12 +01:00
Christian Ebner
f755dc3eaa client: pxar: early return on exclude pattern match
Move the exclude pattern matching further up to avoid unnecessary
instantiation of the metadata object, not needed if the entry was
matched.

No functional change intended.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-02-22 09:03:54 +01:00
Christian Ebner
9052dff2b3 client: pxar: fix minor formatting issues
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-02-22 09:01:27 +01:00
Gabriel Goller
a602a885af fix #4975: client: ignore E2BIG error flag
Some filesystems (f.e. zfs) support xattrs bigger than 64kB, sadly we
can't get them because the kernel vfs limits us. The syscalls listxattr
and getxattr will return a E2BIG error in this case.
Added a flag --ignore-e2big-xattr to the client, this will ignore the
metadata (but still backup the file) if this error occurs.

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-02-15 10:34:10 +01:00
Maximiliano Sandoval
8a95f40add pxar: elide explicit lifetime
Fixes the clippy lint

```
warning: the following explicit lifetimes could be elided: 'b
   --> pbs-client/src/pxar/create.rs:225:33
    |
225 |     fn archive_dir_contents<'a, 'b, T: SeqWrite + Send>(
    |                                 ^^
226 |         &'a mut self,
227 |         encoder: &'a mut Encoder<'b, T>,
    |                                  ^^
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_lifetimes
    = note: `#[warn(clippy::needless_lifetimes)]` on by default
help: elide the lifetimes
    |
225 ~     fn archive_dir_contents<'a, T: SeqWrite + Send>(
226 |         &'a mut self,
227 ~         encoder: &'a mut Encoder<'_, T>,
    |
```

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-02-13 10:21:35 +01:00
Maximiliano Sandoval
5251bf89ba remove redundant guards
Fixes the clippy lint:

```
warning: redundant guard
   --> pbs-datastore/src/chunk_store.rs:325:37
    |
325 |                     Err(ref err) if err == &nix::errno::Errno::ENOENT => {
    |                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#redundant_guards
    = note: `#[warn(clippy::redundant_guards)]` on by default
help: try
    |
325 -                     Err(ref err) if err == &nix::errno::Errno::ENOENT => {
325 +                     Err(nix::errno::Errno::ENOENT) => {
    |
```

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-02-13 10:05:39 +01:00
Wolfgang Bumiller
a9708ec84f client: drop unused code
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-01-19 10:56:46 +01:00
Thomas Lamprecht
6685122c3b tree-wide: fix various typos
found with codespell

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-11-29 18:32:07 +01:00
Wolfgang Bumiller
10d77d9712 fix variables not needing mut warnings
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-11-29 15:34:37 +01:00
Maximiliano Sandoval R
2711e94e3a use IsTerminal trait whenever possible
Continuation of
https://lists.proxmox.com/pipermail/pbs-devel/2023-November/007078.html.

Signed-off-by: Maximiliano Sandoval R <m.sandoval@proxmox.com>
2023-11-27 13:21:45 +01:00
Hannes Laimer
09683f1290 accept a ref to a HttpClient
... since the functions don't actually need to own the value.

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Tested-by: Gabriel Goller <g.goller@proxmox.com>
2023-11-25 17:07:42 +01:00
Max Carrara
722b7bf7ac fix #4779: client: add missing "Connection" header for HTTP2 upgrade
This commit adds the missing "Connection: upgrade" HTTP header [1]
when requesting an upgrade to HTTP 2.
Doing so is mandated in the HTTP Semantics RFC [2], and without this,
(reverse) proxies that strictly follow the standard could potentially
break.

[1]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Upgrade
[2]: RFC 9110, 7.8. Upgrade: “[...] sender of Upgrade MUST also send
     an "Upgrade" connection option in the Connection header [...]”

Reported-By: McTwist <rajb89@hotmail.com>
Signed-off-by: Max Carrara <m.carrara@proxmox.com>
 [ TL: added RFC reference and use case to commit message ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-10-19 16:47:49 +02:00
Thomas Lamprecht
c5e54a5dca client: task polling: rework code to be slightly more readable
the match-arms let one follow the different branches easier than the
relatively crowded if-condition it replaces.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-09-27 18:14:59 +02:00
Gabriel Goller
af7b07479b client: do exit with failure code when task finished with warnings
Warnings in the task log/state normally means that the task actually
did its main job, but there was some detected (potential) issue that
the users should be made aware of. Exiting with an error code in that
case would be a bit odd.

While just exiting with success might not be the best solution either,
it's definitively more correct than a failure-exit-code, so go for
that for now as a stop-gap.

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
 [ TL: rebased on current master (v3 was already applied) and rewrite
   commit message accordingly ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-09-27 18:09:40 +02:00
Gabriel Goller
bc0735fee7 fix #4343: updated view_task_result to bail on task failure
Now we make an additional request on `api2/json/.../tasks/{upid}/status` to
get the `exitstatus` of the task. This allows us to `bail` and thus
get a non-zero exit code in the cli.

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2023-09-18 16:40:02 +02:00
Wolfgang Bumiller
4f133ceef7 minor style cleanup
The match condition has gotten a bit large, and the error case is a
bit more concise with a pattern guard.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-08-24 10:35:05 +02:00
Wolfgang Bumiller
7672150c4c cleanup: drop unnecessary to_owned call
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-08-24 10:28:01 +02:00
Gabriel Goller
b65908fac2 fix #4380: check if file is excluded before running stat()
Passed a closure with the `stat()` function call to `matches()`. This
will traverse through all patterns and try to match using the path only, if a
`file_mode` is needed, it will run the closure. This means that if we exclude
a file with the `MatchType::ANY_FILE_TYPE`, we will skip it without running
`stat()` on it. As we updated the `matches()` function, we also updated all the
invocations of it.
Added `pathpatterns` crate to local overrides in cargo.toml.

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2023-08-24 10:17:41 +02:00
Wolfgang Bumiller
a9f5360b82 shorten code a tiny bit
Ends up with a rather long pattern guard but does avoid the return
case duplication...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-08-17 14:05:16 +02:00
Christian Ebner
70bca12324 fix: #4761: introduce overwrite bitflags for fine grained overwrites
Adds OverwriteFlags for granular control of which entry types should
overwrite entries present on the filesystem during a restore.

The original overwrite flag is refactored in order to cover all of the
other cases.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2023-08-17 14:00:44 +02:00
Christian Ebner
e22da2f40a fix: #4761: unlink existing entries for hard/symlinks when overwrite
Creating symlinks or hardlinks might fail if a directory entry with the
same name is already present on the filesystem during restore.

When the overwrite flag is given, on failure unlink the existing entry
(except directories) and retry hard/symlink creation.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2023-08-17 14:00:43 +02:00
Fabian Grünbichler
66215b38f8 blob/chunk parse errors: add context
to make it more obvious that blob is a chunk here and which one is affected.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-08-08 13:57:04 +02:00
Fabian Grünbichler
7d8d4061c5 Revert "fix #4380: stat() is run when file is executed"
this actually affected the matcher's ability to differentiate between directory
and file patterns, and the alternative would require matching patterns twice
for full coverage, so let's try a different approach altogether.

This reverts commit c8ed10095d.
2023-08-04 10:33:15 +02:00
Gabriel Goller
c8ed10095d fix #4380: stat() is run when file is executed
When executing `proxmox-backup-client backup ...
--exclude "test/test.txt"` it still executed stat() on "test.txt",
which won't work when the current user doesn't have access to the
file or the parent folder. Now we check if the file is excluded,
and if it is not, then we execute stat().

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2023-08-04 09:30:55 +02:00
Fabian Grünbichler
e61aa41076 backup: improve skipped download error message
context always helps when parsing output..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-07-19 11:06:31 +02:00
Maximiliano Sandoval
03e71cc8be fix #4591: pbs-client: backup_writer: improve error reporting
We check if the manifest contains an index for the requested archive, if
it does not we avoid downloading it and report a more helpful error
message.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2023-07-19 10:49:28 +02:00
Maximiliano Sandoval
288893a6a9 pbs-client: backup-writer: use log::warn instead of eprintln!
The errors are not fatal so we only throw a warning.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2023-07-19 10:49:09 +02:00
Max Carrara
68478bb122 pbs-client: pxar: add PxarExtractContext
This enum's purpose is to provide context to errors that occur during
the extraction of a pxar archive, making it possible to handle
extraction errors in a more granular manner.

For now, it's only implemented in `ExtractorIter::next()`, but may be
used in other places if necessary or desired.

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
2023-07-17 12:00:47 +02:00
Max Carrara
c66f2579d5 pbs-client: pxar: refactor body of extract_archive to ExtractorIter
This change factors the body of `extract_archive()` into a separate
struct named `ExtractorIter` which implements the `Iterator` trait.

This refactor has two goals:
  * Make it easier to provide and propagate errors and additional
    information via `anyhow::Context`
  * Introduce a means to handle errors that occur during extraction,
    with the possibility to continue extraction if the handler decides
    that the error is not fatal

The latter point benefits from the information provided by the former;
previously, errors could only be handled in certain locations
(e.g. application of metadata), but not on a "per-entry" basis.

Since `extract_archive()` was already using a "desugared" version of
the iterator pattern to begin with, wrapping its body up in an actual
`Iterator` made the most sense, as it didn't require changing the already
existing control flow that much.

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
2023-07-17 12:00:46 +02:00
Max Carrara
54ef4f157a pbs-client: pxar: preserve error context
In order to preserve the source(s) of errors, `anyhow::Context` is
used instead of propagating errors via `Result::map_err()` and / or
`anyhow::format_err!()`.

This makes it possible to access e.g. an underlying `io::Error` or
`nix::Errno` etc. that caused an execution path to fail.

Certain usages of `anyhow::bail!()` are also changed / replaced
in order to preserve context.

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
2023-07-17 12:00:44 +02:00
Lukas Wagner
08f8a3e5aa api-types: client: datastore: tools: use proxmox-human-bytes crate
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-06-26 13:56:45 +02:00
Thomas Lamprecht
3db2022713 pbs-client: clippy fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-17 17:43:17 +02:00
Fabian Grünbichler
1e5cd060a0 pxar creation: use log crate for error reporting
gives a higher (runtime) control via PBS_LOG, so that users can decide
themselves which messages, sources and levels are interesting for a particular
use case.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-03-27 11:16:09 +02:00
Fabian Grünbichler
8419f1cfca fix #4578: use log crate for pxar create logging
since proxmox-backup-client is used in cron jobs and similar automated
fashions, PBS_LOG= should control the output..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-03-27 11:16:06 +02:00
Wolfgang Bumiller
d97ff8ae2a use new auth api crate
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-03-02 17:01:35 +01:00
Wolfgang Bumiller
5aeeb44a32 adapt to rest-server 0.3 and http 0.8 changes
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-03-02 15:15:36 +01:00
Thomas Lamprecht
c4f3677957 client: task log: use fstrings for shorter code
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-01-19 11:37:24 +01:00
Thomas Lamprecht
22fc132a73 client: task log: only encode upid once
Instead of percent-encoding the UPID on every loop iteration plus on
abort, just encode it once before entering the loop.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-01-19 10:38:19 +01:00
Fabian Grünbichler
158db8c01f fix #4483: fix task log command interrupt handling
`proxmox-backup-client task log ..` and `proxmox-backup-manager task log ..`
are used to view the logs of tasks that have been started by another client, so
interrupting the task progress view should not forward the interrupt to the
running task. other call sites of the same helper(s) that spawn a task and then
print its progress should keep the old behaviour of interrupting the spawned
task on C^c.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-01-19 10:11:25 +01:00
Fiona Ebner
73809d55b7 http client: suppress "storing login ticket"-error when not using a TTY
as a stop-gap measure. Otherwise, task logs for PVE backups started
via non-CLI will have the message
> storing login ticket failed: $XDG_RUNTIME_DIR must be set
show up when running a proxmox-backup-client command (e.g. setting
notes and when uploading the log). This is confusing to users[0].

[0]: https://forum.proxmox.com/threads/120492/

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-01-05 12:29:13 +01:00
Hannes Laimer
2b8b5d0fb3 fix #4387: pbs-client: print task-logs to stdout
... since those logs are the result of the command.
2022-12-20 09:26:56 +01:00
Christoph Heiss
70b22b624d fix #4346: pbs-client: don't fail silently when storing ticket
Instead, report an error if storing the ticket info failed, so that the
user is informed that something went wrong and follow-up commands might
require authentication again.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
2022-12-13 14:32:56 +01:00
Wolfgang Bumiller
f72ccdd65d move pbs_tools::ticket to pbs_ticket
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-12-13 13:58:09 +01:00
Fabian Grünbichler
21d4a68255 sort dependencies
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-12 09:08:56 +01:00
Fabian Grünbichler
1c447cb881 switch remaining member dependencies to workspace
these are only used by a single member at the moment, but we can move them to
the workspace to have a single location for version + base feature set
specification.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-12 09:08:36 +01:00
Fabian Grünbichler
781294e4b5 switch regular dependencies to workspace ones
where applicable.

notable changes:
- serde now uses 'derive' feature across the board
- serde removed from pbs-tools (not used)
- openssl bumped to 0.40 (and patched comment removed)
- removed invalid zstd comment

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-12 09:07:12 +01:00