134 Commits

Author SHA1 Message Date
Gabriel Goller
b65908fac2 fix #4380: check if file is excluded before running stat()
Passed a closure with the `stat()` function call to `matches()`. This
will traverse through all patterns and try to match using the path only, if a
`file_mode` is needed, it will run the closure. This means that if we exclude
a file with the `MatchType::ANY_FILE_TYPE`, we will skip it without running
`stat()` on it. As we updated the `matches()` function, we also updated all the
invocations of it.
Added `pathpatterns` crate to local overrides in cargo.toml.

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2023-08-24 10:17:41 +02:00
Wolfgang Bumiller
a9f5360b82 shorten code a tiny bit
Ends up with a rather long pattern guard but does avoid the return
case duplication...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-08-17 14:05:16 +02:00
Christian Ebner
70bca12324 fix: #4761: introduce overwrite bitflags for fine grained overwrites
Adds OverwriteFlags for granular control of which entry types should
overwrite entries present on the filesystem during a restore.

The original overwrite flag is refactored in order to cover all of the
other cases.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2023-08-17 14:00:44 +02:00
Christian Ebner
e22da2f40a fix: #4761: unlink existing entries for hard/symlinks when overwrite
Creating symlinks or hardlinks might fail if a directory entry with the
same name is already present on the filesystem during restore.

When the overwrite flag is given, on failure unlink the existing entry
(except directories) and retry hard/symlink creation.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2023-08-17 14:00:43 +02:00
Fabian Grünbichler
66215b38f8 blob/chunk parse errors: add context
to make it more obvious that blob is a chunk here and which one is affected.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-08-08 13:57:04 +02:00
Fabian Grünbichler
7d8d4061c5 Revert "fix #4380: stat() is run when file is executed"
this actually affected the matcher's ability to differentiate between directory
and file patterns, and the alternative would require matching patterns twice
for full coverage, so let's try a different approach altogether.

This reverts commit c8ed10095d4cdc01705e5f01f849f8850a0b7395.
2023-08-04 10:33:15 +02:00
Gabriel Goller
c8ed10095d fix #4380: stat() is run when file is executed
When executing `proxmox-backup-client backup ...
--exclude "test/test.txt"` it still executed stat() on "test.txt",
which won't work when the current user doesn't have access to the
file or the parent folder. Now we check if the file is excluded,
and if it is not, then we execute stat().

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2023-08-04 09:30:55 +02:00
Fabian Grünbichler
e61aa41076 backup: improve skipped download error message
context always helps when parsing output..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-07-19 11:06:31 +02:00
Maximiliano Sandoval
03e71cc8be fix #4591: pbs-client: backup_writer: improve error reporting
We check if the manifest contains an index for the requested archive, if
it does not we avoid downloading it and report a more helpful error
message.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2023-07-19 10:49:28 +02:00
Maximiliano Sandoval
288893a6a9 pbs-client: backup-writer: use log::warn instead of eprintln!
The errors are not fatal so we only throw a warning.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2023-07-19 10:49:09 +02:00
Max Carrara
68478bb122 pbs-client: pxar: add PxarExtractContext
This enum's purpose is to provide context to errors that occur during
the extraction of a pxar archive, making it possible to handle
extraction errors in a more granular manner.

For now, it's only implemented in `ExtractorIter::next()`, but may be
used in other places if necessary or desired.

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
2023-07-17 12:00:47 +02:00
Max Carrara
c66f2579d5 pbs-client: pxar: refactor body of extract_archive to ExtractorIter
This change factors the body of `extract_archive()` into a separate
struct named `ExtractorIter` which implements the `Iterator` trait.

This refactor has two goals:
  * Make it easier to provide and propagate errors and additional
    information via `anyhow::Context`
  * Introduce a means to handle errors that occur during extraction,
    with the possibility to continue extraction if the handler decides
    that the error is not fatal

The latter point benefits from the information provided by the former;
previously, errors could only be handled in certain locations
(e.g. application of metadata), but not on a "per-entry" basis.

Since `extract_archive()` was already using a "desugared" version of
the iterator pattern to begin with, wrapping its body up in an actual
`Iterator` made the most sense, as it didn't require changing the already
existing control flow that much.

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
2023-07-17 12:00:46 +02:00
Max Carrara
54ef4f157a pbs-client: pxar: preserve error context
In order to preserve the source(s) of errors, `anyhow::Context` is
used instead of propagating errors via `Result::map_err()` and / or
`anyhow::format_err!()`.

This makes it possible to access e.g. an underlying `io::Error` or
`nix::Errno` etc. that caused an execution path to fail.

Certain usages of `anyhow::bail!()` are also changed / replaced
in order to preserve context.

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
2023-07-17 12:00:44 +02:00
Lukas Wagner
08f8a3e5aa api-types: client: datastore: tools: use proxmox-human-bytes crate
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-06-26 13:56:45 +02:00
Thomas Lamprecht
3db2022713 pbs-client: clippy fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-05-17 17:43:17 +02:00
Fabian Grünbichler
1e5cd060a0 pxar creation: use log crate for error reporting
gives a higher (runtime) control via PBS_LOG, so that users can decide
themselves which messages, sources and levels are interesting for a particular
use case.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-03-27 11:16:09 +02:00
Fabian Grünbichler
8419f1cfca fix #4578: use log crate for pxar create logging
since proxmox-backup-client is used in cron jobs and similar automated
fashions, PBS_LOG= should control the output..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-03-27 11:16:06 +02:00
Wolfgang Bumiller
d97ff8ae2a use new auth api crate
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-03-02 17:01:35 +01:00
Wolfgang Bumiller
5aeeb44a32 adapt to rest-server 0.3 and http 0.8 changes
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-03-02 15:15:36 +01:00
Thomas Lamprecht
c4f3677957 client: task log: use fstrings for shorter code
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-01-19 11:37:24 +01:00
Thomas Lamprecht
22fc132a73 client: task log: only encode upid once
Instead of percent-encoding the UPID on every loop iteration plus on
abort, just encode it once before entering the loop.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2023-01-19 10:38:19 +01:00
Fabian Grünbichler
158db8c01f fix #4483: fix task log command interrupt handling
`proxmox-backup-client task log ..` and `proxmox-backup-manager task log ..`
are used to view the logs of tasks that have been started by another client, so
interrupting the task progress view should not forward the interrupt to the
running task. other call sites of the same helper(s) that spawn a task and then
print its progress should keep the old behaviour of interrupting the spawned
task on C^c.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-01-19 10:11:25 +01:00
Fiona Ebner
73809d55b7 http client: suppress "storing login ticket"-error when not using a TTY
as a stop-gap measure. Otherwise, task logs for PVE backups started
via non-CLI will have the message
> storing login ticket failed: $XDG_RUNTIME_DIR must be set
show up when running a proxmox-backup-client command (e.g. setting
notes and when uploading the log). This is confusing to users[0].

[0]: https://forum.proxmox.com/threads/120492/

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-01-05 12:29:13 +01:00
Hannes Laimer
2b8b5d0fb3 fix #4387: pbs-client: print task-logs to stdout
... since those logs are the result of the command.
2022-12-20 09:26:56 +01:00
Christoph Heiss
70b22b624d fix #4346: pbs-client: don't fail silently when storing ticket
Instead, report an error if storing the ticket info failed, so that the
user is informed that something went wrong and follow-up commands might
require authentication again.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
2022-12-13 14:32:56 +01:00
Wolfgang Bumiller
f72ccdd65d move pbs_tools::ticket to pbs_ticket
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-12-13 13:58:09 +01:00
Fabian Grünbichler
21d4a68255 sort dependencies
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-12 09:08:56 +01:00
Fabian Grünbichler
1c447cb881 switch remaining member dependencies to workspace
these are only used by a single member at the moment, but we can move them to
the workspace to have a single location for version + base feature set
specification.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-12 09:08:36 +01:00
Fabian Grünbichler
781294e4b5 switch regular dependencies to workspace ones
where applicable.

notable changes:
- serde now uses 'derive' feature across the board
- serde removed from pbs-tools (not used)
- openssl bumped to 0.40 (and patched comment removed)
- removed invalid zstd comment

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-12 09:07:12 +01:00
Fabian Grünbichler
1fbdf8ef79 switch proxmox dependencies to workspace
besides harmonizing versions, the only global change is that the tokio-io
feature of pxar is now implied since its default anyway, instead of being
spelled out.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-12 09:05:27 +01:00
Fabian Grünbichler
cb33ed9ca5 workspace: inherit metadata
pbs-buildcfg is the only one that needs to inherit the version as well, since
it stores it in the compiled crate.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-12 09:05:27 +01:00
Wolfgang Bumiller
d9f1ca9a46 tree-wide: bump edition to 2021
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-12-06 13:31:01 +01:00
Wolfgang Bumiller
d701808514 move fuse code from pbs-client to pbs-pxar-fuse
it's used by pxar-bin and proxmox-backup-client for
mounting, but pbs-client is used by more (eg. the
proxmox-backup-qemu library which really doesn't need to
pull in any fuse dependencies)

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-12-06 09:17:04 +01:00
Fabian Grünbichler
16f6766a68 clippy 1.65 fixes
and rustfmt

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-12-05 11:40:02 +01:00
Wolfgang Bumiller
593aa85a9e bump pxar dependency to 0.10.2
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-23 15:47:07 +01:00
Fabian Grünbichler
4b3c010fd5 clippy fixes
the dropped .into() is guarded by the bumped build-dependency on
proxmox-sys 0.4.1, the missing Eq is a new clippy lint.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-10-24 12:10:19 +02:00
Lukas Wagner
de7f180dab pbs-client: fix bash autocompletion for archive-files
Previously, autocompletion of archive names, for instance
in the case of

  $ proxmox-backup-client restore <snapshot> <TAB>

did not work if no namespace was provided via the --ns option.
The fix is to fall back to the root namespace if the option is
not provided by the user.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2022-10-20 13:20:02 +02:00
Wolfgang Bumiller
a616c3dfd2 pbs-client: use new non-Fd open from sys 0.4.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-19 14:28:10 +02:00
Wolfgang Bumiller
5c7ff2afd2 bump sys dep to 0.4.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-19 14:26:00 +02:00
Wolfgang Bumiller
76c2912e64 pbs-client: drop use of deprecated RawFdNum
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-19 14:26:00 +02:00
Wolfgang Bumiller
4cceaf3a2d pbs-client: drop deprecated Fd usage
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-19 13:51:13 +02:00
Wolfgang Bumiller
a63a869e07 client: use BorrowedFd from std
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-19 11:00:36 +02:00
Fabian Grünbichler
237beaed93 cargo fmt
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-10-11 09:37:16 +02:00
Stefan Hanreich
fc65ec4345 fix #4095: make http client use proxy config from 'ALL_PROXY' env var
In order to be able to use a proxy with the proxmox-backup-client, use
ProxyConfig for parsing proxy server config from the environment. Also
added a section in the documentation that describes how to configure the
environment if a proxy server should be used.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-09-16 13:28:24 +02:00
Fabian Grünbichler
f3cfeb20a1 update proxmox-router to 1.3.0
no real change for PBS usage - the ApiHandler enum is marked
non_exhaustive now because it has extra values if the new (enabled by
default) "server" feature is enabled.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-07 14:17:12 +02:00
Fabian Grünbichler
951cbb96a2 update to proxmox-http 0.7
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-07 14:17:00 +02:00
Dominik Csapak
6171c72778 backup-client: mount: fix read of larger files
fuse_lowlevel.h says about read:

 Read should send exactly the number of bytes requested except
 on EOF or error, otherwise the rest of the data will be
 substituted with zeroes.

but we simply forwarded the bytes we got from 'read_at'. The result was
that files were corrupt as soon as read_at returned not the exact number
of bytes requested. such short reads are easy to trigger with large
files (where reading a file has to cross many chunk boundaries).

To fix that, loop over 'read_at' until our buffer is full, or we read
0 bytes, indicating EOF.

reported in the forum:
https://forum.proxmox.com/threads/proxmox-backup-client-mounting-a-pxar-archive-gives-truncated-files.114447/

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>

Added comment, reworded slightly

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-05 15:16:42 +02:00
Fabian Grünbichler
fe2213c986 use short assign-bit-or (clippy fix)
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-08-22 13:15:53 +02:00
Wolfgang Bumiller
2a23675db3 rustfmt
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-08-19 13:38:46 +02:00
Markus Frank
e90d5401ff pbs-client: added options to skip acls/xattrs/ownership/permissions
Also added WITH_OWNER and WITH_PERMISSION to Default-Flags,
because otherwise it would be needed to activly set these flags and most
filesystems that support XATTR and ACL also support
POSIX-Permissions & Ownership.

Signed-off-by: Markus Frank <m.frank@proxmox.com>
2022-08-19 13:36:40 +02:00