5
0
mirror of git://git.proxmox.com/git/proxmox-backup.git synced 2025-01-21 18:03:59 +03:00

61 Commits

Author SHA1 Message Date
Thomas Lamprecht
a6c07eb9c2 limit ZFS Arc to more smalles possible value
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-11 10:21:49 +01:00
Thomas Lamprecht
c190a32cb1 file restore: list api: fix description of "path"
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-11 10:21:49 +01:00
Thomas Lamprecht
5d0d3c61f9 file restore: switch some formats to inline template variables
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-11 10:21:49 +01:00
Dominik Csapak
72220d79d7 file-restore: dynamically increase memory of vm for zpools
when a backup contains a drive with zfs on it, the default memory
size (up to 384 MiB) is often not enough to hold the zfs metadata

to improve that situation, add memory dynamically (1GiB) when a path is
requested that is on zfs. Note that the image must be started with a
kernel capable of memory hotplug.

to achieve that, we also have to add a qmp socket to the vm, so that
we can later connect and add the memory backend and dimm

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-07 13:29:34 +01:00
Dominik Csapak
3e8ee886a5 file-restore: fix deprecated qemu parameters
server and nowait are deprecated, so we should use the longform:
server=on and wait=off

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-04 13:27:07 +01:00
Wolfgang Bumiller
5c7ff2afd2 bump sys dep to 0.4.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-19 14:26:00 +02:00
Fabian Grünbichler
f097eaa80f clippy fixes
and one additional API fn "allow many parameters" addition.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-10-11 09:36:12 +02:00
Dominik Csapak
ba690c4023 file-restore: remove 'json-error' parameter from list_files
we can reuse the 'output_format' here
also remove the 'error: true' here. we can determine it was an error,
by checking if it's an object with a 'message' property

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-10-05 19:34:00 +02:00
Dominik Csapak
cc900ae2f7 file-restore: add 'format' and 'zstd' parameters to 'extract' command
if the target ist stdout, we can now specify the exact format by making use of
the new 'format' parameter of the restore daemons 'extract' api

note that extracting a pxar from a source pxar (container/host backups)
won't work currently since we would have to reencode as pxar first

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 [ T: fixed missing proxmox-compression dependency ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-05 18:40:51 +02:00
Fabian Grünbichler
f3cfeb20a1 update proxmox-router to 1.3.0
no real change for PBS usage - the ApiHandler enum is marked
non_exhaustive now because it has extra values if the new (enabled by
default) "server" feature is enabled.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-07 14:17:12 +02:00
Fabian Grünbichler
e762aea4e0 build: more missing features
these would cause failures when building the sub-crates directly from
their sub-directory.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-05 12:55:33 +02:00
Wolfgang Bumiller
135aaee2d9 bump proxmox-sys dep to 0.4
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-28 13:47:37 +02:00
Wolfgang Bumiller
3f7440916f file-restore: clippy fixes
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-28 09:55:43 +02:00
Wolfgang Bumiller
b58dd1d413 rustfmt
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-30 10:18:32 +02:00
Wolfgang Bumiller
40e47aa9cf file-restore: fix a hyperlink in docs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-30 10:16:03 +02:00
Wolfgang Bumiller
04dffc3ad7 bump proxmox-sys dep to 0.3.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-29 09:45:11 +02:00
Hannes Laimer
095b3c1c18 proxmox-file-restore: replace print with log macro
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-21 10:43:19 +02:00
Hannes Laimer
955aea8ae8 bins: init cli logger
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-21 10:43:19 +02:00
Wolfgang Bumiller
a3593c890e bump proxmox-router dep to 1.2.4
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-06-21 10:43:19 +02:00
Thomas Lamprecht
74cad4a8bd tree wide: typo fixes through codespell
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-07 14:08:09 +02:00
Thomas Lamprecht
fbfb64a6b2 tree wide: clippy lint fixes
most (not all) where done automatically

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-02 15:59:55 +02:00
Fabian Grünbichler
11ca834317 update to nix 0.24 / rustyline 9 / proxmox-sys 0.3
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-06-02 14:33:33 +02:00
Wolfgang Bumiller
8ca7cccf5f file-restore: add namespace support to qemu part
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-12 13:35:34 +02:00
Wolfgang Bumiller
133d718fe4 split the namespace out of BackupGroup/Dir api types
We decided to go this route because it'll most likely be
safer in the API as we need to explicitly add namespaces
support to the various API endpoints this way.

For example, 'pull' should have 2 namespaces: local and
remote, and the GroupFilter (which would otherwise contain
exactly *one* namespace parameter) needs to be applied for
both sides (to decide what to pull from the remote, and what
to *remove* locally as cleanup).

The *datastore* types still contain the namespace and have a
`.backup_ns()` getter.

Note that the datastore's `Display` implementations are no
longer safe to use as a deserializable string.

Additionally, some datastore based methods now have been
exposed via the BackupGroup/BackupDir types to avoid a
"round trip" in code.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller
8c74349b08 api-types: add namespace to BackupGroup
Make it easier by adding an helper accepting either group or
directory

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Dominik Csapak
6ddd69c5ce file-restore: add 'timeout' and 'json-error' parameter
timeout limits the code with the given timeout in seconds, and
'json-error' return json to stdout when the call returns an error like
this:

{
    "msg": "error message",
    "error": true,
    "code": <HTTP_STATUS_CODE>, // if it was an http error
}

with both options set, a client can more easily determine if the call
ran into a timeout (since it will return a 503 error), and can poll
it again

both is done behind new parameters, so that we can stay backwards-compatible

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-27 19:19:57 +02:00
Dominik Csapak
25be1fa0d7 file-restore: factor out 'list_files'
we'll want to reuse that in a later patch

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-27 19:19:57 +02:00
Wolfgang Bumiller
db87d93efc make datastore BackupGroup/Dir ctors private
And use the api-types for their contents.

These are supposed to be instances for a datastore, the pure
specifications are the ones in pbs_api_types which should be
preferred in crates like clients which do not need to deal
with the datastore directly.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 11:56:23 +02:00
Wolfgang Bumiller
ebb85c1ca3 bump proxmox-schema dependency to 1.3.1 for streaming attribute
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 08:20:27 +02:00
Wolfgang Bumiller
fb6e48f402 bump proxmox-router dependency to 1.2
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 08:17:08 +02:00
Wolfgang Bumiller
38774184a9 tree-wide: replace serde_json::from_value(a_value.clone())
`&Value` itself implements `Deserializer` and can therefore
be passed directly to `T::deserialize` without requiring an
intermediate `clone()`. (This also enables optionally
borrowing strings if the result has a short enough lifetime)

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-12 16:12:15 +02:00
Thomas Lamprecht
40ea990c05 file restore: rust fmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-06 16:57:07 +02:00
Wolfgang Bumiller
b300e6fbc2 use BufReader/Writer for Files passed to serde_json::from_reader/writer
As serde_json will otherwise read files 1 byte at a time.
Writing is a bit better, but syntacitcal elements (quotes, braces,
commas) still often show up as single write syscalls, so use BufWriter
there as well.

Note that while we do store the file in the resulting objects, we do not
need to keep the buffered read/writers as we always `seek` to the
beginning on further file operations.

Reported-by: Mark Schouten <mark@tuxis.nl>
Link: https://lists.proxmox.com/pipermail/pbs-devel/2022-April/004909.html
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-06 16:40:35 +02:00
Wolfgang Bumiller
bd4562e4b1 bump proxmox-schema dep to 1.3
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-04 09:50:21 +01:00
Wolfgang Bumiller
9c890d72b9 bump proxmox-async dep to 0.4
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 14:25:37 +01:00
Wolfgang Bumiller
229c1788c1 bump proxmox-lang dep to 1.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 14:24:24 +01:00
Fabian Grünbichler
dcf5a0f62d misc clippy fixes
the trivial ones ;)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-02-08 14:57:16 +01:00
Thomas Lamprecht
1f8b29f578 file restore: scale per-round delay up dynamically
Avoids latency for restore-VMs that are finished fast but not ready
yet the first round while not checking to often for slower ones, iow,
we assume that the start up distribution is looking like a chi-square
Χ² with k=3.

With 25*round we get at max 45 rounds totalling to 25.875 s delay and
1.125 max between-round delay, which still provides an ok reaction
time.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-01-26 16:12:58 +01:00
Thomas Lamprecht
48ce3d00a4 file restore: always wait up to 25s
the timeout for connecting may be much shorter if we get a response
(which doesn't needs to be Ok, e.g., "Connection refused"), so
instead of trying a fixed amount of 60 times lets try for 25s
independent of how often that will be then.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-01-26 16:04:43 +01:00
Fabian Grünbichler
6f1c26b083 tree-wide: is_ok/is_err()
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 15:02:07 +01:00
Fabian Grünbichler
aa174e8e8a tree-wide: drop redundant clones
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 15:02:07 +01:00
Fabian Grünbichler
9a37bd6c84 tree-wide: fix needless borrows
found and fixed via clippy

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 13:55:33 +01:00
Wolfgang Bumiller
821aa8eae6 bump proxmox-schema to 1.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-12-16 11:25:02 +01:00
Dietmar Maurer
fddb9bcc3e remove pbs-tools/src/sys.rs
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-25 09:01:29 +01:00
Dietmar Maurer
25877d05ac update to proxmox-sys 0.2 crate
- imported pbs-api-types/src/common_regex.rs from old proxmox crate
- use hex crate to generate/parse hex digest
- remove all reference to proxmox crate (use proxmox-sys and
  proxmox-serde instead)

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-24 10:32:27 +01:00
Wolfgang Bumiller
154d01b042 d/control and Cargo.toml bumps
* pin-utils isn't used anymore
* proxmox-sys version should also be tracked in Cargo.toml

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-22 10:56:36 +01:00
Dietmar Maurer
92a8f0bc82 depend on proxmox-async 0.2 2021-11-20 17:14:02 +01:00
Dietmar Maurer
9a1b24b6b1 use new proxmox-async crate
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-19 18:03:22 +01:00
Fabian Grünbichler
daaeea8b4b update to base64 0.13
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-11-19 11:58:17 +01:00
Dietmar Maurer
d5790a9f27 use new proxmox-sys crate
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-19 11:06:35 +01:00