Commit Graph

6802 Commits

Author SHA1 Message Date
Dominik Csapak
389f8c134d docs: add documentation about the 'sync-level' tuning
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-10-28 13:04:22 +02:00
Dominik Csapak
8d5039ed57 datastore: improve sync level code a bit
fixups for DatastoreFSyncLevel:
* use derive for Default
* add some more derives (Clone, Copy)

chunk store:
* drop to_owned for chunk_dir_path

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-10-28 13:04:22 +02:00
Fabian Grünbichler
4b3c010fd5 clippy fixes
the dropped .into() is guarded by the bumped build-dependency on
proxmox-sys 0.4.1, the missing Eq is a new clippy lint.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-10-24 12:10:19 +02:00
Aaron Lauterer
fcc8e35499 report: add arcstat
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2022-10-21 17:32:02 +02:00
Stefan Hanreich
3c78a1dec2 fix #4301: correctly pass rate limit parameters to API
With the old code the rate limit parameters got passed in their own
dictionary under the limit key, but the API expects the rate-limit
settings as top-level keys. This commit correctly sets the rate-limit
parameters so the API actually uses them.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
2022-10-20 16:06:09 +02:00
Dominik Csapak
647186ddf9 datastore: implement sync-level tuning for datastores
currently, we don't (f)sync on chunk insertion (or at any point after
that), which can lead to broken chunks in case of e.g. an unexpected
powerloss. To fix that, offer a tuning option for datastores that
controls the level of syncs it does:

* None (default): same as current state, no (f)syncs done at any point
* Filesystem: at the end of a backup, the datastore issues
  a syncfs(2) to the filesystem of the datastore
* File: issues an fsync on each chunk as they get inserted
  (using our 'replace_file' helper) and a fsync on the directory handle

a small benchmark showed the following (times in mm:ss):
setup: virtual pbs, 4 cores, 8GiB memory, ext4 on spinner

size                none    filesystem  file
2GiB (fits in ram)   00:13   0:41        01:00
33GiB                05:21   05:31       13:45

so if the backup fits in memory, there is a large difference between all
of the modes (expected), but as soon as it exceeds the memory size,
the difference between not syncing and syncing the fs at the end becomes
much smaller.

i also tested on an nvme, but there the syncs basically made no difference

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-10-20 14:59:15 +02:00
Dominik Csapak
37495b8eeb pbs-datastore: chunk_store: use replace_file in insert_chunk
it does the same as the current code

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-10-20 14:59:13 +02:00
Dominik Csapak
d2c79c54c2 docs: add information about chunk order option for datastores
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-10-20 14:59:11 +02:00
Dominik Csapak
f963bc0c81 proxmox-tape: inventory: add default to parameters
and convert the 'Option<bool>' to 'bool'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-10-20 13:22:47 +02:00
Dominik Csapak
57ee4e9ea4 ui: tape/ChangerStatus: adding parameter selection to inventory
namely 'catalog' and 'read-all-labels', by always opening a
window (with a drive now autoselected) and the two checkboxes

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-10-20 13:22:46 +02:00
Dominik Csapak
4ebc97ad4b proxmox-tape: add 'catalog' option to 'proxmox-tape inventory'
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-10-20 13:22:43 +02:00
Dominik Csapak
c658ea6185 api/tape/inventory: optionally try to restore catalogs
in a disaster recovery case, it is useful to not only re-invetorize
the labels + media-sets, but also to try to recover the catalogs
from the tape (to know whats on there). This adds an option to
the inventory api call that tries to do a fast catalog restore
from each tape to be inventorized.

also sets the correct default for 'read-all-labels' in the api and
converts to a bool

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-10-20 13:22:42 +02:00
Dominik Csapak
3921deb29a tape: replace '&Path' with 'AsRef<Path>' in function parameters
this way we can omit the pattern
```
let status_path = Path::new(TAPE_STATUS_DIR);
some_function(status_path);
```
and give the TAPE_STATUS_DIR directly. In some instances we now have to
give TAPE_STATUS_DIR more often, but most often we save a few
intermediary Paths.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-10-20 13:22:39 +02:00
Lukas Wagner
de7f180dab pbs-client: fix bash autocompletion for archive-files
Previously, autocompletion of archive names, for instance
in the case of

  $ proxmox-backup-client restore <snapshot> <TAB>

did not work if no namespace was provided via the --ns option.
The fix is to fall back to the root namespace if the option is
not provided by the user.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2022-10-20 13:20:02 +02:00
Wolfgang Bumiller
a616c3dfd2 pbs-client: use new non-Fd open from sys 0.4.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-19 14:28:10 +02:00
Wolfgang Bumiller
6cffe7229f bump d/control
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-19 14:26:00 +02:00
Wolfgang Bumiller
5c7ff2afd2 bump sys dep to 0.4.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-19 14:26:00 +02:00
Wolfgang Bumiller
a61d316c2d client: drop deprecated Fd usage
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-19 14:26:00 +02:00
Wolfgang Bumiller
76c2912e64 pbs-client: drop use of deprecated RawFdNum
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-19 14:26:00 +02:00
Wolfgang Bumiller
4cceaf3a2d pbs-client: drop deprecated Fd usage
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-19 13:51:13 +02:00
Wolfgang Bumiller
61dd5af5aa config: drop deprecated Fd usage
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-19 13:45:53 +02:00
Wolfgang Bumiller
a63a869e07 client: use BorrowedFd from std
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-19 11:00:36 +02:00
Wolfgang Bumiller
62c948a8d9 bump edition in rustfmt.toml
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-13 15:01:11 +02:00
Thomas Lamprecht
8201329a63 cargo: rrd: set license in subcrate too
in preparation of moving this out

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-11 15:46:30 +02:00
Thomas Lamprecht
938dd9278e move proxmox-rest-server to proxmox-rs as separate package
Lives now in the common proxmox rs repo[0] for better reuse with
other projects.

[0]: https://git.proxmox.com/?p=proxmox.git;a=tree;f=proxmox-rest-server;h=8035b65a00271604c229590d0109aba3f75ee784

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-11 15:31:39 +02:00
Dominik Csapak
064a9a6bb1 ui: node certificate: make parent panel scrollable
The `ACME` panel may be rendered out of sight (overflow) when there
are enough SANs on the upper `Certificates` panel to push it down
enough and the browser/display height is too small.

Enable the `scrollable` config for the parent certificate view panel
in the y-axis (vertical) to make ExtJS automatically add a scrollbar
if the content overflows. Avoid enabling it for the x-axis too as
that breaks reflow of the layout if the browser window gets resized.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 [T: fix commit message to one from manager ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-11 11:27:18 +02:00
Fabian Grünbichler
237beaed93 cargo fmt
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-10-11 09:37:16 +02:00
Fabian Grünbichler
f097eaa80f clippy fixes
and one additional API fn "allow many parameters" addition.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-10-11 09:36:12 +02:00
Thomas Lamprecht
c36c901172 prune job: various cleanups, line reduction
no semantic change intended

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-07 15:39:57 +02:00
Thomas Lamprecht
97184e14b4 prune job: avoid duplicate code when assembling CLI keep options
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-07 15:37:32 +02:00
Dominik Csapak
cf91a072ef fix #4274: implement prune notifications
we converted the prune settings of datastores to prune-jobs, but did
not actually implement the notifications for them, even though
we had the notification options in the gui (they did not work).

implement the basic ok/error notification for prune jobs

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-10-07 08:35:56 +02:00
Daniel Tschlatscher
bd25aaed7a Remove BackupFileDownloader.js file and Makefile entry
While finishing up my previous patch series I stumbled upon this file
which is not referenced anywhere in the proxmox-backup repository nor
in any of the other repositories as far as I can tell (except in the
Makefile).
I searched far and wide for any occurences and tested the backup
explorer GUI without it, which still seems to work normally.

Introduced in: 98425309b0
References removed in: 3e395378bc

Therefore I think this can be safely considered dead code and be
removed. It shall remain in the pve-devel archives forevermore anyway.

Signed-off-by: Daniel Tschlatscher <d.tschlatscher@proxmox.com>
2022-10-05 19:35:25 +02:00
Dominik Csapak
ba690c4023 file-restore: remove 'json-error' parameter from list_files
we can reuse the 'output_format' here
also remove the 'error: true' here. we can determine it was an error,
by checking if it's an object with a 'message' property

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-10-05 19:34:00 +02:00
Dominik Csapak
1053a71e81 tape/inventory: make 'load_media_db' a method
and use self.inventory_path. This is only used internally (not pub) so there
is no need to have it as a static function.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-10-05 19:26:36 +02:00
Dylan Whyte
6481fd2498 docs: language fixup
This fixup covers every doc patch since my previous language fixup patch.

Note: not much attention was paid to certificate-management, as it's
derived from pmg, which I had touched up not so long ago.

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
2022-10-05 19:23:17 +02:00
Thomas Lamprecht
7c1029d760 bump version to 2.2.7-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-05 18:59:54 +02:00
Dominik Csapak
cc900ae2f7 file-restore: add 'format' and 'zstd' parameters to 'extract' command
if the target ist stdout, we can now specify the exact format by making use of
the new 'format' parameter of the restore daemons 'extract' api

note that extracting a pxar from a source pxar (container/host backups)
won't work currently since we would have to reencode as pxar first

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 [ T: fixed missing proxmox-compression dependency ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-05 18:40:51 +02:00
Dominik Csapak
702ff41471 restore-daemon: add 'format' and 'zstd' parameters to the 'extract' handler
'format' can be 'plain', 'pxar', 'zip' or 'tar',  and it returns the
content in the given format (with fallback to the old behaviour if not
given)

the 'zstd' denotes if the output should be zstd compressed

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-05 18:40:49 +02:00
Dominik Csapak
1bbf0cd467 pbs-api-types: add FileRestoreFormat type
intended for passing the format to the file-restore client/daemon

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-05 18:40:49 +02:00
Thomas Lamprecht
ed4466d840 prune sim: shorten code for weekdayFlags computation
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-05 16:57:40 +02:00
Thomas Lamprecht
5ad65cc252 prune sim: mask simulation options during update
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-05 16:56:51 +02:00
Thomas Lamprecht
922812ed37 prune sim: style nit
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-05 14:38:59 +02:00
Thomas Lamprecht
c4fe6c1f35 docs: prune sim: explicitly document time range options
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-05 13:18:13 +02:00
Thomas Lamprecht
87cbd8c48a prune sim: rework simulation options layout
move the two columns to a field set each, one for the backup job
config, containing weekdays and schedule, and one for the simulation
time, which contains the end date/time and the duration, which got
moved over, as it better fits along the new date/time.

Note that I used end instead of start as its the later point in time,
so hopefully less confusing, but one could argue either.

Flex and width of the fields got adapted such that they now better
work on smaller resolutions (< 1500px total width) and look better
on both, low and high resolutions/width.

The button got moved into the docked area  mostly due to me being to
lazy to add yet another container indirection for having the hbox
nested inside a vbox, but also as it makes sense to have it big and
centered available.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-05 13:16:22 +02:00
Thomas Lamprecht
3ac9b04a26 docs: prune sim: add h4 to h6 style in css
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-05 13:07:13 +02:00
Thomas Lamprecht
bb044304aa prune sim: move PruneList to more static declaration
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-10-04 16:32:18 +02:00
Matthias Heiserer
f44e2386df prune-simulator: allow setting a custom date/time
With this, the prune simulator will be more flexible, which might clear
up questions such as these:
https://forum.proxmox.com/threads/pbs-prune-simulator-monthly-backups.115081

In order to have the configuration window not take up too much space,
I added another column and moved the border to the left.

Signed-off-by: Matthias Heiserer <m.heiserer@proxmox.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
Reviewed-by: Fiona Ebner <f.ebner@proxmox.com>
2022-09-23 12:32:28 +02:00
Thomas Lamprecht
6088459570 api: disk SMART: fix details for depreacated return value comment
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-09-23 10:33:17 +02:00
Matthias Heiserer
500014e83b fix #4165: SMART: add raw field
This makes it consistent with the naming scheme in PVE/GUI.
Keep value for API stability reasons, and remove it in next major version.

Signed-off-by: Matthias Heiserer <m.heiserer@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Dominik Csapak <d.cspak@proxmox.com>
2022-09-23 10:32:30 +02:00
Fabian Grünbichler
37cfd7b336 daily update: fix subscription check
this wrongly triggered for non-subscribed systems as well.

Fixes: dd16e1dac8
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-09-23 07:58:17 +02:00