5
0
mirror of git://git.proxmox.com/git/proxmox-backup.git synced 2025-01-07 17:18:03 +03:00
Commit Graph

6869 Commits

Author SHA1 Message Date
Thomas Lamprecht
b287a702b7 d/postinst: try to fixup installations with wrong .datastore.lck owner
Must be a rather ancient installation in that case as we create
new lockfiles with backup:backup since a while.

Note that this wasn't all to relevant until recently when we fixed a
TOCTOU for maintenance mode check by locking on datastore create,
which happens in the unpriv. proxy daemon too, e.g., for the
datastore status call.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-28 17:05:28 +01:00
Thomas Lamprecht
551890c8a0 bump version to 2.3.0-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-28 16:11:21 +01:00
Thomas Lamprecht
0ecd7ca107 docs: ransomware: drop extra 'd'
Reported-by: Daniel Tschlatscher <d.tschlatscher@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-28 16:02:27 +01:00
Stefan Sterz
37c64a4a96 docs: add paragraph on verification jobs to ransomware section
Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2022-11-28 16:02:15 +01:00
Stefan Sterz
b27d0f8235 docs: minor re-phrasing and spell checking clean up
Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2022-11-28 16:00:57 +01:00
Dominik Csapak
7faf4b6220 docs: don't mention CLI only anymore for tuning options
since we now have those on the ui

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-28 15:59:55 +01:00
Dominik Csapak
c40a2f8bcc use derive 'Default' for ChunkOrder
instead of hardcoding the default deep inside the code. This makes it
much easier to see what is the actual default

the first instance of ChunkOrder::None was only for the test case, were
the ordering doe not matter

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-28 15:59:55 +01:00
Dominik Csapak
8e8e603cdc ui: datastore tuning: show correct default option
the default in the backend is really 'inode' not 'none', so show that

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-28 15:59:55 +01:00
Thomas Lamprecht
ef4956f5aa ui: datastore: override maintenance icon if currently destroying
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-28 15:56:39 +01:00
Thomas Lamprecht
834bae5575 ui: datastore: allow to destroy all underlying data on removal
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-28 15:56:39 +01:00
Thomas Lamprecht
4fbec4c02a ui: datastore: factor out safe datastore destroy
will extend it for data-destroy flag in a next commit

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-28 15:56:39 +01:00
Wolfgang Bumiller
857f346c22 fix #3335: allow removing datastore contents on delete
Adds an optional 'destroy-data' parameter to the datastore
remove api call.

Based-on: https://lists.proxmox.com/pipermail/pbs-devel/2022-January/004574.html
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-28 14:45:39 +01:00
Wolfgang Bumiller
b9f76a427e api-types: add MaintenanceType::Delete
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-28 14:45:39 +01:00
Wolfgang Bumiller
8729f0a549 api-types: derive Display and FromStr for MaintenanceType
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-28 14:45:39 +01:00
Wolfgang Bumiller
00f4e6bbd3 api-types: make Operation Eq
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-28 14:45:39 +01:00
Thomas Lamprecht
e198127e4c ui: prune job edit: set correct onlineHelp reference
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-28 14:43:13 +01:00
Thomas Lamprecht
89fc8d2d71 docs: add section for prune jobs
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-28 14:42:45 +01:00
Thomas Lamprecht
e59eb74a8b docs: update disks, datastore prune/GC and datastore options screenshots
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-28 14:16:24 +01:00
Thomas Lamprecht
8f373d0718 ui: datastore options: use new onSetValues over setValues
The `onSetValues` (pedant to `onGetValues`) got added relatively
recently to widget toolkits input panel. Using it avoids the need to
hard-code some "dynamic override" calling semantics.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-28 14:00:18 +01:00
Thomas Lamprecht
85f1419313 ui: datastore options: code style line reduction
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-28 13:59:51 +01:00
Thomas Lamprecht
05495245b4 ui: docs: update online help info
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-28 13:52:48 +01:00
Dominik Csapak
2c323b65c6 ui: datastore: add tuning settings to datastore options
Add a simple edit window with 2 combo boxes for `sync-level` and
`chunk-order`.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 [ T: rework commit message/subject a bit ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-28 13:47:10 +01:00
Thomas Lamprecht
b4a81964d0 docs: storage: remove duplicate screenshot, link remaining one
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-28 12:47:16 +01:00
Thomas Lamprecht
d0fecab611 docs: ransomeware: rework structure and expand
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-28 12:11:18 +01:00
Thomas Lamprecht
70abb167cf docs: maintenance: add prune job reference and update title
this section needs general rework/expansion but to be able to link to
it already now add a reference and only do a minimal title update.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-28 12:09:34 +01:00
Thomas Lamprecht
c8f66efd29 docs: ransom ware: fix some typos commented in review
Note that the diff is bigger due to quite some text re-flow.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-28 10:47:27 +01:00
Thomas Lamprecht
d2641fbbd8 docs: ransom ware: reflow text to our 80cc
and drop some trailing whitespace while at it

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-28 10:47:00 +01:00
Noel Ullreich
b8e78fae91 docs: add section on ransomware and PBS help against it
Added a section on ransomware. This includes a bullet point in the
main features section and a section in the backup storage section.
The latter section lists mitigation resources in pbs as well as best
practices.

Updated capitalization to be consistent in main features. Imo, since
these are bullet points and not headings, they should be in lowercase

Signed-off-by: Noel Ullreich <n.ullreich@proxmox.com>
Reviewed-by: Stefan Hanreich <s.hanreich@proxmox.com>
Reviewed-by: Stefan Sterz <s.sterz@proxmox.com>
2022-11-28 10:19:42 +01:00
Dominik Csapak
4694dede0e datastore: make 'filesystem' the default sync-level
rationale is that it makes the backup much safer than 'none', but does not
incur a big of a performance hit as 'file'.

here some benchmark:

data to be backed up:
~14GiB semi-random test images between 12kiB and 4GiB
that results in ~11GiB chunks (more than ram available on the target)

PBS setup:
virtualized (on an idle machine), PBS itself was also idle
8 cores (kvm64 on Intel 12700k) and 8 GiB memory

all virtual disks are on LVM with discard and iothread on
the HDD is a 4TB Seagate ST4000DM000 drive, and the NVME is a 2TB
Crucial CT2000P5PSSD8

i tested each disk with ext4/xfs/zfs (default created with the gui)
with 5 runs each, inbetween the caches are flushed and the filesystem synced
i removed the biggest and smallest result and from the remaining 3
results built the average (percentage is relative to the 'none' result)

result:

test         none     filesystem         file
hdd - ext4   125.67s  140.39s (+11.71%)  358.10s (+184.95%)
hdd - xfs    92.18s   102.64s (+11.35%)  351.58s (+281.41%)
hdd - zfs    94.82s   104.00s (+9.68%)   309.13s (+226.02%)
nvme - ext4  60.44s   60.26s (-0.30%)    60.47s (+0.05%)
nvme - xfs   60.11s   60.47s (+0.60%)    60.49s (+0.63%)
nvme - zfs   60.83s   60.85s (+0.03%)    60.80s (-0.05%)

So all in all, it does not seem to make a difference for nvme drives,
for hdds 'filesystem' increases backup time by ~10%, while
for 'file' it largely depends on the filesystem, but always
in the range of factor ~3 - ~4

Note that this does not take into account parallel actions, such as gc,
verify or other backups.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-28 09:49:55 +01:00
Thomas Lamprecht
f41233d219 d/control: bump version dependency for widget-toolkit 3.5.2
to ensure that the moved CSS rules are installed

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-28 09:23:12 +01:00
Matthias Heiserer
3b2886499f ui: css: move action columns fix to widget-toolkit
Signed-off-by: Matthias Heiserer <m.heiserer@proxmox.com>
2022-11-28 08:32:54 +01:00
Wolfgang Bumiller
52f5b8ad09 datastore: actually hold the active operation file lock on update
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-25 14:11:38 +01:00
Wolfgang Bumiller
dd286be58d fix active state when reading task logs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-25 13:39:59 +01:00
Thomas Lamprecht
8a93ac9ebe bump version to 2.2.8-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-24 17:41:51 +01:00
Thomas Lamprecht
b1dacb0710 prune jobs: switch default to on-error-only notifications
we can still do that as notifications for prune jobs weren't released
yet.

We may want to evaluate if we adapt (some) other notification types
too on next major release.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-24 17:33:28 +01:00
Thomas Lamprecht
c4166d27fc ui: nav tree: move config sets to top of the class
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-24 16:53:57 +01:00
Thomas Lamprecht
1aae684ba6 datastore: write chunk: comment utimesat params
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-24 16:53:57 +01:00
Thomas Lamprecht
2858912686 docs: intro: fix text-width
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-11-24 16:53:57 +01:00
Dominik Csapak
ec2926c2e4 disk: smartctl: ignore bit 2 of exitcode
under some conditions, the smartctl exitcode sets bit 2, even if the
smartctl call succeeded, but has e.g. some warnings derived from the
attributes

we do the same in pve, but it is only the first step in fixing #4353, since
we probably should parse the smartcl output better to include
such warnings

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-11-24 14:47:19 +01:00
Wolfgang Bumiller
6d1f8b4b51 cleanup rrd bounds checks in datastore status call
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-24 13:55:22 +01:00
Wolfgang Bumiller
84283eabb0 rrd: add Entry::get() to access the data
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-24 13:53:49 +01:00
Daniel Tschlatscher
aaddf46a63 gui: change reporting of the estimated_time_full to "Full" if no space
is left in the datastore. Before, the GUI would report "Never" for the
estimated time full, because the value provided in the backend was in
the past. To get around this, the GUI now reports "Full" if the value
for available reaches 0.

Signed-off-by: Daniel Tschlatscher <d.tschlatscher@proxmox.com>
2022-11-24 13:26:29 +01:00
Daniel Tschlatscher
de923258d0 'available' field in rrd data in the API and change usage of 'total'
The API now exposes the field 'available' as well, with which the
unprivileged total is calculated in all corresponsing views in the
frontend.
The rrd charts now also display the total as the unprivileged total
if available, otherwise the absolute total is used.

Signed-off-by: Daniel Tschlatscher <d.tschlatscher@proxmox.com>
2022-11-24 13:26:27 +01:00
Daniel Tschlatscher
f362f8f075 fix #4077: Estimated Full metric on ext4 file systems
The rrd data now includes tracking the available field in disk usage.
The calculation for the estimated_time_full was adapted to use the
total for the unpriviliged user, which is the sum of used + available.

The total for unprivileged users is preferable, because datastores are
always written to by the backup user. Which means that any storage
space reserved for root is unusable for our purposes.

To avoid resetting the estimate when switching to this new version,
the backend will try to use the available value to calculate the
unprivileged total. When that is not an option, it will fall back to
using the absolute total.

Signed-off-by: Daniel Tschlatscher <d.tschlatscher@proxmox.com>
2022-11-24 13:26:24 +01:00
Wolfgang Bumiller
2943d00c84 cleanup for line limit & less indentation
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-24 10:47:09 +01:00
Daniel Tschlatscher
67a5999aa6 make tasklog downloadable in the backup server backend
The read_tasklog API call now stream the whole log file if the query
parameter 'download' is set to true. If the limit parameter is set to
0, all lines in the tasklog will be returned in json format.

To make a file stream and a json response in the same API call work, I
had to use one of the lower level apimethod types from the
proxmox-router. Therefore, the routing declarations and parameter
schemas have been changed accordingly.

Signed-off-by: Daniel Tschlatscher <d.tschlatscher@proxmox.com>
2022-11-24 10:40:05 +01:00
Wolfgang Bumiller
9a087ce7a2 bump d/control
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-23 15:47:33 +01:00
Wolfgang Bumiller
593aa85a9e bump pxar dependency to 0.10.2
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-23 15:47:07 +01:00
Wolfgang Bumiller
45c7d758e5 distinguish between block/chardevs in diff output
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-11-23 10:07:23 +01:00
Lukas Wagner
52189f181f fix #3828: proxmox_backup_debug: Introduce diff archive subcommand.
This new subcommand compares a pxar archive in two different
snapshots and prints a list of added/modified/deleted file
entries.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2022-11-23 09:56:16 +01:00