157 Commits

Author SHA1 Message Date
Thomas Lamprecht
05447e0459 vzdump: import split_list and cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-17 15:45:21 +02:00
Thomas Lamprecht
f839f3d153 vzdump: style fix, trailing comma and closing brace on new line
also sort the hash definition

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-17 15:39:46 +02:00
Aaron Lauterer
df5875b41e vzdump: move remaining guest include logic to single method
The `guest include` logic handling `all` and `exclude` parameters was in
the `PVE::VZDump->exec_backup()` method. Moving this logic into the
`get_included_guests` method allows us to simplify and generalize it.

This helps to make the overall logic easier to test and develop other
features around vzdump backup jobs.

The method now returns a hash with node names as keys mapped to arrays
of VMIDs on these nodes that are included in the vzdump job.

The VZDump API call to create a new backup is adapted to use the new
method to create the list of local VMIDs and the skiplist.

Permission checks are kept where they are to be able to handle missing
permissions according to the current context. The old behavior to die
on a backup job when the user is missing the permission to a guest and
the job is not an 'all' or 'exclude' job is kept.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-06-17 15:16:06 +02:00
Aaron Lauterer
5c4da4c3e8 vzdump: make guest include logic testable
As a first step to make the whole guest include logic more testable the
part from the API endpoint has been moved to its own method with as
little changes as possible.

Everything concerning `all` and `exclude` logic is still in the
PVE::VZDump->exec_backup() method.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-06-08 15:54:42 +02:00
Thomas Lamprecht
9c3a51ee22 vzdump: format size: future proof against huge VM disks
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-08 15:54:35 +02:00
Dietmar Maurer
071e573a13 PVE/VZDump.pm: use new --quiet option for proxmox-backup-client prune 2020-06-03 11:08:50 +02:00
Fabian Grünbichler
7ddcb3afda vzdump: set 'pbs' option when backing up to PBS target
this unifies the logic into a single place instead of all over this
module and the plugins.

it also fixes tons of 'uninitialized value' warnings when backing up
with --dumpdir but no --storage set, since the existing conditions for
PBS targets are missing a definedness check.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-05-06 16:21:41 +02:00
Fabian Grünbichler
7420d7ff95 zstd: add --rsyncable flag
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-05-04 12:19:31 +02:00
Alwin Antreich
3e2c5105f8 Fix #2124: Add support for zstd
This patch adds the zstd to the compression selection for backup on the
GUI and add .zst to the backup file filter. Including zstd as package
install dependency.

Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2020-05-04 10:41:59 +02:00
Thomas Lamprecht
3d45fe518b followup: whitespace and indentation cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-03-12 17:44:26 +01:00
Moayad Almalat
5979889332 fix #2634: print specific error if hook-script is not executable
Signed-off-by: Moayad Almalat <m.almalat@proxmox.com>
Tested-by: Oguz Bektas <o.bektas@proxmox.com>
2020-03-12 16:32:32 +01:00
Dietmar Maurer
1a87db9e56 vzdump: add support for proxmox backup server 2020-02-19 14:02:05 +01:00
Thomas Lamprecht
df2d3636f4 VZDump: sort module usage
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-02-10 11:16:32 +01:00
Stefan Reiter
819e4ff59c fix #2509: add --rsyncable to pigz again
fixes commit e953f92adf72f2ec663e9b5408446c36170faa40

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-12-16 11:39:59 +01:00
Dominic Jäger
ce84a9506d Fix #352: Limit the length of backup logs for mails
When creating a backup the log part can make the mail too big to be
transferred. To ensure delivery, two measures are taken:
1. Always omit the status lines
2. Omit the whole log part if a mail becomes (too) big

Additionally, add a check for missing log files.

Co-developed-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
2019-11-19 15:02:50 +01:00
Fabian Grünbichler
3ac3653e63 use PVE::DataCenterConfig
to make sure that the corresponding cfs_read_file works() works.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-11-18 12:25:35 +01:00
Christian Ebner
2424074ee7 vzdump: move code needed for cfs register of vzdump.cron to guest-common
This moves the cfs register code for vzdump.cron to the
pve-guest-common package. Therefore, it relies on the corresponding
patches in pve-guest-common and pve-docs as build dependencies.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-10-18 08:23:40 +02:00
Stefan Reiter
e953f92adf fix #2314: remove GZIP env var
...and replace instead with command line argument.
Avoids a deprecation warning.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-08-08 13:18:45 +02:00
Tim Marx
f337626155 fix #1278 api: add pool backup option
Signed-off-by: Tim Marx <t.marx@proxmox.com>
2019-06-28 18:05:19 +02:00
Tim Marx
60e049c231 whitespace cleanup
Signed-off-by: Tim Marx <t.marx@proxmox.com>
2019-06-28 18:01:04 +02:00
Thomas Lamprecht
7618913027 perl: fix some common typos found with codespell
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-15 10:40:20 +02:00
Oguz Bektas
6c09101a02 change error message related to maxbackups
make the message a bit more informative (with help from thomas), namely
mentioning the ability to change/increase the limit.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2019-06-11 18:25:10 +02:00
Thomas Lamprecht
1e4583d5de vzdump: new: code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-06-11 17:38:18 +02:00
Thomas Lamprecht
4e0947c88b vzdump: allow plugins to set guest resume time
This refactors things a bit to avoid having the same two lines in 3
places and allows the plugin to set the "guest was resumed" time
stamp at the point it really was resumed, not only once the backup
completed (see #503). Further, if a plugin prints it's own
"resumed/running after X seconds" message, it can unset the vmstop
time and thus avoid printing the message twice.

related to a fix for #503

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-05-15 14:30:45 +02:00
Dominic Jäger
29f26f6ddb Fix #482: Add timestamps to backup create task log
Adding timestamps to the log messages facilitates troubleshooting.

We only log this in the task log, as the syslog and the backup log
(stored together with the backup on the target storage) already have
date/time prefixed, so only the task log missed this info in case of
multiple backups tasks for a single job.

Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
2019-04-24 09:17:06 +00:00
Thomas Lamprecht
9ff893ae52 vzdump: code cleanup empty newlines
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-04-16 14:16:21 +00:00
Thomas Lamprecht
e2b5eb29b5 vzdump: use strftime for backup basename assembly
we already use POSIX strftime a lot in the stack, so nothing new,
also just use the perl built in localtime

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-04-16 14:09:03 +00:00
Alwin Antreich
59abc310f0 Add cephfs to allowed storages for vzdump backup
Signed-off-by: Alwin Antreich <a.antreich@proxmox.com>
2018-07-04 16:57:32 +02:00
Wolfgang Link
6d09915c51 Add CIFS in strorage info to allow backup. 2018-04-05 13:48:52 +02:00
Fabian Grünbichler
998e4eebe7 VZDump: handle timelocal year correctly
while this is unlikely to cause any problems, it is unnecessary to
substract 1900 here - timelocal handles 4-digit years perfectly well.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2018-03-07 11:36:31 +01:00
Thomas Lamprecht
f2e9079f0d remove obsolete Posix use clause
besides the cleanup purpose this fixes an actual problem, perls POSIX
module has the following caveat:

`Everything is exported by default (with a handful of exceptions).
This is an unfortunate backwards compatibility feature and its use is
strongly discouraged. You should either prevent the exporting (by
saying "use POSIX ();", as usual) and then use fully qualified names
(e.g.  "POSIX::SEEK_END"), or give an explicit import list. If you do
neither and opt for the default (as in "use POSIX;"), you will import
hundreds and hundreds of symbols into your namespace.'

see `perldoc POSIX`

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2017-09-25 15:18:25 +02:00
Thomas Lamprecht
a5c9479705 use log method from base VZDump::Plugin
we copied our log method over there to resolve a cyclic dependency,
now use it here to reduce code duplication.
As we are below pve-guest-common in the dpendency hierachy we may use
it, but it does not may (or at least should) use us.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2017-09-22 09:34:09 +02:00
Thomas Lamprecht
8555b100bb vzdump/pigz: use ProcFSTools for core count, not sysconf
A "sysconf(84)" is not really descripitve. There is no POSIX const
entry for _SC_NPROCESSORS_ONLN (number of processors online) as its
not standard but an adoption of glibc.
Better just use our proven ProcFSTool's cpuinfo method for getting
the desired information.

This is kept only for backwards compatibillity as we guaranteed that
we use #cpus/2 if 'pigz' is set to 1, else I would have removed this
alltogether and just directly passed 'pigz' to the command...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2017-09-20 12:34:22 +02:00
Fabian Grünbichler
c3b58274d4 fix #1389: vzdump: handle storage failure in new()
otherwise the mail notification is not triggered

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2017-06-08 16:20:03 +02:00
Dietmar Maurer
9f07075f0a vzdump: do not generate locale specific time stamps
Avoid perl warning: Wide character in print
2017-04-24 07:17:21 +02:00
Thomas Lamprecht
83ec8f816a VZDump: switch ha managed check from enabled to started
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2016-11-22 16:47:19 +01:00
Stefan Priebe
c76d010656 VZDump: die with error if plugin loading fails
Signed-off-by: Stefan Priebe <s.priebe@profihost.ag>
2016-11-17 07:06:38 +01:00
Wolfgang Bumiller
6acb632a54 vzdump: set task mode early to fix a warning
when a vzdump script is set and the backup fails early (eg.
when exceeding the number of backups) run_hook_script()
showed an uninitialized value error trying to use
$task->{mode} which is set only after prepare() was called.

This sets $task->{mode} early, still updated later to $stop
if !$running, and changes the condition for whether
cleanup() should be called to not use $task->{mode} (which
makes no real sense anyway) to using the $cleanup hash like
the rest of the code.
2016-08-19 13:54:38 +02:00
Fabian Grünbichler
0da2260fc2 vzdump: add newline in mail text
for consistency with html version and better readability
2016-07-07 11:12:24 +02:00
Wolfgang Bumiller
44b21574f0 Fix #1048: vzdump: include job-* hook logs in email 2016-07-07 10:52:58 +02:00
Wolfgang Bumiller
6cd975001f vzdump: safer cp call 2016-07-07 10:52:58 +02:00
Fabian Grünbichler
e42ae622cc fix #1005: don't exclude /var/log/?* with stdexcludes
the old behaviour can be easily restored with --exclude-path
and this seemed to surprise a lot of users.
2016-06-08 12:34:04 +02:00
Wolfgang Bumiller
403761c43e Fix #1015: vzdump: send email on early errors
VZDump->new() dies when a tmpdir or dumpdir is configured
but does not exist. At this point the error is not being
reported via email.

This also moves the instantiation of VZDump into the worker
since new() can now call sendmail() on error.

Additionally rather than only showing a single error if both
tmpdir and dumpdir don't exist, both are included in the
message.
2016-06-06 13:38:35 +02:00
Fabian Grünbichler
d50472438d fix #949: add post-restart hook to vzdump 2016-04-22 10:54:18 +02:00
Fabian Grünbichler
74b47fd8e1 Fix 'mailto: ...' in /etc/vzdump.conf
allow comma-separated list of email addresses, like the API2
parameter.
2016-04-14 17:02:37 +02:00
Fabian Grünbichler
4f367c1917 VZDump: add -mailto schema desc 2016-04-14 12:46:21 +02:00
Fabian Grünbichler
3c6feff249 VZDump: remove dead code
remove get_lvm_mapping and get_lvm_device, which are not
used anywhere anymore.
2016-04-14 10:39:59 +02:00
Fabian Grünbichler
e2a2525e60 VZDump: mark 'size' as deprecated, warn if set
this parameter should be removed in a future release, just
warn and change description for now to avoid breakage.
2016-04-14 10:39:45 +02:00
Fabian Grünbichler
2fcb93475f VM -> guest in VZDump schema descriptions 2016-04-14 10:37:26 +02:00
Fabian Grünbichler
af77c16479 Spelling in VZDump descriptions 2016-04-14 10:36:58 +02:00