32 Commits

Author SHA1 Message Date
Lukas Wagner
d915b69907 api: replication: include 'hostname' field for notifications
The field contains the hostname of the host (without any domain part)
which sends the notification. This field can be used in match-field
match rules.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2024-07-04 14:57:08 +02:00
Lukas Wagner
431ae86b08 api: replication: add 'job-id' to notification metadata
This allows users to create notification match rules for specific
replication jobs, if they so desire.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2024-07-04 14:57:08 +02:00
Lukas Wagner
fede7e87e4 notifications: use named templates instead of in-code templates
This commit adapts notification sending for
    - package update
    - replication
    - backups

to use named templates (installed in /usr/share/pve-manager/templates)
instead of passing template strings defined in code to the
notification stack.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Max Carrara <m.carrara@proxmox.com>
Reviewed-by: Max Carrara <m.carrara@proxmox.com>
2024-06-03 14:16:35 +02:00
Lukas Wagner
3f0ffa0efe api: replication: adapt to matcher-based notification system
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-11-17 16:20:03 +01:00
Lukas Wagner
2ab19e8351 api: replication: send notifications via new notification module
If the new 'target-replication' option in datacenter.cfg is set to a
notification target, we send notifications that way. If it is not set,
we continue send a notification to the default target (mail to
root@pam).

There is also a new 'replication' option. It controls whether to send
a notification at all.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2023-08-16 11:10:14 +02:00
Fiona Ebner
e81a10a4ab api: replication job status: allow querying disabled jobs too
Rather than failing with an error claiming that the job doesn't exist.
The disabled status will be visible in the result of the call.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2023-06-07 18:43:32 +02:00
Fabian Grünbichler
178fb4fe2d replication: fix uninitalized warning
if a job has no schedule and is executed via "Schedule now" but fails, the
following will be printed to journal/syslog:

Mar 21 13:05:01 host02 pvescheduler[203343]: send/receive failed, cleaning up snapshot(s)..
Mar 21 13:05:01 host02 pvescheduler[203343]: 100-0: got unexpected replication job error - command 'set -o pipefail && pvesm export local-zfs:vm-100-disk-0 zfs - -with-snapshots 1 -snapshot __replicate_100-0_1679400300__ | /usr/bin/ssh -e none -o 'BatchMode=yes' -o 'HostKeyAlias=host03' root@10.0.74.3 -- pvesm import local-zfs:vm-100-disk-0 zfs - -with-snapshots 1 -snapshot __replicate_100-0_1679400300__ -allow-rename 0' failed: exit code 255
Mar 21 13:05:01 host02 pvescheduler[203343]: Use of uninitialized value in concatenation (.) or string at /usr/share/perl5/PVE/API2/Replication.pm line 107.

defaulting to the fallback schedule '*/15' makes the spurious warning go away.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-03-21 14:49:12 +01:00
Fabian Ebner
b70458b6dd fix #4111: replication: don't send mail when fail count is zero
which can happen when failing to obtain the guest's migration lock.
This led to a lot of mails being sent during migration (timeout for
obtaining lock is only 2 seconds and we run it in a loop).

One could argue that obtaining the lock should increase the fail
count, but without the lock, the job state should not be touched and
even the first three mails upon migration could be considered spam.

Fixes: fa4bb659 ("replication: sent always mail for first three tries and move helper")
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-06-17 17:20:06 +02:00
Thomas Lamprecht
5ac1eaa041 replication: split out error handling and include more info in mail
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-27 10:18:30 +02:00
Thomas Lamprecht
fa4bb659a3 replication: sent always mail for first three tries and move helper
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-27 09:48:28 +02:00
Fabian Ebner
237f00be03 api: replication: don't send mails about failed replication only once
but rather multiple times becoming exponentially less frequent.

Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-26 16:09:43 +02:00
Wolfgang Bumiller
af344184a8 cleanup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2017-12-14 09:07:00 +01:00
Wolfgang Link
64d39c2ef2 Send an email when a replication job fails.
A email notification will be send for each job when the job fails.
This message will only send when an error occurs and the fail count is on 1.

Reviewed-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Acked-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2017-12-13 14:51:07 +01:00
Wolfgang Link
c874209657 Indentation cleanup.
Reviewed-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Acked-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2017-12-13 14:51:07 +01:00
Wolfgang Link
0d083f4b27 Clean up unnecessary code.
The jobid is already parsed by the file parser.
2017-09-25 15:45:28 +02:00
Thomas Lamprecht
959f37afe2 replication: show disabled jobs and allow enabling/disabling
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2017-06-29 10:59:11 +02:00
Dietmar Maurer
a4beaa9422 PVE/API2/Replication.pm: cleanup - avoid modifying jobcfg
This is simply no longer necessary.
2017-06-29 07:45:57 +02:00
Dietmar Maurer
88ea8e67d3 PVE/API2/Replication.pm: add schedule_now API 2017-06-28 12:14:22 +02:00
Dietmar Maurer
5a26b00600 PVE/API2/Replication.pm: call purge_old_states in run_jobs
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2017-06-27 15:48:06 +02:00
Wolfgang Bumiller
5b35845009 tests: update pvesr_lock_path variable location 2017-06-27 15:16:41 +02:00
Wolfgang Bumiller
25420507b2 refuse to create replication jobs for unreplicatable VMs 2017-06-27 06:28:33 +02:00
Dietmar Maurer
2aa029571c PVE/API2/Replication.pm: add $verbose parameter run_jobs() 2017-06-20 08:58:48 +02:00
Dominik Csapak
d09c076a62 change to ReplicaState
this function moved, so the call to it has to move too

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2017-06-12 16:01:55 +02:00
Dietmar Maurer
d092dc4f83 use new replication helpers from pve-guest-common 2017-06-12 06:59:21 +02:00
Dietmar Maurer
810c6776d9 PVE::Replication - remove dependency to PVE::LXC/PVE::QemuServer
So that we can move the whole class to package pve-guest-common.
2017-06-12 05:59:51 +02:00
Dietmar Maurer
fc527b4da1 PVE::API2::Replication: rework replication status API
/nodes/<node>/replication => list status of all jobs

/nodes/<node>/replication/<id>/status => individual job status

/nodes/<node>/replication/<id>/log => job log
2017-06-08 07:16:39 +02:00
Dominik Csapak
4b48563a88 use correct field for child link
namely id instead of vmid, so that the child links are working in the
api inspector and pvesh

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2017-06-02 16:23:30 +02:00
Dietmar Maurer
f5b53d4c31 PVE/API2/Replication.pm: change vmid => guest 2017-06-02 16:22:46 +02:00
Dominik Csapak
c1b20fca93 add vmid parameter to replication status
so that one can limit the status list for one vm/ct

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2017-06-02 16:19:26 +02:00
Dietmar Maurer
5c180db3b5 PVE::Replication - use new calendar events instead of interval
And implement retry algorythm after failure:

  $next_sync = $state->{last_try} + 5*60*$fail_count;

and limit to 3 failures.
2017-05-31 08:23:46 +02:00
Dietmar Maurer
483f89dd47 PVE/Replication.pm: save pid/ptime to running job state
So that we can check which job is running.
2017-05-31 08:23:46 +02:00
Dietmar Maurer
892821fd87 pvesr: add pve storage replication tool
Just added code to configure jobs. Replication itself is not
implemented.
2017-05-31 08:23:46 +02:00