778 Commits

Author SHA1 Message Date
Fabian Grünbichler
939b71ab48 use physical NIC regexp
because in >= Stretch, most systems don't have ethX devices any more.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2017-06-08 15:00:59 +02:00
Fabian Grünbichler
c109a32583 pvesr: prepare_local_job: fixup
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2017-06-08 11:53:30 +02:00
Fabian Grünbichler
0433b86df6 pvesr: prepare_local_job: use PVE::replication::prepare
to reduce code duplication. this slightly changes behaviour
compared to the previous version:

only disks with the correct prefix are cleaned up, not all
disks with __replication* snapshots.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2017-06-08 11:13:47 +02:00
Fabian Grünbichler
c20a8a75dd Replication::prepare: track cleaned up volumes
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2017-06-08 10:45:09 +02:00
Wolfgang Link
ada0438f0c Fix: list_images returns now a array of hashes so adopt the code. 2017-06-08 10:29:11 +02:00
Wolfgang Link
0342d40048 Permit set parent_snapname if it is not set. 2017-06-08 10:27:09 +02:00
Dietmar Maurer
8fde1fb9af PVE/API2/ReplicationConfig.pm: add job creation checks
Check if the target and guest exists to avoid creation of faulty jobs.
2017-06-08 10:01:45 +02:00
Dietmar Maurer
2b134ff10f pvesr: add a message that job removal will take some time 2017-06-08 09:42:12 +02:00
Dietmar Maurer
ddf9146fc5 PVE::Replication::replicate - Set $running to 0 if undef. 2017-06-08 09:31:09 +02:00
Dietmar Maurer
0243416a54 PVE::API2::Tasks - remove dead code 2017-06-08 08:52:22 +02:00
Dietmar Maurer
d03462058a PVE::API2::Tasks - use PVE::Tools::dump_logfile 2017-06-08 07:36:33 +02:00
Dietmar Maurer
b3986ead7e PVE::API2::Replication: rework replication status API
/nodes/<node>/replication => list status of all jobs

/nodes/<node>/replication/<id>/status => individual job status

/nodes/<node>/replication/<id>/log => job log
2017-06-08 07:16:39 +02:00
Dietmar Maurer
bb5cb56104 replication: add replication log files 2017-06-07 17:27:00 +02:00
Dietmar Maurer
b94223b028 replication: improve stale volume detection, allow sync from parent snapshot
We pass a list of storage to scan for stale volumes to prepare_local_job().
So we make sure that we only activate/scan related storages.

Snapshot rollback may remove local replication shapshots. In that case
we still have the $conf->{parent} snapshot on both sides, so we
can use that as base snapshot.
2017-06-07 17:26:56 +02:00
Dietmar Maurer
545e9cb0a7 Remove $vmid form ssh prepare and finalize command. 2017-06-07 17:25:41 +02:00
Dietmar Maurer
eda1b842bb replication: save list of used storage IDs 2017-06-07 11:30:52 +02:00
Dietmar Maurer
3444479e9a PVE::API2::ReplicationConfig - extract guest ID from job ID 2017-06-07 11:30:52 +02:00
Dietmar Maurer
c482645c7a use new PVE::ReplicationState::replication_snapshot_name() 2017-06-07 09:40:08 +02:00
Dietmar Maurer
c823e097ac replication: add last_node to replication state
So that we know which node produced the state. This is not always
the local node, because we copy the state on migrate.
2017-06-06 06:49:05 +02:00
Dietmar Maurer
da95168678 PVE::Replication - do not pass time to logfunc
This is simply not neccessary.
2017-06-06 06:36:03 +02:00
Dominik Csapak
fe5ba666ae use correct field for child link
namely id instead of vmid, so that the child links are working in the
api inspector and pvesh

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2017-06-02 16:23:30 +02:00
Dietmar Maurer
ac9c2e4daa PVE/API2/Replication.pm: change vmid => guest 2017-06-02 16:22:46 +02:00
Dominik Csapak
0409269617 add vmid parameter to replication status
so that one can limit the status list for one vm/ct

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2017-06-02 16:19:26 +02:00
Dietmar Maurer
b541d31729 PVE::Replication - use new PVE::ReplicationState class 2017-06-02 12:28:43 +02:00
Dietmar Maurer
6a171dc1f2 PVE::Replication - simplify code
Move all state writes into run_replication()
2017-06-02 12:28:43 +02:00
Dominik Csapak
dcd587b037 allow width and height parameter for vncshell
so that novnc can request a different screen size

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2017-06-02 09:19:07 +02:00
Wolfgang Bumiller
c5e08dfe7e replication: use a dedicated lock file
Otherwise $update_job_state's file_set_contents() call
drops our lock.
2017-06-02 06:10:55 +02:00
Dietmar Maurer
5134d5ac28 PVE::Replication::replicate - call get_ssh_info on demand
Only call get_ssh_info() when needed, and do not pass $migration_network
for a simply remove_job - this produces just overhead and is not required.
If we always call get_ssh_info() we can never delete jobs using non-existing
targets.
2017-05-31 15:19:26 +02:00
Wolfgang Link
6c796764da Untaint the content of pve-replication-state.json.
JSON::decode will not untaint the raw text, so we get problems with the
last_sync, which we use many times in the code with open3.
2017-05-31 14:35:52 +02:00
Dietmar Maurer
1f7e36396e replication - allow removal of jobs with target on local node 2017-05-31 13:54:40 +02:00
Dietmar Maurer
20698b5604 PVE::API2::ReplicationConfig - implement delete
We just set the remove_job property in the configuration. Actual removal
is done asynchronous inside replicate().
2017-05-31 08:23:47 +02:00
Dietmar Maurer
fdb3516b4a PVE::API2Tools::resolve_proxyto - new helper
Call proxyto_callback if that method is defined.
2017-05-31 08:23:47 +02:00
Dietmar Maurer
a2f081c8c1 PVE::Replication - aquire guest_migration_lock during replication
To block guest migration.
2017-05-31 08:23:47 +02:00
Dietmar Maurer
8497a4c4bd PVE::Replication - implement replicate_volume()
Simply call PVE::Storage::storage_migrate()
2017-05-31 08:23:47 +02:00
Dietmar Maurer
0fb6a611cf PVE::Replication::replicate - implement replicate
The actual volume replication is done in replicate_volume(), which is just
a stub for now.

I also added a regression test replication_test5.pl to verify basic
functions.
2017-05-31 08:23:47 +02:00
Dietmar Maurer
c6c1e03a1c PVE::Replication::remote_finalize_local_job - new helper
Simple wrapper to call "pvesr finalize-local-job" on a remote
cluster node.
2017-05-31 08:23:47 +02:00
Dietmar Maurer
89b18249cc PVE::Replication::remote_prepare_local_job - new helper
Simple wrapper to call "pvesr prepare-local-job" on a remote
cluster node.
2017-05-31 08:23:47 +02:00
Dietmar Maurer
c7fa37c0d8 pvesr finalize-local-job: add helper to cleanup job
This will be called after replication on the target node. For now, this
just removes old snapshots.
2017-05-31 08:23:47 +02:00
Dietmar Maurer
be9adcf2a0 pvesr prepare-local-job: new helper
Prepare for starting a replication job. This is called on the target
node before replication starts. This call is for internal use, and
return a JSON object on stdout. The method first test if VM <vmid>
reside on the local node. If so, stop immediately. After that the
method scans all volume IDs for snapshots, and removes all replications
snapshots with timestamps different than <last_sync>. It also removes
any unused volumes.

Returns a hash with boolean markers for all volumes with existing
replication snapshots.
2017-05-31 08:23:46 +02:00
Dietmar Maurer
7c42d1713c pvesr run: add --verbose flag
We can now pass a $logfunc parameter to replicate() to
produce more verbose logs/output. This is especially useful
for regression testing.
2017-05-31 08:23:46 +02:00
Dietmar Maurer
8ad9177c0d PVE::Replication - use new calendar events instead of interval
And implement retry algorythm after failure:

  $next_sync = $state->{last_try} + 5*60*$fail_count;

and limit to 3 failures.
2017-05-31 08:23:46 +02:00
Dietmar Maurer
b9b5b749cf PVE/Replication.pm: save pid/ptime to running job state
So that we can check which job is running.
2017-05-31 08:23:46 +02:00
Dietmar Maurer
0880c088ff pvesr: add pve storage replication tool
Just added code to configure jobs. Replication itself is not
implemented.
2017-05-31 08:23:46 +02:00
Emmanuel Kasper
91a3065726 Fix #1386: track postfix daemon status via postfix@- 2017-05-29 14:52:24 +02:00
Fabian Grünbichler
ee2278a5c5 ceph: init: rename disable-cephx to disable_cephx
for consistency
2017-05-03 11:37:08 +02:00
Fabian Grünbichler
c976b64aaa ceph: init: add warning to disable-cephx 2017-05-03 10:27:24 +02:00
Alexandre Derumier
d44e688980 ceph: init: add optional disable-cephx option
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2017-05-03 10:27:18 +02:00
Alexandre Derumier
4f017d5854 ceph: init: remove filestore xattr use omap
it's not used anymore since emperor

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2017-05-03 10:26:25 +02:00
Dietmar Maurer
6085a0aaca PVE/APLInfo.pm: avoid locale specific time stamps 2017-04-24 07:24:12 +02:00
Dietmar Maurer
416f30393b vzdump: do not generate locale specific time stamps
Avoid perl warning: Wide character in print
2017-04-24 07:17:21 +02:00