765 Commits

Author SHA1 Message Date
Dietmar Maurer
b94223b028 replication: improve stale volume detection, allow sync from parent snapshot
We pass a list of storage to scan for stale volumes to prepare_local_job().
So we make sure that we only activate/scan related storages.

Snapshot rollback may remove local replication shapshots. In that case
we still have the $conf->{parent} snapshot on both sides, so we
can use that as base snapshot.
2017-06-07 17:26:56 +02:00
Dietmar Maurer
545e9cb0a7 Remove $vmid form ssh prepare and finalize command. 2017-06-07 17:25:41 +02:00
Dietmar Maurer
eda1b842bb replication: save list of used storage IDs 2017-06-07 11:30:52 +02:00
Dietmar Maurer
3444479e9a PVE::API2::ReplicationConfig - extract guest ID from job ID 2017-06-07 11:30:52 +02:00
Dietmar Maurer
c482645c7a use new PVE::ReplicationState::replication_snapshot_name() 2017-06-07 09:40:08 +02:00
Dietmar Maurer
c823e097ac replication: add last_node to replication state
So that we know which node produced the state. This is not always
the local node, because we copy the state on migrate.
2017-06-06 06:49:05 +02:00
Dietmar Maurer
da95168678 PVE::Replication - do not pass time to logfunc
This is simply not neccessary.
2017-06-06 06:36:03 +02:00
Dominik Csapak
fe5ba666ae use correct field for child link
namely id instead of vmid, so that the child links are working in the
api inspector and pvesh

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2017-06-02 16:23:30 +02:00
Dietmar Maurer
ac9c2e4daa PVE/API2/Replication.pm: change vmid => guest 2017-06-02 16:22:46 +02:00
Dominik Csapak
0409269617 add vmid parameter to replication status
so that one can limit the status list for one vm/ct

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2017-06-02 16:19:26 +02:00
Dietmar Maurer
b541d31729 PVE::Replication - use new PVE::ReplicationState class 2017-06-02 12:28:43 +02:00
Dietmar Maurer
6a171dc1f2 PVE::Replication - simplify code
Move all state writes into run_replication()
2017-06-02 12:28:43 +02:00
Dominik Csapak
dcd587b037 allow width and height parameter for vncshell
so that novnc can request a different screen size

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2017-06-02 09:19:07 +02:00
Wolfgang Bumiller
c5e08dfe7e replication: use a dedicated lock file
Otherwise $update_job_state's file_set_contents() call
drops our lock.
2017-06-02 06:10:55 +02:00
Dietmar Maurer
5134d5ac28 PVE::Replication::replicate - call get_ssh_info on demand
Only call get_ssh_info() when needed, and do not pass $migration_network
for a simply remove_job - this produces just overhead and is not required.
If we always call get_ssh_info() we can never delete jobs using non-existing
targets.
2017-05-31 15:19:26 +02:00
Wolfgang Link
6c796764da Untaint the content of pve-replication-state.json.
JSON::decode will not untaint the raw text, so we get problems with the
last_sync, which we use many times in the code with open3.
2017-05-31 14:35:52 +02:00
Dietmar Maurer
1f7e36396e replication - allow removal of jobs with target on local node 2017-05-31 13:54:40 +02:00
Dietmar Maurer
20698b5604 PVE::API2::ReplicationConfig - implement delete
We just set the remove_job property in the configuration. Actual removal
is done asynchronous inside replicate().
2017-05-31 08:23:47 +02:00
Dietmar Maurer
fdb3516b4a PVE::API2Tools::resolve_proxyto - new helper
Call proxyto_callback if that method is defined.
2017-05-31 08:23:47 +02:00
Dietmar Maurer
a2f081c8c1 PVE::Replication - aquire guest_migration_lock during replication
To block guest migration.
2017-05-31 08:23:47 +02:00
Dietmar Maurer
8497a4c4bd PVE::Replication - implement replicate_volume()
Simply call PVE::Storage::storage_migrate()
2017-05-31 08:23:47 +02:00
Dietmar Maurer
0fb6a611cf PVE::Replication::replicate - implement replicate
The actual volume replication is done in replicate_volume(), which is just
a stub for now.

I also added a regression test replication_test5.pl to verify basic
functions.
2017-05-31 08:23:47 +02:00
Dietmar Maurer
c6c1e03a1c PVE::Replication::remote_finalize_local_job - new helper
Simple wrapper to call "pvesr finalize-local-job" on a remote
cluster node.
2017-05-31 08:23:47 +02:00
Dietmar Maurer
89b18249cc PVE::Replication::remote_prepare_local_job - new helper
Simple wrapper to call "pvesr prepare-local-job" on a remote
cluster node.
2017-05-31 08:23:47 +02:00
Dietmar Maurer
c7fa37c0d8 pvesr finalize-local-job: add helper to cleanup job
This will be called after replication on the target node. For now, this
just removes old snapshots.
2017-05-31 08:23:47 +02:00
Dietmar Maurer
be9adcf2a0 pvesr prepare-local-job: new helper
Prepare for starting a replication job. This is called on the target
node before replication starts. This call is for internal use, and
return a JSON object on stdout. The method first test if VM <vmid>
reside on the local node. If so, stop immediately. After that the
method scans all volume IDs for snapshots, and removes all replications
snapshots with timestamps different than <last_sync>. It also removes
any unused volumes.

Returns a hash with boolean markers for all volumes with existing
replication snapshots.
2017-05-31 08:23:46 +02:00
Dietmar Maurer
7c42d1713c pvesr run: add --verbose flag
We can now pass a $logfunc parameter to replicate() to
produce more verbose logs/output. This is especially useful
for regression testing.
2017-05-31 08:23:46 +02:00
Dietmar Maurer
8ad9177c0d PVE::Replication - use new calendar events instead of interval
And implement retry algorythm after failure:

  $next_sync = $state->{last_try} + 5*60*$fail_count;

and limit to 3 failures.
2017-05-31 08:23:46 +02:00
Dietmar Maurer
b9b5b749cf PVE/Replication.pm: save pid/ptime to running job state
So that we can check which job is running.
2017-05-31 08:23:46 +02:00
Dietmar Maurer
0880c088ff pvesr: add pve storage replication tool
Just added code to configure jobs. Replication itself is not
implemented.
2017-05-31 08:23:46 +02:00
Emmanuel Kasper
91a3065726 Fix #1386: track postfix daemon status via postfix@- 2017-05-29 14:52:24 +02:00
Fabian Grünbichler
ee2278a5c5 ceph: init: rename disable-cephx to disable_cephx
for consistency
2017-05-03 11:37:08 +02:00
Fabian Grünbichler
c976b64aaa ceph: init: add warning to disable-cephx 2017-05-03 10:27:24 +02:00
Alexandre Derumier
d44e688980 ceph: init: add optional disable-cephx option
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2017-05-03 10:27:18 +02:00
Alexandre Derumier
4f017d5854 ceph: init: remove filestore xattr use omap
it's not used anymore since emperor

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2017-05-03 10:26:25 +02:00
Dietmar Maurer
6085a0aaca PVE/APLInfo.pm: avoid locale specific time stamps 2017-04-24 07:24:12 +02:00
Dietmar Maurer
416f30393b vzdump: do not generate locale specific time stamps
Avoid perl warning: Wide character in print
2017-04-24 07:17:21 +02:00
Wolfgang Bumiller
d43b427892 statd: rebalance: don't use CpuSet::max_cpuids
We're already limiting CPUs to lxc/cpuset.effective_cpus,
so let's use the highest cpuid from that set as a maximum to
initialize the container count array.
2017-04-20 12:18:55 +02:00
Fabian Grünbichler
ee9dc655cf Ceph: allow bigger size and min_size in API 2017-04-03 12:06:55 +02:00
Fabian Grünbichler
e7fe107f75 fix start on boot
fix the fix for #1024

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2017-03-24 09:52:28 +01:00
Fabian Grünbichler
929382968e ceph: create default keys when creating monitor
this used to be a separate service, but got dropped in
Kraken in favour of calling ceph-create-keys manually when
deploying a node.
2017-03-21 13:50:28 +01:00
Fabian Grünbichler
9554852089 pveceph: use luminous by default 2017-03-21 13:50:28 +01:00
Dietmar Maurer
9b557647fe use 'U' to encode undefined values for RRD graphs
rrdtools 1.5 and newer seems to require this.
2017-03-17 11:27:18 +01:00
Thomas Lamprecht
248b2d3614 remove backup locks when starting all VMs on boot
If on bootup one of our VMs is locked by an backup we safely can
assume that this backup job does not run anymore and that the lock
has no reason anymore and just hinders uptime of services.

As at this time we (the node) have quorum so we may safely assume
that we have a consistent view of the cluster and all our VMs really
belong to us. We just need to ensure that we do not run into an
automatic backup jobs, so execute our code with VZDumps lock or
timeout.
Log in the Task and Sys log that we removed the lock, so that an
admin easily sees that there may be need for cleaning leftovers from
an interrupted backup.

Addresses bug #1024

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2017-03-16 09:04:21 +01:00
Thomas Lamprecht
627fa10467 get_filtered_vmlist: save VM conf in own subhash
small refactoring in get_filtered_vmlist: save a VMs config in its
own subhash to avoid collisions with other data which we want to save
in the vmid list, for now this is only `type` but in the next patch
I want to save also the class

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2017-03-16 09:04:09 +01:00
Thomas Lamprecht
b750c7cb6a include also HA VMs if a vm list is explicit specified
Else the user would not had specified them explicit

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2017-03-16 09:02:05 +01:00
Fabian Grünbichler
9c9480e019 APT: update packages and package names
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2017-03-16 07:06:25 +01:00
Dietmar Maurer
db8df19eb0 pveproxy: do not serve /usr/share/vncterm - no longer required
Note: We removed the java applet
2017-03-14 07:21:09 +01:00
Dominik Csapak
b29450929b fix diffstore issue with clusterlog
extjs cannot "convert" and id from other fields, so the ids in the
diffstore and the realstore are different and we re-add every element on
every update

to mitigate this, we generate the id (which is "uid:hostname") in the
backend, and simply use it in the frontend

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2017-03-10 11:09:46 +01:00
Dominik Csapak
534b84fac6 remove unneeded *Index.pm files
they are not used anymore

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2017-03-09 10:34:48 +01:00