Commit Graph

4632 Commits

Author SHA1 Message Date
Thomas Lamprecht
ca0fc07dd4 fix #844: allow to pre-delay start-all-marked guests on boot
Add a simple ExecStartPre command which reads the local node config,
and if a delay is set the helper sleeps that long then exists.

The systemd-unit approach was chosen as this ensures that we really
only delay when doing the startall on node boot. The pve-guests
service does not allows manual stops, starts or restarts, it can only
be pulled in by the multi-user.target

Mark this command with "-" to tell systemd that errors of it should
not cause an abort, it's a best-effort approach.

The journal from a 2 second delay would look like:
> Nov 19 13:13:48 dev6 systemd[1]: Starting PVE guests...
> Nov 19 13:13:48 dev6 pve-startall-delay[2318]: Delaying on-boot 'startall' command for 2 second(s).
> ...
> Nov 19 13:13:50 dev6 pve-guests[2339]: <root@pam> starting task UPID:dev6:00000924:00000529:5DD3DC7E:startall::root@pam:

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-20 20:20:58 +01:00
Dominik Csapak
6033f465b8 gui: pci passthrough: consider domain in PCISelector
but remove the default domain '0000' before sending to the backend,
and add it if no domain is given in the config

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-11-20 19:43:31 +01:00
Thomas Lamprecht
b775eeb0fc d/control: bump version dependency to qemu-server
for mon_cmd refactoring

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-20 19:43:31 +01:00
Stefan Reiter
447bae61ef refactor: vm_mon_cmd is now Monitor::mon_cmd
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-11-20 18:25:49 +01:00
Thomas Lamprecht
4e3ba05796 allow to run some tests in parallel
the replication test are not yet ready for that, but the others can
be run in parallel

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-19 15:02:50 +01:00
Thomas Lamprecht
9d43b9ca76 followup: cleanup/refactor mail test
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-19 15:02:50 +01:00
Dominic Jäger
7e4113467c Fix #352: Limit the length of backup logs for mails
When creating a backup the log part can make the mail too big to be
transferred. To ensure delivery, two measures are taken:
1. Always omit the status lines
2. Omit the whole log part if a mail becomes (too) big

Additionally, add a check for missing log files.

Co-developed-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
2019-11-19 15:02:50 +01:00
Thomas Lamprecht
353e78aff9 api: node/config: allow to return only a specific config property
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-19 12:04:52 +01:00
Thomas Lamprecht
b89d595297 status/graphite: fix memory leak, avoid cyclic closure reference
The data passed to this closure was never free'd, depending on the
count of VM/CTs one could get >1 MB of RSS (!) memory leaked per
statd status cycle update run...

We could also use Scalar::Util's weaken, to weak a copy of this
variable, but as a simple undef works lets do that with a comment..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-19 09:11:47 +01:00
Thomas Lamprecht
0eb41878a9 fixup: graphite: use correct variable in closure
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-19 09:10:54 +01:00
Thomas Lamprecht
e16c4137f2 Ext. Metrics: module global variable does not help with memory leak
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-18 19:15:13 +01:00
Thomas Lamprecht
15d052e293 statd: increase RSS difference required for restart
it seems that we have a reference leak or the like somewhere in the
(graphite?) status plugin, while the recent transaction based update
mechanism made it slightly better, it's still bad with a lot of VMs..

Until we can track that down, or abandon perl for good, avoid to
frequent restarts by allowing statd to grow 15 MB of memory usage
after initial calibration (it's memory usage at the 10th cycle)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-18 19:07:24 +01:00
Thomas Lamprecht
458fe35e10 statd: report memory usage in KB
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-18 19:04:29 +01:00
Thomas Lamprecht
f3e584e007 ext. metric: move to a transaction model
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-18 19:04:29 +01:00
Fabian Grünbichler
b26df687a4 bump version to 6.0-12
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-18 12:25:35 +01:00
Fabian Grünbichler
fcd5a5efec d/control: bump versioned b-d on libpve-guest-common-perl
otherwise the replication tests fail to load datacenter.cfg

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-11-18 12:25:35 +01:00
Fabian Grünbichler
73bfce2698 d/control: add (build-)depends on libpve-cluster-api-perl
it contains PVE::API2::ClusterConfig

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-11-18 12:25:35 +01:00
Fabian Grünbichler
d20cea0705 d/control: add (build-)depends on libpve-cluster-perl
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-11-18 12:25:35 +01:00
Fabian Grünbichler
b8b2acfd8b test: also mock cfs_read_file
needed for tests to work without a functional running pmxcfs instance.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-11-18 12:25:35 +01:00
Fabian Grünbichler
808c4b03d3 pveproxy.service: take over pvecm call from pve-cluster.service
to ensure certificate/key is present prior to loading them.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-11-18 12:25:35 +01:00
Fabian Grünbichler
c50091d298 use PVE::RRD for RRD data
refactored from PVE::Cluster. same code, same semantics, different file.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-11-18 12:25:35 +01:00
Fabian Grünbichler
4a1cc38481 test: mock PVE::SSHInfo module
since it is now used for getting a cluster node's migration IP address.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-11-18 12:25:35 +01:00
Fabian Grünbichler
a029c045e9 use PVE::DataCenterConfig
to make sure that the corresponding cfs_read_file works() works.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-11-18 12:25:35 +01:00
Fabian Grünbichler
869df6202f takeover CertCache from pve-cluster
same code, same semantics, different file/module

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2019-11-18 12:25:35 +01:00
Thomas Lamprecht
99e3d4f605 move common metric server management part to own module
For now it only handles the plugin registration and the two recently
integrated helpers.
But, this is a prepartation to move the external metrics server
update mechanic from a stateless always-newly-connect-send-disconnect
to a statefull transaction based mechanis; see later patches

keep the PVE::Status::Plugin use in pvestatd, as we read the cfs
hosted status.cfg there, and the parser is defined by the common
status plugin base module.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-16 16:19:42 +01:00
Thomas Lamprecht
6ed1a93245 makefile: sort perlsources module list
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-16 16:10:24 +01:00
Thomas Lamprecht
1f677127e8 ui: ceph/Services: do some housekeeping
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-16 14:31:31 +01:00
Thomas Lamprecht
0a603350e0 bump version to 6.0-12
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-15 19:04:53 +01:00
Thomas Lamprecht
997f8d8435 api: ceph/metadata: add structured node versions
include the version as string and as parts, as we do the split
already. Also include the build commit, so if we re-release a ceph
version, we can differ here too.

Use node as key, to make the new entry a bit more general, could be
easily expanded with other infos, if required.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-15 18:36:37 +01:00
Thomas Lamprecht
9b12740358 api: ceph version: actually get full version
add and change the return signature for the wantarray case, which can
safely done as this is only used once (statd), and there only the
first elemen, the full version string, is used - so no breakage
potential there

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-15 18:35:39 +01:00
Thomas Lamprecht
fb6e1dd479 statd: adapt ceph update error message
"getting ceph services" sound a bit vague, like the download of those
failed, or the like..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-15 11:33:47 +01:00
Thomas Lamprecht
e0aeba5950 statd: refactor out updating ceph metadata
makes no sense to do half in line and half in a extra update_method

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-15 11:30:14 +01:00
Thomas Lamprecht
a5e921bf76 status: cleanup config parser regsistration
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-15 10:49:05 +01:00
Thomas Lamprecht
d35f97aa30 status plugins: add _connect to plugin method interface
in preparation of doing real transactions, with one batch connect +
send + disconnect, and not hundreds of those per update cycle..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-14 19:27:28 +01:00
Thomas Lamprecht
34fd8a7040 status plugins: add update_all and foreach_plug helper
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-14 19:24:24 +01:00
Thomas Lamprecht
c4c2402b77 remove some useless empty lines
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-13 17:05:44 +01:00
Thomas Lamprecht
77123edbd0 statd: refactor update_node_status
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-13 08:42:00 +01:00
Thomas Lamprecht
19978f7ec5 statd: cleanup update_node_status
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-13 08:40:13 +01:00
Aaron Lauterer
e1bd10bea7 ui: vm opts: add hint for spice foldersharing
Spice foldersharing needs the webdavd daemon installed inside the guest.
This patch adds a hint to remind the user to install it in the VM.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2019-11-12 11:28:48 +01:00
Stefan Reiter
82a0daae4d gui: fix onlineHelp for join window
We have an entire chapter written about this, why not link it too.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2019-11-11 11:47:45 +01:00
Thomas Lamprecht
1770237d1a status/graphite: refactor write_graphite to send all at once
Instead of doing multiple sends, for each status metric line one,
assemble it all in a string and send it out in a single go.
Per VM/CT/Node we had >10 lines to send, so this is quite the
reduction. But, also note that thanks to Nagler's delay algorithm
this may not had a big effect for TCP, as it buffered those small
writes anyhow.
For UDP it can reduce the packet count on the line dramatically,
though.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-07 18:59:06 +01:00
Thomas Lamprecht
00892f8614 status/graphite: just use setsockopt to set timeouts
after rethinking this it felt weird, sockets already can to this
themself, so I checked out the IO::Socket::Timeout module, and yeah,
it's just a OOP wrapper for this, hiding the "scary" struct pack.

So instead of adding that as dependency lets do it ourself.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-07 16:27:50 +01:00
Thomas Lamprecht
a8234b22b1 status/graphite: record missing module-use
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-07 16:26:54 +01:00
Dominik Csapak
c2a7475c3c gui: lxc/MPEdit: save and use the mountpoint propertyString
this way, backend only settings do not get lost (like 'size', 'shared')
when editing in the gui

this was most obvious with the new pending options, as every time
we edited a mp, we lost its size, and even setting the options
to exactly the same as the originals, we still had the mp as
'pending', but without the size

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-11-06 21:12:35 +01:00
Dominik Csapak
5f80efc887 gui: qemu/HDEdit: use propertyStringSet helper
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-11-06 21:12:35 +01:00
Dominik Csapak
e90cffd3a7 gui: Utils: add propertyString helper
this helper conditionally sets the given value to the given property
on the given object, optionally a different value

this is useful for our MP/HD Edit panels, where we set the options
of the drive/mp this way for every gui option we have

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-11-06 21:12:35 +01:00
Dominik Csapak
6466578744 gui: lxc/Resources: improve reload behaviour
reload when a new disk was added, and set the button status
on a reload

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-11-06 21:12:35 +01:00
Dominik Csapak
bef2a1f5e3 gui: qemu/HardwareView: use builtin reload function
ObjectGrid (an ancestor of PendingObjectGrid) does already have
a 'reload' function which does exactly the same, so get rid
of the local one here

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2019-11-06 21:12:35 +01:00
Thomas Lamprecht
a10cbb95b4 status/graphite: reduce default timeout to 1 second
This is for TCP only, and TCP needs roughly 1.5 time of the Round
Trip Time for connection setup, So, with 1 second timeout we're still
good for connections with 660 ms latency in-between.

The assumption is that most of the time the status server is
relatively near (same datacenter, or region), and connections to it
are datacenter grade, and not like a spotty GPRS modem.
So, reduce this timeout to ensure that we do not block to long.

If anybody needs higher timeouts they can just change the default
anyway.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-06 21:06:00 +01:00
Thomas Lamprecht
3a03f1160f status/graphite: refactor default assignments, no ternary
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-11-06 21:02:59 +01:00