1840 Commits

Author SHA1 Message Date
Aaron Lauterer
34a2222dd8 ceph tools: add erasure code management functions
Functions to manage erasure code (EC) profiles:
* add
* remove
* check if exists
* get default prefixed name
* get pool properties
* destroy crush rule

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2022-04-28 20:26:38 +02:00
Aaron Lauterer
29fe1eea7a api: ceph: $get_storages check if data-pool too
When removing a pool, we check against any storage that might have that
pool configured.
We need to check if that pool is used as data-pool too.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2022-04-28 20:26:38 +02:00
Thomas Lamprecht
24f3f2bc29 ceph tools: make rados open if not reused shorter
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-28 20:26:38 +02:00
Thomas Lamprecht
0ac20586ac vzdump: transform hook script checks to post-if notation
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-28 18:44:20 +02:00
Dylan Whyte
941ed98edd vzdump: add error if passed in hook script doesn't exist
Previously, if the '--script' argument was passed with a non-existent
file, it would state that a non-executable script was the reason for
failure. This adds a check to see if the hook script exists, in order
to provide a more accurate error message.

Also adds an 'Error:' prefix the 'script not executable' error.

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
2022-04-28 18:38:53 +02:00
Fabian Ebner
010ff16e9f vzdump: check that notes-template is a single line
While vzdump itself wouldn't mind about unescaped newlines, the
parameter isn't supposed to contain any, and when used as part of the
job config, it has to be a single line too, so make it consistent.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-28 14:16:33 +02:00
Fabian Ebner
31213d61d3 vzdump: generate notes: die upon unexpected escape character or variable
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-28 14:16:33 +02:00
Fabian Ebner
e01438a744 partially close #438: vzdump: support setting notes-template
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-28 13:45:53 +02:00
Fabian Grünbichler
c971330601 api2: network: anybridge: re-add regular bridges
commit 89d146f207225bb8ca2e01d7e79000bb37a227d1 introduced permission
checks here that caused all regular bridges to be removed from the
returned list as soon as the SDN package is installed, unless the user
is root@pam or there exists a VNET with the same ID.

this is arguably a breaking change, so limit the priv check to actually
defined VNETs for the time being, and add ALL regular bridges
uncondtionally like before.

get_local_vnets already filters by the same prvs, so we need to get the
full config to find out which IDs are VNETs and which are not.

once/iff we introduce ACL paths for *all* bridges in the future, we can
limit accordingly here.

CC: Alexandre Derumier <aderumier@odiso.com>
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-04-27 13:55:57 +02:00
Thomas Lamprecht
ca65e09904 api: next-id: honor new datacenter.cfg upper/lower range option
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-27 13:52:01 +02:00
Fabian Ebner
bbd4cdd845 vzdump: support setting protected status
Check the number of protected backups early if the protected flag
is set.

Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-27 11:37:26 +02:00
Fabian Ebner
c527d28f0c fix #3955: vzdump: new: also send mail for error during job-init hook
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-27 10:36:19 +02:00
Fabian Ebner
3c5a7616cf vzdump: new: add add_error helper
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-27 10:36:19 +02:00
Thomas Lamprecht
5ac1eaa041 replication: split out error handling and include more info in mail
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-27 10:18:30 +02:00
Thomas Lamprecht
fa4bb659a3 replication: sent always mail for first three tries and move helper
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-27 09:48:28 +02:00
Fabian Ebner
237f00be03 api: replication: don't send mails about failed replication only once
but rather multiple times becoming exponentially less frequent.

Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-26 16:09:43 +02:00
Fabian Ebner
25bea05733 ceph: services: broadcast versions: improve requesting old version
to avoid a "malformed JSON string" warning when there is no old
version present (e.g. after starting a cluster).

get_node_kv will always return something that evaluates to true, so
instead, test if the result has an entry for the current node. Also,
it's enough to request the kv for the current node only.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-26 08:29:58 +02:00
Fabian Grünbichler
725c0555d2 vzdump: always pass full list to prune marking
else this single call site is subtly different from all the rest, which
could cause problems further down the line if we ever change the prune
logic.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-04-06 14:07:21 +02:00
Fabian Ebner
7bffbd2241 vzdump: backup limit: only count unprotected backups
since they are the ones relevant for pruning and protected backups
have their own separate limit.

Since get_backup_file_list is only used in places where the
unprotected backups are needed, adapt the helper accordingly.

If there is a storage, use PVE::Storage::volume_list to count the
unprotected backups. This avoids a direct invocation of the
proxmox-backup-client for PBS and the limit check can also work for
external storage plugins which might not be dir-based or name the
backups differently.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-06 13:40:05 +02:00
Fabian Ebner
0a9ca6caf6 vzdump: backup file list: drop unused parameter
It's not used anymore since 77c7945123724564ea909439ea51b537a5d0b6c2
aligned pruning in a dumpdir with pruning on a storage.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-06 10:00:34 +02:00
Dominik Csapak
81ba0803d5 fix #3976: api/backup: make schedule/starttime truly optional on update
on create we require either starttime (+dow) or a schedule, but when
updating an existing job, this is not necessary

before we changed to schedules, the starttime was not optional either on
update, but i think there is no reason to require the user to send the
schedule/startime along every time.

the gui will send all values every time, so that was never a problem there

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-04-01 17:52:10 +02:00
Dominik Csapak
349fe2a92a Jobs: don't schedule jobs with no computable next event
if we have a schedule that has no 'next event' we should skip the scheduling
instead of schedule every round

this can happen if someone sets an schedule that has no next match.
some examples:
* 2-31 00:00 (there is not February 31st)
* mon 2022-04-02 (this would be a saturday, not monday)
* 1970-1-1 (or every other exact date in the past)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-04-01 16:47:47 +02:00
Fabian Ebner
c53d5c5ec1 api: vzdump: extract config: add content type check
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-01 09:24:35 +02:00
Fabian Ebner
0782c03469 pveam: remove: add content type check
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-01 09:24:35 +02:00
Fabian Ebner
0bd224e56b api: vzdump: extract config: check for VM.Backup privilege
In preparation to have check_volume_access() always allow access for
users with Datastore.Allocate privilege. As to not automatically give
all such users permission to extract the config too.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-01 09:24:35 +02:00
Thomas Lamprecht
c4c3ed73e1 report: lsblk: merge/group parents of sub-trees
can cut down lines outputted, especially on lvm-thin or other systems
that have volumes listed duplicated. E.g.,

Note, if nothing can be merged the output is 1:1 the same as before,
so seems like a win-win to me.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-03-30 15:33:13 +02:00
Thomas Lamprecht
ef01a35173 report: lsblk: output more columns
Output columns that can help on debugging (e.g., physical sector
size, hot-pluggable, rotational, transport) or make it easier to
cross-correlate (model, fs-type)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-03-30 15:33:00 +02:00
Thomas Lamprecht
80e456f655 report: report FS type in df call
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-03-30 15:24:23 +02:00
Thomas Lamprecht
295d8c4351 apl info: trailing whitespace cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-03-29 06:53:28 +02:00
Thomas Lamprecht
42f82359c1 api: cluster: refactor and cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-03-28 14:04:35 +02:00
Wolfgang Bumiller
fd0a63153e api: fill in nodeconfig schema
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-17 14:24:20 +01:00
Wolfgang Bumiller
9556af6134 nodeconfig: use common config parser
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-17 14:24:20 +01:00
Wolfgang Bumiller
9ee9991032 switch to Proxmox::RS::CalendarEvent
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-17 14:02:16 +01:00
Thomas Lamprecht
7851068108 api: sdn: code/whitespace refactor/fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-03-16 17:00:50 +01:00
Alexandre Derumier
89d146f207 api2 : network: anybridge: don't display bridges if user have access to vnets.
This remove vmbr* from bridgeselector if user have access to vnets.
if user need to have also access to vmbr, we can add a permission
in path "/sdn/vnets/vmbrX"

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
2022-03-16 16:43:42 +01:00
Fabian Ebner
d3cd3c9ca3 api: apt: repos: fix interfacing with perlmod
Using
    pvesh create /nodes/pve701/apt/repositories --path
    "/etc/apt/sources.list" --index 0 --enabled 1
reliably leads to
    error: invalid type: string "0", expected usize

Coerce to int to avoid this. I was not able to trigger the issue with
the "enabled" option being a string here (in PMG I was), but be on the
safe side and coerce there too. Otherwise it might get triggered by a
future, completely unrelated change further up in the API call
handling.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-02-17 14:34:18 +01:00
Thomas Lamprecht
0b1d64764f pve6to7: update expected running kernel version to 5.13 or 5.15
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-14 08:19:55 +01:00
Oguz Bektas
dfde1eec16 api: query url metadata: change default user-agent to "Proxmox VE"
To avoid being blacklisted because of the default, quite popular,
libwww-perl user-agent like reported in community forum [0].

[0]: https://forum.proxmox.com/threads/104081/

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
Tested-by: Matthias Heiserer <m.heiserer@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-04 17:49:46 +01:00
Fabian Ebner
38496d8758 api: apt: list_updates: fix dependency detection for virtual package
A virtual package does not have SelectedState Install, but the
dependency will still be satisfied if a package providing it has.

Fixes a bug, wrongly showing that postfix will be installed, when a
different mail-transport-agent is installed and a pve-manager update
is available:
https://forum.proxmox.com/threads/103413/

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-02-04 17:44:03 +01:00
Hannes Laimer
09ba41362c api: add 'type' parameter to pool GET endpoint
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-01-31 13:30:50 +01:00
Thomas Lamprecht
84502c7d1f metrics: influx: preprare value: rename $quote to $force_quote
to avoid implying that it's *only* quoted if that parameter is
truthy.

Also, check the boolean before looks_like_number call, as that is
more expensive to do.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-01-28 11:11:49 +01:00
Markus Frank
9d37535bec fix #3815: metrics: influxdb: coerce guest name always to string
InfluxDB interprets the guest name '66601' as a number and the guest
name 'vm42' as a String. This leads to problematic metrics, that will
be dropped by influxdb. Whichever comes first decides how the
"schema" is defined.

To change that add a $to_quote hashmap to define which value
shouldn't get interpreted as number by always send it quoted.

For now only quote 'name'.

Note, that while the 'nodename' and 'host' properties would have the
same problem, they are tags in InfluxDB which are always interpreted
as strings:
https://docs.influxdata.com/influxdb/v2.1/reference/syntax/line-protocol/

Signed-off-by: Markus Frank <m.frank@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-01-28 11:11:16 +01:00
Fabian Ebner
b241deb731 close #3476: vzdump: add job-init hook
which allows users to prepare the backup storage for activation (e.g.
by waking up a remote node).

Suggested-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Acked-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-01-18 12:20:26 +01:00
Dominik Csapak
c048ad30c8 vzdump: add new 'next-run' field for vzdump job listing
and calculate it by getting the next event after 'now' since
we currently have no way to get the last run time for jobs only running
on different cluster nodes

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-01-13 15:33:41 +01:00
Fabian Grünbichler
ed59fcff47 fix #3789: pass disable TLS 1.2/1.3 options
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-01-13 13:42:58 +01:00
Fabian Grünbichler
64672c2803 fix #3745: handle overridden TLS key location
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-01-13 13:42:58 +01:00
Fabian Grünbichler
ff65c92931 fix #3790: pass TLS 1.3 ciphersuites if set
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-01-13 13:42:58 +01:00
Dominik Csapak
cf2063d4e1 status: influxdb: suppress warning when no token exists
in get_credentials, we try to read the token for the specified server,
and warn if we cannot do that (e.g. because the file does not exist)

since having no token is a valid configuration, suppress that warning on
connection.

reported by a forum-user:
https://forum.proxmox.com/threads/interface-comes-up-with-all-question-marks.83287/page-2#post-438204

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-12-21 15:44:38 +01:00
Thomas Lamprecht
e4c36ea34f jobs: limit ID to maximal 64 characters
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-12-03 10:01:17 +01:00
Dominik Csapak
68df949690 api: journal: stream the journal data to the client
instead of accumulating the whole output of 'mini-journalreader' in
the api call (this can be quite big), use the download mechanic of the
http-server to stream the output to the client.

we lose some error handling possibilities, but we do not have
to allocate anything here, and since perl does not free memory after
allocating[0] this is our desired behaviour.

to keep api compatiblitiy, we need to give the journalreader the '-j'
flag to let it output json.

also tell the http server that the encoding is gzip and pipe
the output through it.

0: https://perldoc.perl.org/perlfaq3#How-can-I-free-an-array-or-hash-so-my-program-shrinks?

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-24 18:19:16 +01:00