Commit Graph

7560 Commits

Author SHA1 Message Date
Dominik Csapak
372709326e examples: add tape write benchmark
A small example that simply writes pseudo-random chunks to a drive.
This is useful to benchmark throughput on tape drives.

The output and behavior is similar to what the pool writer does, but
without writing multiple files, committing or loading data from disk.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-08 09:04:52 +02:00
Dominik Csapak
c343c3f7f6 tape: improve throughput by not unnecessarily syncing/committing
When writing data on tape, the idea was to sync/committing to tape and
the catalog to disk every 128GiB of data. For that the counter
'bytes_written' was introduced and checked after every chunk/snapshot
archive.

Sadly we forgot to reset the counter after doing so, which meant that
after 128GiB was written onto the tape, we synced/committed after every
archive on the tape for the remaining length of the tape.

Since syncing to tape and writing to disk takes a bit of time, the drive
had to slow down every time and reduced the available throughput. (In
our tests here from ~300MB/s to ~255MB/s).

By resetting the value to zero after syncing, we avoid that and increase
throughput performance when backups are bigger than 128GiB on tape.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-08 09:04:42 +02:00
Dietmar Maurer
de2cd9a688 api: delay datastore lookup after permission check
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-04-29 11:20:09 +02:00
Thomas Lamprecht
09da69cc10 update proxmox-metrics dependency to 0.3.1
to ensure that it can handle the recently lifted restrictions on the
organization and bucket parameters correctly by URL encoding them.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-26 17:55:47 +02:00
Gabriel Goller
3e69aba2d8 api-types: remove influxdb bucket name restrictions
Remove the regex for influxdb organizations and buckets. Influxdb does
not place any constraints on these names and allows all characters. This
allows influxdb organization names with slashes.

Also remove a duplicate comment and add some missing ones.

This also aligns the behavior to PVE as there are no restrictions there
either.

The motivation for this patch is this forum post:
https://forum.proxmox.com/threads/influx-db-organization-doesnt-allow-slash.145402/

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2024-04-26 17:54:51 +02:00
Thomas Lamprecht
fea9358b72 update proxmox-sys dependency to 0.5.4
to ensure the next build contains the 78bf05a4 ("fix: use fragmented
block size for space calculation") improvement.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-26 17:54:08 +02:00
Thomas Lamprecht
630be1a577 bump version to 3.2.2-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-25 12:06:33 +02:00
Lukas Wagner
eff279e771 ui: sync job: fix error if local namespace is selected first
When creating a new sync job and a local namespace is configured
without setting a remote first, the createMaxPrefixLength
was passed an array instead of a string/undefined/null, which
triggered a 'ns2.match is not a funtion exception', making the UI
glitchy afterwards.

Fixed by explicitly checking for a string. Verified that the other
user of NamespaceMaxDepthReduced, the prune job edit window, does not
break after the change.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-25 11:50:02 +02:00
Stefan Sterz
5e12ca4ce7 fix: tape ui: unset deleteEmpty in TapeBackupWindow
since the api rejects unknown parameters, deleteEmpty needs to be
unset here, because the endpoint for creating backups does not support
deleting parameters. otherwise a user will get a fairly cryptic error
message in the gui.

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2024-04-25 11:46:43 +02:00
Lukas Wagner
9ac2a76a04 ui: utils: fix defaultMailAuthor
The default mail author for SMTP and Sendmail target is
"Proxmox Backup Server - <hostname>" and not
"Proxmox Backup Server (<hostname>)".

This is just a cosmetical change which affects the empty text for the
'Author' field in the sendmail/smtp edit window.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-25 10:19:54 +02:00
Thomas Lamprecht
725c7bb4fa bump version to 3.2.1-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 22:07:05 +02:00
Stefan Lendl
911e8f98f4 ui: enable vlan widget
* Enabled the "Linux VLAN" option when creating a new interface.
* This requires the updated widget-toolkit to contain vlan field widget.

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 21:49:10 +02:00
Stefan Lendl
6f5757d9ff api: create and update vlan interfaces
* Implement setting vlan-id and vlan-raw-device in the create and update api.
* Checking if the provided vlan-raw-device exists
* Moved VLAN_INTERFACE_REGEX to top level network module to use it in
  the checking functions there. Changed to match with named capture groups.
* Unit tests to verify parsing vlan_id and vlan_raw_device from name.

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 21:49:06 +02:00
Stefan Lendl
4794006f81 fmt: fix intendation in api macro
Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 21:49:01 +02:00
Stefan Lendl
405ad7c825 config: remove unnecessary pub in various methods in NetworkConfig
Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 21:48:58 +02:00
Stefan Lendl
b276acf7ba config: parse vlan interface from config
Support three types of vlan configurations defined in interfaces,
conforming to the PVE configurations:

iface nic.<vlan-id> inet

iface vlan<vlan-id> inet
	vlan-raw-device <nic>

iface <arbitraty-name> inet
	vlan-id <vlan-id>
	vlan-raw-device <nic>

* Add lexer Token enum variants for vlan-id and vlan-raw-device and parse
  them in parse_iface_attributes.
* Add tests to verify this works in the above scenarios

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 21:48:54 +02:00
Stefan Lendl
5f14a0a2ab config: write vlan network interface
* Add vlan_id and vlan_raw_device fields to the Interface api type
* Write to the network config the vlan specific properties for vlan
  interface type
* Add several tests to verify the functionally

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 21:48:50 +02:00
Stefan Lendl
160948c9c1 tests: simple tests for writing the network config
Simple tests for manual and static configurations.

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 21:48:45 +02:00
Stefan Lendl
274c03e297 tests: move network tests to parser.rs
All current tests in network/mod.rs only test parser functionality and
  should therefore live in the parser module.

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 21:48:41 +02:00
Dominik Csapak
a577114a21 api: tape: don't allow overwriting of ids in changer/drive config
by checking the whole section config for an existing id, not only the
ones of the given type.

This prevents creation of a drive config with the same name as an
existing changer and vice versa, as it is confusing that existing things
get deleted, and we can get in the situation that we reference a changer
that does not exist anymore, i.e. consider this:

* create a changer with name `foo`
* create a drive with name `foo` and select changer `foo` for it

this would delete the changer config, but still reference it, leading
to errors when trying to use it.

We could implement support for separate id namespaces in section configs
for different types, but this is much more easier to do and be enough
for now.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-04-24 21:40:36 +02:00
Aaron Lauterer
2e6a4a9d28 installation: add section about unattended/automatic installation
Mention and briefly explain it. The main part of the documentation will
live in the Wiki for now as it applies to not just Proxmox Mail Gateway.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
  [ TL: adapt to changes made in the wiki article ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 21:39:09 +02:00
Stefan Sterz
0b449fe828 auth: request a write lock when exposing the LockedTfaConfig
this function is called every time a user tries to log in to check
whether a tfa challenge is required. since the tfa config may need to
be written by the auth api (e.g. when a recovery key is used) this
needs to use a write lock instead of a read lock in order to avoid
potential races.

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2024-04-24 21:28:11 +02:00
Christoph Heiss
a819f056c2 docs: user-management: add section about AD realm support
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-24 21:06:14 +02:00
Christoph Heiss
1819989bd0 manager: add subcommand for managing AD realms
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-24 21:06:14 +02:00
Christoph Heiss
a8636bbb66 realm sync: add sync job for AD realms
Basically just a thin wrapper over the existing LDAP-based realm sync
job, which retrieves the appropriate config and sets the correct user
attributes.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-24 21:06:14 +02:00
Christoph Heiss
d07013a46c config: domains: add new "ad" section type for AD realms
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-24 21:06:14 +02:00
Christoph Heiss
c7051f3342 api: access: add routes for managing AD realms
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-24 21:06:14 +02:00
Thomas Lamprecht
b18a8170cb d/control: record proxmox-notify build dependency
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 10:46:52 +02:00
Thomas Lamprecht
92cd992f1d d/copyright: update years
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 00:34:19 +02:00
Thomas Lamprecht
6d8b2585b3 bump version to 3.2.0-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:47:30 +02:00
Lukas Wagner
d6ed84f88c api: notification: also list datastores if user has only Backup privs
Use the /admin/datatore API instead of /config/datastore to get a list
of all available datastores - this ensures that users can see
datastores even if they only have Datastore.Backup privs.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
1e5cb74d89 ui: notifications: pull in UX improvements for match rules creation
These changes have not been applied yet in widget toolkit, but
are very valuable for the initial integration in PBS.
We override modified components and replace them with the patched
variants.
The changes change the edit window such that known field names and
values are suggested in a combobox. Also, the 'exact' match mode
can now match multiple values.

This can and *should* be removed once the changes from [1] are
merged into the widget toolkit.

[1] https://lists.proxmox.com/pipermail/pve-devel/2024-April/063539.html

Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
a52f2819e7 ui: util: override default mail author for sendmail/smtp targets
Otherwise, 'Proxmox VE' is shown as the default author in the UI.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
37e8ba56cd docs: add documentation for notification system
Mostly copied from PVE and adapted where it makes sense.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
b77b25abfc docgen: generate synopsis for notifications{-priv, }.cfg
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
ddc94b03c2 proxmox-backup-manager: add CLI for SMTP endpoints
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
fc257af10b proxmox-backup-manager: add CLI for sendmail endpoints
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
44951127b6 proxmox-backup-manager: add CLI for gotify endpoints
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
3f2ecae42a proxmox-backup-manager: add CLI for notification matchers
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
e60e050a1b proxmox-backup-manager: add CLI for notification targets
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
ce15b51507 ui: permissions paths: add /system/notifications to combobox
The /system/notifications ACL path is used for configuring the
notification system.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
be9b09bb0d ui: datastore edit: make new stores use notification system by default
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
ee4b6cf18b ui: utils: add overrides for known notification metadata fields/values
This mechanism allows having nice, translatable notification event
types and fields.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
8dc47a5774 ui: datastore options: add 'notification-mode' parameter
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
461d0276e2 ui: tape restore: add 'notification-mode' parameter
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
168e2f2967 ui: tape backup: add selector for 'notification-mode'
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
5e2ab2765c ui: tape backup job: add selector for notification-mode
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
0098c9f6f2 ui: add notification config panel
This commit adds the same notification configuration panel that we
already use in Proxmox VE.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
a4f1b175d1 server: notifications: send tape notifications via notification system
If the `notification-mode` parameter is set to `legacy-sendmail`, then
we still use the new infrastructure, but don't consider the
notification config and use a hard-coded sendmail endpoint directly.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
1d2069d158 server: notifications: send acme notifications via notification system
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00