Commit Graph

7583 Commits

Author SHA1 Message Date
Lukas Wagner
4ce1962124 docs: document notification-mode and merge old notification section
This new section describes how the notification-mode parameter works.
The section also contains also parts of the old notification section
from the maintenance chapter, reusing the description of the
`notify` and `notify-user` parameters.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
2024-05-22 17:50:03 +02:00
Gabriel Goller
1d0bcd2359 notifications: fix legacy sync notifications
When using the legacy notifications the sync mode would pick up the
settings from the prune-job, which default to Error. This completely
disables notifications for successful sync-jobs when using the legacy
system.

Reported in the forum: https://forum.proxmox.com/threads/147018/

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Max Carrara <m.carrara@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
2024-05-22 17:31:51 +02:00
Wolfgang Bumiller
71c65d2282 bump d/control
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-05-22 16:05:53 +02:00
Wolfgang Bumiller
61f55ceee1 bump proxmox-auth-api to 0.4
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-05-22 16:05:49 +02:00
Stefan Sterz
9ce3d0c88c auth: use auth-api when generating keys and generate ec keys
this commit switches pbs over to generating ed25519 keys when
generating new auth api keys. this also removes the last direct
usages of openssl here and further unifies key handling in the auth
api.

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2024-05-22 16:04:21 +02:00
Stefan Sterz
048a81cc55 auth: move to auth-api's private and public keys when loading keys
this commit moves away from using openssl's `PKey` and uses the
wrappers from proxmox-auth-api. this allows us to handle keys in a
more flexible way and enables as to move to ec based crypto for the
authkey in the future.

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2024-05-22 16:04:19 +02:00
Stefan Sterz
8e77260256 auth: upgrade hashes on user log in
if a users password is not hashed with the latest password hashing
function, re-hash the password with the newest hashing function. we
can only do this on login and after the password has been validated,
as this is the only point at which we have access to the plain text
password and also know that it matched the original password.

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2024-05-22 16:04:18 +02:00
Stefan Sterz
cf71dc2428 auth: move to hmac keys for csrf tokens
previously we used a self-rolled implementation for csrf tokens. while
it's unlikely to cause issues in reality, as csrf tokens are only
valid for a given tickets lifetime, there are still theoretical
attacks on our implementation. so move all of this code into the
proxmox-auth-api crate and use hmac instead.

this change should not impact existing installations for now, as this
falls back to the old implementation if a key is already present. hmac
keys will only be used for new installations and if users manually
remove the old key and

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2024-05-22 16:04:16 +02:00
Thomas Lamprecht
3c23c4c250 ui: garbage-collection: use different state-id for global and per-datastore view
For one these different views have different columns shown, and more
importantly: with the state being shared one could change sorting in
the global view and then have that applied in the per-datastore view
too, even if one cannot sort that view explicitly otherwise as there's
just one row anyway. This small glitch might lead to a bit of
confusion in the worst case and looks unpolished in any way.

Note that I explicitly decided against encoding the datastore in the
state-id for the per-datastore views for now, as most users will want
to adapt layout (like column width) for all per-datastores views.

Having to re-do that for every datastore separately can be quite a
nuisance while the same user wanting different layout for each
datastore in their per-datastore view seems rather to be an edge case.
And we can always change this, so starting out with the slightly more
restricted design that has less browser local data to be saved seems
better w.r.t. maintainability.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-05-21 11:34:21 +02:00
Gabriel Goller
0385762859 fix #5422: ui: garbage-collection: make columns in global view sortable
Make columns sortable in the global 'Prune & GC Jobs' view. In the
per-datastore view the columns will not be sortable as there can only be
one job.

Fixes: db3fd213 ("fix #3217: ui: global prune and gc job view")

Co-authored-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Max Carrara <m.carrara@proxmox.com>
2024-05-21 11:29:31 +02:00
Dominik Csapak
5901050e7a restore daemon: search disk also with truncated serial
the disk serial given to virtio disks only can be 20 characters, so
looking for a disk with a longer serial will always fail (like
'drive-tpmstate0-backup'). If the serial is longer, also try with the
truncated one. Leave the first try in place in case the limit changes.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-16 11:50:45 +02:00
Dominik Csapak
7bc7601f65 restore daemon: log some errors for dir traversal
in case we cannot stat a file in the restore vm, log the path and reason
why. This should normally not happen, but when it does, the path and
error might help us find the issue.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-16 11:50:45 +02:00
Dominik Csapak
31edde560a fix #5465: restore daemon: mount ntfs with utf8 charset
since the change in our restore image to ntfs3, non iso8859-1 filenames
were broken. Fix that by adding the 'iocharset' option to ntfs3.

Leave the ntfs option in place, so that if the image gets booted
with an older kernel for some reason, this still works.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-16 11:50:45 +02:00
Thomas Lamprecht
98e2c16a04 ui: update online help info
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-05-15 19:05:47 +02:00
Dietmar Maurer
00ef50146c api: syslog: fix api macro to return array instead of object.
The implementation already returns Vec, so this change is to generate
correct api documentation.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-05-15 12:17:03 +02:00
Dominik Csapak
6d4b380c3d tape: write informational MAM attributes on tapes
namely:

Vendor: Proxmox
Name: Backup Server
Version: current running package version
User Label Text: the label text
Media Pool: the current media pool

write it on labeling and when writing a new media-set to a tape.

While we currently don't use this info for anything, this can help users
to identify tapes, even with different backup software.

If we need it in the future, we can e.g. make decisions based on these
fields (e.g. the version).

On format, delete them again.

Note that some VTLs don't correctly delete the attributes from the
virtual tapes.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-15 09:35:56 +02:00
Dominik Csapak
9d2fc6565f tape: correct mam format for some attributes
Some MAM attributes are of type 'TEXT' that is not only ascii, but
controlled by an addition field that specifies various 8bit text
formats.

For now, simply assume utf8 as the default is ascii, and we don't expect
any data that is not ASCII anyway.

This will be needed when we'll want to write those attributes.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-15 09:32:10 +02:00
Dominik Csapak
b5af9333f8 tape: include drive activity in status
Since we don't query each drives status seperately, but rely on a single
call to the drives listing parameter for that, we now add the option
to query the activity there too. This makes that data avaiable for us
to show in a seperate (by default hidden) column.

Also we show the activity in the 'State' column when the drive is idle
from our perspective. This is useful when e.g. an LTO-9 tape is loaded
the first time and is calibrating, since that happens automatically.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-14 10:31:33 +02:00
Dominik Csapak
4ebb08a5f0 tape: drive status: make some depend on the activity
when the tape drive has an activity (and the tape is in motion), certain
calls block until the operation is finished. Since we cannot predict how
long it's going to be and it can be quite long in certain cases,
skip those calls when the drive is doing anything.

If we cannot determine the activity, try to do the queries.

We have to extend the check for a loaded drive in the UI, since the
position is not available during any activity.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-14 10:27:23 +02:00
Dominik Csapak
1d6b1e0258 tape: add drive activity to drive status api
and show it in the gui for single drives. Adds the known values for the
activity to the UI.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-14 10:25:42 +02:00
Dominik Csapak
3f1a084b90 tape: add functions to parse drive device activity
we use the VHF part from the DT Device Activity page for that.
This is intended to query the drive for it's current state and activity.

Currently only the activity is parsed and used.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-14 10:11:06 +02:00
Dominik Csapak
4b21a00744 tape: save 'bytes used' in tape inventory
and show them on the ui. This can help uses with seeing how much a tape
is used.

The value is updated on 'commit' and when the tape is changed during a
backup.

For drives not supporting the volume statistics, this is simply skipped.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-14 10:07:57 +02:00
Dietmar Maurer
aea66a8128 tape: cleanup: rename bytes_written to bytes_written_after_sync 2024-05-08 09:16:57 +02:00
Dominik Csapak
372709326e examples: add tape write benchmark
A small example that simply writes pseudo-random chunks to a drive.
This is useful to benchmark throughput on tape drives.

The output and behavior is similar to what the pool writer does, but
without writing multiple files, committing or loading data from disk.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-08 09:04:52 +02:00
Dominik Csapak
c343c3f7f6 tape: improve throughput by not unnecessarily syncing/committing
When writing data on tape, the idea was to sync/committing to tape and
the catalog to disk every 128GiB of data. For that the counter
'bytes_written' was introduced and checked after every chunk/snapshot
archive.

Sadly we forgot to reset the counter after doing so, which meant that
after 128GiB was written onto the tape, we synced/committed after every
archive on the tape for the remaining length of the tape.

Since syncing to tape and writing to disk takes a bit of time, the drive
had to slow down every time and reduced the available throughput. (In
our tests here from ~300MB/s to ~255MB/s).

By resetting the value to zero after syncing, we avoid that and increase
throughput performance when backups are bigger than 128GiB on tape.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-08 09:04:42 +02:00
Dietmar Maurer
de2cd9a688 api: delay datastore lookup after permission check
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-04-29 11:20:09 +02:00
Thomas Lamprecht
09da69cc10 update proxmox-metrics dependency to 0.3.1
to ensure that it can handle the recently lifted restrictions on the
organization and bucket parameters correctly by URL encoding them.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-26 17:55:47 +02:00
Gabriel Goller
3e69aba2d8 api-types: remove influxdb bucket name restrictions
Remove the regex for influxdb organizations and buckets. Influxdb does
not place any constraints on these names and allows all characters. This
allows influxdb organization names with slashes.

Also remove a duplicate comment and add some missing ones.

This also aligns the behavior to PVE as there are no restrictions there
either.

The motivation for this patch is this forum post:
https://forum.proxmox.com/threads/influx-db-organization-doesnt-allow-slash.145402/

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2024-04-26 17:54:51 +02:00
Thomas Lamprecht
fea9358b72 update proxmox-sys dependency to 0.5.4
to ensure the next build contains the 78bf05a4 ("fix: use fragmented
block size for space calculation") improvement.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-26 17:54:08 +02:00
Thomas Lamprecht
630be1a577 bump version to 3.2.2-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-25 12:06:33 +02:00
Lukas Wagner
eff279e771 ui: sync job: fix error if local namespace is selected first
When creating a new sync job and a local namespace is configured
without setting a remote first, the createMaxPrefixLength
was passed an array instead of a string/undefined/null, which
triggered a 'ns2.match is not a funtion exception', making the UI
glitchy afterwards.

Fixed by explicitly checking for a string. Verified that the other
user of NamespaceMaxDepthReduced, the prune job edit window, does not
break after the change.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-25 11:50:02 +02:00
Stefan Sterz
5e12ca4ce7 fix: tape ui: unset deleteEmpty in TapeBackupWindow
since the api rejects unknown parameters, deleteEmpty needs to be
unset here, because the endpoint for creating backups does not support
deleting parameters. otherwise a user will get a fairly cryptic error
message in the gui.

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2024-04-25 11:46:43 +02:00
Lukas Wagner
9ac2a76a04 ui: utils: fix defaultMailAuthor
The default mail author for SMTP and Sendmail target is
"Proxmox Backup Server - <hostname>" and not
"Proxmox Backup Server (<hostname>)".

This is just a cosmetical change which affects the empty text for the
'Author' field in the sendmail/smtp edit window.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-25 10:19:54 +02:00
Thomas Lamprecht
725c7bb4fa bump version to 3.2.1-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 22:07:05 +02:00
Stefan Lendl
911e8f98f4 ui: enable vlan widget
* Enabled the "Linux VLAN" option when creating a new interface.
* This requires the updated widget-toolkit to contain vlan field widget.

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 21:49:10 +02:00
Stefan Lendl
6f5757d9ff api: create and update vlan interfaces
* Implement setting vlan-id and vlan-raw-device in the create and update api.
* Checking if the provided vlan-raw-device exists
* Moved VLAN_INTERFACE_REGEX to top level network module to use it in
  the checking functions there. Changed to match with named capture groups.
* Unit tests to verify parsing vlan_id and vlan_raw_device from name.

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 21:49:06 +02:00
Stefan Lendl
4794006f81 fmt: fix intendation in api macro
Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 21:49:01 +02:00
Stefan Lendl
405ad7c825 config: remove unnecessary pub in various methods in NetworkConfig
Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 21:48:58 +02:00
Stefan Lendl
b276acf7ba config: parse vlan interface from config
Support three types of vlan configurations defined in interfaces,
conforming to the PVE configurations:

iface nic.<vlan-id> inet

iface vlan<vlan-id> inet
	vlan-raw-device <nic>

iface <arbitraty-name> inet
	vlan-id <vlan-id>
	vlan-raw-device <nic>

* Add lexer Token enum variants for vlan-id and vlan-raw-device and parse
  them in parse_iface_attributes.
* Add tests to verify this works in the above scenarios

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 21:48:54 +02:00
Stefan Lendl
5f14a0a2ab config: write vlan network interface
* Add vlan_id and vlan_raw_device fields to the Interface api type
* Write to the network config the vlan specific properties for vlan
  interface type
* Add several tests to verify the functionally

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 21:48:50 +02:00
Stefan Lendl
160948c9c1 tests: simple tests for writing the network config
Simple tests for manual and static configurations.

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 21:48:45 +02:00
Stefan Lendl
274c03e297 tests: move network tests to parser.rs
All current tests in network/mod.rs only test parser functionality and
  should therefore live in the parser module.

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Folke Gleumes <f.gleumes@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 21:48:41 +02:00
Dominik Csapak
a577114a21 api: tape: don't allow overwriting of ids in changer/drive config
by checking the whole section config for an existing id, not only the
ones of the given type.

This prevents creation of a drive config with the same name as an
existing changer and vice versa, as it is confusing that existing things
get deleted, and we can get in the situation that we reference a changer
that does not exist anymore, i.e. consider this:

* create a changer with name `foo`
* create a drive with name `foo` and select changer `foo` for it

this would delete the changer config, but still reference it, leading
to errors when trying to use it.

We could implement support for separate id namespaces in section configs
for different types, but this is much more easier to do and be enough
for now.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-04-24 21:40:36 +02:00
Aaron Lauterer
2e6a4a9d28 installation: add section about unattended/automatic installation
Mention and briefly explain it. The main part of the documentation will
live in the Wiki for now as it applies to not just Proxmox Mail Gateway.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
  [ TL: adapt to changes made in the wiki article ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 21:39:09 +02:00
Stefan Sterz
0b449fe828 auth: request a write lock when exposing the LockedTfaConfig
this function is called every time a user tries to log in to check
whether a tfa challenge is required. since the tfa config may need to
be written by the auth api (e.g. when a recovery key is used) this
needs to use a write lock instead of a read lock in order to avoid
potential races.

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2024-04-24 21:28:11 +02:00
Christoph Heiss
a819f056c2 docs: user-management: add section about AD realm support
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-24 21:06:14 +02:00
Christoph Heiss
1819989bd0 manager: add subcommand for managing AD realms
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-24 21:06:14 +02:00
Christoph Heiss
a8636bbb66 realm sync: add sync job for AD realms
Basically just a thin wrapper over the existing LDAP-based realm sync
job, which retrieves the appropriate config and sets the correct user
attributes.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-24 21:06:14 +02:00
Christoph Heiss
d07013a46c config: domains: add new "ad" section type for AD realms
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-24 21:06:14 +02:00
Christoph Heiss
c7051f3342 api: access: add routes for managing AD realms
Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-24 21:06:14 +02:00