7699 Commits

Author SHA1 Message Date
Fabian Grünbichler
4940514b0f bump version to 3.2.4-1
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-06-05 16:24:36 +02:00
Christian Ebner
9978f6934b datastore: dynamic index: add method to get digest
In preparation for injecting reused payload chunks in payload streams
for regular files with unchanged metaddata. Allows to get the digest
of a dynamic index entry to construct a reusable dynamic entry from
it.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-05 10:47:36 +02:00
Christian Ebner
846e10cdb4 api: datastore: refactor getting local chunk reader
Move the code to get the local chunk reader to a dedicated function
to make it reusable. The same code is required to get the local chunk
reader for the payload stream for split stream archives.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-05 09:59:13 +02:00
Christian Ebner
3e57f3dc91 client: backup: factor out extension from backup target
Instead of composing the backup target name and pushing it to the
backup list, push the archive name and extension separately, only
constructing it while iterating the list later.

By this it remains possible to additionally prefix the extension, as
required with the separate pxar metadata and payload indexes.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-06-05 09:59:13 +02:00
Shannon Sterz
94d6a65dd6 auth: add locking to PbsAuthenticator to avoid race conditions
currently we don't lock the shadow file when removing or storing a
password. by adding locking here we avoid a situation where storing
and/or removing a password concurrently could lead to a race
condition. in this scenario it is possible that a password isn't
persisted or a password isn't removed. we already do this for
the "token.shadow" file, so just use the same mechanism here.

Signed-off-by: Shannon Sterz <s.sterz@proxmox.com>
2024-06-03 10:55:02 +02:00
Fiona Ebner
843211b050 fix #5503: d/control: bump dependency for proxmox-widget-toolkit
With proxmox-widget-toolkit < 4.1.4, loading the UI will fail with
a JavaScript error:

> Uncaught TypeError: Proxmox.Utils.overrideNotificationFieldName is not a function

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
2024-06-03 09:50:19 +02:00
Gabriel Goller
92c0b1866b fix: api: permission using wrong pathname
The read_interface endpoint uses the wrong path identifier. It has been
renamed to 'iface' some time ago but hasn't been changed here.

When a user has a permission on '/' with 'Admin', he wasn't able to
show the config of a single interface, as the non-existent path didn't
match.

Reported-by: https://forum.proxmox.com/threads/permissons-not-working-for-network-settings.147899/

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2024-05-31 11:03:28 +02:00
Wolfgang Bumiller
83e748baf5 fixup build with new acme crate
We missed an API break in the acme crate versioning...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-05-27 10:54:03 +02:00
Fabian Grünbichler
8c0bbc0d97 trivial clippy fixes
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-05-24 12:49:59 +02:00
Fabian Grünbichler
b096c590eb run cargo fmt
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-05-24 12:49:21 +02:00
Thomas Lamprecht
1d4afdccea bump version to 3.2.3-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-05-22 19:32:15 +02:00
Thomas Lamprecht
e50448e4ec tape: rework setting MAM Host type attributes
The product name is Proxmox Backup Server, not just Backup Server,
that makes no sense on its own and it really cannot be expected by
tools extracting any Medium Auxiliary Memory (MAM) info to render it
as `${app_vendor} ${app_name}`.

Drop the comment about ignoring errors, that's pretty clear with
the only-log-error construct.

Instead, add some comments about what the hex numbers refers too and
what their respective length (limit) is. The names where taken from
Table 315 "MAM Host type attributes" in the "IBM LTO SCSI Reference"
for LTO 9.

Slightly off-topic: The tape code really is a mess with sprinkling
those hex numbers hard coded all over the place, often with some
unchecked coupling in other places (like here, the list of set MAM
attrs and the one that get cleared can easily get out of sync..), but
that's for another time to clean-up (I need to cut a release).

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-05-22 19:15:16 +02:00
Thomas Lamprecht
23a9d70d57 build config: add constant for full cargo crate version
and a todo comment to document some cleanup potential

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-05-22 19:02:28 +02:00
Fabian Grünbichler
a55c6efbf7 acme: explicitly ask for custom directory URI
instead of blocking on input without telling the user what's going on.

Reported on the forum: https://forum.proxmox.com/threads/147058/

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-05-22 17:53:40 +02:00
Lukas Wagner
1665eb2e48 ui: datastore options: link to 'notification-mode' section
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2024-05-22 17:50:03 +02:00
Lukas Wagner
c730196684 docs: notifications: rewrite overview for more clarity
Also link to the following subsections where applicable.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2024-05-22 17:50:03 +02:00
Lukas Wagner
4ce1962124 docs: document notification-mode and merge old notification section
This new section describes how the notification-mode parameter works.
The section also contains also parts of the old notification section
from the maintenance chapter, reusing the description of the
`notify` and `notify-user` parameters.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
2024-05-22 17:50:03 +02:00
Gabriel Goller
1d0bcd2359 notifications: fix legacy sync notifications
When using the legacy notifications the sync mode would pick up the
settings from the prune-job, which default to Error. This completely
disables notifications for successful sync-jobs when using the legacy
system.

Reported in the forum: https://forum.proxmox.com/threads/147018/

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Max Carrara <m.carrara@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
2024-05-22 17:31:51 +02:00
Wolfgang Bumiller
71c65d2282 bump d/control
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-05-22 16:05:53 +02:00
Wolfgang Bumiller
61f55ceee1 bump proxmox-auth-api to 0.4
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-05-22 16:05:49 +02:00
Stefan Sterz
9ce3d0c88c auth: use auth-api when generating keys and generate ec keys
this commit switches pbs over to generating ed25519 keys when
generating new auth api keys. this also removes the last direct
usages of openssl here and further unifies key handling in the auth
api.

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2024-05-22 16:04:21 +02:00
Stefan Sterz
048a81cc55 auth: move to auth-api's private and public keys when loading keys
this commit moves away from using openssl's `PKey` and uses the
wrappers from proxmox-auth-api. this allows us to handle keys in a
more flexible way and enables as to move to ec based crypto for the
authkey in the future.

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2024-05-22 16:04:19 +02:00
Stefan Sterz
8e77260256 auth: upgrade hashes on user log in
if a users password is not hashed with the latest password hashing
function, re-hash the password with the newest hashing function. we
can only do this on login and after the password has been validated,
as this is the only point at which we have access to the plain text
password and also know that it matched the original password.

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2024-05-22 16:04:18 +02:00
Stefan Sterz
cf71dc2428 auth: move to hmac keys for csrf tokens
previously we used a self-rolled implementation for csrf tokens. while
it's unlikely to cause issues in reality, as csrf tokens are only
valid for a given tickets lifetime, there are still theoretical
attacks on our implementation. so move all of this code into the
proxmox-auth-api crate and use hmac instead.

this change should not impact existing installations for now, as this
falls back to the old implementation if a key is already present. hmac
keys will only be used for new installations and if users manually
remove the old key and

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2024-05-22 16:04:16 +02:00
Thomas Lamprecht
3c23c4c250 ui: garbage-collection: use different state-id for global and per-datastore view
For one these different views have different columns shown, and more
importantly: with the state being shared one could change sorting in
the global view and then have that applied in the per-datastore view
too, even if one cannot sort that view explicitly otherwise as there's
just one row anyway. This small glitch might lead to a bit of
confusion in the worst case and looks unpolished in any way.

Note that I explicitly decided against encoding the datastore in the
state-id for the per-datastore views for now, as most users will want
to adapt layout (like column width) for all per-datastores views.

Having to re-do that for every datastore separately can be quite a
nuisance while the same user wanting different layout for each
datastore in their per-datastore view seems rather to be an edge case.
And we can always change this, so starting out with the slightly more
restricted design that has less browser local data to be saved seems
better w.r.t. maintainability.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-05-21 11:34:21 +02:00
Gabriel Goller
0385762859 fix #5422: ui: garbage-collection: make columns in global view sortable
Make columns sortable in the global 'Prune & GC Jobs' view. In the
per-datastore view the columns will not be sortable as there can only be
one job.

Fixes: db3fd213 ("fix #3217: ui: global prune and gc job view")

Co-authored-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Max Carrara <m.carrara@proxmox.com>
2024-05-21 11:29:31 +02:00
Dominik Csapak
5901050e7a restore daemon: search disk also with truncated serial
the disk serial given to virtio disks only can be 20 characters, so
looking for a disk with a longer serial will always fail (like
'drive-tpmstate0-backup'). If the serial is longer, also try with the
truncated one. Leave the first try in place in case the limit changes.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-16 11:50:45 +02:00
Dominik Csapak
7bc7601f65 restore daemon: log some errors for dir traversal
in case we cannot stat a file in the restore vm, log the path and reason
why. This should normally not happen, but when it does, the path and
error might help us find the issue.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-16 11:50:45 +02:00
Dominik Csapak
31edde560a fix #5465: restore daemon: mount ntfs with utf8 charset
since the change in our restore image to ntfs3, non iso8859-1 filenames
were broken. Fix that by adding the 'iocharset' option to ntfs3.

Leave the ntfs option in place, so that if the image gets booted
with an older kernel for some reason, this still works.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-16 11:50:45 +02:00
Thomas Lamprecht
98e2c16a04 ui: update online help info
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-05-15 19:05:47 +02:00
Dietmar Maurer
00ef50146c api: syslog: fix api macro to return array instead of object.
The implementation already returns Vec, so this change is to generate
correct api documentation.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-05-15 12:17:03 +02:00
Dominik Csapak
6d4b380c3d tape: write informational MAM attributes on tapes
namely:

Vendor: Proxmox
Name: Backup Server
Version: current running package version
User Label Text: the label text
Media Pool: the current media pool

write it on labeling and when writing a new media-set to a tape.

While we currently don't use this info for anything, this can help users
to identify tapes, even with different backup software.

If we need it in the future, we can e.g. make decisions based on these
fields (e.g. the version).

On format, delete them again.

Note that some VTLs don't correctly delete the attributes from the
virtual tapes.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-15 09:35:56 +02:00
Dominik Csapak
9d2fc6565f tape: correct mam format for some attributes
Some MAM attributes are of type 'TEXT' that is not only ascii, but
controlled by an addition field that specifies various 8bit text
formats.

For now, simply assume utf8 as the default is ascii, and we don't expect
any data that is not ASCII anyway.

This will be needed when we'll want to write those attributes.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-15 09:32:10 +02:00
Dominik Csapak
b5af9333f8 tape: include drive activity in status
Since we don't query each drives status seperately, but rely on a single
call to the drives listing parameter for that, we now add the option
to query the activity there too. This makes that data avaiable for us
to show in a seperate (by default hidden) column.

Also we show the activity in the 'State' column when the drive is idle
from our perspective. This is useful when e.g. an LTO-9 tape is loaded
the first time and is calibrating, since that happens automatically.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-14 10:31:33 +02:00
Dominik Csapak
4ebb08a5f0 tape: drive status: make some depend on the activity
when the tape drive has an activity (and the tape is in motion), certain
calls block until the operation is finished. Since we cannot predict how
long it's going to be and it can be quite long in certain cases,
skip those calls when the drive is doing anything.

If we cannot determine the activity, try to do the queries.

We have to extend the check for a loaded drive in the UI, since the
position is not available during any activity.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-14 10:27:23 +02:00
Dominik Csapak
1d6b1e0258 tape: add drive activity to drive status api
and show it in the gui for single drives. Adds the known values for the
activity to the UI.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-14 10:25:42 +02:00
Dominik Csapak
3f1a084b90 tape: add functions to parse drive device activity
we use the VHF part from the DT Device Activity page for that.
This is intended to query the drive for it's current state and activity.

Currently only the activity is parsed and used.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-14 10:11:06 +02:00
Dominik Csapak
4b21a00744 tape: save 'bytes used' in tape inventory
and show them on the ui. This can help uses with seeing how much a tape
is used.

The value is updated on 'commit' and when the tape is changed during a
backup.

For drives not supporting the volume statistics, this is simply skipped.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-14 10:07:57 +02:00
Dietmar Maurer
aea66a8128 tape: cleanup: rename bytes_written to bytes_written_after_sync 2024-05-08 09:16:57 +02:00
Dominik Csapak
372709326e examples: add tape write benchmark
A small example that simply writes pseudo-random chunks to a drive.
This is useful to benchmark throughput on tape drives.

The output and behavior is similar to what the pool writer does, but
without writing multiple files, committing or loading data from disk.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-08 09:04:52 +02:00
Dominik Csapak
c343c3f7f6 tape: improve throughput by not unnecessarily syncing/committing
When writing data on tape, the idea was to sync/committing to tape and
the catalog to disk every 128GiB of data. For that the counter
'bytes_written' was introduced and checked after every chunk/snapshot
archive.

Sadly we forgot to reset the counter after doing so, which meant that
after 128GiB was written onto the tape, we synced/committed after every
archive on the tape for the remaining length of the tape.

Since syncing to tape and writing to disk takes a bit of time, the drive
had to slow down every time and reduced the available throughput. (In
our tests here from ~300MB/s to ~255MB/s).

By resetting the value to zero after syncing, we avoid that and increase
throughput performance when backups are bigger than 128GiB on tape.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-05-08 09:04:42 +02:00
Dietmar Maurer
de2cd9a688 api: delay datastore lookup after permission check
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-04-29 11:20:09 +02:00
Thomas Lamprecht
09da69cc10 update proxmox-metrics dependency to 0.3.1
to ensure that it can handle the recently lifted restrictions on the
organization and bucket parameters correctly by URL encoding them.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-26 17:55:47 +02:00
Gabriel Goller
3e69aba2d8 api-types: remove influxdb bucket name restrictions
Remove the regex for influxdb organizations and buckets. Influxdb does
not place any constraints on these names and allows all characters. This
allows influxdb organization names with slashes.

Also remove a duplicate comment and add some missing ones.

This also aligns the behavior to PVE as there are no restrictions there
either.

The motivation for this patch is this forum post:
https://forum.proxmox.com/threads/influx-db-organization-doesnt-allow-slash.145402/

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2024-04-26 17:54:51 +02:00
Thomas Lamprecht
fea9358b72 update proxmox-sys dependency to 0.5.4
to ensure the next build contains the 78bf05a4 ("fix: use fragmented
block size for space calculation") improvement.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-26 17:54:08 +02:00
Thomas Lamprecht
630be1a577 bump version to 3.2.2-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-25 12:06:33 +02:00
Lukas Wagner
eff279e771 ui: sync job: fix error if local namespace is selected first
When creating a new sync job and a local namespace is configured
without setting a remote first, the createMaxPrefixLength
was passed an array instead of a string/undefined/null, which
triggered a 'ns2.match is not a funtion exception', making the UI
glitchy afterwards.

Fixed by explicitly checking for a string. Verified that the other
user of NamespaceMaxDepthReduced, the prune job edit window, does not
break after the change.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-25 11:50:02 +02:00
Stefan Sterz
5e12ca4ce7 fix: tape ui: unset deleteEmpty in TapeBackupWindow
since the api rejects unknown parameters, deleteEmpty needs to be
unset here, because the endpoint for creating backups does not support
deleting parameters. otherwise a user will get a fairly cryptic error
message in the gui.

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2024-04-25 11:46:43 +02:00
Lukas Wagner
9ac2a76a04 ui: utils: fix defaultMailAuthor
The default mail author for SMTP and Sendmail target is
"Proxmox Backup Server - <hostname>" and not
"Proxmox Backup Server (<hostname>)".

This is just a cosmetical change which affects the empty text for the
'Author' field in the sendmail/smtp edit window.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-25 10:19:54 +02:00
Thomas Lamprecht
725c7bb4fa bump version to 3.2.1-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-24 22:07:05 +02:00