5
0
mirror of git://git.proxmox.com/git/proxmox-backup.git synced 2025-02-09 09:57:40 +03:00

7545 Commits

Author SHA1 Message Date
Lukas Wagner
57570bda1d notifications: allow sending notifications via proxmox_notify
- Set the context in proxmox_notify
  - Add helper function which queues notifications to a spool
    directory
  - Set up a worker task, running in the privileged process, which
    periodically checks the spool directory for queued notifications

The queuing is needed because on PBS we send most if not all
notifications from the proxy-process running as the `backup` user.
However, to have access to the protected passwords/tokens for various
notification endpoints, we need to read the notification config as
root.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
2764be3db5 server: rename email_notifications module to notifications
The module will be extended to interact with the proxmox_notify crate,
hence the name change seems to be in order.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Lukas Wagner
34ab74a0c9 pbs-config: add module for loading notification config
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-23 23:14:46 +02:00
Maximiliano Sandoval
24b1c641a3 fix #5251: login: set autocomplete on password and user
Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-04-23 17:21:53 +02:00
Fabian Grünbichler
6ae6337398 GC status: reduce code duplication
the schedule handling is the same whether there was a last run or not, so let's
do it once and not twice. the duration can be stored right away, instead of
using an intermediate variable.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-04-22 13:58:08 +02:00
Fabian Grünbichler
814eb2751b ui: don't re-calculate GC duration
it is returned by the API anyway

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-04-22 13:58:08 +02:00
Fabian Grünbichler
113de4515b api: merge garbage-collection-status and -job-status
the latter was newly introduced, and they both return basically the same
information now. the new extended (job) status struct is a strict superset of
the old status struct, so this is not a breaking change API wise.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-04-22 13:58:08 +02:00
Fabian Grünbichler
3ae21d87c1 GC: flatten existing status into job status
to avoid drifting definitions and reduce duplication. with the next major
release, the 'upid' field could then be renamed and aliased to be in line with
the other jobs, which all use 'last-run-upid'. doing it now would break
existing callers of the GC status endpoint (or consumers of the on-disk status
file).

the main difference is that the GC status fields are now not optional (except
for the UPID) in the job status, since flattening an optional value is not
possible. this only affects datastores that were never GCed at all, and only
direct API consumers, since the UI handles those fields correctly.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-04-22 13:58:08 +02:00
Dominik Csapak
b7fce90b12 ui: gc view: remove unnecessary widths in columns
setting `width` and `flex` in a column simultaneously won't work, and
the `flex` value takes priority. So remove the unused `width`
properties.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-04-22 13:58:08 +02:00
Dominik Csapak
7a23a7ca5d ui: prune/gc view: improve sizing & scrolling behaviour
before, this was only used where the top list was a fixed size and only
for one datastore (which limits the number of prune jobs a bit)

since now we show gc jobs for all datastores here too and all their
prune jobs, this panel can get much bigger.

To improve it's scrolling sizing behavior, make the prune jobs panel
`flex: 1`, so it fills out the rest of the view, and add a splitter
between them so one can resize them on the fly. To prevent making one of
the panels too small, set an appropriate minHeight for both and make the
surrounding panel scrollable.

To not save the height into it's state, we have to filter that out for
the GCView.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-04-22 13:58:08 +02:00
Dominik Csapak
ea14e2bb09 ui: gc view: use beforedestroy for stopping the store
because during destroy, the controller (and the relevant function) might
not be there anymore

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-04-22 13:58:08 +02:00
Lukas Wagner
0f1496d138 proxmox-backup-mgr: gc jobs: pretty-print bytes/duration/timestamps
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
2024-04-22 13:58:08 +02:00
Lukas Wagner
84baca8a4c ui: gcview: fix eslint warnings
The ternary ? operator should be at the start of the line if the
the expression is split into multiple lines.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
2024-04-22 13:58:08 +02:00
Stefan Lendl
b08d340237 ui: configure width and flex on GC Jobs columns
table expands to the full width and relevant data is still visible on a
narrow screen.

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewd-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-22 13:58:08 +02:00
Stefan Lendl
6a20762d99 ui: show removed and pending data of last run in bytes
Show the removed and pending data of the last run formatted with
Proxmox.Utils.format_size for better readability identically to data
display in the overview tab.

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewd-by: Gabriel Goller <g.goller@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Suggested-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-22 13:58:08 +02:00
Stefan Lendl
ce70efc36e fix #4723: cli: list gc jobs with proxmox-backup-manager
proxmox-backup-manager garbage-collection list
  to list the garbage collection job status for all datastores,
  including datastores without gc jobs.

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
  [LW: add ref to bugzilla issue to commit message]
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewd-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-22 13:58:08 +02:00
Stefan Lendl
a32f6e2ac5 ui: order Prune & GC before Sync Jobs
Make the order identical to local datastore view.

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewd-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-22 13:58:08 +02:00
Stefan Lendl
9d1e62fa7c ui: hide datastore column in local gc view
Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewd-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-22 13:58:08 +02:00
Stefan Lendl
8e07e44ab2 ui: move prune and gc widget to config
* move datastore/PruneAndGC to config/PruneAndGC
* renaming the widgets to PBS.config.PruneAndGC

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewd-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-22 13:58:08 +02:00
Stefan Lendl
db3fd2132d fix #3217: ui: global prune and gc job view
In the global datastore view, extend the prune view to display gc job
status as a table.  Use the same widget in the local view and dispaly gc
job status as a single row.

The local PruneAndGC view is parameterized (cbind) with the datastore.
At initialization the only row is selected.  This allows the rest of the
grid to act on selected rows and it requires far less special casing if
the datastore is set on the view or not.

Having a single row always selected and therefore highlighted, is
visually not appealing.  Therefore, highlighting of selected rows is
disabled in the local view.

Moved GCView to different file and enhanced it with last, next run,
status and duration. Added button to show task log.

Changed `render_task_status()` to also take in account upids stored in
other 'columns'.

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
  [LW: include ref to bugzilla in commit message]
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
Originally-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewd-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-22 13:58:08 +02:00
Stefan Lendl
fe1d34d2e4 api: garbage collect job status
Adds an api endpoint on the datastore that reports the gc job status
such as:
 - Schedule
 - State (of last run)
 - Duration (of last run)
 - Last Run
 - Next Run (if scheduled)
 - Pending Chunks (of last run)
 - Pending Bytes (of last run)
 - Removed Chunks (of last run)
 - Removed Bytes (of last run)

Adds a dedicated endpoint admin/gc that reports gc job status for all
datastores including the onces without a gc-schedule.

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Originally-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewd-by: Gabriel Goller <g.goller@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-22 13:58:08 +02:00
Dominik Csapak
83daeed72a ui: fix layout reset
we have to iterate over the keys of the state object here, not over the
values. This meant one could not reset the layout from the settings
window.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2024-04-22 13:39:20 +02:00
Dietmar Maurer
50c0246a89 api: assert that maintenance mode transitions are valid
Maintenance mode Delete locks the datastore. It must not be possible to go
back to normal modes, because the datastore may be in undefined state.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-04-22 12:19:22 +02:00
Dietmar Maurer
7767c7cfd5 maintenance: derive Copy for maintenance type and make maintenance mode fields public
Because it is a public api type.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-04-22 12:19:22 +02:00
Dietmar Maurer
81904b652c pbs-api-types: use SchemaDeserializer for maintenance mode
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2024-04-22 12:19:22 +02:00
Christian Ebner
207c0eb470 client: backup writer: fix minor formatting issue
no functional changes

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-04-21 13:28:38 +02:00
Christian Ebner
fad6d21910 client: fix whitespace issue
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-04-21 13:28:38 +02:00
Lukas Wagner
b12ad47242 ui: sync view: increase default width of 'Max. Depth' column
Before, the column was so small that it was practically invisible.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-19 09:07:09 +02:00
Lukas Wagner
a76d418979 ui: sync view: add 'max-depth' to model
Otherwise, the sync job overview does no refresh if 'max-depth' is
changed in the edit window.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-19 09:07:09 +02:00
Lukas Wagner
14e4eff09d ui: sync view: rename column 'Max. Recursion' -> 'Max. Depth'
In the edit dialog we already use 'Max. Depth', so it makes sense
to use the same term in the overview.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2024-04-19 09:07:09 +02:00
Hannes Duerr
99bc0ff3a1 docs: add missing html_css_files entry
Signed-off-by: Hannes Duerr <h.duerr@proxmox.com>
2024-04-18 14:15:44 +02:00
Hannes Duerr
b57d3a928f docs: move custom.js and custom.css into _static folder
The sphinx documentation [0] describes the _static folder as the
location for the custom.js and custom.css so we move the files there, as
we do not need those files outside the directory.
This also removes the error message when building:
WARNING: html_static_path entry '_static' does not exist

[0] https://www.sphinx-doc.org/en/master/development/theming.html#add-your-own-static-files-to-the-build-assets

Signed-off-by: Hannes Duerr <h.duerr@proxmox.com>
2024-04-18 14:15:44 +02:00
Gabriel Goller
52731339c2 pbs-client: fixed typo in error message
Fixed error message on the client: 'dynmamic' -> 'dynamic'.

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2024-04-10 18:08:26 +02:00
Gabriel Goller
432de66a36 api: make prune-group a real workertask
`prune-group` is currently not a real workertask, ie it behaves like one
but doesn't start a thread nor a task to do its work.

Changed it to start a tokio-task, so that we can delete snapshots
asynchronously. The `dry-run` feature still behaves in the same way and
returns early.

This paves the way for the new logging infra (which uses `task_local` to
define a logger) and improves performance of bigger backup-groups.

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2024-04-09 12:43:18 +02:00
Markus Frank
b757c61621 api: datastore create: allow re-using existing dirs if empty & not a mountpoint
When formatting and creating a filesystem on a disk it's important
that the target directory in `/mnt/datastore/<name>` either doesn't
exist yet, or is empty and not a mountpoint of an existing FS. As that
way we ensure that no data is lost, or gets hidden, on creating a new
datastore. Our current check was a bit stricter than required, it
always bailed if the target directory existed, even if it was a plain
& empty directory on the root file-system.

So adapt the check and also check whether an existing target directory
is empty and not already mounted, as then it can be used just fine.

Signed-off-by: Markus Frank <m.frank@proxmox.com>
Tested-by: Christian Ebner <c.ebner@proxmox.com>
 [ TL: reword subject and commit message to include more details ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-04-08 15:10:01 +02:00
Christian Ebner
ceea2e485f client: backup: early check for fixed index type
Early return when the check fails, avoiding constuction of unused
object instances.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-04-04 10:54:06 +02:00
Christian Ebner
fcea0794c8 client: backup writer: only borrow http client
Instead of taking ownership of the http client when starting a new
BackupWriter instance, only borrow the client.

This allows to reuse the http client to later reuse it to start also a
BackupReader instance as required for backup runs with metadata based
file change detection mode, where both must use the same http client.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-04-04 10:51:45 +02:00
Fabian Grünbichler
df5854986c fix #5248: client: allow self-signed/untrusted certificate chains
instead of rejecting any non-leaf certificate not pre-validated by OpenSSL,
treat them as valid but keep track of the fact that the pre-validation result
is no logner trustable.

certificate chains completely trusted by openssl are still accepted like
before, and leaf certificates without a chain are also handled the same (since
the verify callback is only ever called with depth == 0 in that case).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-03-26 14:29:34 +01:00
Thomas Lamprecht
b1d01b82fc bump version to 3.1.5-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-25 17:44:03 +01:00
Christoph Heiss
132e9722dd realm sync: generic-ify LdapSyncSettings and GeneralSyncSettings
Since both only needs a handful of attributes anyway, pass them
explicitly instead of as an LDAP-specific config object, such that these
types can be reused for other realms like the new Active Directory one.

No functional changes.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-25 17:17:12 +01:00
Christoph Heiss
ab09f409be auth: factor out CA store and cert lookup into own fn
This will be needed by the AD authenticator as well, so avoid duplicate
code.

No functional changes.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-25 17:17:12 +01:00
Christoph Heiss
30c34f0b50 api-types: factor out LdapMode -> ConnectionMode conversion into own fn
This will be needed by the AD authenticator as well, so avoid duplicate
code.

No functional changes.

Signed-off-by: Christoph Heiss <c.heiss@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-25 17:17:12 +01:00
Maximiliano Sandoval
6aff2de5d9 api: use if-let pattern for error-only handling
It is more readable than using match. We also inline variables in
eprintln!.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-25 17:17:12 +01:00
Maximiliano Sandoval
87b4b63e55 pxar-fuse: use ReplyBufState::is_full() when possible
Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-25 17:17:12 +01:00
Stefan Lendl
0f7204a4da ui: prune job: disallow setting blank schedule
The schedule value for prune jobs can not be empty.

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-25 17:17:12 +01:00
Gabriel Goller
bc9b899066 docs: added shell prompt
On this `ls` command the shell prompt ('#') was missing.

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-25 17:17:12 +01:00
Gabriel Goller
93f2568fe3 docs: add examples for --exclude parameter
Added two examples for the `--exclude` parameter of the
`proxmox-backup-client backup` command.

Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-25 17:17:12 +01:00
Christian Ebner
f4a8be4bab api: sync job: log stats for removed vanished entities
Extend the current task log summary to include a log entry stating the
number of removed because vanished on the source side snapshots,
backup groups and namespaces.

The additional task log line states, e.g.:
> Summary: removed vanished: snapshots: 2, groups: 1, namespaces: 0

The log line is not shown if the sync jobs `remove_vanished` flag was
not set and therefore no removed vanished stats are present.

Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-25 17:17:12 +01:00
Christian Ebner
ceb639bd0f server: sync job: include removed vanished stats
Include statistics of vanished and therefore removed snapshots, backup
groups and namespaces in the `PullStats`.

In preparation for including these values in the sync jobs task log
output.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-25 17:17:12 +01:00
Christian Ebner
524ed4048b datastore: group: return basic stats on backup group destroy
No functional change intended: In preparation for including the
removed vanished groups and snapshots statistics in a sync jobs task
log output.

Instead of returning a boolean value showing whether all of the
snapshots of the group have been removed, return an instance of
`BackupGroupDeleteStats`, containing the count of deleted and
protected snapshots, the latter not having been removed from the
group.

The `removed_all` method is introduced as replacement for the previous
boolean return value and can be used to check if all snapshots have
been removed. If there are no protected snapshots, the group is
considered to be deleted.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2024-03-25 17:17:12 +01:00