Commit Graph

1707 Commits

Author SHA1 Message Date
Dominik Csapak
cd4e485600 proxmox-rest-server: OutputFormatter: add new format_data_streaming method
that takes the data in form of a `Box<dyn SerializableReturn + Send>`
instead of a Value.

Implement it in json and extjs formatter, by starting a thread and
stream the serialized data via a `BufWriter<SenderWriter>` and use
the Receiver side as a stream for the response body.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 08:13:36 +02:00
Wolfgang Bumiller
bcf6abaa0d bump proxmox-api-macro to 1.0.2-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-12 14:27:11 +02:00
Wolfgang Bumiller
6ce3a73681 bump proxmox-router to 1.2.0-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-12 14:26:07 +02:00
Wolfgang Bumiller
930bb59d84 proxmox-router: depend on proxmox-async 0.4.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-12 14:24:27 +02:00
Wolfgang Bumiller
61d6541ce2 router: deduplicate some code
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-12 14:23:05 +02:00
Dominik Csapak
ca3b25869c proxmox-api-macro: add 'streaming' option
to generate the `Streaming` variants of the ApiHandler

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-12 14:23:05 +02:00
Dominik Csapak
f585722aad proxmox-router: add new ApiHandler variants for streaming serialization
they should behave like their normal variants, but return a
`Box<dyn SerializableReturn + Send>` instead of a value. This is useful
since we do not have to generate the `Value` in-memory, but can
stream the serialization to the client.

We cannot simply use a `Box<dyn serde::Serialize>`, because that trait
is not object-safe and thus cannot be used as a trait-object.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-12 14:23:05 +02:00
Dominik Csapak
2c9272945e promxox-router: add SerializableReturn Trait
this will be useful as a generic return type for api calls which
must implement Serialize.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-12 14:23:05 +02:00
Wolfgang Bumiller
27c8106d7b bump proxmox-async to 0.4.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-12 14:22:59 +02:00
Dominik Csapak
9471ba9969 proxmox-async: add SenderWriter helper
this wraps around a tokio Sender for Vec<u8>, but implements a blocking
write. We can use thas as an adapter for something that only takes a
writer, and can read from it asynchonously

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-12 14:05:14 +02:00
Thomas Lamprecht
9661defb0f tree wide: some stylistic clippy fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-11 08:14:28 +02:00
Thomas Lamprecht
4cdeee64dc sys: make acl constants rustfmt safe
there's not much better one can do here..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 17:39:31 +02:00
Thomas Lamprecht
0a651e00a9 sys: rust fmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 17:39:31 +02:00
Thomas Lamprecht
3a3dd296cc schema: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 12:42:09 +02:00
Thomas Lamprecht
ae9d6e255c lang: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 12:41:44 +02:00
Thomas Lamprecht
0eeb0dd17c http: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 12:41:21 +02:00
Thomas Lamprecht
05cad8926b router: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 12:40:39 +02:00
Thomas Lamprecht
0ec1c684ae shared memory: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 12:37:38 +02:00
Thomas Lamprecht
6f8173f67a tfa: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 12:34:41 +02:00
Thomas Lamprecht
4554034d32 time: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 12:34:04 +02:00
Thomas Lamprecht
800cf63a8a uuid: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 12:34:01 +02:00
Thomas Lamprecht
d3b387f1a7 update gitignore
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 12:29:03 +02:00
Thomas Lamprecht
6b06cc6839 rest server: log rotation: refactor and comment improvements
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-07 14:04:18 +02:00
Thomas Lamprecht
5eaab7864a rest server: log rotation: fix off-by-one for max_days
The entries in a file go from oldest end-time in the first time to
newest end-time in the last line. So, just because the first line is
older than the cut-off time, the remaining one doesn't necessarily
have to be old enough too. What we can know for sure that older than
the current checked rotations of the task archive are definitively up
for deletion.

Another possibility would be to check the last line, but as scanning
backwards is more expensive/complex to do while only being an actual
improvement in a very specific edge case (it's more likely to have a
mixed time-cutoff vs. task-log-file boundary than that those are
aligned)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-07 12:58:32 +02:00
Dominik Csapak
bd8fd62de4 rest-server: add option to rotate task logs by 'max_days' instead of 'max_files'
and use it with the configurable: 'task_log_max_days' of the node config

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-04-06 17:12:49 +02:00
Dominik Csapak
6c856eed5e rest-server: cleanup_old_tasks: improve error handling
by not bubbling up most errors, and continuing on. this avoids that we
stop cleaning up because e.g. one directory was missing.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-04-06 17:10:02 +02:00
Thomas Lamprecht
2bda552b55 rest server: rust fmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-06 16:55:39 +02:00
Wolfgang Bumiller
f04eb949d1 tfa: serde tools: improve variance and dropck
`FoldSeqVisitor` doesn't actually own a `T` and therefore
cannot drop a `T`, we only use it via the `Fn(&mut Out, T)`,
so use `fn(T)` in the `PhantomData` to keep `T`
contravariant.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-22 12:31:54 +01:00
Wolfgang Bumiller
6221d86c64 schema: add another test case
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-09 14:29:53 +01:00
Wolfgang Bumiller
d7283d5aeb schema: don't accept unterminated quoted strings
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-09 13:41:09 +01:00
Wolfgang Bumiller
cdf4326e43 schema: factor out string verify fn and improve docs
We'll need to name the type when we add more perl bindings.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-04 14:51:16 +01:00
Wolfgang Bumiller
c01fbafae0 bump proxmox-schema dep to 1.3
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-04 09:50:21 +01:00
Wolfgang Bumiller
6e4a912bea bump schema to 1.3.0-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-04 09:47:40 +01:00
Wolfgang Bumiller
1e0c04a4ba drop param_bail test
it's not actually testing anything useful

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-04 09:44:51 +01:00
Dominik Csapak
7c7e99dca1 proxmox-schema: add convenience macros for ParameterError
with two variants:

(expr, expr) => assumes that the second is an 'Error'
(expr, (tt)+) => passes the tt through anyhow::format_err

also added tests

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-03-04 09:43:01 +01:00
Wolfgang Bumiller
ec5ff23d70 make property_string module public
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-01 14:57:45 +01:00
Wolfgang Bumiller
fca5218749 schema: add const fn unwrap_*_schema/format methods
'unwrap_' because they will panic and as `const fn` since
panic in const fn is now possible

Note that const evaluation will only be triggered when
actually used in const context, so to ensure *compile time*
checks, use something like this:

    const FOO_SCHEMA: &AllOfSchema =
        SomeType::API_SCHEMA.unwrap_all_of_schema();
    then_use(FOO_SCHEMA);

or to use the list of enum values of an enum string type
with compile time checks:

    const LIST: &'static [EnumEntry] =
        AnEnumStringType::API_SCHEMA
            .unwrap_string_schema()
            .unwrap_format()
            .unwrap_enum_format();
    for values in LIST {
        ...
    }

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-22 09:35:06 +01:00
Wolfgang Bumiller
98811ba9f1 bump proxmox-metrics to 0.1.1-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-22 09:00:39 +01:00
Wolfgang Bumiller
77ea32cc5a metrics: bump async dep to 0.4
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-22 08:59:37 +01:00
Wolfgang Bumiller
53aa06e411 bump proxmox-async dep to 0.4
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 14:25:37 +01:00
Wolfgang Bumiller
6fe5357ce9 bump proxmox-lang dep to 1.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 14:24:24 +01:00
Dominik Csapak
f1681d4b83 use io_format_err, io_bail, io_err_other from proxmox-lang
and move the comment from the local io_bail in pbs-client/src/pxar/fuse.rs
to the only use

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 14:24:13 +01:00
Dominik Csapak
13408babad depend on new 'proxmox-compression' crate
the compression utilities live there now

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 14:23:43 +01:00
Wolfgang Bumiller
1e75baecb0 bump proxmox-async to 0.4.0
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 14:20:13 +01:00
Wolfgang Bumiller
04b3bdb6cd proxmox-compression: update d/control
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 14:17:01 +01:00
Dominik Csapak
b8bf6a5c81 split out compression code into new crate 'proxmox-compression'
this removes quite a bit of dependecies of proxmox-async

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
[set proxmox-lang dep to 1.1]
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 14:10:53 +01:00
Wolfgang Bumiller
d663ff328a formatting fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 13:49:48 +01:00
Wolfgang Bumiller
57d052af36 workspace: set proxmox-lang dep version to 1.1
to ensure the error macros are available

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 13:45:46 +01:00
Wolfgang Bumiller
d8ecb87358 bump proxmox-lang to 1.1.0-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 13:45:10 +01:00
Wolfgang Bumiller
fc4e02a34f lang: remove io_assert
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 13:41:59 +01:00