5
0
mirror of git://git.proxmox.com/git/pxar.git synced 2025-01-03 09:17:38 +03:00
Commit Graph

247 Commits

Author SHA1 Message Date
Christian Ebner
48431e5e38 encoder: move to stack based state tracking
In preparation for the proxmox-backup-client look-ahead caching,
where a passing around of different encoder instances with internal
references is not feasible.

Instead of creating a new encoder instance for each directory level
and keeping references to the parent state, use an internal stack.
Adds additional helper functions to solve borrow issues, when both
the state and writers have to be accessed by a mutable reference.

This is a breaking change in the pxar library API.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-05-23 14:39:11 +02:00
Christian Ebner
a79ae33c14 lib: add type for input/output variant differentiation
Introduce an enum which stores 2 different possible variants of
inputs or outputs to be passed to encoder and decoder/accessor
instances, depending whether to read/write a fully self contained
pxar archive or whether to split off the payload stream into a
separate input/output.

Co-authored-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-05-23 11:26:23 +02:00
Christian Ebner
e5b9854989 decoder: factor out skip part from skip_entry
Make the skip part reusable for a different input.

In preparation for skipping payload paddings in a separated input.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-05-23 11:26:08 +02:00
Fabian Grünbichler
675ecff32f encoder: fix some more typos/phrasing
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2024-04-03 11:11:40 +02:00
Christian Ebner
237917b741 encoder: fix two typos in comments
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-04-03 11:09:26 +02:00
Christian Ebner
f5be8e5c6d format/examples: Fix typo in PXAR_PAYLOAD description
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2024-02-28 19:09:31 +01:00
Wolfgang Bumiller
fab6470854 doc fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-08-04 13:11:03 +02:00
Max Carrara
4355e8e619 decoder: aio: Make TokioReader public
This exposes `decoder::aio::TokioReader<T>` in a similar manner to
`decoder::sync::StandardReader<T>`, which is necessary if one wants
to remain generic over `T: tokio::io::AsyncRead`, e.g.:

> struct FooDecoder<T: tokio::io::AsyncRead> {
>    decoder: aio::Decoder<aio::TokioReader<T>>,
> }

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
2023-08-04 13:09:09 +02:00
Fabian Grünbichler
de472e826d d/changelog: fix typo
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-07-31 15:45:17 +02:00
Christian Ebner
00d36bf97b format: Document source of format const values
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2023-05-17 11:27:08 +02:00
Wolfgang Bumiller
6ad046f9f9 drop custom poll_fn, require rust 1.64
this was added to std in 1.64

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2023-01-13 13:37:43 +01:00
Fabian Grünbichler
375865b729 update d/control
after debcargo update

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2023-01-09 13:33:47 +01:00
Wolfgang Bumiller
729281cd93 bump upload target
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-27 15:31:33 +02:00
Wolfgang Bumiller
a84425dbc3 bump d/control
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-27 15:29:14 +02:00
Wolfgang Bumiller
29cbeed3e1 bump version to 0.10.2-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-27 15:28:22 +02:00
Wolfgang Bumiller
165dcc33a2 clippy fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-27 15:25:50 +02:00
Lukas Wagner
6ec9e99994 derive PartialEq trait for Metadata and related structs
This change is needed in order to compare a file's metadata
in the coming proxmox-backup-debug diff tool.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
2022-10-27 15:24:24 +02:00
Wolfgang Bumiller
a6e6873f70 fix fifo and socket entry kinds in accessor
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-27 15:18:20 +02:00
Wolfgang Bumiller
85f5a177ef test socket/fifo entry kinds in accessor
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-10-27 15:18:20 +02:00
Wolfgang Bumiller
6f3f889e98 update buildbot.yml for tokio 1.18
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-28 14:54:24 +02:00
Wolfgang Bumiller
ddc08ebb8b don't hold temp buffer mutex across await point
the type system guarantees that this can only be accessed by
a single mutable instance, but it's still a bad idea

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-28 14:36:46 +02:00
Wolfgang Bumiller
a50514a920 trivial clippy fixes
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-07-28 14:36:43 +02:00
Wolfgang Bumiller
3a83f493f5 drop imports already in 2021 prelude
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-12-01 14:14:47 +01:00
Wolfgang Bumiller
c40ababaac another clippy fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-12-01 14:10:33 +01:00
Wolfgang Bumiller
81d500297d clippy fixes
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-12-01 14:09:10 +01:00
Wolfgang Bumiller
149f88f3c9 upgrade to edition 2021
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-12-01 13:54:52 +01:00
Wolfgang Bumiller
ceb4a54674 more clippy fixes
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-12 13:53:47 +01:00
Wolfgang Bumiller
609e3a633f use #![deny(unsafe_op_in_unsafe_fn)]
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-12 13:52:17 +01:00
Wolfgang Bumiller
04eccc8e53 buildbot.yml: remove tokio+fs dep
it's provided by the tokio main package and we have an old
real package around which causes buildbot issues

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-05 09:31:01 +02:00
Wolfgang Bumiller
2d8f513a21 add back .buildbot.yml
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-04 15:29:46 +02:00
Wolfgang Bumiller
8184367000 bump d/control
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-04 15:07:54 +02:00
Wolfgang Bumiller
1a2a5570b1 fix deprecated use of std::u64 modules
MAX values in these modules are replaced by associated
constants in the integer types

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-04 15:00:44 +02:00
Wolfgang Bumiller
e5a2495ed3 add more code documentation
all but the `format` module are now #![deny(missing_docs)]

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-17 14:09:19 +02:00
Wolfgang Bumiller
5d4e59c52c make the ReadAtImpl enum variants private
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-17 14:09:19 +02:00
Wolfgang Bumiller
71194b54e4 ditch anyhow crate in examples/tests
mostly to shutup our current buildbot tests

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-17 11:46:53 +02:00
Wolfgang Bumiller
d4a04d53c9 some clippy fixes
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-07 11:57:28 +02:00
Wolfgang Bumiller
b203d38bcd bump version to 0.10.1-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-31 13:56:22 +02:00
Wolfgang Bumiller
07265b3e37 rustfmt
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-31 13:54:18 +02:00
Stefan Reiter
8a9c71c035 decoder/aio: add contents() and content_size() calls
Returns a decoder::Contents without a wrapper type, since in this case
we don't want to hide the SeqRead implementation (as done in
decoder::sync). For conviencience also implement AsyncRead if "tokio-io"
is enabled.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-03-31 13:52:24 +02:00
Wolfgang Bumiller
eae6dc06af rustfmt
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-31 13:52:09 +02:00
Wolfgang Bumiller
e827b69a60 encoder: only flush owned outputs in 'finish'
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-31 13:51:46 +02:00
Dominik Csapak
d995b5319d encoder: flush after writing last entry
some writers may need to be flushed to write out all data

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-25 07:31:16 +01:00
Wolfgang Bumiller
737f75cf97 rustfmt
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-12 10:42:14 +01:00
Wolfgang Bumiller
64f8857c70 bump version to 0.10.0-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-12 10:38:34 +01:00
Wolfgang Bumiller
180186c567 explicitly pad StatxTimestamp with zeros
otherwise we can have random data in there

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-12 10:34:55 +01:00
Wolfgang Bumiller
6e24d08a08 update d/control
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-17 09:55:26 +01:00
Wolfgang Bumiller
82608859c8 bump version to 0.9.0-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-17 09:54:38 +01:00
Wolfgang Bumiller
b8d79e5fc7 add EncoderOutput::to_borrowed()
may more "explicit" than using `.as_mut().into()`, but
otherwise just seems like a useful addition to show the
lifetime constraints on this

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-17 09:51:38 +01:00
Wolfgang Bumiller
f904859816 revert Send requirement from previous patch
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-17 09:23:37 +01:00
Stefan Reiter
d84024338a make aio::Encoder actually behave with async
To really use the encoder with async/await, it needs to support
SeqWrite implementations that are Send. This requires changing a whole
bunch of '&mut dyn SeqWrite' trait objects to instead take a 'T:
SeqWrite' generic parameter directly instead. Most of this is quite
straightforward, though incurs a lot of churn (FileImpl needs a generic
parameter now for example).

The trickiest part is returning a new Encoder instance in
create_directory, as the trait object trick with
SeqWrite::as_trait_object doesn't work if SeqWrite is implemented for
generic '&mut S'.

Instead, work with the generic object directly, and express the
owned/borrowed state in the Encoder (to avoid nested borrowing) as an
enum EncoderOutput.

Add to the aio test to ensure the Encoder is now actually useable.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-17 09:20:40 +01:00