Commit Graph

2696 Commits

Author SHA1 Message Date
Wolfgang Bumiller
c6cccff92e sys: remove deprecations
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-07-11 14:43:37 +02:00
Wolfgang Bumiller
3dde52e5ce log: deny(unsafe_op_in_unsafe_fn) and feature(doc_cfg)
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-07-11 14:43:37 +02:00
Wolfgang Bumiller
cbc30882e7 acme-api: adapt to tracing infrastructure
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-07-11 14:43:37 +02:00
Wolfgang Bumiller
2fd7b13fbe log: reexport the logging macros
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-07-11 14:43:37 +02:00
Gabriel Goller
ddb91a6594 enable tracing logger, remove task_log macros
Enable the tracing-system by setting the LOGGER task local variable
to a instance of a FileLogger and initializing the WARN_COUNTER.
Removed the task_log! macros and some occurences.

Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
[WB: remove flog! import in doctests]
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-07-11 14:43:37 +02:00
Gabriel Goller
0550659cd1 proxmox-log: add tracing infrastructure
Add the `proxmox_log` crate which includes the new logging infra.
Export the `init_logger` function, which creates the `tracing` logger
that includes the default subscriber and two layer.

The first layer comes from the tracing-journald crate and logs
everything that does not come from a worker-task/thread to the syslog.
The second layer filters the exact opposite and writes the logs into the
corresponding task-log file.

Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
2024-07-11 14:43:37 +02:00
Wolfgang Bumiller
f1920d9b94 rest-server: fix a build warning
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-07-11 14:43:37 +02:00
Wolfgang Bumiller
9c3e4d5ccf rest-server: bump version to 0.5.4-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-07-10 12:44:29 +02:00
Wolfgang Bumiller
ce802d8320 rest-server: drop some unnecessary 'pub's
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-07-10 12:37:51 +02:00
Max Carrara
f6bacbb58f fix #5105: rest-server: connection: overhaul TLS handshake check logic
On rare occasions, the TLS "client hello" message [1] is delayed after
a connection with the server was established, which causes HTTPS
requests to fail before TLS was even negotiated. In these cases, the
server would incorrectly respond with "HTTP/1.1 400 Bad Request"
instead of closing the connection (or similar).

The reasons for the "client hello" being delayed seem to vary; one
user noticed that the issue went away completely after they turned off
UFW [2]. Another user noticed (during private correspondence) that the
issue only appeared when connecting to their PBS instance via WAN, but
not from within their VPN. In the WAN case a firewall was also
present. The same user kindly provided tcpdumps and strace logs on
request.

The issue was finally reproduced with the following Python script:

  import socket
  import time

  HOST: str = ...
  PORT: int = ...

  with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
      sock.connect((HOST, PORT))
      time.sleep(1.5) # simulate firewall / proxy / etc. delay
      sock.sendall(b"\x16\x03\x01\x02\x00")
      data = sock.recv(256)
      print(data)

The additional delay before sending the first 5 bytes of the "client
hello" message causes the handshake checking logic to incorrectly fall
back to plain HTTP.

All of this is fixed by the following:

  1. Increase the timeout duration to 10 seconds (from 1)
  2. Instead of falling back to plain HTTP, refuse to accept the
     connection if the TLS handshake wasn't initiated before the
     timeout limit is reached
  3. Only accept plain HTTP if the first 5 bytes do not correspond to
     a TLS handshake fragment [3]
  4. Do not take the last number of bytes that were in the buffer into
     account; instead, only perform the actual handshake check if
     5 bytes are in the peek buffer using some of tokio's low-level
     functionality

Regarding 1.: This should be generous enough for any client to be able
to initiate a TLS handshake, despite its surrounding circumstances.

Regarding 4.: While this is not 100% related to the issue, peeking into
the buffer in this manner should ensure that our implementation here
remains correct, even if the kernel's underlying behaviour regarding
edge-triggering is changed [4]. At the same time, there's no need for
busy-waiting and continuously yielding to the event loop anymore.

[1]: https://www.rfc-editor.org/rfc/rfc8446.html#section-4.1.2
[2]: https://forum.proxmox.com/threads/disable-default-http-redirects-on-8007.142312/post-675352
[3]: https://www.rfc-editor.org/rfc/rfc8446.html#section-5.1
[4]: https://lwn.net/Articles/864947/

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
2024-07-10 12:22:17 +02:00
Max Carrara
847ca5d14d rest-server: connection: log peer address on error
.. in order to make debugging easier and logs more helpful.

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
2024-07-10 12:22:17 +02:00
Max Carrara
0d3e7c8eaf rest-server: connection: clean up accept data flow
This adds the structs `AcceptState` and `AcceptFlags` and adapts
relevant method signatures of `AcceptBuilder` accordingly. This makes
it easier to add further parameters in the future.

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
2024-07-10 12:22:17 +02:00
Wolfgang Bumiller
9f33be3078 http: bump version to 0.9.2-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-07-10 12:17:59 +02:00
Wolfgang Bumiller
2269153522 compression: bump version to 0.2.3-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-07-10 12:15:54 +02:00
Maximiliano Sandoval
4d1c4ec829 http: teach the Client how to decode deflate content
The Backup Server can compress the content using deflate so we teach the
client how to decode it.

If a request is sent with the `Accept-Encoding` [2] header set to
`deflate`, and the response's `Content-Encoding` [1] header is equal to
`deflate` we wrap the Body stream with a stream that can decode `zlib`
on the run.

Note that from the `Accept-Encoding` docs [2], the `deflate` encoding is
actually `zlib`.

This can be also tested against
http://eu.httpbin.org/#/Response_formats/get_deflate by adding the
following test:

```rust
    #[tokio::test]
    async fn test_client() {
        let client = Client::new();
        let headers = HashMap::from([(
            hyper::header::ACCEPT_ENCODING.to_string(),
            "deflate".to_string(),
        )]);
        let response = client
            .get_string("https://eu.httpbin.org/deflate", Some(&headers))
            .await;
        assert!(response.is_ok());
    }
```

at `proxmox-http/src/client/simple.rs` and running

```
cargo test --features=client,client-trait
```

[1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding
[2] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Encoding

Suggested-by: Lukas Wagner <l.wagner@proxmox.com>
Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Reviewed-by: Max Carrara <m.carrara@proxmox.com>
Tested-by: Max Carrara <m.carrara@proxmox.com>
2024-07-10 11:37:36 +02:00
Maximiliano Sandoval
8b8957b5ba compression: deflate: add test module
We test the deflate encoder against the deflate decoder using (or not)
zlib and with different small buffer sizes. We also test compression and
decompression against the flate2 crate.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Reviewed-by: Max Carrara <m.carrara@proxmox.com>
Tested-by: Max Carrara <m.carrara@proxmox.com>
2024-07-10 11:37:36 +02:00
Maximiliano Sandoval
e6ca8d6049 compression: deflate: add a decoder
Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Reviewed-by: Max Carrara <m.carrara@proxmox.com>
Tested-by: Max Carrara <m.carrara@proxmox.com>
2024-07-10 11:37:36 +02:00
Maximiliano Sandoval
3f09610d86 compression: deflate: add builder pattern
This allows creating a encoder in a more general way and allows to
specify whether we want to set zlib headers. This is useful to compress
HTTP traffic, as per [1].

[1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding#directives

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Reviewed-by: Max Carrara <m.carrara@proxmox.com>
Tested-by: Max Carrara <m.carrara@proxmox.com>
2024-07-10 11:37:36 +02:00
Maximiliano Sandoval
70a71e0c8e compression: deflate: move encoder into a mod
This allows to add a decompression mod inside the deflate mod. This does
not touch the public API.

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Reviewed-by: Max Carrara <m.carrara@proxmox.com>
Tested-by: Max Carrara <m.carrara@proxmox.com>
2024-07-10 11:37:36 +02:00
Dietmar Maurer
7ae351854e apt-api-types: derive PartialEq for all types
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-07-09 13:15:16 +02:00
Dietmar Maurer
54dcb0942c syslog-api: add helper for mini-journalreader
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-07-09 11:29:50 +02:00
Wolfgang Bumiller
3479a9afe4 apt: bump version to 0.11.0-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-07-08 15:21:49 +02:00
Wolfgang Bumiller
67671399e8 apt-api-types: bump version to 1.0.0-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-07-08 15:20:40 +02:00
Dietmar Maurer
75c62574c6 apt: avoid global apt config
Because it was only used for the test setup. Instead, we simply
add an apt_lists_dir parameter where we need it.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-07-08 15:17:10 +02:00
Dietmar Maurer
f451a643ae apt: add cache feature
Save/read package state from a file, and add the api functions to manipulate
that state.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-07-08 15:17:10 +02:00
Dietmar Maurer
f536a91b2f apt: use api types from apt-api-types crate
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-07-08 15:17:10 +02:00
Dietmar Maurer
e56e39185a apt: avoid direct impl on api types (use traits instead)
So that we can use api types from expternal crate proxmox-apt-api-types.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-07-08 15:17:10 +02:00
Dietmar Maurer
960131925e apt-api-types: use serde-plain to display/parse enums
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-07-08 15:17:10 +02:00
Dietmar Maurer
b11dbfd105 apt-api-types: new crate
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2024-07-08 15:17:10 +02:00
Wolfgang Bumiller
9ec5a48701 access-control: bump to 0.2.0
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-07-04 14:33:42 +02:00
Wolfgang Bumiller
e8b5ad6b45 access-control: use ConfigDigest for digests
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-07-04 14:31:24 +02:00
Wolfgang Bumiller
3545d67b1f auth-api: bump version to 0.4.2
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-07-03 15:26:23 +02:00
Wolfgang Bumiller
2ca71fe601 tfa: bump version to 5.0.0
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-07-03 15:25:39 +02:00
Maximiliano Sandoval
5079ff6a3c tfa: webauthn: serialize OriginUrl following RFC6454
We serialize `OriginUrl` using the ASCII serialization mentioned at
[RFC6454] section 6.2 or [1]. Note that the unicode serialization is not
used widely adopted [2].

Note that `url::Url` serialize with a trailign slash, e.g.
https://foo.bar serializes as https://foo.bar/ which is not the origin
for this domain.

[RFC6454] https://www.rfc-editor.org/rfc/rfc6454
[1] https://html.spec.whatwg.org/multipage/browsers.html#ascii-serialisation-of-an-origin
[2] https://html.spec.whatwg.org/multipage/browsers.html#unicode-serialisation-of-an-origin

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-07-03 15:20:30 +02:00
Wolfgang Bumiller
0652d81977 tree-wide: enable doc_cfg and doc_auto_cfg for docs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-07-02 11:59:53 +02:00
Wolfgang Bumiller
635c8bcbed client: clippy: factor out complex type
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-06-28 11:38:59 +02:00
Wolfgang Bumiller
1f9cb87576 sys: process_locker: explicitly don't truncate the lock file
clippy rightfully complains about a create() with an unspecified
truncation behavior. This file has no contents so let's just not
truncate it in case we ever want to also have data in it...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-06-28 11:18:04 +02:00
Wolfgang Bumiller
b2dd0117d1 acme-api: allow clippy::manual_map where .map doesn't make sense
The code chooses whichever one of a multitude of functions returns
Some, switching to .map for the final else would make it less
readable.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-06-28 11:14:30 +02:00
Wolfgang Bumiller
a3c1d9d456 network-api: drop unnecessary string allocations
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2024-06-28 11:09:48 +02:00
Maximiliano Sandoval
52cf0c05f5 auth-api: do not clone struct implementing Copy
Fixes the clippy warning:

warning: using `clone` on type `Option<&dyn AuthContext>` which implements the `Copy` trait
   --> proxmox-auth-api/src/api/mod.rs:111:5
    |
111 | /     AUTH_CONTEXT
112 | |         .lock()
113 | |         .unwrap()
114 | |         .clone()
    | |________________^
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#clone_on_copy
    = note: `#[warn(clippy::clone_on_copy)]` on by default
help: try dereferencing it
    |
111 ~     (*AUTH_CONTEXT
112 +         .lock()
113 +         .unwrap())
    |

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-06-28 11:09:48 +02:00
Maximiliano Sandoval
229cc6ae02 acl: directly return struct rather than a binding
Fixes the following clippy warning:

warning: returning the result of a `let` binding from a block
   --> proxmox-access-control/src/acl.rs:687:13
    |
686 |             let config = TestAcmConfig { roles };
    |             ------------------------------------- unnecessary `let` binding
687 |             config
    |             ^^^^^^
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#let_and_return
    = note: `#[warn(clippy::let_and_return)]` on by default
help: return the expression directly
    |
686 ~
687 ~             TestAcmConfig { roles }
    |

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-06-28 10:22:58 +02:00
Maximiliano Sandoval
a4b57d6c3c acme: derive Default for Status
Fixes the clippy warning:

warning: this `impl` can be derived
  --> proxmox-acme/src/order.rs:36:1
   |
36 | / impl Default for Status {
37 | |     fn default() -> Self {
38 | |         Status::New
39 | |     }
40 | | }
   | |_^
   |
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#derivable_impls
   = note: `#[warn(clippy::derivable_impls)]` on by default
   = help: remove the manual implementation...
help: ...and instead derive it...
   |
12 + #[derive(Default)]
13 | pub enum Status {
   |
help: ...and mark the default variant
   |
15 ~     #[default]
16 ~     New,
   |

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-06-28 10:22:58 +02:00
Maximiliano Sandoval
baccbda477 acme: elide explicit lifetimes
Fixes the clippy warning:

warning: the following explicit lifetimes could be elided: 'a
  --> proxmox-acme/src/async_client.rs:65:30
   |
65 |     pub async fn new_account<'a>(
   |                              ^^
66 |         &'a mut self,
   |          ^^
   |
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_lifetimes
   = note: `#[warn(clippy::needless_lifetimes)]` on by default

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-06-28 10:22:58 +02:00
Maximiliano Sandoval
6b1e4b83bb http: remove unnecessary cast
Fixes the clippy warning:

warning: casting to the same type is unnecessary (`usize` -> `usize`)
   --> proxmox-http/src/websocket/mod.rs:446:40
    |
446 |             mask.copy_from_slice(&data[mask_offset as usize..payload_offset as usize]);
    |                                        ^^^^^^^^^^^^^^^^^^^^ help: try: `mask_offset`
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_cast
    = note: `#[warn(clippy::unnecessary_cast)]` on by default

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-06-28 10:22:58 +02:00
Maximiliano Sandoval
b006e66361 http: remove redundant redefinition of binding
Fixes the clippy error:

error: redundant redefinition of a binding `data`
   --> proxmox-http/src/websocket/mod.rs:375:9
    |
375 |         let data = data;
    |         ^^^^^^^^^^^^^^^^
    |
help: `data` is initially defined here
   --> proxmox-http/src/websocket/mod.rs:369:27
    |
369 |     pub fn try_from_bytes(data: &[u8]) -> Result<Option<FrameHeader>, WebSocketError> {
    |                           ^^^^
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#redundant_locals
    = note: `#[deny(clippy::redundant_locals)]` on by default

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-06-28 10:22:58 +02:00
Maximiliano Sandoval
82bc4db723 network-api: remove useless uses of format!
Fixes the clippy warning:

warning: useless use of `format!`
   --> proxmox-network-api/src/config/mod.rs:632:13
    |
632 | /             format!(
633 | |                 r#"
634 | | iface enp3s0 inet static
635 | |     address 10.0.0.100/16
636 | |     gateway 10.0.0.1"#
637 | |             )
    | |_____________^
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#useless_format
help: consider using `.to_string()`
    |
632 ~             r#"
633 + iface enp3s0 inet static
634 ~     address 10.0.0.100/16
635 ~     gateway 10.0.0.1"#.to_string()

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-06-28 10:22:58 +02:00
Maximiliano Sandoval
d965713565 shared-memory: remove unneeded generic parameter
Fixes the clippy warning:

warning: type parameter `T` goes unused in function definition
  --> proxmox-shared-memory/tests/raw_shared_mutex.rs:80:19
   |
80 | fn create_test_dir<T: Init>(filename: &str) -> Option<PathBuf> {
   |                   ^^^^^^^^^ help: consider removing the parameter
   |
   = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#extra_unused_type_parameters
   = note: `#[warn(clippy::extra_unused_type_parameters)]` on by default

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-06-28 10:22:58 +02:00
Maximiliano Sandoval
a87d52dad3 remove unneeded returns
Fixes the clippy warning:

warning: unneeded `return` statement
   --> proxmox-tfa/src/api/mod.rs:468:17
    |
468 | /                 return TfaResult::Failure {
469 | |                     needs_saving: true,
470 | |                     tfa_limit_reached,
471 | |                     totp_limit_reached,
472 | |                 };
    | |_________________^
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_return
    = note: `#[warn(clippy::needless_return)]` on by default
help: remove `return`
    |
468 ~                 TfaResult::Failure {
469 +                     needs_saving: true,
470 +                     tfa_limit_reached,
471 +                     totp_limit_reached,
472 ~                 }
    |

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-06-28 10:22:58 +02:00
Maximiliano Sandoval
d07a0243f4 use const blocks in thread_local! calls
Fixes the clippy warning:

warning: initializer for `thread_local` value can be made `const`
   --> proxmox-router/src/cli/command.rs:221:71
    |
221 |     static HELP_CONTEXT: RefCell<Option<Arc<CommandLineInterface>>> = RefCell::new(None);
    |                                                                       ^^^^^^^^^^^^^^^^^^ help: replace with: `const { RefCell::new(None) }`
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#thread_local_initializer_can_be_made_const
    = note: `#[warn(clippy::thread_local_initializer_can_be_made_const)]` on by default

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-06-28 10:22:58 +02:00
Maximiliano Sandoval
e3602c1943 acl: remove null pointer cast
Fixes the clippy warning:

warning: casting raw pointers to the same type and constness is unnecessary (`*mut fs::acl::libc::c_void` -> `*mut fs::acl::libc::c_void`)
   --> proxmox-sys/src/fs/acl.rs:130:23
    |
130 |         let mut ptr = ptr::null_mut() as *mut c_void;
    |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try: `ptr::null_mut()`
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_cast
    = note: `#[warn(clippy::unnecessary_cast)]` on by default

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
2024-06-28 10:22:58 +02:00