IF YOU WOULD LIKE TO GET AN ACCOUNT, please write an
email to Administrator. User accounts are meant only to access repo
and report issues and/or generate pull requests.
This is a purpose-specific Git hosting for
BaseALT
projects. Thank you for your understanding!
Только зарегистрированные пользователи имеют доступ к сервису!
Для получения аккаунта, обратитесь к администратору.
this lock can be quite contended, until the surrounding code is properly split
to reduce this contention it should help to give the worker task
creation/cleanup code a bit more breathing room.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
this mimics how the count is updated when spawning a new task - the lock scope
needs to cover the count update itself, else there's a race when multiple
worker's log their result at the same time..
Co-developed-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
if setting up a new worker fails after it has been inserted into the
WORKER_TASK_LIST, we need to clean it up instead of bubbling up the error right
away, else we "leak" the worker task and it never finishes..
a worker task that never finishes will indefinitely block shutdown
of the rest server process, including the "old" process when reloading
the rest server.
this issue was found in the wild on a system with lock contention on the
file-based lock covering task index updating leading to lock acquiring
timeouts.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Add a custom handlebars escape function. It's nearly identical to the
default `html_escape` fn [0], but it does not escape the '='. This is
needed to support base64 encoded values.
[0]: https://docs.rs/handlebars/latest/handlebars/fn.html_escape.html
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
[ TL: use full width for comment ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
I considered keeping it as log::trace level, but IMO that's just not
worth it, as just the peek_len is not giving one much more and can
also be basically also gathered through strace.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
When a connection is closed by the client before we have enough data
to determine if it contains a TLS Handshake or not, the socket stays
in a readable state.
While we setup a tokio backed timeout of 10s for the connection
build-up here, this timeout does not trigger on said early connection
abort from the client side, causing then the async_io loop to
endlessly loop around peeking into the client, which always returns
the last available bytes before the connection was closed. This in
turn causes 100% CPU usage for one of the PBS threads.
The timeout not triggering is rather odd, and does indicate some
potential for further improvement in tokio itself, but our
questionable use of the WouldBlock error does violate the API
contract, so this is not a clear cut.
Such an early connection abort is often triggered by monitoring
solutions, which use it to relatively cheaply check if TCP on a port
still works as "is service up" heuristic.
To fix this, save the amount of bytes peek returned and if they did
not change between invocations of the callback, we can assume that the
connection was closed and thus exit the connection attempt with an
error.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
[ TL: reword commit message and change error to ConnectionAborted ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
To avoid openssl's unhelpful error messages when the proxy.key or
proxy.pem files have the wrong permissions, we open the files. To load
the private key, we can simply read from the file and pass it to the
`set_private_key` openssl function. Sadly such a function does not exist
for loading certificate chains, so we have to open and close the file
before calling the `set_certificate_chain_file` fn.
Motivation: https://forum.proxmox.com/threads/proxmox-backup-tailscale-proxmox-backup-proxy-service-wont-boot.153204
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
These are Iterators or Streams which continuously produce output. They
can either be formatted, in which they are serialized like the as
usually, or, if the client caccepts `application/json-seq` via an
`Accept` header, it will be streamed as a sequence directly.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
This does not "stream", but rather skips the intermediate step to
serialize the entire output into a local json string.
We now reserve the "Stream*" prefix for actual *streaming*, that is,
producing an API response which gets streamed continuously as it is
asynchronously produced.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Fixes the clippy warning:
warning: doc list item missing indentation
--> proxmox-subscription/src/subscription_info.rs:179:9
|
179 | /// (this mode is used to decide whether to refresh the subscription information)
| ^
|
= help: if this is supposed to be its own paragraph, add a blank line
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#doc_lazy_continuation
= note: `#[warn(clippy::doc_lazy_continuation)]` on by default
help: indent this line
|
179 | /// (this mode is used to decide whether to refresh the subscription information)
| +
Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Used only for read/writing a pid_t (an integer) to a socket.
The standard to_ne_bytes()/from_ne_bytes() should be sufficient here,
we already have libc::pid_t which we can use to get the correct type
namespace for accessing ::from_ne_bytes().
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
As per [RFC9110] the Deflate encoding is a "zlib" data format. This
makes the rest-server compatible with the http-client.
[RFC9110] https://www.rfc-editor.org/rfc/rfc9110#field.content-encoding
Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
Reviewed-by: Max Carrara <m.carrara@proxmox.com>
Tested-by: Max Carrara <m.carrara@proxmox.com>
Since hyper can spawn() more tasks, when we stop passing `WorkerTask`
references down the stack, we still need to be able to *inherit* the
current logging context. Hyper provides a way to replace its used
`spawn()` method, so we need to provide a way to reuse the logging
context.
Instead of having the `FileLogger` and warn counter separately
available with local-only access, put them behind an Arc<Mutex<>>.
Previously they already *were* behind an Arc<Mutex<>> as part of the
WorkerTaskState.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Enable the tracing-system by setting the LOGGER task local variable
to a instance of a FileLogger and initializing the WARN_COUNTER.
Removed the task_log! macros and some occurences.
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
[WB: remove flog! import in doctests]
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
On rare occasions, the TLS "client hello" message [1] is delayed after
a connection with the server was established, which causes HTTPS
requests to fail before TLS was even negotiated. In these cases, the
server would incorrectly respond with "HTTP/1.1 400 Bad Request"
instead of closing the connection (or similar).
The reasons for the "client hello" being delayed seem to vary; one
user noticed that the issue went away completely after they turned off
UFW [2]. Another user noticed (during private correspondence) that the
issue only appeared when connecting to their PBS instance via WAN, but
not from within their VPN. In the WAN case a firewall was also
present. The same user kindly provided tcpdumps and strace logs on
request.
The issue was finally reproduced with the following Python script:
import socket
import time
HOST: str = ...
PORT: int = ...
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.connect((HOST, PORT))
time.sleep(1.5) # simulate firewall / proxy / etc. delay
sock.sendall(b"\x16\x03\x01\x02\x00")
data = sock.recv(256)
print(data)
The additional delay before sending the first 5 bytes of the "client
hello" message causes the handshake checking logic to incorrectly fall
back to plain HTTP.
All of this is fixed by the following:
1. Increase the timeout duration to 10 seconds (from 1)
2. Instead of falling back to plain HTTP, refuse to accept the
connection if the TLS handshake wasn't initiated before the
timeout limit is reached
3. Only accept plain HTTP if the first 5 bytes do not correspond to
a TLS handshake fragment [3]
4. Do not take the last number of bytes that were in the buffer into
account; instead, only perform the actual handshake check if
5 bytes are in the peek buffer using some of tokio's low-level
functionality
Regarding 1.: This should be generous enough for any client to be able
to initiate a TLS handshake, despite its surrounding circumstances.
Regarding 4.: While this is not 100% related to the issue, peeking into
the buffer in this manner should ensure that our implementation here
remains correct, even if the kernel's underlying behaviour regarding
edge-triggering is changed [4]. At the same time, there's no need for
busy-waiting and continuously yielding to the event loop anymore.
[1]: https://www.rfc-editor.org/rfc/rfc8446.html#section-4.1.2
[2]: https://forum.proxmox.com/threads/disable-default-http-redirects-on-8007.142312/post-675352
[3]: https://www.rfc-editor.org/rfc/rfc8446.html#section-5.1
[4]: https://lwn.net/Articles/864947/
Signed-off-by: Max Carrara <m.carrara@proxmox.com>
This adds the structs `AcceptState` and `AcceptFlags` and adapts
relevant method signatures of `AcceptBuilder` accordingly. This makes
it easier to add further parameters in the future.
Signed-off-by: Max Carrara <m.carrara@proxmox.com>
Adds a privileged_addr to ApiConfig, and some helpers for
hyper (both server and client)
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
