f2ccb5aed7
io_uring: make io_kiocb_to_cmd() typesafe
...
We need to make sure (at build time) that struct io_cmd_data is not
casted to a structure that's larger.
Signed-off-by: Stefan Metzmacher <metze@samba.org >
Link: https://lore.kernel.org/r/c024cdf25ae19fc0319d4180e2298bade8ed17b8.1660201408.git.metze@samba.org
Signed-off-by: Jens Axboe <axboe@kernel.dk >
2022-08-12 17:01:00 -06:00
cc18cc5e82
io_uring: mem-account pbuf buckets
...
Potentially, someone may create as many pbuf bucket as there are indexes
in an xarray without any other restrictions bounding our memory usage,
put memory needed for the buckets under memory accounting.
Cc: <stable@vger.kernel.org >
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com >
Link: https://lore.kernel.org/r/d34c452e45793e978d26e2606211ec9070d329ea.1659622312.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk >
2022-08-04 08:35:07 -06:00
b8c015598c
io_uring: allow 0 length for buffer select
...
If user gives 0 for length, we can set it from the available buffer size.
Signed-off-by: Dylan Yudaken <dylany@fb.com >
Link: https://lore.kernel.org/r/20220630091231.1456789-2-dylany@fb.com
Signed-off-by: Jens Axboe <axboe@kernel.dk >
2022-07-24 18:39:16 -06:00
795bbbc8a9
io_uring: kbuf: inline io_kbuf_recycle_ring()
...
Make io_kbuf_recycle_ring() inline since it is the fast path of
provided buffer.
Signed-off-by: Hao Xu <howeyxu@tencent.com >
Link: https://lore.kernel.org/r/20220623130126.179232-1-hao.xu@linux.dev
Signed-off-by: Jens Axboe <axboe@kernel.dk >
2022-07-24 18:39:16 -06:00
024b8fde33
io_uring: kbuf: kill __io_kbuf_recycle()
...
__io_kbuf_recycle() is only called in io_kbuf_recycle(). Kill it and
tweak the code so that the legacy pbuf and ring pbuf code become clear
Signed-off-by: Hao Xu <howeyxu@tencent.com >
Link: https://lore.kernel.org/r/20220622055551.642370-1-hao.xu@linux.dev
Signed-off-by: Jens Axboe <axboe@kernel.dk >
2022-07-24 18:39:15 -06:00
27a9d66fec
io_uring: kill extra io_uring_types.h includes
...
io_uring/io_uring.h already includes io_uring_types.h, no need to
include it every time. Kill it in a bunch of places, it prepares us for
following patches.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com >
Link: https://lore.kernel.org/r/94d8c943fbe0ef949981c508ddcee7fc1c18850f.1655384063.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk >
2022-07-24 18:39:14 -06:00
f09c8643f0
io_uring: kbuf: add comments for some tricky code
...
Add comments to explain why it is always under uring lock when
incrementing head in __io_kbuf_recycle. And rectify one comemnt about
kbuf consuming in iowq case.
Signed-off-by: Hao Xu <howeyxu@tencent.com >
Link: https://lore.kernel.org/r/20220617050429.94293-1-hao.xu@linux.dev
Signed-off-by: Jens Axboe <axboe@kernel.dk >
2022-07-24 18:39:14 -06:00
53ccf69bda
io_uring: don't inline io_put_kbuf
...
io_put_kbuf() is huge, don't bloat the kernel with inlining.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com >
Link: https://lore.kernel.org/r/2e21ccf0be471ffa654032914b9430813cae53f8.1655371007.git.asml.silence@gmail.com
Reviewed-by: Hao Xu <howeyxu@tencent.com >
Signed-off-by: Jens Axboe <axboe@kernel.dk >
2022-07-24 18:39:13 -06:00
3b77495a97
io_uring: split provided buffers handling into its own file
...
Move both the opcodes related to it, and the internals code dealing with
it.
Signed-off-by: Jens Axboe <axboe@kernel.dk >
2022-07-24 18:39:12 -06:00
