linux/io_uring
Pavel Begunkov 49f1c68e04 io_uring: optimise submission side poll_refs
The final poll_refs put in __io_arm_poll_handler() takes quite some
cycles. When we're arming from the original task context task_work won't
be run, so in this case we can assume that we won't race with task_works
and so not take the initial ownership ref.

One caveat is that after arming a poll we may race with it, so we have
to add a bunch of io_poll_get_ownership() hidden inside of
io_poll_can_finish_inline() whenever we want to complete arming inline.
For the same reason we can't just set REQ_F_DOUBLE_POLL in
__io_queue_proc() and so need to sync with the first poll entry by
taking its wq head lock.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/8825315d7f5e182ac1578a031e546f79b1c97d01.1655990418.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2022-07-24 18:39:16 -06:00
..
2022-07-24 18:39:15 -06:00
2022-07-24 18:39:15 -06:00
2022-07-24 18:39:11 -06:00
2022-07-24 18:39:12 -06:00
2022-07-24 18:39:14 -06:00
2022-07-24 18:39:15 -06:00
2022-07-24 18:39:15 -06:00
2022-07-24 18:39:15 -06:00
2022-07-24 18:39:15 -06:00