Revision tags: v6.6.35, v6.6.34, v6.6.33, v6.6.32, v6.6.31, v6.6.30, v6.6.29, v6.6.28, v6.6.27, v6.6.26, v6.6.25, v6.6.24, v6.6.23 |
|
#
1319dbd6 |
| 25-Mar-2024 |
Jens Axboe <axboe@kernel.dk> |
io_uring: use the right type for work_llist empty check
[ Upstream commit 22537c9f79417fed70b352d54d01d2586fee9521 ]
io_task_work_pending() uses wq_list_empty() on ctx->work_llist, but it's not an
io_uring: use the right type for work_llist empty check
[ Upstream commit 22537c9f79417fed70b352d54d01d2586fee9521 ]
io_task_work_pending() uses wq_list_empty() on ctx->work_llist, but it's not an io_wq_work_list, it's a struct llist_head. They both have ->first as head-of-list, and it turns out the checks are identical. But be proper and use the right helper.
Fixes: dac6a0eae793 ("io_uring: ensure iopoll runs local task work as well") Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org>
show more ...
|
Revision tags: v6.6.35, v6.6.34, v6.6.33, v6.6.32, v6.6.31, v6.6.30, v6.6.29, v6.6.28, v6.6.27, v6.6.26, v6.6.25, v6.6.24, v6.6.23 |
|
#
1319dbd6 |
| 25-Mar-2024 |
Jens Axboe <axboe@kernel.dk> |
io_uring: use the right type for work_llist empty check
[ Upstream commit 22537c9f79417fed70b352d54d01d2586fee9521 ]
io_task_work_pending() uses wq_list_empty() on ctx->work_llist, but it's not an
io_uring: use the right type for work_llist empty check
[ Upstream commit 22537c9f79417fed70b352d54d01d2586fee9521 ]
io_task_work_pending() uses wq_list_empty() on ctx->work_llist, but it's not an io_wq_work_list, it's a struct llist_head. They both have ->first as head-of-list, and it turns out the checks are identical. But be proper and use the right helper.
Fixes: dac6a0eae793 ("io_uring: ensure iopoll runs local task work as well") Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org>
show more ...
|
Revision tags: v6.6.35, v6.6.34, v6.6.33, v6.6.32, v6.6.31, v6.6.30, v6.6.29, v6.6.28, v6.6.27, v6.6.26, v6.6.25, v6.6.24, v6.6.23 |
|
#
1319dbd6 |
| 25-Mar-2024 |
Jens Axboe <axboe@kernel.dk> |
io_uring: use the right type for work_llist empty check
[ Upstream commit 22537c9f79417fed70b352d54d01d2586fee9521 ]
io_task_work_pending() uses wq_list_empty() on ctx->work_llist, but it's not an
io_uring: use the right type for work_llist empty check
[ Upstream commit 22537c9f79417fed70b352d54d01d2586fee9521 ]
io_task_work_pending() uses wq_list_empty() on ctx->work_llist, but it's not an io_wq_work_list, it's a struct llist_head. They both have ->first as head-of-list, and it turns out the checks are identical. But be proper and use the right helper.
Fixes: dac6a0eae793 ("io_uring: ensure iopoll runs local task work as well") Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org>
show more ...
|
Revision tags: v6.6.35, v6.6.34, v6.6.33, v6.6.32, v6.6.31, v6.6.30, v6.6.29, v6.6.28, v6.6.27, v6.6.26, v6.6.25, v6.6.24, v6.6.23 |
|
#
1319dbd6 |
| 25-Mar-2024 |
Jens Axboe <axboe@kernel.dk> |
io_uring: use the right type for work_llist empty check
[ Upstream commit 22537c9f79417fed70b352d54d01d2586fee9521 ]
io_task_work_pending() uses wq_list_empty() on ctx->work_llist, but it's not an
io_uring: use the right type for work_llist empty check
[ Upstream commit 22537c9f79417fed70b352d54d01d2586fee9521 ]
io_task_work_pending() uses wq_list_empty() on ctx->work_llist, but it's not an io_wq_work_list, it's a struct llist_head. They both have ->first as head-of-list, and it turns out the checks are identical. But be proper and use the right helper.
Fixes: dac6a0eae793 ("io_uring: ensure iopoll runs local task work as well") Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org>
show more ...
|
Revision tags: v6.6.35, v6.6.34, v6.6.33, v6.6.32, v6.6.31, v6.6.30, v6.6.29, v6.6.28, v6.6.27, v6.6.26, v6.6.25, v6.6.24, v6.6.23 |
|
#
1319dbd6 |
| 25-Mar-2024 |
Jens Axboe <axboe@kernel.dk> |
io_uring: use the right type for work_llist empty check
[ Upstream commit 22537c9f79417fed70b352d54d01d2586fee9521 ]
io_task_work_pending() uses wq_list_empty() on ctx->work_llist, but it's not an
io_uring: use the right type for work_llist empty check
[ Upstream commit 22537c9f79417fed70b352d54d01d2586fee9521 ]
io_task_work_pending() uses wq_list_empty() on ctx->work_llist, but it's not an io_wq_work_list, it's a struct llist_head. They both have ->first as head-of-list, and it turns out the checks are identical. But be proper and use the right helper.
Fixes: dac6a0eae793 ("io_uring: ensure iopoll runs local task work as well") Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org>
show more ...
|
Revision tags: v6.6.35, v6.6.34, v6.6.33, v6.6.32, v6.6.31, v6.6.30, v6.6.29, v6.6.28, v6.6.27, v6.6.26, v6.6.25, v6.6.24, v6.6.23 |
|
#
1319dbd6 |
| 25-Mar-2024 |
Jens Axboe <axboe@kernel.dk> |
io_uring: use the right type for work_llist empty check
[ Upstream commit 22537c9f79417fed70b352d54d01d2586fee9521 ]
io_task_work_pending() uses wq_list_empty() on ctx->work_llist, but it's not an
io_uring: use the right type for work_llist empty check
[ Upstream commit 22537c9f79417fed70b352d54d01d2586fee9521 ]
io_task_work_pending() uses wq_list_empty() on ctx->work_llist, but it's not an io_wq_work_list, it's a struct llist_head. They both have ->first as head-of-list, and it turns out the checks are identical. But be proper and use the right helper.
Fixes: dac6a0eae793 ("io_uring: ensure iopoll runs local task work as well") Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org>
show more ...
|
Revision tags: v6.6.35, v6.6.34, v6.6.33, v6.6.32, v6.6.31, v6.6.30, v6.6.29, v6.6.28, v6.6.27, v6.6.26, v6.6.25, v6.6.24, v6.6.23 |
|
#
1319dbd6 |
| 25-Mar-2024 |
Jens Axboe <axboe@kernel.dk> |
io_uring: use the right type for work_llist empty check
[ Upstream commit 22537c9f79417fed70b352d54d01d2586fee9521 ]
io_task_work_pending() uses wq_list_empty() on ctx->work_llist, but it's not an
io_uring: use the right type for work_llist empty check
[ Upstream commit 22537c9f79417fed70b352d54d01d2586fee9521 ]
io_task_work_pending() uses wq_list_empty() on ctx->work_llist, but it's not an io_wq_work_list, it's a struct llist_head. They both have ->first as head-of-list, and it turns out the checks are identical. But be proper and use the right helper.
Fixes: dac6a0eae793 ("io_uring: ensure iopoll runs local task work as well") Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org>
show more ...
|
Revision tags: v6.6.16, v6.6.15, v6.6.14, v6.6.13, v6.6.12, v6.6.11, v6.6.10, v6.6.9, v6.6.8 |
|
#
303c0a13 |
| 19-Dec-2023 |
Jens Axboe <axboe@kernel.dk> |
io_uring/unix: drop usage of io_uring socket
Commit a4104821ad651d8a0b374f0b2474c345bbb42f82 upstream.
Since we no longer allow sending io_uring fds over SCM_RIGHTS, move to using io_is_uring_fops(
io_uring/unix: drop usage of io_uring socket
Commit a4104821ad651d8a0b374f0b2474c345bbb42f82 upstream.
Since we no longer allow sending io_uring fds over SCM_RIGHTS, move to using io_is_uring_fops() to detect whether this is a io_uring fd or not. With that done, kill off io_uring_get_socket() as nobody calls it anymore.
This is in preparation to yanking out the rest of the core related to unix gc with io_uring.
Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org>
show more ...
|
#
7cbd3aa5 |
| 29-Jan-2024 |
Jens Axboe <axboe@kernel.dk> |
io_uring/poll: add requeue return code from poll multishot handling
Commit 704ea888d646cb9d715662944cf389c823252ee0 upstream.
Since our poll handling is edge triggered, multishot handlers retry int
io_uring/poll: add requeue return code from poll multishot handling
Commit 704ea888d646cb9d715662944cf389c823252ee0 upstream.
Since our poll handling is edge triggered, multishot handlers retry internally until they know that no more data is available. In preparation for limiting these retries, add an internal return code, IOU_REQUEUE, which can be used to inform the poll backend about the handler wanting to retry, but that this should happen through a normal task_work requeue rather than keep hammering on the issue side for this one request.
No functional changes in this patch, nobody is using this return code just yet.
Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
show more ...
|
Revision tags: v6.6.7, v6.6.6, v6.6.5, v6.6.4, v6.6.3 |
|
#
4ca5f54a |
| 27-Nov-2023 |
Jens Axboe <axboe@kernel.dk> |
io_uring: enable io_mem_alloc/free to be used in other parts
commit edecf1689768452ba1a64b7aaf3a47a817da651a upstream.
In preparation for using these helpers, make them non-static and add them to o
io_uring: enable io_mem_alloc/free to be used in other parts
commit edecf1689768452ba1a64b7aaf3a47a817da651a upstream.
In preparation for using these helpers, make them non-static and add them to our internal header.
Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
show more ...
|
Revision tags: v6.6.2, v6.5.11, v6.6.1, v6.5.10, v6.6, v6.5.9, v6.5.8, v6.5.7, v6.5.6 |
|
#
1658633c |
| 02-Oct-2023 |
Jens Axboe <axboe@kernel.dk> |
io_uring: ensure io_lockdep_assert_cq_locked() handles disabled rings
io_lockdep_assert_cq_locked() checks that locking is correctly done when a CQE is posted. If the ring is setup in a disabled sta
io_uring: ensure io_lockdep_assert_cq_locked() handles disabled rings
io_lockdep_assert_cq_locked() checks that locking is correctly done when a CQE is posted. If the ring is setup in a disabled state with IORING_SETUP_R_DISABLED, then ctx->submitter_task isn't assigned until the ring is later enabled. We generally don't post CQEs in this state, as no SQEs can be submitted. However it is possible to generate a CQE if tagged resources are being updated. If this happens and PROVE_LOCKING is enabled, then the locking check helper will dereference ctx->submitter_task, which hasn't been set yet.
Fixup io_lockdep_assert_cq_locked() to handle this case correctly. While at it, convert it to a static inline as well, so that generated line offsets will actually reflect which condition failed, rather than just the line offset for io_lockdep_assert_cq_locked() itself.
Reported-and-tested-by: syzbot+efc45d4e7ba6ab4ef1eb@syzkaller.appspotmail.com Fixes: f26cc9593581 ("io_uring: lockdep annotate CQ locking") Cc: stable@vger.kernel.org Signed-off-by: Jens Axboe <axboe@kernel.dk>
show more ...
|
Revision tags: v6.5.5, v6.5.4, v6.5.3, v6.5.2, v6.1.51, v6.5.1, v6.1.50, v6.5, v6.1.49, v6.1.48 |
|
#
093a650b |
| 24-Aug-2023 |
Pavel Begunkov <asml.silence@gmail.com> |
io_uring: force inline io_fill_cqe_req
There are only 2 callers of io_fill_cqe_req left, and one of them is extremely hot. Force inline the function.
Signed-off-by: Pavel Begunkov <asml.silence@gma
io_uring: force inline io_fill_cqe_req
There are only 2 callers of io_fill_cqe_req left, and one of them is extremely hot. Force inline the function.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/ffce4fc5e3521966def848a4d930586dfe33ae11.1692916914.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
show more ...
|
#
ec26c225 |
| 24-Aug-2023 |
Pavel Begunkov <asml.silence@gmail.com> |
io_uring: merge iopoll and normal completion paths
io_do_iopoll() and io_submit_flush_completions() are pretty similar, both filling CQEs and then free a list of requests. Don't duplicate it and mak
io_uring: merge iopoll and normal completion paths
io_do_iopoll() and io_submit_flush_completions() are pretty similar, both filling CQEs and then free a list of requests. Don't duplicate it and make iopoll use __io_submit_flush_completions(), which also helps with inlining and other optimisations.
For that, we need to first find all completed iopoll requests and splice them from the iopoll list and then pass it down. This adds one extra list traversal, which should be fine as requests will stay hot in cache.
CQ locking is already conditional, introduce ->lockless_cq and skip locking for IOPOLL as it's protected by ->uring_lock.
We also add a wakeup optimisation for IOPOLL to __io_cq_unlock_post(), so it works just like io_cqring_ev_posted_iopoll().
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/3840473f5e8a960de35b77292026691880f6bdbc.1692916914.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
show more ...
|
#
59fbc409 |
| 24-Aug-2023 |
Pavel Begunkov <asml.silence@gmail.com> |
io_uring: optimise extra io_get_cqe null check
If the cached cqe check passes in io_get_cqe*() it already means that the cqe we return is valid and non-zero, however the compiler is unable to optimi
io_uring: optimise extra io_get_cqe null check
If the cached cqe check passes in io_get_cqe*() it already means that the cqe we return is valid and non-zero, however the compiler is unable to optimise null checks like in io_fill_cqe_req().
Do a bit of trickery, return success/fail boolean from io_get_cqe*() and store cqe in the cqe parameter. That makes it do the right thing, erasing the check together with the introduced indirection.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/322ea4d3377d3d4efd8ae90ab8ed28a99f518210.1692916914.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
show more ...
|
#
20d6b633 |
| 24-Aug-2023 |
Pavel Begunkov <asml.silence@gmail.com> |
io_uring: refactor __io_get_cqe()
Make __io_get_cqe simpler by not grabbing the cqe from refilled cached, but letting io_get_cqe() do it for us. That's cleaner and removes some duplication.
Signed-
io_uring: refactor __io_get_cqe()
Make __io_get_cqe simpler by not grabbing the cqe from refilled cached, but letting io_get_cqe() do it for us. That's cleaner and removes some duplication.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/74dc8fdf2657e438b2e05e1d478a3596924604e9.1692916914.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
show more ...
|
#
b24c5d75 |
| 24-Aug-2023 |
Pavel Begunkov <asml.silence@gmail.com> |
io_uring: simplify big_cqe handling
Don't keep big_cqe bits of req in a union with hash_node, find a separate space for it. It's bit safer, but also if we keep it always initialised, we can get rid
io_uring: simplify big_cqe handling
Don't keep big_cqe bits of req in a union with hash_node, find a separate space for it. It's bit safer, but also if we keep it always initialised, we can get rid of ugly REQ_F_CQE32_INIT handling.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/447aa1b2968978c99e655ba88db536e903df0fe9.1692916914.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
show more ...
|
#
a0727c73 |
| 24-Aug-2023 |
Pavel Begunkov <asml.silence@gmail.com> |
io_uring: improve cqe !tracing hot path
While looking at io_fill_cqe_req()'s asm I stumbled on our trace points turning into the chunk below:
trace_io_uring_complete(req->ctx, req, req->cqe.user_da
io_uring: improve cqe !tracing hot path
While looking at io_fill_cqe_req()'s asm I stumbled on our trace points turning into the chunk below:
trace_io_uring_complete(req->ctx, req, req->cqe.user_data, req->cqe.res, req->cqe.flags, req->extra1, req->extra2);
io_uring/io_uring.c:898: trace_io_uring_complete(req->ctx, req, req->cqe.user_data, movq 232(%rbx), %rdi # req_44(D)->big_cqe.extra2, _5 movq 224(%rbx), %rdx # req_44(D)->big_cqe.extra1, _6 movl 84(%rbx), %r9d # req_44(D)->cqe.D.81184.flags, _7 movl 80(%rbx), %r8d # req_44(D)->cqe.res, _8 movq 72(%rbx), %rcx # req_44(D)->cqe.user_data, _9 movq 88(%rbx), %rsi # req_44(D)->ctx, _10 ./arch/x86/include/asm/jump_label.h:27: asm_volatile_goto("1:" 1:jmp .L1772 # objtool NOPs this # ...
It does a jump_label for actual tracing, but those 6 moves will stay there in the hottest io_uring path. As an optimisation, add a trace_io_uring_complete_enabled() check, which is also uses jump_labels, it tricks the compiler into behaving. It removes the junk without changing anything else int the hot path.
Note: apparently, it's not only me noticing it, and people are also working it around. We should remove the check when it's solved generically or rework tracing.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/555d8312644b3776f4be7e23f9b92943875c4bc7.1692916914.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
show more ...
|
Revision tags: v6.1.46 |
|
#
b6b2bb58 |
| 11-Aug-2023 |
Pavel Begunkov <asml.silence@gmail.com> |
io_uring: never overflow io_aux_cqe
Now all callers of io_aux_cqe() set allow_overflow to false, remove the parameter and not allow overflowing auxilary multishot cqes.
When CQ is full the function
io_uring: never overflow io_aux_cqe
Now all callers of io_aux_cqe() set allow_overflow to false, remove the parameter and not allow overflowing auxilary multishot cqes.
When CQ is full the function callers and all multishot requests in general are expected to complete the request. That prevents indefinite in-background grows of the overflow list and let's the userspace to handle the backlog at its own pace.
Resubmitting a request should also be faster than accounting a bunch of overflows, so it should be better for perf when it happens, but a well behaving userspace should be trying to avoid overflows in any case.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/bb20d14d708ea174721e58bb53786b0521e4dd6d.1691757663.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
show more ...
|
#
056695bf |
| 11-Aug-2023 |
Pavel Begunkov <asml.silence@gmail.com> |
io_uring: remove return from io_req_cqe_overflow()
Nobody checks io_req_cqe_overflow()'s return, make it return void.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kerne
io_uring: remove return from io_req_cqe_overflow()
Nobody checks io_req_cqe_overflow()'s return, make it return void.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/8f2029ad0c22f73451664172d834372608ee0a77.1691757663.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
show more ...
|
#
00b0db56 |
| 11-Aug-2023 |
Pavel Begunkov <asml.silence@gmail.com> |
io_uring: open code io_fill_cqe_req()
io_fill_cqe_req() is only called from one place, open code it, and rename __io_fill_cqe_req().
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: htt
io_uring: open code io_fill_cqe_req()
io_fill_cqe_req() is only called from one place, open code it, and rename __io_fill_cqe_req().
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/f432ce75bb1c94cadf0bd2add4d6aa510bd1fb36.1691757663.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
show more ...
|
Revision tags: v6.1.45, v6.1.44, v6.1.43, v6.1.42, v6.1.41, v6.1.40, v6.1.39 |
|
#
17bc2837 |
| 07-Jul-2023 |
Jens Axboe <axboe@kernel.dk> |
io_uring: have io_file_put() take an io_kiocb rather than the file
No functional changes in this patch, just a prep patch for needing the request in io_file_put().
Signed-off-by: Jens Axboe <axboe@
io_uring: have io_file_put() take an io_kiocb rather than the file
No functional changes in this patch, just a prep patch for needing the request in io_file_put().
Signed-off-by: Jens Axboe <axboe@kernel.dk>
show more ...
|
#
569f5308 |
| 09-Aug-2023 |
Pavel Begunkov <asml.silence@gmail.com> |
io_uring: fix false positive KASAN warnings
io_req_local_work_add() peeks into the work list, which can be executed in the meanwhile. It's completely fine without KASAN as we're in an RCU read secti
io_uring: fix false positive KASAN warnings
io_req_local_work_add() peeks into the work list, which can be executed in the meanwhile. It's completely fine without KASAN as we're in an RCU read section and it's SLAB_TYPESAFE_BY_RCU. With KASAN though it may trigger a false positive warning because internal io_uring caches are sanitised.
Remove sanitisation from the io_uring request cache for now.
Cc: stable@vger.kernel.org Fixes: 8751d15426a31 ("io_uring: reduce scheduling due to tw") Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/c6fbf7a82a341e66a0007c76eefd9d57f2d3ba51.1691541473.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
show more ...
|
Revision tags: v6.1.38, v6.1.37, v6.1.36, v6.4 |
|
#
0fdb9a19 |
| 23-Jun-2023 |
Pavel Begunkov <asml.silence@gmail.com> |
io_uring: make io_cq_unlock_post static
io_cq_unlock_post() is exclusively used in io_uring/io_uring.c, mark it static and don't expose to other files.
Signed-off-by: Pavel Begunkov <asml.silence@g
io_uring: make io_cq_unlock_post static
io_cq_unlock_post() is exclusively used in io_uring/io_uring.c, mark it static and don't expose to other files.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/3dc8127dda4514e1dd24bb32035faac887c5fa37.1687518903.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
show more ...
|
#
91c7884a |
| 23-Jun-2023 |
Pavel Begunkov <asml.silence@gmail.com> |
io_uring: remove IOU_F_TWQ_FORCE_NORMAL
Extract a function for non-local task_work_add, and use it directly from io_move_task_work_from_local(). Now we don't use IOU_F_TWQ_FORCE_NORMAL and it can be
io_uring: remove IOU_F_TWQ_FORCE_NORMAL
Extract a function for non-local task_work_add, and use it directly from io_move_task_work_from_local(). Now we don't use IOU_F_TWQ_FORCE_NORMAL and it can be killed.
As a small positive side effect we don't grab task->io_uring in io_req_normal_work_add anymore, which is not needed for io_req_local_work_add().
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/2e55571e8ff2927ae3cc12da606d204e2485525b.1687518903.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
show more ...
|
Revision tags: v6.1.35 |
|
#
3beed235 |
| 20-Jun-2023 |
Christoph Hellwig <hch@lst.de> |
io_uring: remove io_req_ffs_set
Just checking the flag directly makes it a lot more obvious what is going on here.
Signed-off-by: Christoph Hellwig <hch@lst.de> Link: https://lore.kernel.org/r/2023
io_uring: remove io_req_ffs_set
Just checking the flag directly makes it a lot more obvious what is going on here.
Signed-off-by: Christoph Hellwig <hch@lst.de> Link: https://lore.kernel.org/r/20230620113235.920399-5-hch@lst.de Signed-off-by: Jens Axboe <axboe@kernel.dk>
show more ...
|