0f314c3b | 08-Mar-2024 |
Pavel Begunkov <asml.silence@gmail.com> |
io_uring: fix mshot io-wq checks
[ Upstream commit 3a96378e22cc46c7c49b5911f6c8631527a133a9 ]
When checking for concurrent CQE posting, we're not only interested in requests running from the poll h
io_uring: fix mshot io-wq checks
[ Upstream commit 3a96378e22cc46c7c49b5911f6c8631527a133a9 ]
When checking for concurrent CQE posting, we're not only interested in requests running from the poll handler but also strayed requests ended up in normal io-wq execution. We're disallowing multishots in general from io-wq, not only when they came in a certain way.
Cc: stable@vger.kernel.org Fixes: 17add5cea2bba ("io_uring: force multishot CQEs into task context") Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/d8c5b36a39258036f93301cd60d3cd295e40653d.1709905727.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org>
show more ...
|
4cbc5e93 | 15-Mar-2024 |
Pavel Begunkov <asml.silence@gmail.com> |
io_uring: fix poll_remove stalled req completion
[ Upstream commit 5e3afe580a9f5ca173a6bd55ffe10948796ef7e5 ]
Taking the ctx lock is not enough to use the deferred request completion infrastructure
io_uring: fix poll_remove stalled req completion
[ Upstream commit 5e3afe580a9f5ca173a6bd55ffe10948796ef7e5 ]
Taking the ctx lock is not enough to use the deferred request completion infrastructure, it'll get queued into the list but no one would expect it there, so it will sit there until next io_submit_flush_completions(). It's hard to care about the cancellation path, so complete it via tw.
Fixes: ef7dfac51d8ed ("io_uring/poll: serialize poll linked timer start with poll removal") Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/c446740bc16858f8a2a8dcdce899812f21d15f23.1710514702.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org>
show more ...
|
0b6f39c1 | 13-Mar-2024 |
Gabriel Krisman Bertazi <krisman@suse.de> |
io_uring: Fix release of pinned pages when __io_uaddr_map fails
[ Upstream commit 67d1189d1095d471ed7fa426c7e384a7140a5dd7 ]
Looking at the error path of __io_uaddr_map, if we fail after pinning th
io_uring: Fix release of pinned pages when __io_uaddr_map fails
[ Upstream commit 67d1189d1095d471ed7fa426c7e384a7140a5dd7 ]
Looking at the error path of __io_uaddr_map, if we fail after pinning the pages for any reasons, ret will be set to -EINVAL and the error handler won't properly release the pinned pages.
I didn't manage to trigger it without forcing a failure, but it can happen in real life when memory is heavily fragmented.
Signed-off-by: Gabriel Krisman Bertazi <krisman@suse.de> Fixes: 223ef4743164 ("io_uring: don't allow IORING_SETUP_NO_MMAP rings on highmem pages") Link: https://lore.kernel.org/r/20240313213912.1920-1-krisman@suse.de Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org>
show more ...
|
59a53469 | 01-Mar-2024 |
Dan Carpenter <dan.carpenter@linaro.org> |
io_uring/net: fix overflow check in io_recvmsg_mshot_prep()
[ Upstream commit 8ede3db5061bb1fe28e2c9683329aafa89d2b1b4 ]
The "controllen" variable is type size_t (unsigned long). Casting it to int
io_uring/net: fix overflow check in io_recvmsg_mshot_prep()
[ Upstream commit 8ede3db5061bb1fe28e2c9683329aafa89d2b1b4 ]
The "controllen" variable is type size_t (unsigned long). Casting it to int could lead to an integer underflow.
The check_add_overflow() function considers the type of the destination which is type int. If we add two positive values and the result cannot fit in an integer then that's counted as an overflow.
However, if we cast "controllen" to an int and it turns negative, then negative values *can* fit into an int type so there is no overflow.
Good: 100 + (unsigned long)-4 = 96 <-- overflow Bad: 100 + (int)-4 = 96 <-- no overflow
I deleted the cast of the sizeof() as well. That's not a bug but the cast is unnecessary.
Fixes: 9b0fc3c054ff ("io_uring: fix types in io_recvmsg_multishot_overflow") Signed-off-by: Dan Carpenter <dan.carpenter@linaro.org> Link: https://lore.kernel.org/r/138bd2e2-ede8-4bcc-aa7b-f3d9de167a37@moroto.mountain Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org>
show more ...
|
51d28472 | 27-Feb-2024 |
Jens Axboe <axboe@kernel.dk> |
io_uring/net: move receive multishot out of the generic msghdr path
[ Upstream commit c55978024d123d43808ab393a0a4ce3ce8568150 ]
Move the actual user_msghdr / compat_msghdr into the send and receiv
io_uring/net: move receive multishot out of the generic msghdr path
[ Upstream commit c55978024d123d43808ab393a0a4ce3ce8568150 ]
Move the actual user_msghdr / compat_msghdr into the send and receive sides, respectively, so we can move the uaddr receive handling into its own handler, and ditto the multishot with buffer selection logic.
Signed-off-by: Jens Axboe <axboe@kernel.dk> Stable-dep-of: 8ede3db5061b ("io_uring/net: fix overflow check in io_recvmsg_mshot_prep()") Signed-off-by: Sasha Levin <sashal@kernel.org>
show more ...
|
7b8fa7a0 | 31-Jan-2024 |
Jens Axboe <axboe@kernel.dk> |
io_uring: remove unconditional looping in local task_work handling
[ Upstream commit 9fe3eaea4a3530ca34a8d8ff00b1848c528789ca ]
If we have a ton of notifications coming in, we can be looping in her
io_uring: remove unconditional looping in local task_work handling
[ Upstream commit 9fe3eaea4a3530ca34a8d8ff00b1848c528789ca ]
If we have a ton of notifications coming in, we can be looping in here for a long time. This can be problematic for various reasons, mostly because we can starve userspace. If the application is waiting on N events, then only re-run if we need more events.
Fixes: c0e0d6ba25f1 ("io_uring: add IORING_SETUP_DEFER_TASKRUN") Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org>
show more ...
|
6fc19b3d | 19-Dec-2023 |
Jens Axboe <axboe@kernel.dk> |
io_uring: drop any code related to SCM_RIGHTS
This is dead code after we dropped support for passing io_uring fds over SCM_RIGHTS, get rid of it.
Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-
io_uring: drop any code related to SCM_RIGHTS
This is dead code after we dropped support for passing io_uring fds over SCM_RIGHTS, get rid of it.
Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org>
show more ...
|
22ccf61c | 29-Jan-2024 |
Jens Axboe <axboe@kernel.dk> |
io_uring/net: limit inline multishot retries
Commit 76b367a2d83163cf19173d5cb0b562acbabc8eac upstream.
If we have multiple clients and some/all are flooding the receives to such an extent that we c
io_uring/net: limit inline multishot retries
Commit 76b367a2d83163cf19173d5cb0b562acbabc8eac upstream.
If we have multiple clients and some/all are flooding the receives to such an extent that we can retry a LOT handling multishot receives, then we can be starving some clients and hence serving traffic in an imbalanced fashion.
Limit multishot retry attempts to some arbitrary value, whose only purpose serves to ensure that we don't keep serving a single connection for way too long. We default to 32 retries, which should be more than enough to provide fairness, yet not so small that we'll spend too much time requeuing rather than handling traffic.
Cc: stable@vger.kernel.org Depends-on: 704ea888d646 ("io_uring/poll: add requeue return code from poll multishot handling") Depends-on: 1e5d765a82f ("io_uring/net: un-indent mshot retry path in io_recv_finish()") Depends-on: e84b01a880f6 ("io_uring/poll: move poll execution helpers higher up") Fixes: b3fdea6ecb55 ("io_uring: multishot recv") Fixes: 9bb66906f23e ("io_uring: support multishot in recvmsg") Link: https://github.com/axboe/liburing/issues/1043 Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
show more ...
|
7cbd3aa5 | 29-Jan-2024 |
Jens Axboe <axboe@kernel.dk> |
io_uring/poll: add requeue return code from poll multishot handling
Commit 704ea888d646cb9d715662944cf389c823252ee0 upstream.
Since our poll handling is edge triggered, multishot handlers retry int
io_uring/poll: add requeue return code from poll multishot handling
Commit 704ea888d646cb9d715662944cf389c823252ee0 upstream.
Since our poll handling is edge triggered, multishot handlers retry internally until they know that no more data is available. In preparation for limiting these retries, add an internal return code, IOU_REQUEUE, which can be used to inform the poll backend about the handler wanting to retry, but that this should happen through a normal task_work requeue rather than keep hammering on the issue side for this one request.
No functional changes in this patch, nobody is using this return code just yet.
Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
show more ...
|
eac52bfe | 29-Jan-2024 |
Jens Axboe <axboe@kernel.dk> |
io_uring/net: un-indent mshot retry path in io_recv_finish()
Commit 91e5d765a82fb2c9d0b7ad930d8953208081ddf1 upstream.
In preparation for putting some retry logic in there, have the done path just
io_uring/net: un-indent mshot retry path in io_recv_finish()
Commit 91e5d765a82fb2c9d0b7ad930d8953208081ddf1 upstream.
In preparation for putting some retry logic in there, have the done path just skip straight to the end rather than have too much nesting in here.
No functional changes in this patch.
Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
show more ...
|
0848bf7e | 29-Jan-2024 |
Jens Axboe <axboe@kernel.dk> |
io_uring/poll: move poll execution helpers higher up
Commit e84b01a880f635e3084a361afba41f95ff500d12 upstream.
In preparation for calling __io_poll_execute() higher up, move the functions to avoid
io_uring/poll: move poll execution helpers higher up
Commit e84b01a880f635e3084a361afba41f95ff500d12 upstream.
In preparation for calling __io_poll_execute() higher up, move the functions to avoid forward declarations.
No functional changes in this patch.
Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
show more ...
|
0241f4c2 | 04-Jan-2024 |
Jens Axboe <axboe@kernel.dk> |
io_uring: ensure local task_work is run on wait timeout
commit 6ff1407e24e6fdfa4a16ba9ba551e3d253a26391 upstream.
A previous commit added an earlier break condition here, which is fine if we're usi
io_uring: ensure local task_work is run on wait timeout
commit 6ff1407e24e6fdfa4a16ba9ba551e3d253a26391 upstream.
A previous commit added an earlier break condition here, which is fine if we're using non-local task_work as it'll be run on return to userspace. However, if DEFER_TASKRUN is used, then we could be leaving local task_work that is ready to process in the ctx list until next time that we enter the kernel to wait for events.
Move the break condition to _after_ we have run task_work.
Cc: stable@vger.kernel.org Fixes: 846072f16eed ("io_uring: mimimise io_cqring_wait_schedule") Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
show more ...
|
c239b77e | 21-Dec-2023 |
Jens Axboe <axboe@kernel.dk> |
io_uring/rw: ensure io->bytes_done is always initialized
commit 0a535eddbe0dc1de4386046ab849f08aeb2f8faf upstream.
If IOSQE_ASYNC is set and we fail importing an iovec for a readv or writev request
io_uring/rw: ensure io->bytes_done is always initialized
commit 0a535eddbe0dc1de4386046ab849f08aeb2f8faf upstream.
If IOSQE_ASYNC is set and we fail importing an iovec for a readv or writev request, then we leave ->bytes_done uninitialized and hence the eventual failure CQE posted can potentially have a random res value rather than the expected -EINVAL.
Setup ->bytes_done before potentially failing, so we have a consistent value if we fail the request early.
Cc: stable@vger.kernel.org Reported-by: xingwei lee <xrivendell7@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
show more ...
|