diff options
| author | Pavel Begunkov <asml.silence@gmail.com> | 2021-01-07 03:15:41 +0000 |
|---|---|---|
| committer | Jens Axboe <axboe@kernel.dk> | 2021-01-07 07:48:09 -0700 |
| commit | 80c18e4ac20c9cde420cb3ffab48c936147cf07d (patch) | |
| tree | a72690e3c6f2bc6d59994d642936e35289104dff /fs/io_uring.c | |
| parent | 3e2224c5867fead6c0b94b84727cc676ac6353a3 (diff) | |
| download | lwn-80c18e4ac20c9cde420cb3ffab48c936147cf07d.tar.gz lwn-80c18e4ac20c9cde420cb3ffab48c936147cf07d.zip | |
io_uring: trigger eventfd for IOPOLL
Make sure io_iopoll_complete() tries to wake up eventfd, which currently
is skipped together with io_cqring_ev_posted() for non-SQPOLL IOPOLL.
Add an iopoll version of io_cqring_ev_posted(), duplicates a bit of
code, but they actually use different sets of wait queues may be for
better.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Diffstat (limited to 'fs/io_uring.c')
| -rw-r--r-- | fs/io_uring.c | 13 |
1 files changed, 11 insertions, 2 deletions
diff --git a/fs/io_uring.c b/fs/io_uring.c index 27a8c226abf8..91e517ad1421 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -1713,6 +1713,16 @@ static void io_cqring_ev_posted(struct io_ring_ctx *ctx) eventfd_signal(ctx->cq_ev_fd, 1); } +static void io_cqring_ev_posted_iopoll(struct io_ring_ctx *ctx) +{ + if (ctx->flags & IORING_SETUP_SQPOLL) { + if (waitqueue_active(&ctx->wait)) + wake_up(&ctx->wait); + } + if (io_should_trigger_evfd(ctx)) + eventfd_signal(ctx->cq_ev_fd, 1); +} + /* Returns true if there are no backlogged entries after the flush */ static bool __io_cqring_overflow_flush(struct io_ring_ctx *ctx, bool force, struct task_struct *tsk, @@ -2428,8 +2438,7 @@ static void io_iopoll_complete(struct io_ring_ctx *ctx, unsigned int *nr_events, } io_commit_cqring(ctx); - if (ctx->flags & IORING_SETUP_SQPOLL) - io_cqring_ev_posted(ctx); + io_cqring_ev_posted_iopoll(ctx); io_req_free_batch_finish(ctx, &rb); if (!list_empty(&again)) |