diff options
| author | Pavel Begunkov <asml.silence@gmail.com> | 2021-01-19 13:32:46 +0000 |
|---|---|---|
| committer | Jens Axboe <axboe@kernel.dk> | 2021-02-01 10:02:42 -0700 |
| commit | 9affd664f0e0512d8997dbdddb1448a4faf9bc82 (patch) | |
| tree | 69504bfc48aa05eed32f6c3b3919fa90667d1915 /fs/io_uring.c | |
| parent | a38d68db6742c19a74141c0f56785ef67f51c504 (diff) | |
| download | lwn-9affd664f0e0512d8997dbdddb1448a4faf9bc82.tar.gz lwn-9affd664f0e0512d8997dbdddb1448a4faf9bc82.zip | |
io_uring: don't flush CQEs deep down the stack
io_submit_flush_completions() is called down the stack in the _state
version of io_req_complete(), that's ok because is only called by
io_uring opcode handler functions directly. Move it up to
__io_queue_sqe() as preparation.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Diffstat (limited to 'fs/io_uring.c')
| -rw-r--r-- | fs/io_uring.c | 13 |
1 files changed, 10 insertions, 3 deletions
diff --git a/fs/io_uring.c b/fs/io_uring.c index f676b198ee1b..935a16a682a2 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -1941,8 +1941,7 @@ static void io_req_complete_state(struct io_kiocb *req, long res, req->result = res; req->compl.cflags = cflags; list_add_tail(&req->compl.list, &cs->list); - if (++cs->nr >= 32) - io_submit_flush_completions(cs); + cs->nr++; } static inline void __io_req_complete(struct io_kiocb *req, long res, @@ -6577,7 +6576,15 @@ again: io_queue_linked_timeout(linked_timeout); } else if (likely(!ret)) { /* drop submission reference */ - req = io_put_req_find_next(req); + if (cs) { + io_put_req(req); + if (cs->nr >= 32) + io_submit_flush_completions(cs); + req = NULL; + } else { + req = io_put_req_find_next(req); + } + if (linked_timeout) io_queue_linked_timeout(linked_timeout); |