diff options
| author | Pavel Begunkov <asml.silence@gmail.com> | 2021-09-24 21:59:58 +0100 |
|---|---|---|
| committer | Jens Axboe <axboe@kernel.dk> | 2021-10-19 05:49:53 -0600 |
| commit | 4652fe3f10e57abc3fc6ac11c431b2ef39f78c03 (patch) | |
| tree | 77b98209eb96be4dda19c4d032bca7323f775bc6 /fs/io_uring.c | |
| parent | 2a56a9bd64dbdff4fe5cbe00b20014da07694a78 (diff) | |
| download | lwn-4652fe3f10e57abc3fc6ac11c431b2ef39f78c03.tar.gz lwn-4652fe3f10e57abc3fc6ac11c431b2ef39f78c03.zip | |
io_uring: split slow path from io_queue_sqe
We don't want the slow path of io_queue_sqe to be inlined, so extract a
function from it.
text data bss dec hex filename
91950 13986 8 105944 19dd8 ./fs/io_uring.o
91758 13986 8 105752 19d18 ./fs/io_uring.o
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/fb01253911f8fb374268f65b1ba939b54ca6583f.1632516769.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Diffstat (limited to 'fs/io_uring.c')
| -rw-r--r-- | fs/io_uring.c | 15 |
1 files changed, 11 insertions, 4 deletions
diff --git a/fs/io_uring.c b/fs/io_uring.c index f0fb458e81f3..e5a414c208bd 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -6935,12 +6935,10 @@ issue_sqe: } } -static inline void io_queue_sqe(struct io_kiocb *req) +static void io_queue_sqe_fallback(struct io_kiocb *req) __must_hold(&req->ctx->uring_lock) { - if (likely(!(req->flags & (REQ_F_FORCE_ASYNC | REQ_F_FAIL)))) { - __io_queue_sqe(req); - } else if (req->flags & REQ_F_FAIL) { + if (req->flags & REQ_F_FAIL) { io_req_complete_fail_submit(req); } else if (unlikely(req->ctx->drain_active) && io_drain_req(req)) { return; @@ -6954,6 +6952,15 @@ static inline void io_queue_sqe(struct io_kiocb *req) } } +static inline void io_queue_sqe(struct io_kiocb *req) + __must_hold(&req->ctx->uring_lock) +{ + if (likely(!(req->flags & (REQ_F_FORCE_ASYNC | REQ_F_FAIL)))) + __io_queue_sqe(req); + else + io_queue_sqe_fallback(req); +} + /* * Check SQE restrictions (opcode and flags). * |