| Age | Commit message (Collapse) | Author |
|---|
|
git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux
Pull more block updates from Jens Axboe:
"Bigger than usual for an end-of-merge-window pull request, main reason
being that the MD side I had missed a pull on, and both NVMe and block
got a good pile of LLM driven fixes where some of them may as well
just go in now rather than wait a full cycle. This contains:
- NVMe fixes via Keith:
- nvme-tcp fallout from the parallel queue allocation: initialize
pf_cache_lock and the header digest before requests are set up,
and don't dereference a NULL queue->ctrl on reconnect
- nvme-fabrics queue mapping fixes: reset io_queues on reconnect,
limit each queue type to the online CPUs, and clear a poll map
with no poll queues
- nvme-tcp io_work priority inversion with send_mutex held
(Seiji), error recovery on read_sock failure and skipping TLS
control records while waiting for the ICResp (Martin B), TLS
handshake timeout validation, and an address attribute race
- nvme-rdma NULL metadata_sgl dereference with T10-PI
- nvme-pci early boot hang with the quirks parameter
- nvme-multipath ANA log bounds underflow, and an nvme-fc
discovery reference leak
- nvmet hardening: rdma inline SGL out-of-bounds write and short
connect requests, fc use-after-free in LS create connection and
teardown on admin queue failure, pci-epf transport error
reporting and SGL length checks, zns NUMD wrap, fc stale read
data, auth state and status fixes, ports list and qid_max
locking, rdma double puts and irq-safe locking, and a double
free on metadata SGL allocation failure
- nvmet fixes from Josef: big discard length truncation, NULL
derefs for PCI endpoint targets and debugfs host_traddr, stop
listening when a tcp port is removed, and don't BUG() in
nvme-loop when a scan runs beside a reset
- MD fixes via Yu Kuai:
- raid5 validation: reject too few disks, invalid reshape disks, a
per-device size below one chunk, and an inconsistent reshape
state from the superblock instead of BUG(), plus journal payload
size checks
- raid5 error path and lifetime fixes: extra_page leak and NULL
derefs in free_conf/setup_conf, draining released_stripes before
destroying the cache, RCU for the big_stripe_tree lookup, and
per-array io_unit slab names
- md core: reject v0.90 arrays with too many devices, validate
bblog_size, take reconfig_mutex when enabling PPL, fix the
mddev->private NULL deref when stopping an array under IO, and
don't hand out the array before mddev->kobj is added
- raid10 missing barrier wakeup, raid0 device count validation, a
bogus mddev_unlock() in lbs_store(), and shared helpers
- blk-mq fixes from Josef:
- don't trust the bio cookie in blk_mq_poll()
- keep tag sharing fair after a queue update
- don't bring back flags of a completed request from the timeout
handler
- honor a restarted timer
- initialize the dma state for P2P
- quiesce a queue joining a quiesced tag set
- set up and exit hctxs under tag_list_lock
- free the tag bitmaps in the SRCU callback
- blk-cgroup and friends from Josef:
- keep the old blkg while reassociating a bio
- free struct ioc after a grace period
- don't truncate v1 iops limits
- fix root io.stat on 32-bit
- don't free a blkg with its punt work queued
- flush iolatency enable_work after going offline
- pass dead group I/O counts on and make reset_stats reset them
- don't enter reclaim under rq_qos_mutex
- Other core fixes from Josef:
- a disk use-after-free in sync_bdevs()
- don't losing the status of a failed flush request
- 64-bit sector math for mac partitions
- apply the segment size to the integrity buffer
- blk-iocost: abort a weight transfer on invalid donation weights,
and publish hweights as one atomic value
- Fix a NULL deref and use-after-free in __blk_crypto_evict_key(),
reachable via dm-inlinecrypt
- Fix recursive i_rwsem locking in blkdev_write_iter(), from two
fixes for the same race landing in the window
- Advance the iterator after copy-in for SG_DXFER_TO_FROM_DEV, fixing
an infinite loop in blk_rq_map_user_iov()
- Check the open mode for LOOP_SET_STATUS in the compat ioctl
- rnbd-srv use-after-free and kobject fixes
- drbd timer and req_lock fixes
- zloop cleanups and error path fixes
- cdrom header and kernel-doc cleanups"
* tag 'block-7.4-20261029' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux: (109 commits)
blk-crypto: fix NULL deref and UAF in __blk_crypto_evict_key()
zloop: check that the zone backing file support ->{read,write}_iter
zloop: fput file after failed zone init
zloop: share more code for opening a zone backing file
zloop: remove a duplicate call to zloop_get_block_size
block: fix kernel-doc warnings in blk-map.c and blk-mq-dma.c
blk-iocost: publish hweights as one atomic value
blk-iocost: abort weight transfer on invalid donation weights
rnbd-srv: publish the sess_dev at the end of OPEN
rnbd-srv: fix use-after-free on two CLOSE messages
block/rnbd-srv: drop the sess_dev kobject unlocked
block/rnbd-srv: don't free the device on a sysfs error
drbd: take req_lock to skip the initial sync as peer
drbd: stop the start_resync timer of a dying device
block: apply the segment size to the integrity buffer
blk-mq: free the bitmaps of tags in the SRCU callback
block: partitions/mac: compute the sectors in 64 bits
blk-mq: set up a queue under tag_list_lock
blk-mq: exit the hctxs of a queue under tag_list_lock
blk-mq: quiesce a queue that joins a quiesced tag set
...
|
|
nvme updates for Linux 7.4
- Fix regressions from the parallel nvme-tcp I/O queue setup: request
PDU sizing with header digest, a NULL controller dereference when
reconnecting with more queues, and early lock initialization
- Handle TLS control records in nvme-tcp: skip records arriving ahead
of the ICResp, and start error recovery when the socket read fails
- Fix an nvme-tcp io_work priority inversion stalling I/O
- Fix nvme-tcp address attribute race with queue teardown
- Validate the TLS handshake timeout parameters for host and target
- Fix fabrics connects with isolated CPUs and queue mapping bugs
- Fix early boot hang and buffer size for the nvme.quirks parameter
- Fix nvme-rdma NULL metadata SGL dereference with T10-PI
- Fix ANA underflow check and an nvme-fc discovery reference leak
- Fix nvme-loop BUG() when a namespace scan races a reset
- Target port and configfs locking fixes, and port removal races
- Target auth fixes for CHALLENGE failure and negotiation handling
- Target fc fixes for a connection use-after-free, read data on failed
commands, and admin queue allocation failure teardown
- Target rdma fix for short connection requests
- Target rdma fix for inline SGL out-of-bounds write
- Target rdma refcount, locking, and module unload fixes
- Target pci-epf SGL length and error status fixes
- Target NULL dereference fixes for pci-epf and debugfs
- Target ZNS NUMD overflow check
- Target Connect fix for a controller leak with a zero queue size
- Target fixes for a metadata SGL double-free and a leaked blk plug
- Target fix for truncated large discards
- Misc cleanups
* tag 'nvme-7.4-2026-10-28' of git://git.infradead.org/nvme: (44 commits)
nvmet: don't clear sq->ctrl for a Connect with sqsize 0
nvmet-rdma: fix double puts after a failed RDMA WRITE
nvmet-rdma: disable interrupts under rsp_wr_wait_lock
nvmet-rdma: flush nvmet-wq on module exit
nvme-loop: don't BUG() if a scan runs beside a reset
nvmet-tcp: don't read a removed port at Connect time
nvmet: fix NULL dereference in debugfs host_traddr
nvmet-tcp: stop listening when a port is removed
nvmet: fix NULL dereference of PCI endpoint targets
nvmet-auth: take only AUTH_Negotiate for a negotiation
nvmet: don't truncate the length of big discards
nvme-tcp: validate the TLS handshake timeout
nvmet-tcp: validate the TLS handshake timeout
nvmet: Free the old negotiation state when restarting authentication
nvmet: Finish the blk plug when integrity allocation fails
nvme-tcp: fix NULL queue->ctrl dereference on reconnect
nvme-tcp: Validate the queue state under the queue lock in the address attribute
nvme-tcp: fix io_work priority inversion when send_mutex is held
nvme-tcp: fix request PDU size when header digest is enabled
nvme-tcp: skip TLS control records while waiting for the ICResp
...
|
|
tls_handshake_timeout is writable but stored as a signed int. Multiplying
the value by MSEC_PER_SEC and HZ therefore uses signed arithmetic.
With HZ=1000, setting it to 2147484 overflows both multiplications. If
the handshake request is accepted and its completion remains pending,
schedule_timeout() reports a wrong timeout value with a stack dump and
returns zero. The handshake then fails immediately with -ETIMEDOUT,
while the handshake agent was given about 24.9 days. Negative values
such as -1 have the same kernel-side effect.
Store the parameter as an unsigned int and reject values that cannot be
represented both by tls_handshake_args::ta_timeout_ms and by a safe
jiffies delay. Take one snapshot so a concurrent parameter update cannot
give the handshake agent and completion wait different timeouts.
Document the accepted range for built-in use.
wait_for_completion_interruptible_timeout() returns the remaining
jiffies as a long after a successful completion. Keep that result in a
long because an accepted timeout can exceed INT_MAX jiffies on 64-bit
systems.
Negative and oversized values now fail with -EINVAL. Keep zero valid,
preserving the existing immediate-timeout behavior.
Fixes: be8e82caa685 ("nvme-tcp: enable TLS handshake upcall")
Cc: stable@vger.kernel.org
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Yingjie Wang <1075151112@qq.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
Since commit 61b0ed3be81b ("nvme-tcp: parallelize I/O queue allocation
and startup"), a reconnect that gets more I/O queues than before grows
the tag set before the new queues are allocated. queue->ctrl is only
set by nvme_tcp_alloc_queue(), so I/O submitted in between reaches
nvme_tcp_queue_rq() on a queue that was never allocated and
dereferences a NULL queue->ctrl:
BUG: kernel NULL pointer dereference, address: 00000000000002ac
RIP: 0010:nvme_tcp_queue_rq+0x33/0x3b0
Add nvme_tcp_init_queue() for the queue fields that do not depend on
the connection and call it for every queue when allocating the
controller. Such requests are then requeued like on any other queue
that is not live. send_mutex and io_work move there as well, and
nvme_tcp_exit_queue() destroys the helper's mutexes when the controller
is freed.
Fixes: 61b0ed3be81b ("nvme-tcp: parallelize I/O queue allocation and startup")
Suggested-by: Keith Busch <kbusch@kernel.org>
Reviewed-by: Nilay Shroff <nilay@linux.ibm.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Signed-off-by: Christian Aurich Zanettini Martins <christian.aurichzm@gmail.com>
[kbusch: dropped the comment above nvme_tcp_init_queue()]
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
Reading the "address" sysfs attribute calls kernel_getsockname() on the
admin queue socket, which can race with the queue teardown during
connection error recovery: a reader can observe NVME_TCP_Q_LIVE set,
then block on the queue lock while nvme_tcp_free_queue() releases the
socket and clears queue->sock without holding the lock. Once the
reader acquires the lock it dereferences the NULL socket pointer.
Commit 76d54bf20cdc ("nvme-tcp: don't access released socket during
error recovery") had the queue liveness check inside the lock.
Commit 782373ba2766 ("nvme: tcp: avoid race between queue_lock lock
and destroy") moved it back out because the lock itself is destroyed
in nvme_tcp_free_queue(), so a reader arriving late trips
mutex_destroy().
Check the queue liveness inside the queue lock, and clear queue->sock
under the same lock. The teardown clears NVME_TCP_Q_LIVE under the
lock well before the socket is released, so a reader either sees the
bit cleared or a socket that cannot be released while it holds the
lock. To also close the lock lifetime race, the queue locks are now
owned by the controller: they are initialized with the queue array and
only destroyed when the controller is freed, which cannot happen while
a sysfs attribute read is still in progress. The queue count is
computed by the NVME_TCP_OPT_NR_QUEUES() macro from the original
connect options for both initialization and destruction, as reconnects
may shrink ctrl->queue_count.
Fixes: 782373ba2766 ("nvme: tcp: avoid race between queue_lock lock and destroy")
Cc: stable@vger.kernel.org
Signed-off-by: Guixin Liu <kanie@linux.alibaba.com>
Reviewed-by: Nilay Shroff <nilay@linux.ibm.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
Commit f42d4796ee10 ("nvme-tcp: fix I/O stalls on congested sockets")
added a check in nvme_tcp_io_work() that sets pending to true when the
queue has pending requests and the socket is writeable. The check is
placed after the mutex_trylock() block, so it also runs when
mutex_trylock(&queue->send_mutex) failed and io_work did not send
anything.
When .queue_rq sends a request directly from its own context, it holds
send_mutex and can give up the CPU in lock_sock_nested() when another
task needs the CPU, for example the io_work kworker of the same queue,
which runs on the same CPU. Then io_work cannot take send_mutex, but
the requests are still on req_list and the socket is writeable, so
pending is set to true. io_work loops until the deadline and queues
itself again on io_cpu, without sleeping.
This is a priority inversion. io_work runs on the high priority
nvme_tcp_wq and cannot make progress until send_mutex is released, but
it keeps running. The task holding send_mutex runs at normal priority
and needs the CPU to release it. The scheduler keeps giving the CPU to
io_work, so the task holding send_mutex may not run again, and the
requests on the queue are not sent and time out.
This is the same problem that commit 70f437fb4395 ("nvme-tcp: fix
io_work priority inversion") fixed. That commit removed
"pending = !llist_empty(&queue->req_list)" from the path where
mutex_trylock() fails, and let the task holding send_mutex queue
io_work again in nvme_tcp_queue_request() after it releases the mutex.
The new check brings back the same behavior in a different form.
Two things make this easier to hit. nvme_tcp_queue_request() sends
directly only when it runs on io_cpu, so the task that takes send_mutex
is on the same CPU as the io_work kworker of that queue when it takes
the mutex. Also, when the tasks are in a cgroup with the cpu controller
enabled, the whole group competes with the nice -20 kworker as one
entity, so each task in the group gets even less CPU time.
We hit this on a 6 CPU VM running a database on nvme-tcp storage. In
the vmcore, 2 tasks held send_mutex of 2 queues and were runnable in
lock_sock_nested() for 103 seconds, while the kworkers for those queues
kept running io_work on their CPUs. All database tasks were in one
cgroup, which got only about 0.3% of the CPU against the nice -20
kworker. Requests timed out, and blk_mq_timeout_work() waited in
synchronize_srcu() for the tasks still in .queue_rq.
Only do the pending check if io_work could take send_mutex. If it could
not, the task holding send_mutex sends what it can in
nvme_tcp_send_all(), and queues io_work again after it releases the
mutex if requests are still pending, so no request is left behind.
Fixes: f42d4796ee10 ("nvme-tcp: fix I/O stalls on congested sockets")
Cc: stable@vger.kernel.org # v6.16+
Signed-off-by: Seiji Nishikawa <snishika@redhat.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
Since commit 61b0ed3be81b ("nvme-tcp: parallelize I/O queue allocation
and startup") the I/O tag set is allocated before the I/O queues.
nvme_tcp_init_request() sizes the PDU with nvme_tcp_hdgst_len(queue),
but queue->hdr_digest is only set later by nvme_tcp_alloc_queue(), so
every I/O PDU is allocated without room for the header digest.
With hdr_digest enabled, the digest is then written 4 bytes past the
PDU, over the header of the next PDU in the page_frag cache, and the
target drops the connection on the first I/O:
nvmet_tcp: unexpected pdu type 159
nvmet: ctrl 1 fatal error occurred!
Queues that a later reconnect adds for the first time have the same
problem.
Set queue->hdr_digest when allocating the controller, together with
pf_cache_lock, so it is valid before any request is initialized.
Fixes: 61b0ed3be81b ("nvme-tcp: parallelize I/O queue allocation and startup")
Signed-off-by: Christian Aurich Zanettini Martins <christian.aurichzm@gmail.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
A peer may send a NewSessionTicket as soon as the handshake completes,
ahead of the ICResp. nvme_tcp_init_connection() fails the connection on
any record that is not application data, so such a target cannot be
reached at all. An OpenSSL-based target does this by default.
RFC 9846 section 4.7.1 requires a client that does not support session
resumption to ignore a NewSessionTicket. Discard such a record and read
again, up to a fixed number of records. Report an alert, and fail the
connection for any other record type.
The record type is now checked before the length. A control record
shorter than the ICResp was reported as -ECONNRESET, which hid the real
cause.
Fixes: 2837966ab2a8 ("nvme-tcp: control message handling for recvmsg()")
Assisted-by: LLM
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Martin Belanger <martin.belanger@dell.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
nvme_tcp_recv_skb() disables the queue and starts error recovery for
errors it detects itself. An error returned by the transport reaches
nvme_tcp_try_recv() without either action.
nvme_tcp_io_work() returns on a negative result and does not requeue
itself, so the queue stops receiving and remains stalled until a command
times out.
Report the error and start error recovery when the queue is still
enabled, which indicates that the error was not handled by the actor.
Do this under the socket lock, as nvme_tcp_recv_skb() does.
Fixes: be8e82caa685 ("nvme-tcp: enable TLS handshake upcall")
Cc: stable@vger.kernel.org
Assisted-by: LLM
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Martin Belanger <martin.belanger@dell.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
Since commit 2a8513091d2f ("nvme-rdma: parallelize I/O queue allocation
and startup") the I/O tag set is allocated before the I/O queues.
nvme_rdma_init_request() only sets req->metadata_sgl if
queue->pi_support is set, but pi_support is only set later by
nvme_rdma_alloc_queue(), so every I/O request is left with a NULL
metadata_sgl.
With an RDMA device that supports integrity and a namespace formatted
with protection information, nvme_rdma_dma_map_req() then dereferences
the NULL pointer on the first integrity I/O, already during the
namespace scan when there is more than one I/O queue.
Queues that a later reconnect adds for the first time have the same
problem.
Check queue_idx and the controller's max_integrity_segments instead,
which is what nvme_rdma_alloc_queue() derives pi_support from.
Fixes: 2a8513091d2f ("nvme-rdma: parallelize I/O queue allocation and startup")
Signed-off-by: Christian Aurich Zanettini Martins <christian.aurichzm@gmail.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
nvme_fc_nvme_discovery_store() takes an lport and rport reference before
checking whether the rport is already queued on the local discovery
list. On the retry path (goto restart), rports that were queued in an
earlier pass are skipped by the list_empty() check, but the references
taken for them in the current pass are never dropped, leaking one lport
and one rport reference per retry.
Check list_empty() first and only take references for rports that are
actually added to the local discovery list.
Reviewed-by: Laurence Oberman <loberman@redhat.com>
Signed-off-by: Martin George <marting@netapp.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
The current 128 byte buffer for the nvme.quirks command line parameter
is a bit too small. If a user needs to specify multiple quirks or use
multiple VID:DID combinations, the configuration string can easily
exceed this limit.
Increase the size of the quirks_param array from 128 to 256 bytes to
comfortably accommodate longer and more complex quirk configurations.
Reviewed-by: Laurence Oberman <loberman@redhat.com>
Signed-off-by: Maurizio Lombardi <mlombard@redhat.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
Passing nvme.quirks on the kernel command line causes a silent boot
hang when the NVMe driver is compiled as a built-in module.
This happens because the parameter is currently registered using
module_param_cb(). For built-in modules, this macro flags the parameter
to be evaluated at initcall level -1. Consequently, quirks_param_set() is
called before the memory allocators are initialized.
Fix this by replacing module_param_cb() with device_param_cb(). This
defers the parameter parsing to initcall level 6, guaranteeing that
the system memory allocators are fully functional before the quirk
string is parsed and allocated.
Fixes: 7bb8c40f5ad8 ("nvme: add support for dynamic quirk configuration via module parameter")
Reviewed-by: Laurence Oberman <loberman@redhat.com>
Signed-off-by: Maurizio Lombardi <mlombard@redhat.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
The ANA log bounds checks subtract the required size from ana_log_size
in unsigned arithmetic, which can wrap when a rogue or buggy target
reports an oversized nnsids. Compare as "ana_log_size - offset < required"
instead, so the subtraction can't underflow and bypass the check.
Signed-off-by: Martin George <marting@netapp.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux
Pull io_uring dma-buf support from Jens Axboe:
"This adds native dma-buf support to the I/O path, exposed through
io_uring. It's been in development for a while and should be ready to
go now.
- dma-buf file I/O infrastructure: a dma_buf_io_ctx created through a
new ->init_dma_buf_io_ctx file operation, and a dma_buf_io_map
holding the device DMA mapping. The two are split so a mapping can
be invalidated and re-established underneath a live context, with
the map counting in-flight requests so it can be torn down safely.
- A new iov_iter type for dmabuf maps, and a dma map backed bio type
for the block layer, with raw block device support on top.
- nvme-pci backend implementing dma-buf backed requests, including
SGL support.
- io_uring dmabuf backed registered buffers, built on a reworked
buffer registration structure with an extended update path, regbuf
import flags, an uncloneable flag for buffer types that can't be
cloned, and cloning support for registered dma-bufs"
* tag 'for-7.4/io_uring-dmabuf-20261023' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux:
io_uring/dma-buf: don't fail fixed buffers without dma-buf support
nvme-pci: detach dma-bufs only on removal
block: reject dma-bufs larger than 4GiB
io_uring/dma-buf: allow cloning registered dma-bufs
dma-buf-io: document dma_buf_io_init_map
dma-buf-io: fix typos in kernel docs
block: fix indention for dma-buf checks
io_uring/rsrc: add dmabuf backed registered buffers
io_uring/rsrc: add regbuf import flags
io_uring/rsrc: add uncloneable regbuf flag
io_uring/rsrc: extend buffer update
io_uring/rsrc: introduce buf registration structure
nvme-pci: add SGL support for the dmabuf path
nvme-pci: implement dma-buf backed requests
nvme-pci: rename nvme_pci_sgl_set_data to nvme_pci_dma_iter_set_sgl
block: add dma-buf support for raw bdev
block: introduce dma map backed bio type
block: always adjust bi_offset on bio_advance_iter
iov_iter: add iterator type for dmabuf maps
dma-buf: introduce initial file I/O infrastructure
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux
Pull block updates from Jens Axboe:
- NVMe updates via Keith:
- Apple M4 (t8132) ANS2 NVMe support
- nvme-multipath sysfs attributes for path_state, iopolicies, and
fail_on_ctrl_loss, plus path state evaluation fixes
- nvme-tcp parallelized I/O queue allocation and startup, and using
the caller's network namespace
- nvmet passthru UAF fix and rework, PR preempt use-after-free
fixes, auth hardening, and traffic based keep alive support
- DH-HMAC-CHAP secret attributes readable by root only
- Various other fixes and cleanups
- swim driver overhaul and fixes, plus floppy fixes, via a pull from
Denis
- Block device buffered and direct I/O rework from Tal: use
iomap_dirty_folio, take i_rwsem for the write fallback and splice
read, honor IOCB_NOWAIT in the buffered read path, and reject atomic
write fallback to buffered I/O
- Large zone revalidation rework: serialize revalidation, handle
capacity changes by dropping and retrying zone write plugs, propagate
read-only and offline conditions, and treat offline and read-only
zones as dead
- ublk robustness work around canceled FETCH rounds and going live over
canceled io commands, dropping the default I/O scheduler, and a batch
of selftest additions
- blk-iocost flush cost support and zone append accounting
- rust block fixes for Send/Sync bounds on GenDisk and TagSet, and
import style cleanups
- Atomic write fixes capping size by the PI buffer constraints and
failing short atomic pins
- Skip the redundant flush for O_DSYNC direct writes, and only use
REQ_FUA when the device supports it
- Store GPT attributes as a raw value
- Remove support for swapping loop fds
- Various other fixes and cleanups all over
* tag 'for-7.4/block-20261021' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux: (144 commits)
block: restore the iterator when rejecting a misaligned bio
blk-mq: clear stale inactive flag on hctx reuse
block, bfq: initialize active lists for all actuators
zloop: print device that had errors
ublk: pin module while release work is pending
blk-mq: drop the tag_set argument from blk_mq_free_rq_map/tags
blk-mq: share one tags_srcu instance across all tag sets
blk-iocost: fix stale comment in ioc_rqos_throttle()
blk-iocost: account zone append completions in latency stats
blk-iocost: charge zone appends as writes
blk-iocost: add flush cost support with the flushiops model parameter
ublk: use global index for reported zones
selftests: ublk: add test for going live over canceled io commands
selftests: ublk: stop a batch thread when PREP_IO_CMDS fails
selftests: ublk: move the control command helpers into ctrl.c
ublk: refuse to go live after an io command was canceled
ublk: let STOP_DEV cancel the server's commands before its release
ublk: allow a NULL io->cmd in the cancel callback
ublk: reset the FETCH round under ub->mutex
ublk: reset the FETCH round in release also without a disk
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux
Pull io_uring updates from Jens Axboe:
"The bulk of this is a big robustness pass over cancelation, io-wq,
timeouts, and poll. Outside of that:
- Large series from Josef hardening the cancelation, io-wq, timeout,
poll, and wait paths. Fixes for lost wakeups and lost cancels as
io-wq arms poll/timeouts, work stranding on worker exit, CQE32
resize handling, napi entry timeouts, and a pile of smaller
correctness issues across the board
- Rework of how io-wq posts completions, posting from the last
request reference and dropping the request file before the
completion, and running ring release cancelations synchronously -
canceling and waiting for all in-flight requests and dropping
registered files and buffers at release time
- Don't charge the SQ/CQ rings or provided buffer rings to
RLIMIT_MEMLOCK, which the region API conversion had started doing
- Split io_uring_cmd_set_res() out from io_uring_cmd_done() so a
command can set the result from its ->end_io() handler, used by
nvme
- CQE32 ring fixes for the free entry check, aux CQE big_cqe zeroing,
and the dummy skip CQE
- Timeout fixes for sequence handling and multishot updates
- net fixes for MSG_TRUNC buffer overconsumption and keeping recv
buffer select length within MAX_RW_COUNT
- Wake real-time waiters from hard interrupt context
- Don't spin on local work an exiting task can't run, reachable by an
unprivileged user
- Various other fixes and cleanups"
* tag 'for-7.4/io_uring-20261021' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux: (75 commits)
io_uring: end the drain when the drain request completes
io_uring/futex: return a wait's data to the cache when it fails to arm
io_uring/wait: wake real-time waiters from hard interrupt context
io_uring/waitid: return -EAGAIN for a non-blocking pidfd
io_uring/net: keep recv buffer select length within MAX_RW_COUNT
io_uring/waitid: prevent cancel reference wrap
io_uring/poll: don't loop on HUP with multiple references
io_uring/timeout: don't lose a cancel as io-wq arms a timeout
io_uring/timeout: don't flush with the sequence behind the last flush
io_uring/timeout: don't put a sequence target behind the last flush
io_uring/poll: avoid retry wakeup loss when dropping ownership
io_uring: look up the file before queueing to io-wq
io_uring/timeout: don't end a linked timeout with 1
io_uring/timeout: no ABS update of multishot timeouts
io_uring/net: don't retry recv on uncommitted buffers
io_uring: don't pass restart error codes to userspace
io_uring/openclose: keep the lock in io_pipe_fixed()
io_uring/sqpoll: let the thread sleep on disabled rings
io_uring/kbuf: handle index wrap in last buffer check
io_uring/zcrx: fix buffer offsets in areas above 4GB
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/mszyprowski/linux
Pull dma-mapping updates from Marek Szyprowski:
- several SWIOTLB initialization cleanups (Aneesh Kumar K.V, Francesco
Valla)
- exact page allocation for DMA pages (Qingfang Deng)
- tools/dma build fixes (Sasha Levin)
- minor fixes for various corner cases (Marek Szyprowski, Leon
Romanovsky) and code cleanup (John Garry)
* tag 'dma-mapping-7.4-2026-10-21' of git://git.kernel.org/pub/scm/linux/kernel/git/mszyprowski/linux:
dma-debug: Skip cacheline tracking for MMIO mappings
tools/dma: fix building from the top level with O=
tools/dma: fix building with O=
dma: swiotlb: Track whether the pool size was explicitly set
dma: swiotlb: Consolidate slab rounding
dma: swiotlb: Rename swiotlb_size_or_default()
dma: swiotlb: use KiB in restricted pool allocation log
dma-direct: Restore arch_dma_alloc() for the DMA_ATTR_NO_KERNEL_MAPPING case
MAINTAINERS: Add files to the DMA MAPPING HELPERS entry
dma-mapping: use exact allocation for DMA pages
dma-mapping: rename dma_opt_mapping_size()
|
|
Detaching the dma-bufs on every shutdown permanently prevents dma-buf
I/O after resuming from a suspend. The mappings remain valid across
suspend, so detach these only when the device is being removed.
Fixes: ce5e386c1108 ("nvme-pci: implement dma-buf backed requests")
Signed-off-by: Keith Busch <kbusch@kernel.org>
Link: https://patch.msgid.link/20261021184658.644645-3-kbusch@meta.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
nvme_auth_submit() issues both AUTH Send and AUTH Receive commands,
but the failure messages are hardcoded to "auth_send failed", which
can be misleading when an AUTH Receive fails. So print the correct
direction instead.
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Signed-off-by: Martin George <marting@netapp.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
blk_mq_alloc_tag_set sets the nr_queues of every map to nr_hw_queues.
When nr_poll_queues is set but no queues are left for polling,
nvmf_map_queues skips the poll map and leaves it this way, with all CPUs
mapped to hctx 0. The queue then claims poll support, polled I/O ends
up on hctx 0, which isn't a poll queue, and hctx 0 is marked as a poll
hctx, so even the Connect command of the first I/O queue is marked as
polled.
Clear the poll map when no poll queues are mapped, so polled I/O is
rejected with -EOPNOTSUPP.
Fixes: 1a9460cef571 ("nvme-tcp: support simple polling")
Fixes: 5651cd3c4336 ("nvme-rdma: fix queue mapping when queue count is limited")
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Tested-by: Maurizio Lombardi <mlombard@redhat.com>
Signed-off-by: Daniel Wagner <wagi@kernel.org>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
nvmf_nr_io_queues limits each queue type to the number of online CPUs
when asking for I/O queues, but nvmf_set_io_queues hands out the
granted queues without these limits, and only nr_io_queues is clamped
when parsing the options.
E.g. with 2 CPUs, nr_io_queues=2, nr_write_queues=10 and
nr_poll_queues=10, 2 + 2 + 2 = 6 queues are requested, but handed out
as 4 default, 2 read and 0 poll queues. blk_mq_map_queues can't create
more CPU masks than there are CPUs, so default queues 2 and 3 take over
the CPUs of default queues 0 and 1, and connecting the I/O queue of
default queue 1 fails with -EXDEV.
Apply the same limits in nvmf_set_io_queues. A queue type with more
queues than online CPUs always has a hardware context without an online
CPU, so no working setup changes. The options are not modified, so the
limits follow CPU hotplug on reconnect.
Fixes: 6486199378a5 ("nvme-tcp: fix queue mapping when queue count is limited")
Fixes: a249d3066de6 ("nvme-fabrics: add queue setup helpers")
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Tested-by: Maurizio Lombardi <mlombard@redhat.com>
Signed-off-by: Daniel Wagner <wagi@kernel.org>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
nvmf_set_io_queues only sets the io_queues entries of the queue types it
hands out, but the array is reused on every reconnect. When the
controller grants fewer queues on a reconnect, stale entries survive and
nvmf_map_queues maps CPUs to hardware contexts that don't exist anymore,
which crashes blk_mq_map_swqueue with a NULL pointer dereference. E.g.
with nr_io_queues=8, nr_write_queues=8 and nr_poll_queues=8, 24 queues
are mapped 8/8/8 (default/read/poll). If the controller grants only 16
queues on the reconnect, the poll map still starts at hardware context
16.
Clear the array before computing the new mapping.
Fixes: 6486199378a5 ("nvme-tcp: fix queue mapping when queue count is limited")
Fixes: 5651cd3c4336 ("nvme-rdma: fix queue mapping when queue count is limited")
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Tested-by: Maurizio Lombardi <mlombard@redhat.com>
Signed-off-by: Daniel Wagner <wagi@kernel.org>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
Since commit 61b0ed3be81b ("nvme-tcp: parallelize I/O queue allocation
and startup") the I/O tag set is allocated before the I/O queues.
blk_mq_alloc_tag_set calls nvme_tcp_init_request for every request,
which takes pf_cache_lock of the request's queue, but nvme_tcp_alloc_queue
initializes that mutex only later. With CONFIG_DEBUG_MUTEXES the first
connect triggers DEBUG_LOCKS_WARN_ON(lock->magic != lock) in
__mutex_lock and lock debugging is disabled.
Initialize pf_cache_lock for all queues when allocating the controller
and keep it for the lifetime of the controller.
Fixes: 61b0ed3be81b ("nvme-tcp: parallelize I/O queue allocation and startup")
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Tested-by: Maurizio Lombardi <mlombard@redhat.com>
Tested-by: Saravanan D <saravanand@crusoe.ai>
Signed-off-by: Daniel Wagner <wagi@kernel.org>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
Cross-merge networking fixes after downstream PR (net-7.3-rc8).
No conflicts. Adjacent changes:
drivers/net/ethernet/microchip/lan865x/lan865x.c
2a3f6d2a9ac3 ("microchip: lan865x: move setting of netops to fix crash")
15358c2c3dca ("net: ethernet: oa_tc6: return ERR_PTR from oa_tc6_init()")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
This turns down to cause up to 15% performance regressions on standard
large transfer workloads. Revert it and hope the IOVA allocator can
eventually be fixed to remove the need for the original workaround.
This reverts commit 3710e2b056cb92ad816e4d79fa54a6a5b6ad8cbd.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
The dhchap_secret and dhchap_ctrl_secret sysfs attributes have mode
0644. Their show functions return the secret, so any local user can
read the DH-HMAC-CHAP secrets of a connected controller.
Define them with DEVICE_ATTR_ADMIN_RW(), which sets mode 0600. The
macro takes the show and store functions from the attribute name, so
rename them. Writing the attributes was already limited to root and is
not changed.
Fixes: f50fff73d620 ("nvme: implement In-Band authentication")
Cc: stable@vger.kernel.org
Reviewed-by: Daniel Wagner <dwagner@suse.de>
Assisted-by: LLM
Signed-off-by: Martin Belanger <martin.nvme.dev@gmail.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
Add SGL support in addition to PRP for dmabuf-backed requests,
building the descriptor list from the mapping's sg_table.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Anuj Gupta <anuj20.g@samsung.com>
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://patch.msgid.link/86fb4caeddaa58fafad106548f82950782ea04fd.1791975048.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Enable BIO_DMABUF_MAP backed requests. On registration we map the
dma-buf and store it as a prp list, which is then used to initialise
requests. All attached contexts are stored in a new list dmabuf_ctxs,
and additions/removals are synchronised with dmabuf_lock.
Suggested-by: Keith Busch <kbusch@kernel.org>
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/9540e74b5d2dd1a5dc0d436bc0c2a6f5e18fc389.1791975048.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Rename the blk_dma_iter based nvme_pci_sgl_set_data() to
nvme_pci_dma_iter_set_sgl().
Suggested-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Anuj Gupta <anuj20.g@samsung.com>
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://patch.msgid.link/42788b01948b58b891476006cb7ee9f9f5e917db.1791975048.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
nvme_query_fdp_granularity() parses the FDP Configurations log using
sizes the controller returns, without checking that they fit the buffer
it was given. The log size is only bounded from above before
kvmalloc(), so a size below the log header makes
le16_to_cpu(h->numfdpc) read past the allocation, and a size of zero
yields ZERO_SIZE_PTR and faults. Each configuration descriptor is then
dereferenced (dsze while walking, and nrg and runs on the selected one)
without checking that the whole descriptor lies within the buffer, so a
descriptor at or past the end is read out of bounds. The out-of-bounds
runs value is then exposed to userspace through
/sys/block/nvmeXnY/queue/write_stream_granularity.
A device-supplied dsze that is not a multiple of eight also leaves the
following descriptors unaligned, and the structure is not packed, so
reading nrg and runs from one can fault on architectures without
efficient unaligned access.
A malicious or buggy controller triggers this during namespace scan with a
short FDP Configurations log:
[ 1.104828] BUG: KASAN: slab-out-of-bounds in nvme_alloc_ns+0x39a6/0x4480
[ 1.104848] Read of size 8 at addr ffff888009fd8fe0 by task kworker/u8:3/52
[ 1.104852]
[ 1.104857] CPU: 1 UID: 0 PID: 52 Comm: kworker/u8:3 Not tainted 7.3.0-rc6-00063-g0c2669a9f4a1 #1 PREEMPT(lazy)
[ 1.104862] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014
[ 1.104866] Workqueue: async async_run_entry_fn
[ 1.104873] Call Trace:
[ 1.104876] <TASK>
[ 1.104878] dump_stack_lvl+0x66/0xa0
[ 1.104886] print_report+0xd0/0x630
[ 1.104893] ? nvme_alloc_ns+0x39a6/0x4480
[ 1.104897] ? __virt_addr_valid+0x209/0x3f0
[ 1.104903] ? nvme_alloc_ns+0x39a6/0x4480
[ 1.104907] kasan_report+0xe4/0x120
[ 1.104913] ? nvme_alloc_ns+0x39a6/0x4480
[ 1.104919] nvme_alloc_ns+0x39a6/0x4480
[ 1.104926] ? __pfx_nvme_alloc_ns+0x10/0x10
[ 1.104931] ? check_prev_add+0xfd/0xe80
[ 1.104942] ? lock_acquire+0x18d/0x300
[ 1.104948] ? nvme_find_get_ns+0xbb/0x2e0
[ 1.104952] ? find_held_lock+0x2b/0x80
[ 1.104956] ? nvme_find_get_ns+0x221/0x2e0
[ 1.104962] ? nvme_find_get_ns+0x231/0x2e0
[ 1.104967] ? nvme_scan_ns+0x2e1/0x8d0
[ 1.104971] ? kfree+0x2e5/0x510
[ 1.104976] ? __lock_acquire+0x570/0x1b00
[ 1.104982] nvme_scan_ns+0x56a/0x8d0
[ 1.104987] ? __pfx_nvme_scan_ns+0x10/0x10
[ 1.104994] ? lockdep_hardirqs_on_prepare+0xdc/0x190
[ 1.104999] ? trace_hardirqs_on+0x18/0x160
[ 1.105005] ? kvm_clock_get_cycles+0x19/0x40
[ 1.105011] ? __pfx_nvme_scan_ns_async+0x10/0x10
[ 1.105015] async_run_entry_fn+0x8d/0x290
[ 1.105020] process_one_work+0x8ac/0x1b00
[ 1.105029] ? __pfx_process_one_work+0x10/0x10
[ 1.105034] ? lock_acquire+0x18d/0x300
[ 1.105040] ? lock_is_held_type+0x8f/0x100
[ 1.105046] ? __pfx_async_run_entry_fn+0x10/0x10
[ 1.105050] worker_thread+0x4ee/0xe70
[ 1.105056] ? lockdep_hardirqs_on_prepare+0xdc/0x190
[ 1.105061] ? trace_hardirqs_on+0x18/0x160
[ 1.105066] ? __pfx_worker_thread+0x10/0x10
[ 1.105071] ? __pfx_worker_thread+0x10/0x10
[ 1.105076] kthread+0x2ce/0x3b0
[ 1.105080] ? __pfx_kthread+0x10/0x10
[ 1.105085] ret_from_fork+0x52e/0x780
[ 1.105090] ? __pfx_ret_from_fork+0x10/0x10
[ 1.105094] ? __switch_to+0x572/0xde0
[ 1.105101] ? __pfx_kthread+0x10/0x10
[ 1.105105] ret_from_fork_asm+0x1a/0x30
[ 1.105114] </TASK>
[ 1.105116]
[ 1.105117] Allocated by task 52:
[ 1.105119] kasan_save_stack+0x33/0x60
[ 1.105123] kasan_save_track+0x14/0x30
[ 1.105126] __kasan_kmalloc+0x8f/0xa0
[ 1.105129] __kvmalloc_node_noprof+0x2e1/0x820
[ 1.105134] nvme_alloc_ns+0x1097/0x4480
[ 1.105138] nvme_scan_ns+0x56a/0x8d0
[ 1.105142] async_run_entry_fn+0x8d/0x290
[ 1.105145] process_one_work+0x8ac/0x1b00
[ 1.105148] worker_thread+0x4ee/0xe70
[ 1.105152] kthread+0x2ce/0x3b0
[ 1.105155] ret_from_fork+0x52e/0x780
[ 1.105159] ret_from_fork_asm+0x1a/0x30
[ 1.105163]
[ 1.105164] The buggy address belongs to the object at ffff888009fd8fc0
[ 1.105164] which belongs to the cache kmalloc-32 of size 32
[ 1.105167] The buggy address is located 8 bytes to the right of
[ 1.105167] allocated 24-byte region [ffff888009fd8fc0, ffff888009fd8fd8)
[ 1.105171]
[ 1.105172] The buggy address belongs to the physical page:
[ 1.105175] page: refcount:0 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x9fd8
[ 1.105179] flags: 0x100000000000000(node=0|zone=1)
[ 1.105183] page_type: f5(slab)
[ 1.105188] raw: 0100000000000000 ffff888008c41780 dead000000000100 dead000000000122
[ 1.105192] raw: 0000000000000000 0000000000400040 00000000f5000000 0000000000000000
[ 1.105194] page dumped because: kasan: bad access detected
[ 1.105195]
[ 1.105196] Memory state around the buggy address:
[ 1.105198] ffff888009fd8e80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[ 1.105201] ffff888009fd8f00: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[ 1.105203] >ffff888009fd8f80: fc fc fc fc fc fc fc fc 00 00 00 fc fc fc fc fc
[ 1.105205] ^
[ 1.105207] ffff888009fd9000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 fc
[ 1.105210] ffff888009fd9080: fc fc fc fc fc fc fc fc 00 00 00 00 00 00 00 00
[ 1.105212] ==================================================================
Reject a log smaller than its header plus one configuration descriptor,
zero the buffer so a short transfer leaves no uninitialized tail to parse,
require each descriptor's declared size to be at least a descriptor and to
leave room for the next one before walking it, and read the descriptor
fields with the unaligned accessors.
Fixes: 30b5f20bb2dda ("nvme: register fdp parameters with the block layer")
Cc: stable@vger.kernel.org
Suggested-by: Kanchan Joshi <joshi.k@samsung.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Kanchan Joshi <joshi.k@samsung.com>
Signed-off-by: Yehyeong Lee <yhlee@isslab.korea.ac.kr>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
nvme_user_cmd() handles NVME_IOCTL_IO_CMD on namespace block devices,
including partitions. It receives flags identifying a partition, but
passes 0 to nvme_cmd_allowed(), bypassing the partition guard and
allowing unprivileged passthrough commands to access sectors outside
the partition.
Pass flags to the permission check, rejecting unprivileged commands on
partitions.
Fixes: 313c08c72ee7 ("nvme: don't allow unprivileged passthrough on partitions")
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Yao Sang <sangyao@kylinos.cn>
Reviewed-by: Kanchan Joshi <joshi.k@samsung.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
nvme_auth_process_dhchap_success1() returns success as soon as the
controller clears the rvalid flag in its DH-HMAC-CHAP success1 message,
without looking at whether the host had asked the controller to
authenticate. A host configured with a controller secret sends a
challenge in its reply, computes the expected controller response, and
sets chap->bi_directional; if the controller then answers with
rvalid = 0 the response is never compared, the host still sends
success2, and the queue is marked as authenticated.
A malicious or man-in-the-middle controller therefore only has to clear
one flag to turn the configured bidirectional authentication into a
one-way one, without holding any secret of its own. The target side
already treats the mirror image of this as fatal: when the host supplied
a challenge but no controller key is configured, nvmet_auth_success1()
fails the authentication rather than answering with rvalid = 0.
Refuse the connection if the controller returns no response although
bidirectional authentication was requested. chap->bi_directional is the
right condition rather than ctrl->ctrl_key, because secure concatenation
sends a challenge only to derive the PSK and deliberately clears
bi_directional; a one-way setup without a controller secret keeps
returning success as before.
Fixes: f50fff73d620 ("nvme: implement In-Band authentication")
Cc: stable@vger.kernel.org
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Yehyeong Lee <yhlee@isslab.korea.ac.kr>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
When no usable path exists, I/O on a multipath namespace is queued
until a path returns. With ctrl_loss_tmo=-1 that can be forever:
during a long fabric outage any process waiting on the I/O is stuck in
D state. We hit this on virtualization hosts, where a SIGKILLed VM
process cannot exit while draining I/O to an unreachable NVMe/TCP
target.
Nothing can fail this I/O without tearing something down: controller
deletion takes every namespace on the controller with it.
Add a fail_on_ctrl_loss attribute on the ns-head disk: a persistent
per-namespace policy to fail parked and newly arriving I/O instead of
queueing it when no usable path exists. It is the namespace-scoped
counterpart to the controller-scoped fast_io_fail_tmo: the trigger is
an event userspace observes (a consumer known to be gone, for us a
SIGKILLed VM the host must reap), not a duration picked up front, and
sibling namespaces behind the same controllers keep queueing and ride
out the outage.
A CONNECTING controller stops counting as an available path, and with
no controllers left the policy overrides the delayed_removal_secs
queue-if-no-path window; the two are opposites, so fail_on_ctrl_loss
takes precedence when both are set. ANA change and controller resetting
still queue, since both are transient. Controller state is untouched and
reconnects continue. Like dm-multipath, the policy is transport
agnostic.
Assisted-by: Claude:claude-fable-5
Reviewed-by: Nilay Shroff <nilay@linux.ibm.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Krishna Iyer <kiyer@crusoe.ai>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
nvme_available_path() has two problems that keep I/O queued when it
should be failed:
1. When fast_io_fail_tmo expires, NVME_CTRL_FAILFAST_EXPIRED is set and
the path is skipped, but the function still falls through to
nvme_mpath_queue_if_no_path(). If delayed_removal_secs is configured
the I/O is requeued indefinitely, defeating fast_io_fail_tmo. Only
fall through to queue_if_no_path when there really are no paths;
if a path exists but its failfast timer has expired, fail instead.
2. A LIVE controller counts as a usable path regardless of the namespace
ANA state. A path whose ANA state is inaccessible or persistent-loss
cannot serve I/O, so it should not count as available. ANA change is
transient and bounded by ANATT, so keep queueing while it resolves.
Also kick the requeue work from nvme_update_ns_ana_state() when a path
does not transition to live, so parked I/O is re-evaluated when an ANA
transition leaves the namespace inaccessible.
Move nvme_state_is_live() above nvme_available_path() so it can be used
there.
Assisted-by: Claude:claude-fable-5
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Krishna Iyer <kiyer@crusoe.ai>
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Reviewed-by: Nilay Shroff <nilay@linux.ibm.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
A discovery subsystem never exposes namespaces, so it has no multipath
head and no I/O paths to select between. Nevertheless the per-subsystem
iopolicy attribute is registered unconditionally and reads back the
driver-wide default which is 'numa' for the discovery subsystems too.
This can be misleading as it suggests a path selection policy is in
effect when nothing is ever selected, and writing the attribute silently
mutates state that can never be used.
Report "none" instead for a discovery subsystem, and reject writes
with -EOPNOTSUPP. Report "none" from iopolicies as well, so that the
value read from iopolicy always remains a member of the set advertised
by iopolicies; otherwise userspace validating the current policy
against the available ones would reject its own subsystem.
And while at it, document iopolicies too, which was not covered when it
was added previously.
Reviewed-by: Laurence Oberman <loberman@redhat.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Martin George <marting@netapp.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
When a remote invalidation names an rkey that is not the one we handed out
for this request, nvme_rdma_process_nvme_rsp() logs and schedules error
recovery, then falls through to nvme_rdma_end_request(). The request is
completed with req->mr never invalidated, and nvme_rdma_unmap_data() hands
it back with ib_mr_pool_put(), which only puts it on a list. The rkey
stays valid and still maps the pages the completed read filled, so a
controller that sent the bogus invalidation can write into those pages
after pread() has returned.
Return instead, the way the local invalidation path below already does,
and leave the request for error recovery to cancel.
Fixes: 3ef0279bb0031 ("nvme-rdma: Check remotely invalidated rkey matches our expected rkey")
Cc: stable@vger.kernel.org
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Yehyeong Lee <yhlee@isslab.korea.ac.kr>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
Similar to commit 2a8513091d2f ("nvme-rdma: parallelize I/O queue
allocation and startup"), refactor nvme tcp I/O queue setup to use async
API, combining allocation and startup into a single parallel operation
per queue. This reduces connection and reconnection setup time when
there are delays in establishing connections, which is especially
important for high-core-count hosts.
Key changes:
- Use async API to facilitate parallel calls for io queue setup.
- Add nvme_tcp_queue_setup_ctx for propagating errors from async
workers.
- Remove nvme_tcp_start_io_queues() and __nvme_tcp_alloc_io_queues();
their logic is folded into nvme_tcp_setup_io_queues() and
nvme_tcp_configure_io_queues().
- Allocate the io tag set before the queues so that the queue range is
known, and only set up the reconnect grow case if the queue count
actually increased.
- Serialize the cpu scan and claim in nvme_tcp_set_queue_io_cpu() with a
spinlock, as concurrent callers would otherwise select the same cpu.
The per-cpu counters no longer need to be atomics.
Testing on a 64-core host with 64 IO-queues shows nvme-tcp connection
time reduced from 61ms to 11ms.
Signed-off-by: Surabhi Gogte <sgogte@purestorage.com>
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
Split the I/O queue setup helpers apart so that the individual steps can
be called directly from nvme_tcp_configure_io_queues():
- Queue count negotiation moves out of nvme_tcp_alloc_io_queues() into a
new nvme_tcp_set_io_queue_count(), leaving the allocator with just the
per-queue allocation loop.
- TLS PSK validation moves out of __nvme_tcp_alloc_io_queues() into a
new nvme_tcp_tls_check_psk().
- nvme_tcp_configure_io_queues() now calls the three steps directly in
the same order as before.
Signed-off-by: Surabhi Gogte <sgogte@purestorage.com>
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
nvme_tcp_alloc_queue() creates each socket using
current->nsproxy->net_ns, which is correct for the initial connect that
runs in userspace context. However, the reconnect and error recovery
paths run from a workqueue where current netns is a kernel thread in
init_net, so sockets created during reconnect end up in a different
namespace than the one the controller was originally connected from.
Fix this by capturing the caller's network namespace at controller
allocation time. Use ctrl->net for all socket creation so that every
queue, on initial connect, reset, or reconnect, is created in the same
namespace.
Fixes: 3f2304f8c6d6 ("nvme-tcp: add NVMe over TCP host driver")
Signed-off-by: Surabhi Gogte <sgogte@purestorage.com>
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
Add support for ANS2 NVMe on the Apple t8132 (M4) SoC.
This variant requires the IO queues to be registered using additional
MMIO writes. The qsize of the create_{cq,sq} command is now ignored, and
instead a write to mmio_nvme+0x1210 (which was previously known as
APPLE_ANS_MAX_PEND_CMDS_CTRL) sets the IO queue sizes.
Signed-off-by: Yureka Lilian <yureka@cyberchaos.dev>
Reviewed-by: Sven Peter <sven@kernel.org>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux
Pull block fixes from Jens Axboe:
- NVMe fixes via Keith:
- Fix swapped ZRASF zone state values for full and read only
- nvme-multipath revalidate head zones after unfreezing the head
queue
- Revert "nvme: do not reset controllers in NVME_CTRL_NEW state",
which broke things
- nvmet fixes for invalid NSIDs on feature 82h and using the local
P2P device on metadata allocation failure
- zloop: zero out only the unread tail of a short read
* tag 'block-7.3-20261010' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux:
Revert "nvme: do not reset controllers in NVME_CTRL_NEW state"
zloop: zero out only the unread tail of short reads
nvmet: use the local P2P device on metadata allocation failure
nvme-multipath: revalidate head zones after unfreezing the head queue
nvme: fix swapped ZRASF zone state values for full and read only
nvmet: return Invalid Field for invalid NSIDs feature 82h
|
|
Cross-merge networking fixes after downstream PR (net-7.3-rc7).
Conflicts:
include/net/strparser.h
0984ebc631792 ("strparser: make sure __strp_recv isn't running before tearing down the parser")
02fd0a2111374 ("net: strparser: removed the aborted bit and aborts counter")
https://lore.kernel.org/asYfmLsfy-SCxXjr@sirena.co.uk
Adjacent changes:
Documentation/networking/strparser.rst
5f193cdabf25 ("docs: networking: strparser: remove general mode")
0984ebc63179 ("strparser: make sure __strp_recv isn't running before tearing down the parser")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Use the new helpers instead of open coding the SOCK_NOSPACE
manipulation, so that TCP can later maintain a cheaper private
copy of this bit.
No functional change intended.
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
Signed-off-by: Eric Dumazet <edumazet@kernel.org>
Link: https://patch.msgid.link/20260929071743.23624-8-edumazet@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
This reverts commit c20bba234f3e1dcd81f7a2ec21ea7bf01a4cf35f.
The standard transport drivers were okay with the patch, but the Apple
driver needs a bit of work before it can handle the state transition
requirement.
Link: https://lore.kernel.org/linux-nvme/94912d46-381d-4988-800a-6ef3b595233f@cyberchaos.dev/
Reported-by: Yureka Lilian <yureka@cyberchaos.dev>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
When a namespace on a multipath controller is updated,
nvme_update_ns_info() freezes the head disk queue, commits the new
limits, and then calls nvme_mpath_revalidate_zones() before it
unfreezes the queue again.
That is the wrong way round for blk_revalidate_disk_zones(). It starts
a limits update, which takes q->limits_lock, and it freezes the queue
itself while updating the zone resources. The block layer takes
limits_lock before freezing the queue, never the other way around, so
calling it with the head queue already frozen reverses that order.
syzbot has hit this twice. One report goes through q->limits_lock. The
other one is on linux-next, where blk_revalidate_disk_zones() also
takes disk->zone_revalidate_mutex and holds it across alloc_workqueue()
the first time a disk's zone resources are set up. Lockdep then sees:
q_usage_counter(io) (frozen head queue, nvme_update_ns_info())
--> &disk->zone_revalidate_mutex
--> wq_pool_mutex --> fs_reclaim --> q_usage_counter(io)
WARNING: possible circular locking dependency detected
kworker/u8:10/3352 is trying to acquire lock:
(&disk->zone_revalidate_mutex){+.+.}-{4:4}, at: blk_revalidate_disk_zones+0x1c5/0x1650
but task is already holding lock:
(&q->q_usage_counter(io)#75){++++}-{0:0}, at: nvme_update_ns_info+0x3ac/0x1200
...
blk_revalidate_disk_zones+0x1c5/0x1650 block/blk-zoned.c:2560
nvme_mpath_revalidate_zones+0x106/0x1c0 drivers/nvme/host/multipath.c:301
nvme_update_ns_info+0x984/0x1200 drivers/nvme/host/core.c:2620
The rest of the driver already does this correctly:
nvme_update_ns_info_block() unfreezes ns->disk->queue before calling
blk_revalidate_disk_zones(), and nvme_mpath_set_live() revalidates the
head zones without freezing the queue. Do the same here, and only
revalidate the head zones once the queue is unfrozen and the limits
update has succeeded.
Fixes: 224041412693 ("nvme-multipath: revalidate zones for namespace heads")
Reported-by: syzbot+b0910be96b7c31314822@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=b0910be96b7c31314822
Reported-by: syzbot+2e02ccadb3c5522a5c59@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=2e02ccadb3c5522a5c59
Link: https://lore.kernel.org/all/2bfc96f2-7d0d-47e0-936e-8810abb31a9f@acm.org/
Cc: Shuah Khan <shuah@kernel.org>
Cc: Brigham Campbell <me@brighamcampbell.com>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
Reviewed-by: Yao Sang <sangyao@kylinos.cn>
Tested-by: Yao Sang <sangyao@kylinos.cn>
Signed-off-by: Palla Raghunath <raghunathpalla.0209@gmail.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux
Pull block fixes from Jens Axboe:
- NVMe fixes via Keith:
- Fix an out-of-bounds write in nvmet_auth_challenge(), where
sizeof() on a void pointer undercounted the challenge header and
let a short AUTH_RECEIVE buffer pass the check
- nvme-multipath fixes for an ANA log bounds check underflow, the
command effects log lifetime for multipath heads, and only
setting BLK_FEAT_ZONED after the zone info is known.
- nvmet fixes for ns->enabled teardown ordering, rejecting I/O
after the percpu ns reference is killed, device path preservation
on allocation failure, and too-short SGL segments in pci-epf
- nvme-tcp: revert the per-socket dynamic lockdep keys, and delay
the socket reclassification
- A DMA pool alignment quirk for the Micron 4100AT
- Controller state/reset race fixes, and -Wformat-security
workarounds
- blk-mq: set RQF_USE_SCHED when the operation is known, and allow
cached requests to be used for flush operations
- Reject polled dio with user integrity metadata
- Save the IRQ state in blkg_tryget_closest()
- Set the zone write granularity in virtio_blk
- ublk selftest fixes
* tag 'block-7.3-20261002' of git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux: (23 commits)
virtio_blk: set the zone write granularity
nvme-multipath: set BLK_FEAT_ZONED only after the zone info is known
nvme: fix command effects log lifetime for multipath heads
nvmet: don't allow I/O admission after percpu ns reference is killed
nvmet: defer setting ns->enabled to false in nvmet_ns_disable()
nvmet: copy the hostid into the ctrl before creating PR pc_refs
nvmet-auth: fix out-of-bounds write in nvmet_auth_challenge()
nvmet: pci-epf: reject too-short SGL segments
nvme-multipath: fix underflow in ANA log bounds checks
nvme: work around all -Wformat-security warnings
nvme: work around -Wformat-security warning
nvme: do not reset controllers in NVME_CTRL_NEW state
nvme-tcp: delay nvme_tcp_reclassify_socket()
Revert "nvme-tcp: lockdep: use dynamic lockdep keys per socket instance"
drbd: remove unused drbd_nl_mcgrps[] array
blk-mq: allow cached requests to be used for flush operations
blk-mq: set RQF_USE_SCHED when the operation is known
block: reject polled dio with user integrity metadata
selftests: ublk: fix unused_result error
blk-cgroup: save IRQ state in blkg_tryget_closest()
...
|
|
The namespace head is marked zoned at allocation time based only on the
command set identifier. If the zone info query reports a zero zone size,
the path namespace is registered without zoned limits while the head
still advertises the zoned capability with a zone size of zero, and
reporting zones or writing to the head then shifts by ilog2(0).
Drop the zoned feature from the head allocation and let the head limits
refresh stack it in from the path namespace instead.
Fixes: 28982ad73d6a ("nvme: set BLK_FEAT_ZONED for ZNS multipath disks")
Fixes: 3838e80fcfb3 ("nvme: skip the zoned limits update if the zone info query failed")
Cc: stable@vger.kernel.org
Signed-off-by: Guixin Liu <kanie@linux.alibaba.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
KASAN reported a use-after-free when an I/O passthrough command was sent
through a multipath namespace head after the controller path that first
created the head had been removed:
BUG: KASAN: slab-use-after-free in nvme_command_effects+0x192/0x200 [nvme_core]
Read of size 4 at addr ffff888141b14400 by task nvme/19811
nvme_command_effects+0x192/0x200 [nvme_core]
nvme_cmd_allowed+0x7e/0x1b0 [nvme_core]
nvme_user_cmd.constprop.0+0x1b5/0x450 [nvme_core]
nvme_ns_head_chr_ioctl+0xf4/0x2a0 [nvme_core]
Move the log cache to the subsystem, with one entry per command set. Both
controllers and namespace heads hold subsystem references, keeping their
log pointers valid until subsystem release.
Fixes: be93e87e7802 ("nvme: support for multiple Command Sets Supported and Effects log pages")
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Yao Sang <sangyao@kylinos.cn>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
|
The number of ANA groups advertised through Identify Controller sizes the
ANA log buffer. The ANA log header and group descriptors independently
supply the number of groups and namespace IDs to parse.
Both bounds checks subtract an untrusted object size from ana_log_size
before comparing the current offset. If the object is larger than the
buffer, the size_t subtraction underflows and lets the parser read beyond
ana_log_buf.
Check the current offset before the first subtraction and compare each
object size with the remaining buffer instead. This rejects inconsistent
ANA data before dereferencing a truncated group descriptor or walking an
oversized namespace ID array.
Fixes: 0d0b660f214d ("nvme: add ANA support")
Cc: stable@vger.kernel.org
Assisted-by: LLM
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Daehyeon Ko <4ncienth@gmail.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
|
