linux-next.git/kernel/workqueue.c, branch stable

Merge tag 'wq-for-7.4' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq

2026-10-22T20:25:52+00:00

Pull workqueue updates from Tejun Heo: - Flushing a workqueue now visits only the queues used since the last flush instead of one per possible CPU. Idle flushes get 10 to 100 times faster on large machines, which XFS hits on every log force. - Per-cpu workqueues are built through the same attribute path as unbound ones and get a max_active of their own, continuing the work to turn them into an affinity scope of unbound workqueues. - New tracepoints for work items turning CPU-intensive, rescuer mayday and rescue events, and BH workers yielding with work pending. - A KCSAN annotation and small cleanups * tag 'wq-for-7.4' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq: workqueue: mark pool->nr_running as __data_racy workqueue: BUG_ON() a pwq release before workqueue_init() workqueue: Derive the standard attrs from the workqueue flags workqueue: Give percpu workqueues their own max_active workqueue: Make flush_workqueue() visit only pwqs active since the last flush workqueue: Maintain pwq->total_in_flight workqueue: Add for_each_node_with_fallback() workqueue: Fix typo "upto" in comment workqueue: Add workqueue_bh_budget_yield tracepoint workqueue: Add workqueue_mayday and workqueue_rescued tracepoints workqueue: Add workqueue_cpu_intensive tracepoint workqueue: rename the pwq slot helpers shared with percpu workqueue: build percpu pwqs through the attrs path workqueue: make the default pwq optional workqueue: resolve the backing pool in alloc_pwq() workqueue: move the unbound-only attrs check to apply_workqueue_attrs()

workqueue: mark pool->nr_running as __data_racy

2026-10-14T17:03:54+00:00

wq_worker_running() increments pool->nr_running with only preemption disabled, while kick_pool_pick() reads it via need_more_worker() under pool->lock, so the two can run concurrently on different CPUs. KCSAN reports: BUG: KCSAN: data-race in kick_pool_pick / wq_worker_running write to 0xffff888059c2af24 of 4 bytes by task 3007 on cpu 0: wq_worker_running+0x114/0x130 schedule_timeout+0x16e/0x180 io_schedule_timeout+0x4c/0x70 submit_bio_wait+0xd7/0x110 blkdev_issue_flush+0x4c/0x70 ext4_sync_file+0x60d/0x6c0 aio_fsync_work+0x7f/0x270 process_one_work+0x4ae/0xa00 read to 0xffff888059c2af24 of 4 bytes by task 3001 on cpu 1: kick_pool_pick+0x7a/0x3a0 __queue_work+0x72d/0xcf0 queue_work_on+0x9d/0xb0 schedule_on_each_cpu+0x11a/0x2d0 vmstat_refresh+0x23/0x140 proc_sys_call_handler+0x2b9/0x4f0 The race is benign. The increment in wq_worker_running() is the only lockless write to nr_running; decrements and the reset are done under pool->lock, so a reader holding the lock sees either the old or the new count. A stale 0 costs at most a spurious wakeup of an idle worker. A nonzero count is correct because the incrementing worker is running and re-checks the worklist under pool->lock before it stops counting. The increment can also go 1 -> 2, e.g. when a worker sleeping in a work item wakes up after kick_pool() woke another worker. The lockless increment is part of how nr_running is designed to work, as the comment above the field documents. Mark the field __data_racy so that every reader is covered, not just need_more_worker(). Reported-by: syzbot+8922e6b8343f70be92e8@syzkaller.appspotmail.com Closes: https://syzkaller.appspot.com/bug?extid=8922e6b8343f70be92e8 Signed-off-by: Nguyen Ngoc Thang Signed-off-by: Tejun Heo

workqueue: Fix NULL current_pwq deref in mem-reclaim helper

2026-10-07T17:57:17+00:00

current_is_workqueue_mem_reclaim() uses current_wq_worker() to decide whether %current is a workqueue worker and then checks the current workqueue flags. However, current_wq_worker() only implies that %current is a kworker; it does not imply that the worker is currently executing a work item. worker->current_pwq is set while process_one_work() runs the work function and cleared afterwards. A kworker outside work-item execution can therefore have PF_WQ_WORKER set with current_pwq cleared. Guard the flag check with worker->current_pwq. Without a current_pwq, the worker is not executing on a WQ_MEM_RECLAIM workqueue, so the helper should return false. Fixes: da729ddd4a1b ("NFS/localio: issue IO inline when not in a memory-reclaim context") Signed-off-by: Pavankumar Kondeti Signed-off-by: Tejun Heo

Merge tag 'wq-for-7.3-rc6-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq

2026-10-07T00:24:23+00:00

Pull workqueue fixes from Tejun Heo: - Fix a NULL dereference in the chained work check when a kworker queues work on a draining or destroying workqueue outside work item execution. * tag 'wq-for-7.3-rc6-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq: workqueue: Fix NULL current_pwq deref in chained work check

workqueue: Fix NULL current_pwq deref in chained work check

2026-09-28T17:46:45+00:00

current_wq_worker() only tells us that %current is a kworker. It does not guarantee that the worker is currently executing a work item, as worker->current_pwq is populated only while process_one_work() is running the work function. is_chained_work() can be reached while queueing on a draining or destroying workqueue. If a kworker outside work-item execution reaches that path, current_wq_worker() returns a worker but worker->current_pwq is NULL, and the chained-work test can fault before it emits the intended warning. Guard the chained-work test with worker->current_pwq. A kworker without current_pwq is not executing work on the target workqueue, so the helper should return false and let the draining/destroying warning fire. Fixes: 8d03ecfe4718 ("workqueue: reimplement is_chained_work() using current_wq_worker()") Cc: stable@vger.kernel.org Signed-off-by: Pavankumar Kondeti Signed-off-by: Tejun Heo

Merge tag 'wq-for-7.3-rc4-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq

2026-09-27T15:55:24+00:00

Pull workqueue fix from Tejun Heo: - Fix a NULL dereference in the flush dependency check when a worker flushes outside a work item, such as from the OOM path during worker creation * tag 'wq-for-7.3-rc4-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq: workqueue: Fix NULL current_pwq deref in flush dependency check

workqueue: Fix NULL current_pwq deref in flush dependency check

2026-09-27T08:18:50+00:00

check_flush_dependency() uses current_wq_worker() to determine whether the caller is a workqueue worker and then dereferences worker->current_pwq to test whether the current workqueue is WQ_MEM_RECLAIM. current_wq_worker() only means that %current has PF_WQ_WORKER set. A kworker can reach check_flush_dependency() while it is not executing a work item. One such path is worker_thread() acting as the pool manager, where create_worker() does GFP_KERNEL allocation and the allocation path invokes the OOM notifier. In that state worker->current_pwq is NULL because current_pwq is set only by process_one_work() and cleared again after the work function returns. [ 416.760634][ T375] Call trace: [ 416.760638][ T375] check_flush_dependency+0x80/0x120 (P) [ 416.760648][ T375] __flush_work+0x98/0x224 [ 416.760657][ T375] flush_work+0x30/0x44 [ 416.760665][ T375] ... [ 416.760710][ T375] blocking_notifier_call_chain+0x58/0xa0 [ 416.760719][ T375] out_of_memory+0xb4/0x458 [ 416.760730][ T375] __alloc_pages_may_oom+0x11c/0x1a8 [ 416.760739][ T375] __alloc_pages_slowpath+0x314/0x46c [ 416.760746][ T375] __alloc_frozen_pages_noprof+0x110/0x1a4 [ 416.760753][ T375] new_slab+0x12c/0x484 [ 416.760759][ T375] ___slab_alloc+0x7a8/0xc7c [ 416.760765][ T375] __slab_alloc+0x74/0xd8 [ 416.760772][ T375] __kmalloc_cache_node_noprof+0x2ac/0x304 [ 416.760779][ T375] alloc_worker+0x28/0x60 [ 416.760785][ T375] create_worker+0x4c/0x20c [ 416.760790][ T375] worker_thread+0xe8/0x2b8 [ 416.760796][ T375] kthread+0x1a8/0x200 [ 416.760805][ T375] ret_from_fork+0x10/0x20 Guard the WQ_MEM_RECLAIM-worker warning with worker->current_pwq. If the kworker is not currently executing a work item, there is no current workqueue to diagnose with that warning. The PF_MEMALLOC warning is left unchanged so explicit reclaim context flushing a !WQ_MEM_RECLAIM target is still reported. Fixes: fca839c00a12 ("workqueue: warn if memory reclaim tries to flush !WQ_MEM_RECLAIM workqueue") Cc: stable@vger.kernel.org Assisted-by: LLM Signed-off-by: Pavankumar Kondeti Signed-off-by: Tejun Heo

workqueue: BUG_ON() a pwq release before workqueue_init()

2026-09-11T14:48:38+00:00

put_pwq() bounces the release to pwq_release_worker to get out from under pool->lock. workqueue_init() creates that kthread_worker well after workqueue_init_early() has built the system workqueues, so a pwq allocation failing partway through apply_wqattrs_prepare() unwinds through apply_wqattrs_cleanup() and dereferences a NULL worker. There is no way back from that. Running the release inline is not an option, that is what the bounce exists for, and workqueue_init_early() already BUG_ON()s a system workqueue it failed to allocate. BUG_ON() the missing worker instead of oopsing inside kthread_queue_work(). Suggested-by: Tejun Heo Link: https://lore.kernel.org/all/144b3dc986135ebe34f671cdd02bd8aa@kernel.org/ Signed-off-by: Breno Leitao Signed-off-by: Tejun Heo

workqueue: Derive the standard attrs from the workqueue flags

2026-09-10T21:54:04+00:00

alloc_and_link_pwqs() picks one of three static attrs arrays indexed by WQ_HIGHPRI, and branches to a separate apply_workqueue_attrs_locked() call for percpu, ordered and unbound workqueues. The arrays differ only in the nice level and the ordered bit, which the flags already say. Build the attrs from the flags instead. alloc_wq_std_attrs() sets nice for WQ_HIGHPRI and ordered for __WQ_ORDERED, which collapses the three branches into one call and drops the three arrays along with their workqueue_init_early() loop. The ordering WARN now sits behind its own __WQ_ORDERED test rather than the unbound branch. __WQ_ORDERED is internal and every macro that sets it pairs it with WQ_UNBOUND, so the WARN covers the same workqueues as before. The attrs are allocated per creation now. apply_wqattrs_prepare() already allocates one on the same path, including in workqueue_init_early(), so that is not a new constraint, and it only copies what it is given, so the caller can free them once apply_workqueue_attrs_locked() returns. Signed-off-by: Breno Leitao Signed-off-by: Tejun Heo

workqueue: Give percpu workqueues their own max_active

2026-09-10T21:47:51+00:00

wq->max_active means two different things, depending on how the workqueue is backed. On a concurrency managed per-cpu pool it caps pwq->nr_active, so it is a per-cpu limit. On an unbound pool it feeds wq_node_nr_active() and is shared by the whole workqueue. Tejun once said [1]: It kinda sucks that max_active's meaning is different across the boundary tho. Maybe percpu max_active should be separate into its own field, idk. Add wq->percpu_max_active for the first meaning and leave max_active as the system wide one. wq_adjust_max_active() updates whichever of the two applies, and min_active stays with max_active as only the unbound side uses it. Suggested-by: Tejun Heo Link: https://lore.kernel.org/all/amESSqf0TMmzhFGz@slm.duckdns.org/ [1] Signed-off-by: Breno Leitao Signed-off-by: Tejun Heo