workqueue: Add workqueue_cpu_intensive tracepoint

When a concurrency-managed per-CPU work item runs continuously without
sleeping for longer than wq_cpu_intensive_thresh_us, wq_worker_tick() marks
the worker as WORKER_CPU_INTENSIVE and kicks it out of concurrency
management so that pending work items on the pool are not starved.

While CONFIG_WQ_CPU_INTENSIVE_REPORT logs rate-limited warnings and
pwq->stats[PWQ_STAT_CPU_INTENSIVE] maintains a cumulative counter, there is
currently no tracepoint emitted at the moment of this transition.

Therefore, add the workqueue_cpu_intensive tracepoint, recording the
work_struct pointer and callback function pointer, workqueue name,
executing CPU, and the runtime duration consumed in microseconds.

This enables eBPF profilers, bpftrace, and Ftrace to immediately detect and
attribute CPU-hogging work items in real time.

Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
Signed-off-by: Tejun Heo <tj@kernel.org>

1 files changed, 7 insertions, 0 deletions

diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index d4d6bb8a23f5..22fb41b6ec26 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -56,6 +56,7 @@
 #include <linux/kvm_para.h>
 #include <linux/delay.h>
 #include <linux/irq_work.h>
+#include <linux/math64.h>
 
 #include "workqueue_internal.h"
 
@@ -1535,6 +1536,7 @@ void wq_worker_tick(struct task_struct *task)
 	struct worker *worker = kthread_data(task);
 	struct pool_workqueue *pwq = worker->current_pwq;
 	struct worker_pool *pool = worker->pool;
+	u64 delta;
 
 	if (!pwq)
 		return;
@@ -1575,6 +1577,11 @@ void wq_worker_tick(struct task_struct *task)
 		pwq->stats[PWQ_STAT_CM_WAKEUP]++;
 
 	raw_spin_unlock(&pool->lock);
+
+	delta = READ_ONCE(worker->task->se.sum_exec_runtime) - worker->current_at;
+	trace_workqueue_cpu_intensive(pwq, worker->current_work,
+				      worker->current_func,
+				      div_u64(delta, NSEC_PER_USEC));
 }
 
 /**