linux-next.git/kernel/sched/debug.c, branch master

sched/debug: Remove unused schedstats

2026-06-30T08:56:57+00:00

nr_migrations_cold, nr_wakeups_passive and nr_wakeups_idle are not being updated anywhere. So remove them. These are per process stats. So updating sched stats version isn't necessary. Signed-off-by: Shrikanth Hegde Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: K Prateek Nayak Tested-by: K Prateek Nayak Link: https://patch.msgid.link/20260625124648.802832-2-sshegde@linux.ibm.com

sched/eevdf: Move to a single runqueue

2026-06-30T08:56:53+00:00

Change fair/cgroup to a single runqueue. Infamously fair/cgroup isn't working for a number of people; typically the complaint is latencies and/or overhead. The latency issue is due to the intermediate entries that represent a combination of tasks and thereby obfuscate the runnability of tasks. The approach here is to leave the cgroup hierarchy as is; including the intermediate enqueue/dequeue but move the actual EEVDF runqueue outside. This means things like the shares_weight approximation are fully preserved. That is, given a hierarchy like: R | se--G1 / \ G2--se se--G3 / \ | T1--se se--T2 se--T3 This is fully maintained for load tracking, however the EEVDF parts of cfs_rq/se go unused for the intermediates and are instead connected like: _R_ / | \ T1 T2 T3 Since the effective weight of the entities is determined by the hierarchy, this gets recomputed on enqueue,set_next_task and tick. Notably, the effective weight (se->h_load) is computed from the hierarchical fraction: se->load / cfs_rq->load. Since EEVDF is now exclusively operating on rq->cfs, it needs to consider cfs_rq->h_nr_queued rather than cfs_rq->nr_queued. Similarly, only tasks can get delayed, simplifying some of the cgroup cleanup. One place where additional information was required was set_next_task() / put_prev_task(), where we need to track 'current' both in the hierarchical sense (cfs_rq->h_curr) and in the flat sense (cfs_rq->curr). As a result of only having a single level to pick from, much of the complications in pick_next_task() and preemption go away. Since many of the hierarchical operations are still there, this won't immediately fix the performance issues, but hopefully it will fix some of the latency issues. TODO: split struct cfs_rq / struct sched_entity TODO: try and get rid of h_curr Signed-off-by: Peter Zijlstra (Intel) Link: https://patch.msgid.link/20260605124052.227463677%40infradead.org

sched/fair: Change the default cgroup_mode to concur

2026-06-30T08:56:53+00:00

For all the reasons described in the preceding patches, the way cgroup weight is computed is problematic. However, changing it is bound to also lead to trouble. Esp. since people might have taken to inflating the weight value where they can. Since things are configurable, change the default and hope this serves more people than it hurts, esp. in the longer run. Specifically, this prepares for a flattened runqueue, where the hierarchical weight becomes far more important (F_g^d terms), so getting rid of small F_g is imperative. Signed-off-by: Peter Zijlstra (Intel) Link: https://patch.msgid.link/20260605124052.080482755%40infradead.org

sched/fair: Add cgroup_mode: tasks

2026-06-30T08:56:53+00:00

Since we are exploring this space; include a scheme that scales by total number of runnable tasks. This results in: F_g_n' = M * F_g_n This will obviously have: avg(F_g_n') > 1, (it will be ~M/N in fact). And while that sounds odd, it actually has a fairly straight foward meaning for "cpu.weight": average weight per member task. This is an entirely valid and workable option, it is however wildly different from the traditional meaning. Included for completeness (and curiosity). Signed-off-by: Peter Zijlstra (Intel) Link: https://patch.msgid.link/20260605124051.921991975%40infradead.org

sched/fair: Add cgroup_mode: concur

2026-06-30T08:56:52+00:00

Improve upon the previous scheme ("max") by no longer assuming maximal concurrency. Instead scale by: 'min(nr_tasks, nr_cpus)'. This handles the low concurrency cases more gracefully: F_g_n' = min(M, N) * F_g_n Notably this is the first mode where: avg(F_g_n) = 1 In the single task case it reduces to ("smp") and then it nicely scales up until it hits N, where it behaves like ("max"). This is no longer clipped at nice -20. Strictly speaking it isn't different from the normal SMP scenario where all tasks are extremely unbalanced. There are no unnatural inflations in this scheme. The meaning of "cpu.weight" would be: weight per active CPU. NOTE: Compute the group wide number of tasks by extending the tg->load_avg computation with tg->runnable_avg, since cfs_rq->runnable_avg is based on cfs_rq->h_nr_running. Signed-off-by: Peter Zijlstra (Intel) Link: https://patch.msgid.link/20260605124051.740585993%40infradead.org

sched/fair: Add cgroup_mode: max

2026-06-30T08:56:52+00:00

In order to avoid the average CPU fraction avg(F_g_n) becoming tiny '1/N', assume each cgroup is maximally concurrent and distrubute 'N*weight', such that: F_g_n' = N * F_g_n Giving: avg(F_g_n') = N*avg(F_g_n) ~ N * 1/N = 1 And while this sounds like it solves things, remember what that ~ meant. There is the corner case when a cgroup is minimally loaded, eg a single runnable task, therefore limit the CPU fraction to that of a nice -20 task to avoid getting too much load. This last bit is what makes it different from a previous proposal to allow raising cpu.weight to '100 * N', that would not limit the mininal concurrency case and results in a very large F_g_n. And just like F_g_n << 1 is problematic, so is F_g_n >> 1 for the exact same reasons (it would drown the kthreads, but it also risks overflowing the load values). So while this might appear to be a better scheme than the current default scheme, it doesn't really handle less than maximal concurrency nicely -- it clips and introduces artificially large weights. So where the traditional SMP mode works well when nr_tasks << nr_cpus, MAX doesn't work well in that regime and vice-versa. The meaning of "cpu.weight" would be: weight per allowed CPU. Included for completeness (and infrastructure). Signed-off-by: Peter Zijlstra (Intel) Link: https://patch.msgid.link/20260605124051.589618504%40infradead.org

sched/fair: Add cgroup_mode: up

2026-06-30T08:56:52+00:00

Instead of calculating the proportional fraction of the group weight for each CPU, just give each CPU the full measure, ignoring these pesky SMP problems. This makes the SMP cgroup fraction (F_g_n) equal to 1, and ensures a single task in a cgroup competes on equal footing to a task in a level above. However, as already explored, this is not a very good policy because it gets the SMP weight distribution wrong. Included for completeness. Signed-off-by: Peter Zijlstra (Intel) Link: https://patch.msgid.link/20260605124051.450303977%40infradead.org

sched/fair: Add cgroup_mode switch

2026-06-30T08:56:51+00:00

The effective task weight (W_t') for a task in cgroup g on CPU n is given by: W_t W_t' = W_g * F_g_n * ---------- \Sum W_t_n Where W_g is the group's weight (cpu.weight), F_g_n is the fraction of the group weight for CPU n and W_t/W is the relative weight of this task against all other tasks in the same group on the same CPU. Furthermore, this makes: \Sum W_t_n F_g_n = ---------- \Sum W_t The fraction of weight inside the group of CPU n against the whole group. The problem is with F_g_n, the primary goal of this fraction is to make sure that the relative weight of tasks, when distributed over CPUs is maintained. For example, consider 4 (equal weight) tasks and 2 CPUs with a 1:3 distribution, then if F_g_n would simply be 1 (no weight re-distribution) the effective relative weights (W_t') of the tasks in our group would be: CPU0 CPU1 W_g W_g/3 W_g/3 W_g/3 IOW, the lucky task on CPU0 would get an equal amount of weight as all 3 tasks on CPU1 combined. However, with the weight redistribution, this becomes: CPU0 CPU1 W_g/4 W_g/4 W_g/4 W_g/4 All tasks are equal weight (as intended). However, as is already evident from this example, the more CPUs you add, the smaller F_g_n becomes, which creates a disparity against tasks not in our group. Specifically: avg(F_g_n) ~ 1/N This leads to a weight mismatch in the hierarchy. IOW tasks cannot compete fairly across hierarchy levels. *Notably*, what is meant by avg(F_g_n) being proportional to 1/N is that when there are at least N runnable tasks, the average of this fraction tends to 1/N. For a hierarchy of depth d, this gets even worse, since that gets terms on the order of: avg(F_g_n)^d ~ 1/(N^d) Given fixed point arithmetic, this also leads to numerical trouble. However, the meaning of "cpu.weight" is simple and intiutive: the total weight of the cgroup. But as explored above, there is deception in this simplicity. Prepare to add a few alternative methods for distributing weight. Signed-off-by: Peter Zijlstra (Intel) Link: https://patch.msgid.link/20260605124051.338602724%40infradead.org

sched/fair: Remove task_group->se pointer array

2026-06-02T10:26:11+00:00

Now that struct sched_entity is co-located with struct cfs_rq for non-root task groups, the task_group->se pointer array is redundant. The associated sched_entity can be loaded directly from the cfs_rq. This patch performs the access conversion with the helpers: - is_root_task_group(tg): checks if a task group is the root task group. It compares the task group's address with the global root_task_group variable. - tg_se(tg, cpu): retrieves the cfs_rq and returns the address of the co-located se. This function checks if tg is the root task group to ensure behaving the same of previous tg->se[cpu]. Replaces all accesses that use the tg->se[cpu] pointer array with calls to the new tg_se(tg, cpu) accessor. - cfs_rq_se(cfs_rq): simplifies access paths like cfs_rq->tg->se[...] to use the co-located sched_entity. This function also checks if tg is the root task group to ensure same behavior. Since tg_se is not in very hot code paths, and the branch is a register comparison with an immediate value (`&root_task_group`), the performance impact is expected to be negligible. Signed-off-by: Zecheng Li Signed-off-by: Zecheng Li Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: K Prateek Nayak Reviewed-by: Josh Don Link: https://patch.msgid.link/20260522141623.600235-3-zli94@ncsu.edu

sched/deadline: Reject debugfs dl_server writes for offline CPUs

2026-05-29T10:43:15+00:00

Writing runtime or period via the per-CPU dl_server debugfs files (/sys/kernel/debug/sched/{fair,ext}_server/cpu*/{runtime,period}) on an offline CPU can trigger two distinct kernel issues: 1) Divide-by-zero in dl_server_apply_params(): Oops: divide error: 0000 [#1] SMP NOPTI RIP: 0010:dl_server_apply_params+0x239/0x3a0 Call Trace: sched_server_write_common.isra.0+0x21a/0x3c0 full_proxy_write+0x78/0xd0 vfs_write+0xe7/0x6e0 Both __dl_sub() and __dl_add() divide by cpus internally, which can be 0 once the CPU has been removed from any active root-domain span (this has been latent since the debugfs interface was introduced). 2) WARN_ON_ONCE in dl_server_start(): WARNING: kernel/sched/deadline.c:1805 at dl_server_start+0x232/0x270 Commit ee6e44dfe6e5 ("sched/deadline: Stop dl_server before CPU goes offline") added this check to catch enqueueing the server on an offline rq. There's no meaningful semantics for re-configuring the per-CPU dl_server bandwidth while the CPU is offline, so simply reject the write with -EBUSY so userspace gets a clear error. Closes: https://lore.kernel.org/all/20260526092228.3B6891F00A3A@smtp.kernel.org/ Fixes: d741f297bcea ("sched/fair: Fair server interface") Reported-by: Sashiko Signed-off-by: Andrea Righi Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Juri Lelli Tested-by: abaci-kreproducer Link: https://patch.msgid.link/20260526100502.575774-1-arighi@nvidia.com