summaryrefslogtreecommitdiff
path: root/drivers/gpu/drm/amd/amdkfd
AgeCommit message (Collapse)Author
4 daysdrm/amdkfd: Check bounds on CRIU restore queue type and mqd sizeDavid Francis
We weren't checking whether the values provided in the private data in kfd CRIU restore were within bounds. For queue type, add a KFD_QUEUE_TYPE_MAX and ensure the provided type is less than it. For mqd_size, add new function mqd_size_from_queue_type and confirm that the provided mqd_size matches expectations. Reviewed-by: David Yat Sin <david.yatsin@amd.com> Signed-off-by: David Francis <David.Francis@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> (cherry picked from commit f19d8086f6644083c913d70bfdeee20e1b6f46a5) Cc: stable@vger.kernel.org
4 daysdrm/amdkfd: Check bounds in allocate_event_notification_slotDavid Francis
The valid event ids go from 0 to KFD_SIGNAL_EVENT_LIMIT allocate_event_notification_slot has an option to specify an event id to allocate at, used by CRIU. We weren't checking the bounds on that value. Check them. v2: Lower bounds check is unecessary because of idr_alloc already rejecting negative numbers. Upper bounds check should be KFD_SIGNAL_EVENT_LIMIT since the signal mode mappings might not yet exist Signed-off-by: David Francis <David.Francis@amd.com> Reviewed-by: David Yat Sin <david.yatsin@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> (cherry picked from commit 6853f1f6cbbeb3f53ebbbd7286536aeb2c5d5f50) Cc: stable@vger.kernel.org
4 daysamdkfd: properly free secondary context idZhu Lingshan
Function kfd_process_free_id() should skip over the primary kfd process because its context id is fixed assigned, not allocated through the ida table. This function should only work on secondary contexts. Fixes: fac682a1d1af ("amdkfd: identify a secondary kfd process by its id") Signed-off-by: Zhu Lingshan <lingshan.zhu@amd.com> Reviewed-by: Felix Kuehling <felix.kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> (cherry picked from commit 8799ba6fb6a48438aea20c82e74c2f2a3d2b2e7a) Cc: stable@vger.kernel.org
4 daysdrm/amdkfd: Don't acquire buffers during CRIU queue restore.David Francis
kfd_criu_restore_queue's call of kfd_queue_acquire_buffers was failing for multiple reasons - The ctl_stack_size set by the CRIU plugin doesn't match what is expected by acquire_buffers - The svm buffer cannot be acquired at this point because CRIU may not have restored it, or may have restored it to a different address. The only reason acquire_buffers was necessary here was to avoid a null ptr dereference in init_user_queue. Just put in a check for that dereference; it doesn't appear to come up in real use cases right now. That is, there is no usage of CRIU with shared MES. This is a partial revert of commit 20a5e7ffdfec ("drm/amdkfd: Properly acquire queue buffers in CRIU restore") Fixes: 20a5e7ffdfec ("drm/amdkfd: Properly acquire queue buffers in CRIU restore") Reviewed-by: David Yat Sin <david.yatsin@amd.com> Signed-off-by: David Francis <David.Francis@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> (cherry picked from commit 1cafa8b29e029eac3ddf64604f891b35dbf6262b) Cc: stable@vger.kernel.org
4 daysdrm/amdkfd: Check bounds on CRIU restore event idDavid Francis
The valid amdkfd event ids go from 0 to KFD_SIGNAL_EVENT_LIMIT - 1. During CRIU restore, ensure that the provided event ids are in that range. v2: No need for lower bound check since idr_alloc rejects negative inputs v3: Also change error message to reflect new error condition Reviewed-by: David Yat Sin <david.yatsin@amd.com> Signed-off-by: David Francis <David.Francis@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> (cherry picked from commit 5c6c247992d4d9200e073b83f4ec6c703c096845)
11 daysdrm/amdgpu: Fix AMDGPU_GTT_MAX_TRANSFER_SIZE for non-4K systemsDonet Tom
Running RCCL unit tests on a system with a 64K PAGE_SIZE triggers the following warning and causes the test to terminate on latest upstream kernel: WARNING: drivers/gpu/drm/amd/amdgpu/amdgpu_object.c:1335 at amdgpu_bo_release_notify+0x1bc/0x280 [amdgpu], CPU#18: rccl-UnitTests/33151 Call trace: amdgpu_bo_release_notify ttm_bo_release amdgpu_gem_object_free drm_gem_object_free amdgpu_bo_unref amdgpu_bo_create amdgpu_bo_create_user amdgpu_gem_object_create amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu kfd_ioctl_alloc_memory_of_gpu kfd_ioctl sys_ioctl The warning is triggered because amdgpu_ttm_next_clear_entity() returns NULL when a clear buffer operation is requested. This happens because the GART window allocation for the default_entity, clear_entity and move_entity fails during initialization. Commit [1] introduced separate GART windows for the default_entity, clear_entity and move_entity of each SDMA instance. Their sizes are derived from AMDGPU_GTT_MAX_TRANSFER_SIZE, which is currently defined as 1024 pages. This implicitly assumes a 4K PAGE_SIZE, where 1024 pages correspond to a 4MB transfer. On a 64K PAGE_SIZE system, however, the same value expands to 64MB. The default_entity and clear_entity each allocate one AMDGPU_GTT_MAX_TRANSFER_SIZE GART window, while the move_entity allocates two such windows. This results in 16MB of GART space per SDMA instance on a 4K PAGE_SIZE system, but 256MB per SDMA instance on a 64K PAGE_SIZE system. On an MI210 system with five SDMA instances and a 512MB GART aperture, the total GART space required becomes 1.25GB, exceeding the available GART aperture. Consequently, GART window allocation fails, amdgpu_ttm_next_clear_entity() returns NULL, and the above warning is triggered. Redefine AMDGPU_GTT_MAX_TRANSFER_SIZE in bytes instead of page units. Where a page count is required, convert it using PAGE_SHIFT. This preserves the existing 4MB transfer size across all PAGE_SIZE configurations while keeping GART window allocations within the available GART aperture. [1] https://lore.kernel.org/all/20260408100327.1372-3-pierre-eric.pelloux-prayer@amd.com/#t Closes: https://gitlab.freedesktop.org/drm/amd/-/work_items/5435 Fixes: 897ee11ec020 ("drm/amdgpu: create multiple clear/move ttm entities") Signed-off-by: Donet Tom <donettom@linux.ibm.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> (cherry picked from commit 27213b776a666d3030de5acc3cd75278197b0494) Cc: stable@vger.kernel.org
11 daysdrm/amdkfd: Use kvcalloc to allocate arraysDavid Francis
There were a few instances in kfd_chardev.c of kvzalloc being used to allocate memory for an array. Switch those to kvcalloc, which - is the standard way of allocating a zero-initialized array - does a check for the mul overflowing Signed-off-by: David Francis <David.Francis@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> (cherry picked from commit 60b048c93f7a3add39757ad65fe2bb6e58eeae23) Cc: stable@vger.kernel.org
11 daysdrm/amdgpu: add support for GC IP version 11.7.1Granthali Vinodkumar Dhandar
Initialize GC IP 11_7_1 Signed-off-by: Granthali Vinodkumar Dhandar <granthali.vinodkumardhandar@amd.com> Reviewed-by: Mario Limonciello <mario.limonciello@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> (cherry picked from commit a928d8d81ec5cdb5a8944d08136720811efad0f6)
11 daysdrm/amdgpu: add support for GC IP version 11.7.0Granthali Vinodkumar Dhandar
Initialize GC IP 11_7_0 Signed-off-by: Granthali Vinodkumar Dhandar <granthali.vinodkumardhandar@amd.com> Reviewed-by: Mario Limonciello <mario.limonciello@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> (cherry picked from commit cf591e67c095542a16475df293ec7bc9a118e4ee)
11 daysdrm/amdkfd: clamp v9 CRIU control stack checkpoint copy to BO sizeYongqiang Sun
CRIU checkpoint copies the MQD control stack using cp_hqd_cntl_stack_size from hardware without bounding it to the allocated BO region. If the HW field is larger than the queue's control stack allocation, memcpy reads past the BO into adjacent GTT memory and can leak kernel data to userspace. Store the page-aligned control stack BO size in mqd_manager and clamp checkpoint copies and reported checkpoint sizes to min(cp_hqd_cntl_stack_size, mm->ctl_stack_size). Apply the same bound for multi-XCC v9.4.3 checkpoint layout. Signed-off-by: Yongqiang Sun <Yongqiang.Sun@amd.com> Reviewed-by: David Francis <David.Francis@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> (cherry picked from commit 6c2abd0ec09e86c6323010673766f76050e28aa3) Cc: stable@vger.kernel.org
11 daysdrm/amdkfd: Guard m->cp_hqd_eop_control setting by q->eop_ring_buffer_sizeXiaogang Chen
To avoid wraparound if the value is 0. Signed-off-by: Xiaogang Chen <xiaogang.chen@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> (cherry picked from commit c0cae35661868af207077a4306bc42c7c972947c) Cc: stable@vger.kernel.org
2026-06-17drm/amdkfd: Use exclusive bounds for SVM split alignment checksGerhard Schwanzer
SVM ranges use inclusive page indices: prange->last is the last page in the range. The split-remap logic introduced by commit 448ee45353ef ("drm/amdkfd: Use huge page size to check split svm range alignment") uses ALIGN_DOWN(prange->last, 512) to determine whether the original range can contain a 2MB huge-page mapping. That aligns the last page itself down. Thus a range ending one page before the next 2MB boundary is classified as if the final 2MB block did not exist. When such a range is split inside that final block, the split head or tail can be left off the remap list even though it was derived from an original range that may have PMD mappings. Use prange->last + 1 as the exclusive upper bound when computing the original range's last 2MB-aligned boundary. Then use the actual split boundary for the head and tail alignment checks: tail->start for a tail split, and new_start for a head split. new_start is equivalent to head->last + 1 and directly names the exclusive end of the split head. Using head->last for the head-side check can both remap a head that ends exactly one page before a 2MB boundary and miss a head whose split boundary is one page after such a boundary. Philip Yang pointed out in the review of the original change that this condition should use head->last + 1 or new_start. Xiaogang Chen identified the inclusive-last cause and posted the candidate fix in the regression thread. With the culprit change active and the local revert not applied, the unchanged C/HSA reproducer completes 10/10 runs with this change on an RX 7600 XT. Fixes: 448ee45353ef ("drm/amdkfd: Use huge page size to check split svm range alignment") Closes: https://gitlab.freedesktop.org/drm/amd/-/work_items/4914 Link: https://lore.kernel.org/stable/IA1PR12MB85172F7FE9157C092EDA46A0E3112@IA1PR12MB8517.namprd12.prod.outlook.com/ Link: https://lore.kernel.org/all/32ce2b72-aa16-4202-9f99-92e3cd4408bc@amd.com/ Suggested-by: Xiaogang Chen <xiaogang.chen@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Gerhard Schwanzer <geschw@pm.me> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> (cherry picked from commit a60ea15807126b148a328051636977a33ad0e9bb) Cc: stable@vger.kernel.org
2026-06-17drm/amdkfd: Use memdup_array_user to copy data from/to user space at kfd ioctlsXiaogang Chen
Several kfd ioctls need transfer array data from/to user space. Kfd driver uses kmalloc_array with user provided size. That can oversize alloc or 32-bit wrap with hostile value. Replace it by memdup_array_user that does overflow checking and allocates through dedicated slab caches, also physical continuous as kmalloc. Signed-off-by: Xiaogang Chen <xiaogang.chen@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> (cherry picked from commit 4eca4742eb215951f9739ffe0122d179d545a7a4)
2026-06-17drm/amdkfd: check find_first_zero_bit before __set_bit on kfd->doorbell_bitmapXiaogang Chen
If inx from find_first_zero_bit is beyond range not need set doorbell_bitmap. Signed-off-by: Xiaogang Chen <xiaogang.chen@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> (cherry picked from commit 2664ce9143d174651a793d96a6a2326050c4f45a)
2026-06-17drm/amdkfd: Let driver decide buffer size at AMDKFD_IOC_GET_DMABUF_INFO ioctlXiaogang Chen
amdkfd driver needs allocate buffer to return bo metadata to user space. The buffer size is controlled by user currently. It is a potential security issue that hostile value (e.g. 2 GiB) lets any render-group user trigger order-MAX allocation/OOM in kernel context. This patch first finds bo metadata size. If the size is smaller than user provided value drive can safely allocate buffer in kernel space and copy to user space buffer. If not, driver will let user know, not allocate and copy. User will redo with new buffer in user space. This patch lets driver decide buffer allocation size to avoid potential hostile size from user space. Signed-off-by: Xiaogang Chen <xiaogang.chen@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> (cherry picked from commit f54ce9e8cbd3abe0eda3a285f54dc4f572fe589a)
2026-06-17drm/amdkfd: Fix NULL deref during sysfs teardownGeoffrey McRae
Move kfd_process_remove_sysfs() earlier in kfd_process_wq_release() so that all sysfs/procfs entries are removed before tearing down PDDs and dropping lead_thread. The per-process sysfs attributes are backed by struct kfd_process_device, and their show/store callbacks dereference PDD fields. Since sysfs removal waits for active callbacks to complete, removing these entries first closes a race where userspace reads sdma_* and stats_* files after PDD teardown. Previously this cleanup ran after kfd_process_destroy_pdds(), which resets p->n_pdds to 0. This meant kfd_process_remove_sysfs() could no longer walk the PDD array, so the per-PDD sysfs cleanup did not run as intended. This race caused NULL pointer dereferences observed in kfd_sdma_activity_worker and kfd_procfs_stats_show. Also harden kfd_process_remove_sysfs() against partially initialized or already-freed objects: - Check kobj_queues before removing PASID and deleting it - Guard kobj_stats and kobj_counters before use These checks prevent invalid dereferences during cleanup. Cc: Felix Kuehling <Felix.Kuehling@amd.com> Cc: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Geoffrey McRae <geoffrey.mcrae@amd.com> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> (cherry picked from commit 674c692702341fed321720b4b92036c5934fb485)
2026-06-17drm/amdkfd: fix list_del corruption in kfd_criu_resume_svmMario Limonciello
The cleanup tail of kfd_criu_resume_svm() walks svms->criu_svm_metadata_list and kfree()s each struct criu_svm_metadata without removing it from the list. The list head is left pointing at freed kmalloc-96 objects. A second AMDKFD_IOC_CRIU_OP from the same process re-enters: list_empty() reads the dangling ->next (use-after-free), the loop walks freed entries, and each is kfree()'d again (double-free). This is reachable by an unprivileged render-group user via /dev/kfd with no capabilities required. Add list_del() before the kfree() so the list is properly emptied. The list_for_each_entry_safe() iterator already caches the next pointer, so unlinking during the walk is safe. Fixes: 2a909ae71871 ("drm/amdkfd: CRIU resume shared virtual memory ranges") Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Mario Limonciello <mario.limonciello@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> (cherry picked from commit 6322d278a298e2c1430b9d2697743d3a04b788b1)
2026-06-17drm/amdkfd: Fix SMI event PID reporting for containersAndrew Martin
SMI events were reporting incorrect PIDs in containerized environments, causing test failures where container processes expected to see their namespace-local PIDs but instead received global host PIDs. The issue had two root causes: 1. Event functions were called from kernel context (page fault handlers, migration workers) where 'current' refers to the kernel worker thread, not the userspace GPU process that triggered the event. 2. PID conversion used task_tgid_vnr() which returns the PID in the caller's namespace (init namespace for kernel threads), not the task's own namespace. This patch updates the SMI event interface: - Change 8 event function signatures to accept task_struct pointer instead of pid_t, allowing proper namespace-aware PID conversion - Convert PIDs using task_tgid_nr_ns(task, task_active_pid_ns(task)) which returns the PID as the process sees it via getpid() - Update 10 call sites to pass p->lead_thread (the GPU process) instead of p->lead_thread->pid or current (kernel worker) This ensures SMI events report container-local PIDs, which is critical for containerized GPU workloads to correctly correlate events with their processes. Tested-by: Andrew Martin <andmarti@amd.com> Assisted-by: Claude:Sonnet 4-5 Signed-off-by: Andrew Martin <andrew.martin@amd.com> Reviewed-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> (cherry picked from commit 60271ec06e04ba5d69d68714f3abdf637d86c257)
2026-06-17drm/amdkfd: Properly acquire queue buffers in CRIU restoreDavid Francis
When kfd_queue_acquire_buffers() was split off from set_queue_properties_from_user(), set_queue_properties_from_criu() was missed. Thus, set_queue_properties_from_criu() is not filling out the buffer fields of queue_properties, which can come up when subsequent code expects them to be non-null. Add the proper call to kfd_queue_acquire_buffers(), and also use the right cast types in set_queue_properties_from_criu() (which were missed at the same time) Signed-off-by: David Francis <David.Francis@amd.com> Reviewed-by: Kent Russell <kent.russell@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com> (cherry picked from commit 88ed96abbbe27b70193544fbc1ee06448c274714)
2026-06-04drm/amdkfd: always resume_all after suspend_allAlex Deucher
Need to restore any good queues even if the suspend_all failed for some. Always run remove_queue as that will schedule a GPU reset is removing the queue fails. v2: move resume_all after remove Fixes: eb067d65c33e ("drm/amdkfd: Update BadOpcode Interrupt handling with MES") Reviewed-by: Amber Lin <Amber.Lin@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-06-04drm/amdkfd: Fix infinite loop parsing CRAT with zero subtype lengthYongqiang Sun
Malformed ACPI CRAT tables can advertise a zero or undersized subtype length. The parser then fails to advance the cursor and loops forever while the remaining image still looks large enough for a generic header. Validate sub_type_hdr->length on each iteration before parsing or advancing. Return -EINVAL and warn when length is zero or smaller than the generic subtype header. Signed-off-by: Yongqiang Sun <Yongqiang.Sun@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-06-04drm/amdkfd: fix sysfs topology prop length on buffer truncationYongqiang Sun
sysfs_show_gen_prop() accumulated snprintf()'s return value into the offset. snprintf() reports bytes that would have been written, not bytes actually written, so a truncated sysfs show could over-report its length. Use sysfs_emit_at(), which returns only the bytes written. Signed-off-by: Yongqiang Sun <Yongqiang.Sun@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-06-04drm/amdkfd: Unwind debug trap enable on copy_to_user failureYongqiang Sun
If kfd_dbg_trap_enable() fails while copying runtime_info to userspace, it had already activated the trap, set debug_trap_enabled, taken an extra process reference, and opened the debug event file. Return -EFAULT without unwinding that state, leaving inconsistent trap state and a refcount imbalance that could break later DISABLE/ENABLE. On copy_to_user failure, deactivate the trap and undo the rest of the enable setup before returning. Signed-off-by: Yongqiang Sun <Yongqiang.Sun@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-06-04drm/amdkfd: Add bounds check for AMDKFD_IOC_WAIT_EVENTSSunday Clement
The kfd_wait_on_events ioctl passes a user-supplied num_events parameter directly to alloc_event_waiters() which calls kcalloc() without validation. This allows unprivileged users with /dev/kfd access to trigger large kernel memory allocations, potentially causing memory exhaustion and denial of service via the OOM killer. Add a check to reject num_events values exceeding KFD_SIGNAL_EVENT_LIMIT (4096), which is the maximum number of events a single process can create. Signed-off-by: Sunday Clement <Sunday.Clement@amd.com> Reviewed-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-06-04drm/amdkfd: fix SMI event cross-process information leakYongqiang Sun
kfd_smi_ev_enabled() skips the suser privilege check when pid=0. PROCESS_START, PROCESS_END, and VMFAULT events are emitted with pid=0 while carrying another process's PID and command name, so any /dev/kfd user in the render group can monitor all GPU workloads. Pass the target process PID into kfd_smi_event_add() for these events so the existing per-client filter restricts delivery to the owning process or CAP_SYS_ADMIN subscribers. Signed-off-by: Yongqiang Sun <Yongqiang.Sun@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-06-03amd/amdkfd: Fix profiler lock init orderTvrtko Ursulin
A call chain at driver probe exists where profiler lock is used before it is initialized: [ 12.131440] kfd kfd: Allocated 3969056 bytes on gart [ 12.131561] kfd kfd: Total number of KFD nodes to be created: 1 [ 12.132691] ------------[ cut here ]------------ [ 12.132703] DEBUG_LOCKS_WARN_ON(lock->magic != lock) [ 12.132705] WARNING: kernel/locking/mutex.c:625 at __mutex_lock+0x616/0x1150, CPU#0: (udev-worker)/569 ... [ 12.133051] Call Trace: [ 12.133055] <TASK> [ 12.133059] ? mark_held_locks+0x40/0x70 [ 12.133068] ? init_mqd+0xe1/0x1b0 [amdgpu 5154987db73e842b9b4f761e2bd86e17c7ada65c] [ 12.133671] ? _raw_spin_unlock_irqrestore+0x4c/0x60 [ 12.133683] ? init_mqd+0xe1/0x1b0 [amdgpu 5154987db73e842b9b4f761e2bd86e17c7ada65c] [ 12.134235] init_mqd+0xe1/0x1b0 [amdgpu 5154987db73e842b9b4f761e2bd86e17c7ada65c] [ 12.134781] init_mqd_hiq+0x12/0x30 [amdgpu 5154987db73e842b9b4f761e2bd86e17c7ada65c] [ 12.135340] kq_initialize.constprop.0+0x309/0x400 [amdgpu 5154987db73e842b9b4f761e2bd86e17c7ada65c] [ 12.135898] kernel_queue_init+0x44/0x80 [amdgpu 5154987db73e842b9b4f761e2bd86e17c7ada65c] [ 12.136439] pm_init+0x70/0x100 [amdgpu 5154987db73e842b9b4f761e2bd86e17c7ada65c] [ 12.136984] start_cpsch+0x1dc/0x280 [amdgpu 5154987db73e842b9b4f761e2bd86e17c7ada65c] [ 12.137525] kgd2kfd_device_init+0x70f/0xd10 [amdgpu 5154987db73e842b9b4f761e2bd86e17c7ada65c] [ 12.138070] amdgpu_amdkfd_device_init+0x172/0x230 [amdgpu 5154987db73e842b9b4f761e2bd86e17c7ada65c] [ 12.138618] amdgpu_device_init+0x246a/0x2960 [amdgpu 5154987db73e842b9b4f761e2bd86e17c7ada65c] The human readable call chain is: kgd2kfd_device_init kfd_init_node kfd_resume node->dqm->ops.start Where start can be start_cpsch, which calls pm_init, etc, which ends up calling kq->mqd_mgr->init_mqd, which takes the profiler lock: init_mqd() { ... mutex_lock(&mm->dev->kfd->profiler_lock); ... Fix it by initializing the mutext at the top of kgd2kfd_device_init(). Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@igalia.com> Fixes: a789761de305 ("amd/amdkfd: Add kfd_ioctl_profiler to contain profiler kernel driver changes") Cc: Benjamin Welton <benjamin.welton@amd.com> Cc: Perry Yuan <perry.yuan@amd.com> Cc: Kent Russell <kent.russell@amd.com> Cc: Yifan Zhang <yifan1.zhang@amd.com> Cc: Alex Deucher <alexander.deucher@amd.com> Cc: Felix Kuehling <Felix.Kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-06-03drm/amdkfd: Fix buffer overflow in SDMA queue checkpoint/restore on GFX11Andrew Martin
The v11 MQD manager incorrectly assigned the CP-compute variants of checkpoint_mqd/restore_mqd for KFD_MQD_TYPE_SDMA queues. These functions use sizeof(struct v11_compute_mqd) (2048 bytes) instead of sizeof(struct v11_sdma_mqd) (512 bytes), causing a 1536-byte overflow. During CRIU checkpoint of an SDMA queue on Navi3x: - checkpoint_mqd() reads 2048 bytes from a 512-byte SDMA MQD buffer, leaking 1536 bytes of adjacent GTT memory to userspace During CRIU restore: - restore_mqd() writes 2048 bytes into a 512-byte SDMA MQD buffer, corrupting 1536 bytes of adjacent GTT memory (often the ring buffer or neighboring MQDs) This is a copy-paste regression unique to v11. All other ASIC backends (cik, vi, v9, v10, v12) correctly use the SDMA-specific variants. Add checkpoint_mqd_sdma() and restore_mqd_sdma() functions that properly handle the smaller v11_sdma_mqd structure, matching the pattern used in other MQD managers. Fixes: cc009e613de6 ("drm/amdkfd: Add KFD support for soc21 v3") Assisted-by: Claude:Sonnet 4-5 Signed-off-by: Andrew Martin <andrew.martin@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-06-03drm/amdkfd: fix NULL dereference in get_queue_ids()Muhammad Bilal
When usr_queue_id_array is NULL and num_queues is non-zero, get_queue_ids() returns NULL. The callers check only IS_ERR() on the return value; since IS_ERR(NULL) == false the check passes, and suspend_queues() calls q_array_invalidate() which immediately dereferences NULL while iterating num_queues times. Userspace can trigger this via kfd_ioctl_set_debug_trap() by supplying num_queues > 0 with a zero queue_array_ptr, causing a kernel panic. A NULL usr_queue_id_array with num_queues == 0 is a legitimate no-op (q_array_invalidate never executes, and resume_queues already guards all queue_ids dereferences behind a NULL check). Return ERR_PTR(-EINVAL) only when num_queues is non-zero and the pointer is absent; both callers already propagate IS_ERR() returns correctly to userspace. Fixes: a70a93fa568b ("drm/amdkfd: add debug suspend and resume process queues operation") Signed-off-by: Muhammad Bilal <meatuni001@gmail.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-06-03drm/amdgpu: add support for SDMA IP version 6.4.0Pratik Vishwakarma
This initializes SDMA IP version 6.4.0. Signed-off-by: Pratik Vishwakarma <Pratik.Vishwakarma@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-06-03drm/amdgpu: Add support for GC IP version 11.5.6Pratik Vishwakarma
Initialize GC IP 11_5_6 Signed-off-by: Pratik Vishwakarma <Pratik.Vishwakarma@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-06-03drm/amdkfd: fix UAF race in destroy_queue_cpschAlysa Liu
wait_on_destroy_queue() drops locks to wait for queue resume, allowing a concurrent destroy to free the queue. Use is_being_destroyed flag to serialize destruction. Reviewed-by: Amir Shetaia <Amir.Shetaia@amd.com> Signed-off-by: Alysa Liu <Alysa.Liu@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-05-27drm/amdkfd: fix a vulnerability of integer overflow in kfd debuggerEric Huang
get_queue_ids() computes array_size = num_queues * sizeof(uint32_t), which could overflow on 32-bit size_t build. using array_size() instead, it saturates to SIZE_MAX on overflow. Signed-off-by: Eric Huang <jinhuieric.huang@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-05-27drm/amdkfd: Check for pdd drm file first in CRIU restore pathDavid Francis
CRIU restore ioctls are meant to be called by CRIU with no existing drm file. There's an error path for if the drm file unexpectedly exists. It was positioned so it was missing a fput(drm_file). Do that check earlier, as soon as we have the pdd. Signed-off-by: David Francis <David.Francis@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-05-27drm/amdkfd: fix NULL pointer bug in svm_range_set_attrEric Huang
The process_info could be NULL if user doesn't call kfd_ioctl_acquire_vm before calling kfd_ioctl_svm. Signed-off-by: Eric Huang <jinhuieric.huang@amd.com> Reviewed-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-05-27drm/amdkfd: Fix UML build guards for x86_64-only codeAlex Hung
cpu_data().topo.apicid and kfd_fill_iolink_info_for_cpu() rely on x86-specific structs not present on UML. The kfd_topology.c and kfd_crat.c were guarded by CONFIG_X86_64 alone, causing build failures when CONFIG_DRM_AMDGPU is selected on UML. Update guards to '#if defined(CONFIG_X86_64) && !defined(CONFIG_UML)' to ensure x86_64-only paths are excluded on UML builds. Fixes: af3f2f5db265 ("drm/amdgpu: Remove UML build exclusion from Kconfig") Reported-by: kernel test robot <lkp@intel.com> Closes: https://lore.kernel.org/oe-kbuild-all/202605140506.TI8zPIBG-lkp@intel.com/ Cc: Harry Wentland <harry.wentland@amd.com> Assisted-by: Copilot:Claude-Sonnet-4.6 Reviewed-by: Felix Kuehling <felix.kuehling@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Hung <alex.hung@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-05-19drm/amdgpu: unmap all user mappings of framebuffer and doorbell before mode1 ↵Yifan Zhang
reset During Mode 1 reset, the ASIC undergoes a reset cycle and becomes temporarily inaccessible via PCIe. Any attempt to access framebuffer or MMIO registers during this window can result in uncompleted PCIe transactions, leading to NMI panics or system hangs. To prevent this, Unmap all of the applications mappings of the framebuffer and doorbell BARs before mode1 reset. Also prevent new mappings from coming in during the reset process. v2: remove inode in kfd_dev (Christian) v3: correct unmap offset (Felix), remove prevent new mappings part to avoid deadlock (Christian) Reviewed-by: Felix Kuehling <felix.kuehling@amd.com> Signed-off-by: Yifan Zhang <yifan1.zhang@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-05-18drm/amdkfd: Check bounds for allocate_sdma_queue restore_sdma_idDavid Francis
allocate_sdma_queue has an option where the sdma queue id can be specified (used by CRIU). We weren't bounds-checking that value. Confirm it's less than the maximum number of queues. Signed-off-by: David Francis <David.Francis@amd.com> Reviewed-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-05-18drm/amdkfd: Check bounds on allocate_doorbellDavid Francis
allocated_doorbell has an option to set the doorbell id to a specific value (used by CRIU). This value was not bounds checked. Check to confirm it's less than KFD_MAX_NUM_OF_QUEUES_PER_PROCESS. Signed-off-by: David Francis <David.Francis@amd.com> Reviewed-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-05-18drm/amdkfd: Fix OOB memory exposure in get_wave_state()Sunday Clement
The get_wave_state() function for v9 trusts cp_hqd_cntl_stack_size and cp_hqd_cntl_stack_offset values read directly from the MQD, which are written by GPU microcode and fully attacker-controlled on the CRIU-restore path (via AMDKFD_IOC_RESTORE_PROCESS with H3). this leads to an unbounded copy_to_user() that can leak adjacent GTT/kernel memory. If offset > size, integer underflow produces a ~4 GiB read length, if size is set to 1 MiB against a 4 KiB allocation, we leak 1 MiB of adjacent kernel memory (other queues' MQDs, ring buffers, KASLR pointers). Fix by clamping both cp_hqd_cntl_stack_size to the actual allocated buffer size (q->ctl_stack_size) and cp_hqd_cntl_stack_offset to the clamped size before performing arithmetic and copy_to_user(). This ensures we never read beyond the allocated kernel BO regardless of attacker-supplied MQD field values. Signed-off-by: Sunday Clement <Sunday.Clement@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-05-18drm/amdkfd: Enable SDMA queue reset on gfx v12.1Amber Lin
After suspend/resume sdma_gang is supported on MES 12.1, SDMA queue reset is supported too. Signed-off-by: Amber Lin <Amber.Lin@amd.com> Reviewed-by: Michael Chen<michael.chen@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-05-11drm/amdkfd: fix unhalt_cpsch warning during module unloadPerry Yuan
Downgrade unhalt_cpsch warning to dev_dbg when sched is already stopped Signed-off-by: Perry Yuan <perry.yuan@amd.com> Reviewed-by: Yifan Zhang <yifan1.zhang@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-05-11drm/amdgpu: Move KFD sched stop/start into PTL control pathPerry Yuan
Move amdgpu_amdkfd_stop/start_sched calls from kfd_ptl_control() into amdgpu_ptl_perf_monitor_ctrl() so all PTL callers (KFD ioctl, sysfs, GFX init) get consistent scheduling management. Add amdgpu_amdkfd_stop/start_sched_all() wrappers to stop and restart KFD scheduling on all nodes without assuming node ID ordering. v3: * call start/stop for PTL Set Only v2: * move the stop/start sched function to amdgpu_ptl_perf_monitor_ctrl(Lijo) * add wrapper amdgpu_amdkfd_stop_sched_all and amdgpu_amdkfd_start_sched_all (Lijo) Signed-off-by: Perry Yuan <perry.yuan@amd.com> Reviewed-by: Yifan Zhang <yifan1.zhang@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-05-11drm/amdgpu: add SPI idle check for GC 9.4.4 in gfx_v9_4_3_is_idle()Perry Yuan
GC 9.4.4 uses SPI busy status for idle detection instead of GRBM GUI_ACTIVE. Add version check to use SPI_BUSY for 9.4.4 while keeping GRBM_STATUS GUI_ACTIVE check for other GC versions. v2: move this check into amdgpu_ptl_perf_monitor_ctrl(Lijo) Signed-off-by: Perry Yuan <perry.yuan@amd.com> Reviewed-by: Yifan Zhang <yifan1.zhang@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-05-11drm/amdgpu: Wait for GFX idle before PTL state transitionPerry Yuan
Ensure GFX engine is idle before switching PTL state to prevent register access violations and CP hang. This addresses the race condition where in-flight GPU commands could conflict with PTL state changes. Signed-off-by: Perry Yuan <perry.yuan@amd.com> Reviewed-by: Yifan Zhang <yifan1.zhang@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-05-11drm/amdgpu: Track PTL disable requests by sourcePerry Yuan
Use a bitmap to track PTL disable requests from sysfs and profiler. PTL is only re-enabled once all sources have released their disable requests, avoiding premature enablement. Signed-off-by: Perry Yuan <perry.yuan@amd.com> Reviewed-by: Yifan Zhang <yifan1.zhang@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-05-11drm/amdkfd: suspend scheduler during PTL re-enablingPerry Yuan
Stop the scheduler before releasing the PTL disable request to ensure the GPU is quiescent during the PTL state transition. This prevents potential queue preemption failures and GPU resets caused by modifying PTL state while waves are executing v1->v2: only stop/start the scheduler when the PTL state actually needs to transition(Yifan) Signed-off-by: Perry Yuan <perry.yuan@amd.com> Reviewed-by: Yifan Zhang <yifan1.zhang@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-05-11drm/amdkfd: Add PTL control IOCTL Option and unify refcount logicPerry Yuan
Introduce a new IOCTL option to allow userspace explicit control over the Peak Tops Limiter (PTL) state for profiling Link: https://github.com/ROCm/rocm-systems/tree/develop/projects/rocprofiler-sdk Signed-off-by: Perry Yuan <perry.yuan@amd.com> Reviewed-by: Yifan Zhang <yifan1.zhang@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-05-11drm/amdgpu: add psp interfaces for peak tops limiter driverPerry Yuan
Introduce a Peak Tops Limiter (PTL) driver that dynamically caps engine frequency to ensure delivered TOPS never exceeds a defined TOPS_limit. This initial implementation provides core data structures and kernel-space interfaces (set/get, enable/disable) to manage PTL state. PTL performs a firmware handshake to initialize its state and update predefined format types. It supports updating these format types at runtime while user-space tools automatically switch PTL state, and also allows explicitly switching PTL state via newly added commands. Signed-off-by: Perry Yuan <perry.yuan@amd.com> Reviewed-by: Lijo Lazar <lijo.lazar@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-05-11amd/amdkfd: Add kfd_ioctl_profiler to contain profiler kernel driver changesBenjamin Welton
kfd_ioctl_profiler takes a similar approach to that of kfd_ioctl_dbg_trap (which contains debugger related IOCTL services) where kfd_ioctl_profiler will contain all profiler related IOCTL services. The IOCTL is designed to be expanded as needed to support additional profiler functionality. The current functionality of the IOCTL is to allow for profilers which need PMC counters from GPU devices to both signal to other profilers that may be on the system that the device has active PMC profiling taking place on it (multiple PMC profilers on the same device can result in corrupted counter data) and to setup the device to allow for the collection of SQ PMC data on all queues on the device. For PMC data for the SQ block (such as SQ_WAVES) to be available to a profiler, mmPERFCOUNT_ENABLE must be set on the queues. When profiling a single process, the profiler can inject PM4 packets into each queue to turn on PERFCOUNT_ENABLE. When profiling system wide, the profiler does not have this option and must have a way to turn on profiling for queues in which it cannot inject packets into directly. Accomplishing this requires a few steps: 1. Checking if the user has the necessary permissions to profile system wide on the device. This check uses the same check that linux perf uses to determine if a user has the necessary permissions to profile at this scope (primarily if the process has CAP_SYS_PERFMON or is root). 2. Locking the device for profiling. This is done by setting a lock bit on the device struct and storing the process that locked the device. 3. Iterating all queues on the device and issuing an MQD Update to enable perfcounting on the queues. 4. Actions to cleanup if the process exits or releases the lock. The IOCTL also contains a link to the existing PC Sampling IOCTL as well. This is per a suggestion that we should potentially remove the PC Sampling IOCTL to have it be a part of the profiler IOCTL. This is a future change. In addition, we do expect to expand the profiler IOCTL to include additional profiler functionality in the future (which necessitates the use of a version number). v2: sqaush in proper IOCTL number Proposed userpace support: https://github.com/ROCm/rocm-systems/commit/40abc95a6463a61bb318a67efd6d9cc3e5ee8839 Signed-off-by: Benjamin Welton <benjamin.welton@amd.com> Signed-off-by: Perry Yuan <perry.yuan@amd.com> Acked-by: Kent Russell <kent.russell@amd.com> Reviewed-by: Yifan Zhang <yifan1.zhang@amd.com> Acked-by: Alex Deucher <alexander.deucher@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
2026-05-05drm/amdkfd: remove obsolete codes for kfd_mmapEric Huang
kfd_reserved_mem_mmap is only for mapping CWSR on APU in IOMMUv2 mode, which is no longer supported, and qpd->cwsr_base has been set before calling kfd_process_init_cwsr_apu, which is the only caller for KFD_MMAP_TYPE_RESERVED_MEM, so kfd_process_init_cwsr_apu is not functional anymore, remove them together. On the other hand, it will fix a vulnerability issue to abuse KFD_MMAP_TYPE_RESERVED_MEM of kfd_mmap from userspace. Signed-off-by: Eric Huang <jinhuieric.huang@amd.com> Reviewed-by: Felix Kuehling <felix.kuehling@amd.com> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>