diff options
| author | Linus Torvalds <torvalds@linux-foundation.org> | 2026-10-23 18:29:02 -1000 |
|---|---|---|
| committer | Linus Torvalds <torvalds@linux-foundation.org> | 2026-10-23 18:29:02 -1000 |
| commit | 5b0764a53235d3c77e1ed674e836e1b3fe1a4b1f (patch) | |
| tree | 61666e82efa4901ddb00101ec9d36bab2d0b6662 /Documentation/gpu/nova/core | |
| parent | 17bb0e2f6c8043f70a693a5f3797471cbed354c8 (diff) | |
| parent | fad06cfaf385787b9ffa2f73f049d42dbe1de57f (diff) | |
| download | lwn-master.tar.gz lwn-master.zip | |
Pull drm updates from Dave Airlie:
"Highlights:
- core: a lot of atomic refactoring in progress, panic got split into
a core/helpers model
- amdgpu: enables HDMI 2.1 FRL and other good things finally, UAlink
over Ethernet support
- xe: has a bunch of CRI support, ras error for bad vram, memory
offlining support.
- msm: enables a bunch more GPUs.
Otherwise it's a the usual mountain of stuff all over the place.
Details:
rust:
- I/O tree shared with driver-core
- DMA tree shared with driver-core
- allow aux registration data closures to return covariant sub-fields
- Implement Send and Sync for MapleTree
- add pci resource flags and IORESOURCE_MEM_64
drm/rust:
- add dma_fence abstraction
- init DRM registrations and their data in place
- drop unused flags argument to registration
vgaarb:
- rework API to pass a private ptr instead of pci device
sound:
- adapt to omapdrm changes
gem:
- add callback for handle count going to 0
gpusvm:
- share one HMM fault and keep single mappings inline
pagemap:
- Prevent CPU stalls during unbounded pagemap teardown
dma-fence:
- fix docs
vga_switcheroo:
- disconnect from fbcon
- update fbcon from within fbdev emulation
atomic:
- move DRM from state reset callbacks to atomic_create_state
- remove drm_simple_encoder_init
panic:
- split into core and helpers
buddy:
- improve dirty tracking
clients:
- improve vmap handling
- fbdev: Move helpers from drm_kms_helper.ko to drm_client_lib.ko
display:
- export HDMI SCDC status data via debugfs
- rework SCDC for better HDMI 2.0 support
- dp/mst: improve DPCD compliance
edid:
- parse AMD VSDB
- parse ALLM/VRR capabilities
- support detecting panel size
log:
- add drm_warn_ratelimited
sched:
- cleanup interface
- add missing locking
- fix run-queue cleanup
- fix locking
sysfb:
- remove obsolete quirk for Lenovo D330
- fix panel orientation for non-native display modes
- drop drm_class_device_register
colorop:
- provide DRM_COLOR_OP_FIXED_MATRIX
bridge:
- analogix: Support eDP lane mapping
- ti-sn65dsi83: Support LVDS Channel B on SN65DSI84
- Select DRM_PANEL_BRIDGE for various drivers that require it
- add chrontel ch7218a support
panel:
- panel-edp: Support MNE007QS3-F, TM140VDXP15, and KD116N36-30NB-A001
- samsung-s6d16d0: Use mipi_dsi_*_multi() functions
- support Ilitek ILI7836A OLED plus DT bindings
- support Novatek NT36532 plus DT bindings
- convert several drivers to managed cleanup
- fix Kconfig selections
- edp:
- Add support for BOE NV140FHM-T0A
- Support CSOT MNE007QB1-1
- NV140WU2-M70
- ebbg-ft8719: Fix MODULE_LICENSE() string; Improve initialization
- samsung: Share test-key helpers
- ilitek-ili9882t: Support Ilitek IL79600A plus DT bindings
- simple: Support Truly TFT320240-229-E plus DT bindings; Support BOE
DV215FHM-R01 plus DT bindings
- Refactor struct drm_panel to contain struct drm_bridge plus fallout
in various panel drivers
- Add support for BOE Novatek NT51021, Retroid Pocket 6, Raydium
RM69220 DDIC panels, Sharp LQ138P1JX61 eDP, BOE NV140FHD-N43, AUO
B140HAN04.0, AUO 0x0db3, panels.
amdgpu:
- use AMD VSDB for freesync
- implement YUV-to-RGB with fixed-matrix colorop
- enable FRL for HDMI
- HDMI 2.1 freesync support
- HDMI 2.1 ALLM support
- HDMI 2.1 VRR support
- UALoE (UALink over Ethernet) support
- add syncobj_points to userq signal ioctl
- drop unused kunit tests
- NBIO/GC/MMHUB/SMU updates/fixes
- GPU spatial partitioning updates
- display kunit fixes
- userq fixes
- RGB quantization fixes
- DC RMCM refactor
- DP MST fixes
- USB4 fixes
- add new trace events
- Rework display instance handling
- stack frame size fixes for various compilers/configs
- VCN 4/5 reset fixes
- freesync updates
- cursor refactor
- Fix direct scanout alpha on some DRM_FORMATs
- Expand Mac reserved memory workaround
- Make runtime pm more fine grained
amdkfd:
- updates and fixes
- GC 10 CU occupancy support
radeon:
- eDP fix for DCE3.2 iMacs
xe:
- add drm_ras netlink error event support
- add memory page offlining support
- introduce error threshold to drm_ras
- fix stale GTT mappings on unmap
- reject page faults for non-page fault mode scratch VM
- enable CRI package and VRAM temperatures
- introduce cold reset recovery
- handle CRI+ power interval encoding
- VF migration for multi-queue
- handle exec queue teardown after hot unplug
- alerts and controller enabling modifications
- use IBCv3 on PTL
- VRAM health check
- report GT TDR and VM rebind faults using SIGID
- add multi_queue_active_lrca debugfs
- shrinker: Runtime PM reference for non-system memory
- guc: Add simple KLV decoder helpers
- debugfs: Expose per-engine idle status
- Keep walking on SVM eviction failure
- Nuke PTs only after unlinking contested VMAs
- limit sg segment size to PAGE_SIZE on Xen PV
- keep VF LMEM bar size low if no VFs
- CPU binds and ULLS on migration queue
- New PTL PCI ID
- Enable PXP for NVL-S
- Do not dma_fence_put zero fences in err_out path
- Refresh TTM LRU on SVM prefetch
- export device UID in sysfs
i915:
- rename GEN6_PCODE_DATA to match spec
- fix incorrect RCU teardown order
- fix potential UAF in command parsing
- avoid WARN splat during RPS boosting race
- use simpler min functions
- Fixes to make i915 work well with PREEMPT_RT
i915/display:
- Enable CMRR (Content Match Refresh Rate) in fixed-RR VRR path
- Switch DG2 MPLL to DPLL framework
- DP-HDMI PCON DSC fixes
- limit invalid joiner combinations
- CDCLK related fixes and refactor around PCODE integration
- Gate UHBR SST SDP splitting on sink caps
- add HDMI TMDS audio bandwidth check
- vswing/preemph override
- enable AS SDP Skip frames
- Handle negative configuration index on DP link caps for robustness
- Preparation for enabling SDR plane color pipeline
- An eDP quirk for HP Pavilion Plus
- Introduce PME (Power Management Event) support on display hotplug
- Handle VSC SDP revision 7 in unpack
msm:
- CI:
- Dropped sc7180 trogdor lazor from the CI
- Core:
- Fixed msm_ioremap for resources without a name
- Changed msm.separate_gpu_kms to default to auto selection
- GPU:
- Enable CLX (Current Limit eXtension) for gen8 to avoid exceeding
limits on CX/GX rails
- New GPUs: a830 (Pakala), a845 (Maili), a850 (Hawi), x2-85
(Mahua)
- Fix VBIF halt mask on a306, which was preventing runpm suspend
- Enable THP support for GEM buffers
- Add modparam to disable waiting on fences in shrinker
- Drop use of drm_prime_sg_to_page_array()
- a5xx: don't read the GPMU registers on GPUs without a GPMU
- a2xx/a3xx/a5xx: hangcheck progress detection
- a6xx: gmuwrapper runpm fix
- Calculate reported GMEM size based on active slices
- Split a6xx/a7xx/a8xx catalog
- DP:
- Picked up some preparation patches for the MST support
- Fixed reported status of the link training
- DPU:
- Fixed CRTC bandwidth computation
- Fixed DSC programming
- Fixed encoder disablement path
- DSI:
- Fixed VM leak in error path
- Indention fix
- MDP5:
- Added MSM8952 configuration
- Dropped incorrect cursor pipe on MSM8976/56
- HDMI:
- Small fixes
nouveau:
- improve runtime PM on R570 GSP firmware
- various fixes throughout the driver
- dispnv50: support 2.147 GHz pixel clock in GB20x
nova:
- add initial uapi concept
- add initial PRAMIN access support
- add GpuMm centralized memory manager
- add GPU VM and BAR1 support
- add GPU interrupts and GSP events support
- GSP messaging fixes
tyr:
- use bitfield!
- zero unsupported and newly added GPU info fields
- limit doorbell array to 17 entries
exynos:
- FIMD fixes
- Fix clock leak on resume error path
- Fix Sparse warnings by adding missing __iomem annotation
- VIDI cleanup
- Replace kmalloc() and copy_from_user() with memdup_user()
amdxdna:
- fix BO handling
- Support AIE2 rev 9
- Fix UAF access in scheduler fences
- Update workqueue interface
- expose busy time via fdinfo
ast:
- support 256-byte EDID
hyperv_drm:
- remove support for pre win10 hosts
komeda:
- fix GLB_CORE_ID usage
omap:
- report HDMI hotplug events to ASoC HDMI codec
panthor:
- provide gpu_cache_flush tracepoint
- improve dma_fence signalling latency
- improve locking
- export scheduling information via debugfs
- Validate user-space input against firmware limits
qaic:
- reject BOs that exceed maximum page count
- add missing include statements
verisilicon:
- fix hardware cursor offsets
vkms:
- implement YUV-to-RGB with fixed-matrix colorop
gma500:
- avoid unsupported ARGB formats in primary plane
hibmc:
- handle vblank interrupts correctly
ivpu:
- add DRM_IVPU_CAP_CMDQ_SET_PRIORITY
- support command-queue priority ioctl
- Simplify NULL-pointer handling in cleanup code
arcpgu:
- replace simple-display pipeline with regular atomic helpers
aspeed:
- replace simple-display pipeline with regular atomic helpers
gm12u320:
- replace simple-display pipeline with regular atomic helpers
imagination:
- various improvement to firmware interaction
- add job to pvr_job_submit_fw trace event
mali-dp:
- fix runtime PM leak
mcde:
- replace simple-display pipeline with regular atomic helpers
p111:
- replace simple-display pipeline with regular atomic helpers
- drop unsupported alpha formats from primary plane
rcar-du:
- support Renesas R-Car V4H DSC
- dsi: Support larger transfers with AXI memory
- Support RZ/G3L SoC
repaper:
- replace simple-display pipeline with regular atomic helpers
rockchip:
- dsi: improve PHY timings
ssd130x:
- support Solomon SSD1351 OLED plus DT bindings
- ssd133x: upgrade color format to RGB565
- ssd132x: Set correct hardware window when clearing screen
tve200:
- replace simple-display pipeline with regular atomic helpers
xen:
- replace simple-display pipeline with regular atomic helpers
v3d:
- improve job handling under perfmon
vc4:
- reliably reset V3D
- drop brcm,vc4-v3d compatible
virtio:
- fix damage handling
- Support ARGB8888 for virtio primary planes.
host1x:
- Support partitioning
imx:
- Replace simple-display with regular atomic helpers
mxsfb:
- Support bus formats plus DT bindings
tegra:
- improve syncobj handling on submit
panfrost:
- add Mali-G68 support
- improve GPU reset handling
rockchip:
- add HDMI 2.0 support to dw-hdmi-qp
- improve HPD handling"
* tag 'drm-next-2026-10-21' of https://gitlab.freedesktop.org/drm/kernel: (2324 commits)
drm/adp: Switch to drm_atomic_helper_connector_create_state
drm/malidp: Convert to atomic_create_state
drm/vc4: Switch to drm_atomic_helper_connector_create_state
drm/sysfb: Switch to drm_atomic_helper_connector_create_state
drm/solomon: ssd130x: Switch to drm_atomic_helper_connector_create_state
drm/imx: ipuv3: Switch to drm_atomic_helper_connector_create_state
drm/hyperv: Switch to drm_atomic_helper_connector_create_state
drm/bridge: megachips: Switch to drm_atomic_helper_connector_create_state
drm/bridge: lontium-lt8912b: Switch to drm_atomic_helper_connector_create_state
drm/arm: komeda: Switch to drm_atomic_helper_connector_create_state
drm/appletbdrm: Fix incorrect error message on encoder attach failure
drm/sun4i: hdmi: Convert to drmm_connector_hdmi_init()
drm/rockchip: rk3066_hdmi: Advertise HDMI 1.4 capabilities
drm/bridge: lontium-lt9611: Advertise HDMI 1.4 capabilities
drm/bridge: ite-it66121: Advertise HDMI 1.2 capabilities
drm/bridge: ite-it6263: Advertise HDMI 1.3 capabilities
drm/bridge: ite-it6263: Drop redundant .mode_valid hook
drm/bridge: inno-hdmi: Advertise HDMI 1.2 capabilities
drm/bridge: adv7511: Advertise HDMI 1.2 capabilities
drm/sched: Document entity->stopped is protected by entity->lock
...
Diffstat (limited to 'Documentation/gpu/nova/core')
| -rw-r--r-- | Documentation/gpu/nova/core/fsp.rst | 2 | ||||
| -rw-r--r-- | Documentation/gpu/nova/core/interrupts.rst | 603 | ||||
| -rw-r--r-- | Documentation/gpu/nova/core/pramin.rst | 128 | ||||
| -rw-r--r-- | Documentation/gpu/nova/core/todo.rst | 2 |
4 files changed, 734 insertions, 1 deletions
diff --git a/Documentation/gpu/nova/core/fsp.rst b/Documentation/gpu/nova/core/fsp.rst index 52d618d22bb8..60c647181da0 100644 --- a/Documentation/gpu/nova/core/fsp.rst +++ b/Documentation/gpu/nova/core/fsp.rst @@ -106,6 +106,8 @@ All FSP messages share a common header format consisting of two 32-bit words: - Bit 30: EOM (End of Message) - Bits 29:28: Packet sequence number - Bits 23:16: Source Endpoint ID +- Bits 15:8: Destination Endpoint ID +- Bits 3:0: MCTP header version (1) **NVDM header** (NVIDIA Vendor Defined Message): diff --git a/Documentation/gpu/nova/core/interrupts.rst b/Documentation/gpu/nova/core/interrupts.rst new file mode 100644 index 000000000000..e42286915711 --- /dev/null +++ b/Documentation/gpu/nova/core/interrupts.rst @@ -0,0 +1,603 @@ +.. SPDX-License-Identifier: GPL-2.0 +.. SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + +============================================= +GPU interrupt handling: GIN and the GSP event +============================================= + +This document describes how nova-core receives interrupts from the GPU on Turing +and later chipsets. It covers the GPU Interrupt and Notification unit (GIN), +which is the GPU's interrupt controller, and the GSP event, the interrupt that +nova-core services in normal operation. + +Throughout, *CPU* means the CPU and the nova-core driver running on it. The GPU +also has on-chip processors that run their own firmware and receive their own +interrupts. The GSP (GPU System Processor) is one of them. + +Register names are the names from the GPU hardware reference headers. The +pre-Hopper headers call the controller ``NV_CTRL`` and the Hopper-plus headers +call it ``NV_GIN``. This document calls it GIN throughout, because the tree that +nova-core services is the same on every supported chipset. OpenRM, NVIDIA's +open-source GPU kernel driver, is cited wherever nova-core follows it. + +Terminology +=========== + +The three levels of the controller, innermost first: + +leaf + One ``LEAF`` register. Each of its 32 bits is the pending bit of one + interrupt source. A Turing, Ampere, or Ada tree has 8 leaves. A Hopper or + Blackwell tree has 16. + +subtree + Two consecutive leaves, summarized by one bit of ``TOP``. A subtree is the + unit of enabling at ``TOP``, and under MSI-X it is the unit of delivery: + every interrupt from one subtree arrives on one MSI-X entry. + +tree + One ``TOP`` register and the leaves under it. Every PCIe function has its + own tree, and nova-core services the CPU tree of one function. + +The hardware headers, OpenRM, and the Linux PCI API all use the word "vector", +each for a different number. This document gives each one its own name, and a +bare "vector" always means a GIN vector. + +GIN vector + The GPU-internal interrupt source number. It addresses one bit of one leaf. + A 16-leaf tree holds vectors 0 through 511, and an 8-leaf tree holds 0 + through 255. The CPU doorbell is vector 129 and the GSP event is vector + 155. + +MSI-X entry + An index into the device's MSI-X table. One entry serves one subtree. + +PCI vector + One of the interrupts that ``pci_alloc_irq_vectors()`` allocates: an MSI-X + entry, or the single MSI message. + +Linux IRQ number + What ``request_irq()`` takes, obtained from ``pci_irq_vector()`` for a PCI + vector. Linux's ``struct msix_entry`` calls this number ``.vector`` as + well. + +The remaining terms, each named for the register or the specification that +defines it: + +enable, disable a vector + Writes to ``LEAF_EN_SET`` and ``LEAF_EN_CLEAR``. + +enable, disable a subtree + Writes to ``TOP_EN_SET`` and ``TOP_EN_CLEAR``. + +serviced subtree + A subtree that nova-core enables and has a handler for. + +rearm + Restoring PCI interrupt delivery after servicing an interrupt. See + "Rearming PCI interrupt delivery". + +mask + Reserved for the two places where hardware and the PCI specification use + the word: the MSI-X per-entry Vector Control mask bit, which Linux + controls, and the falcon interrupt masks. It never names a GIN enable. + +latched, pending + Two names for one state, a set ``LEAF`` bit. The vector's source sets the + bit whether or not the vector is enabled. + +clear a vector + Write a 1 to the vector's bit in ``LEAF``. OpenRM calls the same operation + ``intrClearLeafVector_HAL``. + +pending bits + The plain 32-bit value read from a ``LEAF`` register. + +notification + An interrupt whose only content is that something happened, such as a + posted message. Servicing a notification means reading what it announces. + The unit that raised it needs no attention. The GSP event is one. + +unit + Any block that raises an interrupt. "Engine" is reserved for the blocks + that do user work: GR, CE, NVDEC, and the like. + +falcon + One of the GPU's microcontrollers (see + Documentation/gpu/nova/core/falcon.rst). The GSP runs on the RISC-V core + inside its falcon. A falcon latches each of its interrupt causes and routes + it either to the host, meaning the CPU, or to its own core. + +The GIN controller +================== + +A GPU has many interrupt sources: the GSP, the copy engines, the graphics +engine, video decode and encode, the MMU fault path, timers, and others. GIN +records which of them are pending and raises the PCI interrupt to the CPU. + +Trees +----- + +GIN keeps one tree for each destination it can deliver an interrupt to. The CPU +has one tree per PCIe function, so the physical function and each virtual +function have their own. The GSP has a tree, and so do the other on-chip +processors that receive interrupts. Every tree has the same two-level layout, +and a function reaches its own tree through the per-function register aperture. + +nova-core services the CPU tree of one function. A virtual function's tree +belongs to that function's driver, and a processor's tree belongs to the +firmware running on that processor. + +The two-level tree +------------------ + +A tree is a set of ``LEAF`` registers and one ``TOP`` register. + +* ``LEAF(i)`` is a 32-bit register that holds the pending bits of vectors + ``32i`` through ``32i + 31``. A set bit is a pending vector. +* ``TOP`` is a 32-bit read-only register. Bit ``N`` summarizes subtree ``N``, + which is ``LEAF(2N)`` and ``LEAF(2N + 1)``. The bit is set when an enabled + vector is pending in either leaf. + +A tree with L leaves has L / 2 subtrees and uses TOP bits 0 through L / 2 - 1. +The other TOP bits read 0. The leaves and subtrees that a chipset has are its +implemented leaves and subtrees, and "Per-architecture differences" gives the +counts. For an 8-leaf tree:: + + TOP bit 0 -> subtree 0 -> LEAF(0), LEAF(1) vectors 0..63 + TOP bit 1 -> subtree 1 -> LEAF(2), LEAF(3) vectors 64..127 + TOP bit 2 -> subtree 2 -> LEAF(4), LEAF(5) vectors 128..191 + TOP bit 3 -> subtree 3 -> LEAF(6), LEAF(7) vectors 192..255 + + LEAF(4), one bit per vector, holds vectors 128..159: + + bit 1 = vector 129 (CPU doorbell) + bit 27 = vector 155 (GSP event) + +A vector's number fixes its place in the tree:: + + leaf = vector / 32 + bit = vector % 32 + subtree = leaf / 2 + +Registers +--------- + +nova-core defines the tree's registers in the ``irq`` module's ``regs.rs``. The +leaf registers are arrays indexed by leaf number. + +* ``LEAF(i)`` reads as the pending bits of leaf ``i``. Writing a 1 to a bit + clears that vector, and a 0 leaves the bit as it was. +* ``LEAF_EN_SET(i)`` and ``LEAF_EN_CLEAR(i)`` enable and disable the vectors + of leaf ``i``, one bit per vector. +* ``TOP_EN_SET`` and ``TOP_EN_CLEAR`` enable and disable subtrees, one bit per + subtree. +* ``LEAF_TRIGGER`` takes a vector number and latches that vector, exactly as + the vector's own source would. It is write-only. The self-test uses it. + +nova-core does not read ``TOP``. "Servicing the tree" says why. + +Every set and clear register acts per bit: a 1 performs the action for that +bit, and a 0 leaves the bit alone. No register needs a read-modify-write. + +GIN delivers a vector to the CPU only when its leaf enable bit and its +subtree's TOP enable bit are both set. The enables do not affect the latch. The +source of a disabled vector still sets its ``LEAF`` bit. ``TOP`` does not show +that bit, so reading the leaf is the only way to see it. + +How a unit interrupt reaches the CPU +------------------------------------ + +A unit does not write a ``LEAF`` register. Each unit has an interrupt control +register that GSP firmware programs. The control register holds the unit's +vector, the GFID that identifies the PCIe function whose tree receives the +interrupt, and one enable bit per destination: the CPU, the GSP, and the other +on-chip processors. When the unit has an event:: + + 1. The unit sends GIN an interrupt message carrying the vector, the GFID, + and the destination enables from its control register. + 2. In the tree of each destination that the message selects, GIN sets bit + (vector % 32) of LEAF(vector / 32). + 3. If the vector and its subtree are enabled in the CPU tree, GIN raises + the PCI interrupt. + +Because firmware assigns the vectors, nova-core does not hardcode which vector +belongs to which unit, with two exceptions. The hardware headers of every +supported chipset define the GSP event as vector 155, and GSP firmware's own +interrupt table uses that definition. The CPU doorbell is vector 129. Pre-Hopper +hardware fixes that number, and GSP firmware keeps it on Hopper and later. +nova-core names both by number. A driver can fetch the full unit-to-vector +table from the GSP by RPC, and nouveau does. nova-core does not, because a +fixed vector needs no lookup. + +Edge-triggered delivery +----------------------- + +A ``LEAF`` bit is a latch. Its source sets it on a rising edge, and it stays set +until the CPU clears it. A source that stays high does not set the bit again. + +GIN raises the PCI interrupt for a subtree when the subtree's enabled pending +state goes from low to high:: + + Per vector, in leaf i at bit b: + LEAF(i)[b] AND LEAF_EN(i)[b] + + Per subtree N, across leaves 2N and 2N + 1: + OR of every enabled pending bit -> TOP[N] + + Delivery for subtree N: + TOP[N] AND TOP_EN[N] -> rising edge -> PCI interrupt + +``TOP_EN`` applies after the summary, so disabling a subtree stops delivery +without changing what ``TOP`` reports. + +Three consequences: + +* Code that must find every pending vector reads the leaves. A vector that + latched while disabled is not in ``TOP``. +* Writing ``TOP_EN_SET`` for a subtree with an enabled pending bit produces a + new edge. GIN delivers an interrupt for a pending bit left uncleared as soon + as its subtree is enabled again. +* A source that holds its signal high produces no new edge after the CPU + clears the leaf bit. Such a source has to re-emit its interrupt. The falcons + do that through ``INTR_RETRIGGER`` (see "Retriggering a falcon"). + +Delivery over PCI +================= + +GIN delivers the tree's interrupts to the CPU as MSI or MSI-X, whichever Linux +grants. nova-core requests MSI-X first and falls back to MSI, and never uses +INTx. + +MSI has a single message, and every subtree raises that one message, so one +Linux IRQ serves the whole tree. + +MSI-X gives each subtree its own table entry, at the index equal to the subtree +number. Linux masks every entry until a driver requests its Linux IRQ number, +and a masked entry sends no message: the GPU records the interrupt in the MSI-X +pending bit array, where it stays until Linux unmasks the entry. An entry that +the driver never requests is never unmasked. A driver that enables a subtree +without requesting that subtree's entry loses every interrupt from that +subtree, with nothing reported: the leaf and TOP registers show the vector +pending and enabled while no handler runs. + +The serviced-subtree invariant +------------------------------ + +Every subtree enabled at ``TOP`` has an allocated PCI vector with a registered +handler. + +MSI satisfies this with its single message. MSI-X needs one allocated entry per +serviced subtree, and a PCI allocation cannot be sparse, so nova-core requests +entries 0 through the highest serviced subtree:: + + MSI-X, with subtree 2 serviced: + + subtree 0 -> entry 0 allocated, no handler, stays masked + subtree 1 -> entry 1 allocated, no handler, stays masked + subtree 2 -> entry 2 handler here, and its rearm covers subtree 2 + + MSI, with any serviced set: + + every serviced subtree -> the one allocated PCI vector, whose + handler's rearm covers the whole serviced set + +An allocated entry whose subtree nova-core does not service costs nothing. The +entry stays masked, and a disabled subtree raises no interrupt. + +nova-core services one subtree. The GSP event, vector 155, is in leaf 4, which +is in subtree 2. OpenRM's headers place its UVM_SHARED interrupt category in +subtree 2 on every chipset that nova-core supports. The self-test doorbell, +vector 129, is in the same leaf, and the test allocates its own vectors for it. + +Rearming PCI interrupt delivery +------------------------------- + +A message-signaled interrupt is delivered once per edge, and the PCI side +delivers no further interrupt until the CPU rearms it. The rearm operation +depends on the GPU family and on the interrupt type that Linux granted: + +================== ===== =========================================== +Architecture Type Rearm operation +================== ===== =========================================== +Turing through Ada MSI write the configuration-mirror EOI register +Hopper and later MSI clear then set the serviced TOP enables +Any MSI-X clear then set the handler's own TOP enable +================== ===== =========================================== + +The end-of-interrupt register is ``NV_XVE_CYA_2`` in the BAR0 mirror of PCI +configuration space, and the value written does not matter. The ``TOP_EN`` +cycle produces a new delivery edge. The MSI forms cover every serviced subtree, +because one message serves all of them. The MSI-X form covers one subtree, +because each serviced subtree has its own entry and its own handler. + +A handler rearms once per delivered interrupt, on every path, including the +path where it finds its vector not pending. A handler that skips the rearm +receives no further interrupts. + +OpenRM makes the same split. It writes the configuration-space EOI for MSI on +pre-Hopper chipsets, and cycles the TOP enables of the subtrees that it +services for Hopper-plus MSI and for MSI-X. + +Servicing the tree +================== + +Servicing a leaf has a required order: read its pending bits, then clear them. +Clearing a leaf before reading it discards every vector latched in it, and no +register reports the loss. In nova-core, reading a leaf produces the handle that +clears it, so that the wrong order does not compile. The handle clears exactly +the bits that it read, so a vector that latched after the read stays pending. + +A handler clears its bit before it services the vector. Clearing afterwards +would discard an interrupt that the source raised while the handler ran. + +nova-core services the tree in two ways. + +The notification path services one vector. It reads the vector's leaf, clears +only the vector's bit, and rearms. The subtree stays enabled, and a vector +pending beside it in the same leaf keeps its bit set for the code that services +that vector. The GSP event handler takes this path, and so does the self-test +handler. + +The startup drain walks the whole tree, because it must clear whatever is +pending across every subtree rather than one known vector. It disables the +serviced subtrees at ``TOP``, reads and clears every implemented leaf, and +leaves the subtrees disabled for its caller to enable once the caller is ready +for deliveries. The drain reads every leaf rather than descending from ``TOP``, +because sources latch vectors during boot while those vectors are disabled, and +``TOP`` does not show them. OpenRM's stall-interrupt path reads every leaf for +the same reason. + +The two paths as register operations:: + + Startup drain, run once during probe: + write TOP_EN_CLEAR = serviced stop new deliveries + for each implemented leaf i: + pending = read LEAF(i) + write LEAF(i) = pending clear what was read + (returns with TOP_EN still clear) + + Notification, the subtree stays enabled: + pending = read LEAF(leaf) is the handler's bit set? + write LEAF(leaf) = bit clear that one bit + rearm PCI interrupt delivery + +The drain clears every pending bit, including bits that nova-core never +services. An uncleared bit holds its subtree in the pending state, and enabling +that subtree again would deliver an interrupt for a vector that no handler +services. + +The drain's ``TOP_EN_CLEAR`` is not a rearm, and pre-Hopper MSI rearms through +the configuration mirror, which the drain never writes. An interrupt delivered +before probe had no handler to rearm it, so the tree reset rearms explicitly +after the drain. A TOP-enable rearm leaves the serviced subtrees enabled, so +the reset then disables them again. + +nova-core does not serialize access to the tree. The GSP event handler +touches only its own leaf, and the drain runs during probe, before that handler +is registered. + +Per-architecture differences +============================ + +The tree is the same on every supported GPU except for its size, which changes +at Hopper: + +=================== ====== ======== ==================== +GPUs Leaves Subtrees Implemented subtrees +=================== ====== ======== ==================== +Turing, Ampere, Ada 8 4 ``0x0f`` +Hopper, Blackwell 16 8 ``0xff`` +=================== ====== ======== ==================== + +The interrupt HAL provides the leaf count, and the subtree count and the +implemented-subtree set derive from it. A subtree that the chipset does not +implement has no TOP bit, so building a tree that services one fails with +``EINVAL``. Vectors 129 and 155 are in the 8-leaf tree, so every supported +chipset has them. + +OpenRM's headers assign every interrupt category of a 16-leaf tree to leaves 0 +through 11. The drain reads all 16, because a vector can be latched in any +implemented leaf. + +The HAL performs the rearm as well (see "Rearming PCI interrupt delivery"). Two +falcon properties also differ by family, and the falcon HAL carries them: +Turing falcons have no ``INTR_RETRIGGER``, and the RISC-V routing registers +moved at GA102 (see "Retriggering a falcon"). + +The GSP event +============= + +When the GSP has output for the CPU, it writes messages into the GSP-to-CPU +queue in shared memory and raises SWGEN0, one of the software-generated +interrupt causes of the GSP falcon. SWGEN0 is routed to the host, at vector +155, leaf 4 bit 27, in subtree 2. + +The queue carries notifications (log records, error records, lifecycle events) +and command replies. A thread waiting for a reply reads the queue itself, so +the interrupt is only the trigger to drain the queue (see "Draining the +GSP-to-CPU queue"). + +The falcon latches every cause that it raises, SWGEN0 among them, in its +``IRQSTAT`` register. The handler services the host-routed causes and clears +their latches, and then it writes ``INTR_RETRIGGER`` so that the falcon re-emits +any cause that latched in the meantime. "Retriggering a falcon" has the details. + +Draining the queue takes the command-queue mutex and walks shared memory, so it +cannot run in hard interrupt context. nova-core registers a threaded handler, +under the name ``nova-core`` in ``/proc/interrupts``. The top half runs in hard +interrupt context and reads and writes only registers, and it wakes the IRQ +thread to drain the queue:: + + GSP writes messages into the GSP-to-CPU queue + GSP raises SWGEN0 + GIN sets bit 27 of LEAF(4), and subtree 2 becomes pending + PCI interrupt -> Linux IRQ -> top half, in hard interrupt context: + read LEAF(4), and if bit 27 is clear, rearm and return + clear bit 27 (the subtree stays enabled) + read the falcon causes routed to the host, clearing SWGEN0 if set + for every other host cause: log it, clear its latch, and read the + host causes back + if the clear ended all of them: retrigger the falcon + otherwise: disable vector 155 at its leaf and skip the retrigger + rearm PCI interrupt delivery + wake the IRQ thread if SWGEN0 was set + IRQ thread, which may sleep: + take the command-queue mutex and drain the GSP-to-CPU queue + +A halt and a posted message can be pending together, so the top half services +every cause that the status reports. + +A drain fails when a message's framing or checksum is bad (see "Draining the +GSP-to-CPU queue"). The message stays at the queue head, so every later drain +would fail the same way, and the IRQ thread disables vector 155 and logs the +failure, which leaves the queue unserviced until the device is reset. + +The handler, the self-test, and the rest of the driver read BAR0 through one +shared mapping. nova-core unregisters an interrupt handler when the device +unbinds, so a handler runs only while the mapping exists. + +Retriggering a falcon +--------------------- + +A falcon signals the tree when its set of host-routed causes goes from empty to +non-empty. A cause left latched keeps the set non-empty, so no later cause +signals the tree, and the vector is lost. For a cause that stays latched, the +handler can clear the tree leaf first or the falcon latch first, and the loss +is the same. + +``IRQSTAT`` latches every cause in the falcon, including the causes routed to +the falcon's own RISC-V core and owned by the firmware running on it. A host +handler owns only the causes that ``PRISCV_RISCV_IRQMASK`` and +``PRISCV_RISCV_IRQDEST`` both select, so it intersects ``IRQSTAT`` with both +before it reads or clears a cause. OpenRM computes the same intersection in +``kflcnRiscvReadIntrStatus``. GA100 keeps the Turing offsets of the two routing +registers and GA102 moves them, so the offsets change at GA102 rather than at +the Ampere boundary. The handler masks no cause: ``PRISCV_RISCV_IRQMASK`` is +read-only to the host, and ``FALCON_IRQMASK`` has no effect on host routing on +a RISC-V falcon. + +``INTR_RETRIGGER`` makes the falcon re-emit its host-routed causes into the +tree, which supplies the transition that clearing the leaf lost. The handler +writes ``INTR_RETRIGGER`` only on a path where it ended every cause that it +read, because a re-emitted cause that stays set arrives again at once and +on every pass after that. + +``IRQSCLR`` ends a latch and does not end the source behind it, so a cause +driven from outside the falcon stays set after the write. On Blackwell the +fault-containment and ECC causes are driven that way: they appear in +``IRQSTAT`` but come from ``PRISCV_RISCV_FAULT_CONTAINMENT_SRCSTAT`` and +``PGSP_ECC_INTR_STATUS``, and only a device reset ends them. So the handler +clears the latch of every host cause other than SWGEN0, reads the host causes +back, and retriggers only when the read-back is empty. When a cause is still +set, the handler disables vector 155 instead and reports that the device needs +a reset. Disabling loses no notification: the cause that is still set holds the +host-routed set non-empty, so the falcon would signal nothing further either +way. OpenRM makes the same choice, and ``kgspService_TU102`` skips +``kflcnIntrRetrigger`` once it has recorded a fatal error. + +The handler cannot distinguish a fault cause that arrives after the clear from +one that the clear failed to end, so it disables the vector in that case too. +Both mean that the GSP has faulted. + +Turing falcons have no ``INTR_RETRIGGER``, so a Turing handler cannot re-create +a transition that it has lost. It must leave no host cause latched: it reads the +host-routed status once and takes every cause that the status reports, rather +than stopping at the first one that it recognizes. One window stays open. A +cause that arrives after the handler has read the status is not in the value +that the handler clears, so it stays latched after the leaf has been cleared, +and no later cause from that falcon signals the tree. OpenRM has the same window +on Turing, where ``kflcnIntrRetrigger`` does nothing. + +Enabling the GSP event +---------------------- + +SWGEN0 is a latch, and the GSP drives no new edge into the tree while it stays +set. nova-core's GSP boot code consumes the GSP's notifications by polling the +queue, which leaves the latch set and leaves pending bits in the tree. The +handoff from polling to interrupts has a required order:: + + disable every implemented vector drop enables left by boot, or by a + driver that ran before this one + drain the tree clear stale pending bits + rearm PCI interrupt delivery required under pre-Hopper MSI, which + no later step rearms + disable subtree 2 at TOP a TOP-enable rearm enabled it again + clear the SWGEN0 latch so the next message makes an edge + register the threaded handler no delivery can reach it yet + enable vector 155 at LEAF(4) the subtree is still disabled + enable subtree 2 at TOP deliveries become possible here + drain the GSP-to-CPU queue messages posted before the clear + +The first four steps are the tree reset. nova-core runs them, and clears the +latch, before it registers the handler. Registering unmasks the PCI interrupt, +and GIN would then deliver a vector that boot left enabled to a handler that +services one vector and has no way to service any other. OpenRM clears every +leaf enable at the same point for the same reason. + +nova-core clears the latch after the drain. If nova-core cleared the latch +first, a message posted before the drain could set it again, along with bit 27 +of LEAF(4). The drain would then clear the leaf bit while the latch stays set, +and no later message would signal the tree. Clearing after the drain can +instead leave the leaf bit pending with the latch already clear. Enabling the +subtree then delivers one interrupt whose ``IRQSTAT`` reads zero. The top half +clears the leaf bit, rearms, and does not wake the IRQ thread, and the queue +drain that follows reads the message. + +Clearing the latch makes the first interrupt possible. A message that the GSP +posted before that clear produces no interrupt, so the sequence ends by +draining the queue. + +The subtree is enabled at ``TOP`` only once the handler is registered. A +TOP-enable rearm sets the enables that it cycles, so the reset disables the +serviced subtrees again after its rearm, and the tree reaches the registration +with subtree 2 disabled under every rearm method. On teardown nova-core disables +the vector at its leaf, so that no vector in the subtree can be delivered, then +calls ``free_irq()``, and disables the subtree last. Disabling the subtree +earlier would let a handler still in flight enable it again through the +``TOP_EN`` cycle of its rearm, which would leave the subtree enabled with no +handler registered. The driver's ``Gpu`` object, which owns the tree, holds the +subtree enable, and a handler's registration holds only the enable of its own +vector, so that tearing down one handler does not disable a subtree that another +handler shares. nova-core tears down the registration before it frees the queue +that the handler drains and the falcon that the handler reads, and before it +unloads the GSP. + +Draining the GSP-to-CPU queue +----------------------------- + +The queue carries command replies and unsolicited events, and a message's +function code says which it is. + +* A function code that matches the awaited reply: the message is decoded and + returned to the caller that sent the command. +* Anything else is an event. An OS error record and a robust-channel record + are logged at error level, and an unrecognized function code at warning + level. The other known events (GSP logs, libos prints, assertion records, + lifecycle notices) need no action and get no line of their own, because the + receive trace at debug level already records every message's arrival with + its sequence number, function code, and length. + +The sequence number takes no part in the match, because the GSP does not echo +the command's sequence number on every reply. On r570 the reply to +``UnloadingGuestDriver`` carries sequence 0. + +The read pointer advances past every message that passes framing and checksum +validation, whether it matched or was an event. A matched message that is too +short to decode is the exception, and it stays at the queue head. + +A message's length is inside the region that the checksum covers, so once the +framing or the checksum fails there is no trustworthy length with which to skip +the message. Such a message also stays at the queue head, and every later +receive fails on it with ``EIO`` until the device is reset. + +The polling path and the IRQ thread both read the queue under the command-queue +mutex. Replies and events share one queue and one read pointer, so one lock is +held across the whole drain. A thread waiting for a reply logs each event that +arrives before the reply and keeps waiting. One deadline of 5 seconds applies to +the whole wait, rather than a fresh timeout after each message, and the thread +holds the mutex for the whole wait, so that no other caller consumes the message +that it waits for. + +With one lock, a drain waits for an in-flight command's receive to finish or +time out. For log and error records that delay does not matter. diff --git a/Documentation/gpu/nova/core/pramin.rst b/Documentation/gpu/nova/core/pramin.rst new file mode 100644 index 000000000000..f50b052d73ba --- /dev/null +++ b/Documentation/gpu/nova/core/pramin.rst @@ -0,0 +1,128 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================= +PRAMIN aperture mechanism +========================= + +.. note:: + The following description is approximate and current as of the Ampere + family. It may change for future generations and is intended to assist in + understanding the driver code. + +Introduction +============ + +PRAMIN is a hardware aperture mechanism that provides CPU access to GPU Video +RAM (VRAM) before the GPU's Memory Management Unit (MMU) and page tables are +initialized. This 1 MiB sliding window, located at a fixed offset within BAR0, +is essential for setting up page tables and other critical GPU data structures +without relying on the GPU's MMU. + +Architecture Overview +===================== + +The PRAMIN aperture mechanism is logically implemented by the GPU's PBUS (PCIe +Bus Controller Unit) and provides a CPU-accessible window into VRAM through the +PCIe interface:: + + +-----------------+ PCIe +------------------------------+ + | CPU |<----------->| GPU | + +-----------------+ | | + | +----------------------+ | + | | PBUS | | + | | (Bus Controller) | | + | | | | + | | +--------------+ <------------ [1] + | | | PRAMIN | | | + | | | Window | | | + | | | (1 MiB) | | | + | | +--------------+ | | + | | | | | + | +---------|------------+ | + | | | + | v | + | +----------------------+ <------- [2] + | | VRAM | | + | | (Several GiB) | | + | | | | + | | FB[0x0000000000] | | + | | ... | | + | | FB[0xFFFFFFFFFF] | | + | +----------------------+ | + +------------------------------+ + + [1] Window starts at BAR0 + 0x700000. + [2] Program PRAMIN to any 64 KiB-aligned VRAM boundary. + +PBUS is responsible for, among other things, handling MMIO +accesses to the BAR registers. + +PRAMIN Window Operation +======================= + +The PRAMIN window provides a 1 MiB sliding aperture that can be repositioned +over the entire VRAM address space using the ``NV_PBUS_BAR0_WINDOW`` register. + +Window Control Mechanism +------------------------- + +:: + + NV_PBUS_BAR0_WINDOW Register (0x1700): + +-------+--------+--------------------------------------+ + | 31:26 | 25:24 | 23:0 | + | RSVD | TARGET | BASE_ADDR | + | | | (bits 39:16 of VRAM address) | + +-------+--------+--------------------------------------+ + + The 24-bit BASE_ADDR field encodes bits [39:16] of the target VRAM address, + providing 40-bit (1 TiB) address space coverage with 64 KiB alignment. + + TARGET field (bits 25:24): + - 0x0: VRAM (Video Memory) + - 0x1: Reserved (unused) + - 0x2: SYS_MEM_COH (Coherent System Memory) + - 0x3: SYS_MEM_NONCOH (Non-coherent System Memory) + +.. note:: + Nova only uses TARGET=VRAM (0x0) for video memory access. The SYS_MEM + target values are documented here for hardware completeness but are + not used by the driver. + +64 KiB Alignment Requirement +---------------------------- + +The PRAMIN window must be aligned to 64 KiB boundaries in VRAM. This is enforced +by the ``BASE_ADDR`` field representing bits [39:16] of the target address:: + + VRAM Address Calculation: + actual_vram_addr = (BASE_ADDR << 16) + pramin_offset + Where: + - BASE_ADDR: 24-bit value from NV_PBUS_BAR0_WINDOW[23:0] + - pramin_offset: 20-bit offset within the PRAMIN window [0x00000-0xFFFFF] + + Example Window Positioning: + +---------------------------------------------------------+ + | VRAM Space | + | | + | 0x0000000000 +-----------------+ <-- 64 KiB aligned | + | | PRAMIN Window | | + | | (1 MiB) | | + | 0x00000FFFFF +-----------------+ | + | | + | | ^ | + | | | Window can slide | + | v | to any 64 KiB-aligned boundary | + | | + | 0x0123400000 +-----------------+ <-- 64 KiB aligned | + | | PRAMIN Window | | + | | (1 MiB) | | + | 0x01234FFFFF +-----------------+ | + | | + | ... | + | | + | 0xFFFFF00000 +-----------------+ <-- 64 KiB aligned | + | | PRAMIN Window | | + | | (1 MiB) | | + | 0xFFFFFFFFFF +-----------------+ | + +---------------------------------------------------------+ diff --git a/Documentation/gpu/nova/core/todo.rst b/Documentation/gpu/nova/core/todo.rst index d5130b2b08fb..a01c362b1be0 100644 --- a/Documentation/gpu/nova/core/todo.rst +++ b/Documentation/gpu/nova/core/todo.rst @@ -33,7 +33,7 @@ A good example from nova-core would be the ``Chipset`` enum type, which defines the value ``AD102``. When probing the GPU the value ``0x192`` can be read from a certain register indication the chipset AD102. Hence, the enum value ``AD102`` should be derived from the number ``0x192``. Currently, nova-core uses a custom -implementation (``Chipset::from_u32`` for this. +implementation (``Chipset::from_u32``) for this. Instead, it would be desirable to have something like the ``FromPrimitive`` trait [1] from the num crate. |
