| Age | Commit message (Collapse) | Author |
|---|
|
The vector arithmetic and the leaf and subtree sets touch no hardware,
so KUnit can cover them without a GPU. A wrong leaf, bit or subtree for
a vector would otherwise show up only as a lost interrupt on hardware.
Add the nova_core_gin_tree suite. It checks that a leaf index stops at
the widest supported tree, that a leaf count implies the right number of
subtrees and vectors, that a leaf count enumerates every leaf in order,
that a vector maps to the right leaf, bit and subtree, and that a vector
beyond an 8-leaf tree is rejected there and accepted in a 16-leaf tree.
It also exercises the subtree set operations and checks that every
supported chipset implements the subtree that carries the GSP event.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260930034148.590687-15-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
The GSP posts each message for the CPU to the GSP-to-CPU queue and then
raises SWGEN0, a software-generated interrupt cause of its falcon. GIN
delivers SWGEN0 to the CPU on vector 155.
A thread waiting for a command reply was the only reader of the queue,
so an event posted between commands stayed unread until the next
command was sent.
Register a threaded handler on the GSP vector. Draining the queue takes
the command-queue mutex, which can sleep, so the top half reads and
writes only registers and wakes the IRQ thread to drain the queue.
Every existing read of the queue asks for one message type and waits on
a deadline, so add a drain that logs and consumes whatever the GSP has
already posted and returns at once.
When a falcon cause other than SWGEN0 stays set after its latch is
cleared, as the Blackwell fault-containment and ECC causes do, disable
the GSP vector rather than retrigger the falcon. A retrigger would
re-emit the cause at once, and the CPU would take the same interrupt
again and again.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260930034148.590687-14-jhubbard@nvidia.com
[acourbot: trim comment leaking unneeded implementation details.]
[acourbot: make Subtree crate-visible here instead of in earlier patch.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
A GPU interrupt can be lost in the MSI or MSI-X allocation, in the GIN
tree's enables, or in the rearm, and every one of those failures looks
the same: no interrupt arrives, and no register or log line says which
one broke.
Add a probe-time self-test, built under NOVA_CORE_SELFTESTS, that
latches the CPU doorbell vector through the GIN software trigger and
waits for a registered handler to service it. One delivery would pass
with a broken rearm, because the first message-signaled interrupt
arrives whether or not the driver rearms, so the test triggers twice and
waits for the first handler to finish before the second trigger. It runs
after GFW boot and before GSP boot, on a quiesced tree, and fails probe
unless both deliveries arrive, each finds only the doorbell pending, and
the leaf ends clear.
The doorbell has the same vector on every supported GPU, so the test
names it without asking GSP-RM. It allocates the PCI vectors for the
doorbell's subtree and releases them before returning, so under MSI-X
the delivery also exercises that subtree's table entry.
Assisted-by: LLM
Co-developed-by: Joel Fernandes <joelagnelf@nvidia.com>
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260930034148.590687-9-jhubbard@nvidia.com
[acourbot: move relevant bits of previous patch into this one.]
[acourbot: remove reference to self-test section of documentation.]
[acourbot: replace completions with atomic polling code.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
Servicing a GIN leaf has a required order: read its pending bits, then
clear them. Clearing a leaf first discards every vector latched in it,
and the hardware keeps no record of what was discarded. The interrupt
never arrives, and no register shows that it was ever pending.
Every subtree enabled at TOP also needs an allocated PCI vector with a
handler registered on it. Under MSI-X each subtree has its own table
entry. Linux masks every entry until a driver requests its IRQ, and a
masked entry sends no message. An enabled subtree whose entry was never
requested raises interrupts that never reach a handler, while the leaf
and TOP registers show them pending and enabled. Under MSI the whole
tree raises a single message, so one entry serves every subtree.
Add the CPU interrupt tree of one PCIe function. The tree allocates the
PCI vectors for the subtrees that it services and hands out the request
for each, so that the chipset is given once and the vectors live as
long as the tree. Reading a leaf yields the handle that clears it, so
that the wrong order does not compile. Building a tree fails if it
names a subtree that the GPU does not implement. A reset disables every
vector, clears every pending bit, rearms delivery, and leaves the
serviced subtrees disabled at TOP, so that a handler registered
afterwards receives no interrupt left over from boot.
Request MSI-X entries 0 through the highest serviced subtree, since an
MSI-X allocation cannot be sparse, and fall back to a single MSI
message, never to INTx.
[jhubbard: reworked on top of the GIN vector types: a leaf read yields
the handle that clears it, enables are guarded, the leaf count and the
rearm come from the interrupt HAL, the drain reads every implemented
leaf rather than descending from TOP, the tree owns its PCI vectors,
and a reset returns the tree to a known state before a handler is
registered]
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Will Pierce <wpierce@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260930034148.590687-7-jhubbard@nvidia.com
[acourbot: keep item that doesn't need to be exported private.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
The GIN CPU interrupt tree differs by GPU family in two ways:
* The size of the tree. Turing, Ampere and Ada implement 8 leaves.
Hopper and later families implement 16.
* The write that rearms delivery. A message-signaled interrupt is
delivered once per edge, and the PCI side delivers nothing more until
the CPU rearms it. Before Hopper, MSI rearms by writing the
end-of-interrupt register in the BAR0 mirror of PCI configuration
space. On Hopper and later, MSI rearms by clearing and then setting
the TOP enables of every serviced subtree, which produces a new edge.
MSI-X rearms the same way on every family, but for the handler's own
subtree only, since each subtree has its own table entry.
Add an interrupt HAL that provides the leaf count and performs the rearm
for each family. Name the interrupt type with two variants, MSI and
MSI-X, since nova-core never allocates the level-triggered INTx that the
kernel's PCI interrupt type also names.
The HAL has no user yet.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Will Pierce <wpierce@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260930034148.590687-6-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
GIN is the GPU's interrupt controller. It latches each interrupt source
in a two-level tree of LEAF registers summarized by TOP, and raises the
PCI interrupt when an enabled vector in an enabled subtree becomes
pending. A message-signaled interrupt is delivered once per edge, and
pre-Hopper MSI rearms delivery by writing the end-of-interrupt register
in the BAR0 mirror of PCI configuration space.
Add the CPU tree registers that receiving GSP interrupts and running the
software-triggered self-test need: the leaf pending and enable arrays,
the TOP enables, and the leaf trigger. Add the end-of-interrupt
register, NV_XVE_CYA_2, alongside them.
Use the NV_VIRTUAL_FUNCTION_PRIV_CPU_INTR_* names on every chipset. The
pre-Hopper headers call the same tree NV_CTRL, but each function reaches
its own tree through this aperture on both families. Declare the leaf
arrays at 16 entries, the widest tree that any supported chipset
implements.
Leave the read-only TOP summary undeclared. A vector that latched while
disabled does not appear in TOP, so nova-core never descends from it and
reads every implemented leaf instead.
Declare each leaf field as a set of vectors within one leaf and each TOP
field as a set of subtrees, so that a set of subtrees cannot be written
to a leaf register, nor the reverse. The trigger register's vector field
is 12 bits wide, wider than any GIN vector, so a vector converts into it
infallibly.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Will Pierce <wpierce@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260930034148.590687-5-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
GIN, the GPU Interrupt and Notification unit, is the GPU's interrupt
controller. Each interrupt source has a GIN vector number, and the
controller latches a pending vector in a two-level tree: one bit of a
LEAF register, summarized two leaves at a time by one bit of the TOP
register. A vector's number fixes its position in that tree:
leaf = vector / 32
bit = vector % 32
subtree = leaf / 2
A tree implements either 8 or 16 leaves, depending on the GPU family.
The leaf count sets both the number of subtrees and the highest vector
that the tree carries.
Without distinct types, a vector, a leaf index, a set of vectors within
one leaf, a subtree and a set of subtrees are all plain integers. A
caller can pass one where another belongs, and a register field can
accept the wrong one.
Add a type for each of those, and for the leaf count. A vector converts
to its own leaf, bit and subtree. Its constructor rejects, at build
time, a number beyond the widest supported tree, and a validation
method rejects, at run time, a number beyond the leaves that the
current tree implements. A leaf count yields the set of subtrees that
it implements.
The module has no user yet.
Suggested-by: Danilo Krummrich <dakr@kernel.org>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260930034148.590687-4-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
