| Age | Commit message (Collapse) | Author |
|---|
|
Use KVec::zeroed() to create the message buffer in recv_msg() instead of
creating an empty vector and resizing it.
Signed-off-by: Thorsten Blum <blum@kernel.org>
Link: https://patch.msgid.link/20261006065437.58408-3-blum@kernel.org
[ Avoid the unnecessary turbofish syntax. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
|
|
Use KVVec::zeroed() instead of KVVec::from_elem() to allocate and
zero-initialize the command and continuation payload buffers.
Also use KVVec::zeroed() in the test helper read_payload().
Signed-off-by: Thorsten Blum <blum@kernel.org>
Link: https://patch.msgid.link/20261006115311.92873-3-blum@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
|
|
bar1_size() reports the length of PCI resource 1 as the size of BAR1.
That is only correct when BAR0 is a 32-bit BAR. On Blackwell, and on some
Ampere GA100 and Hopper SKUs, BAR0 is 64-bit and takes resource slots 0
and 1, pushing BAR1 to slot 2. Slot 1 is then the upper half of BAR0
rather than a BAR of its own, so NOVA_GETPARAM_VRAM_BAR_SIZE reports the
wrong size on those GPUs.
Probe handles this with the new bar1_resource_index(), which it uses to
map BAR1, so use the same lookup here.
Fixes: 7edc491f6fca ("drm: nova: Use nova-core to read VRAM_BAR_SIZE parameter")
Signed-off-by: Alistair Popple <apopple@nvidia.com>
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20261001005615.77130-1-apopple@nvidia.com
[ Import the driver module. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
|
|
A module-wide new import becomes unused if the `NOVA_CORE_SELFTESTS`
config option is disabled:
warning: unused import: `Spec`
--> ../drivers/gpu/nova-core/driver.rs:29:9
|
29 | Spec, //
| ^^^^
|
Fix this by fully qualifying the unique use of `Spec` when the tests are
enabled.
Fixes: 9cb3f5d56e81 ("gpu: nova-core: add an interrupt delivery self-test")
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202610031630.JOlyp48X-lkp@intel.com/
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Link: https://patch.msgid.link/20261004-nova-int-fixup-v1-1-9e4bce097f94@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
The vector arithmetic and the leaf and subtree sets touch no hardware,
so KUnit can cover them without a GPU. A wrong leaf, bit or subtree for
a vector would otherwise show up only as a lost interrupt on hardware.
Add the nova_core_gin_tree suite. It checks that a leaf index stops at
the widest supported tree, that a leaf count implies the right number of
subtrees and vectors, that a leaf count enumerates every leaf in order,
that a vector maps to the right leaf, bit and subtree, and that a vector
beyond an 8-leaf tree is rejected there and accepted in a 16-leaf tree.
It also exercises the subtree set operations and checks that every
supported chipset implements the subtree that carries the GSP event.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260930034148.590687-15-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
The GSP posts each message for the CPU to the GSP-to-CPU queue and then
raises SWGEN0, a software-generated interrupt cause of its falcon. GIN
delivers SWGEN0 to the CPU on vector 155.
A thread waiting for a command reply was the only reader of the queue,
so an event posted between commands stayed unread until the next
command was sent.
Register a threaded handler on the GSP vector. Draining the queue takes
the command-queue mutex, which can sleep, so the top half reads and
writes only registers and wakes the IRQ thread to drain the queue.
Every existing read of the queue asks for one message type and waits on
a deadline, so add a drain that logs and consumes whatever the GSP has
already posted and returns at once.
When a falcon cause other than SWGEN0 stays set after its latch is
cleared, as the Blackwell fault-containment and ECC causes do, disable
the GSP vector rather than retrigger the falcon. A retrigger would
re-emit the cause at once, and the CPU would take the same interrupt
again and again.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260930034148.590687-14-jhubbard@nvidia.com
[acourbot: trim comment leaking unneeded implementation details.]
[acourbot: make Subtree crate-visible here instead of in earlier patch.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
A falcon latches each interrupt cause that is raised in its IRQSTAT
register. On a RISC-V falcon, each cause is routed either to the host,
meaning the CPU, or to the falcon's own core, and two routing registers
hold that routing. The falcon signals the interrupt tree only when its
set of host-routed causes goes from empty to non-empty, so a handler
that clears the tree leaf while a cause is still latched has to write
INTR_RETRIGGER, which makes the falcon re-emit its causes. Turing
falcons have no INTR_RETRIGGER, and the routing registers have
different offsets from GA102 on.
Add IRQSTAT, INTR_RETRIGGER, and the two routing registers, the last at
both sets of offsets, and add two falcon HAL methods: one intersects an
IRQSTAT value with the routing registers, and one retriggers the
falcon, or does nothing on Turing. GA100 keeps the Turing HAL, as it
does for boot, and the retrigger register is a property of that HAL's
instance.
The two methods have no user yet.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260930034148.590687-13-jhubbard@nvidia.com
[acourbot: remove implementation-specific comments on interface doc.]
[acourbot: make constructor take boolean instead of having two methods.]
[acourbot: add NVIDIA copyright headers to modified files.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
The GSP posts unsolicited events on the same queue as command replies,
so a caller waiting for one message consumes whatever arrives first and
reads again.
Every read started a fresh five-second timeout, so a steady stream of
events extended the wait without bound. The two boot-time waits for an
unsolicited event also released the queue mutex between reads, so a
command sent from another thread could consume the event and leave the
waiter to time out.
Compute one deadline when the wait begins and pass the time remaining to
each read, and hold the queue mutex across the whole wait. Put the loop
in one helper that the command reply wait and both boot-time event waits
share.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260930034148.590687-12-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
The GSP posts unsolicited events on the same queue as command replies,
so a receive that asks for one message type has to report that the
message at the queue head was a different one.
That case returned ERANGE, which means a value outside a valid range and
says nothing about a message.
Return ENOMSG, no message of the desired type, instead.
Suggested-by: Gary Guo <gary@garyguo.net>
Suggested-by: Alexandre Courbot <acourbot@nvidia.com>
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260930034148.590687-11-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
The GSP posts unsolicited messages on the same queue that carries
command replies: log records, OS error and robust-channel records, and
lifecycle notices.
nova-core discarded every message that was not the reply that a caller
was waiting for, and an unrecognized function code aborted the in-flight
command. The GSP's error reports never reached the kernel log.
Log every non-reply message according to its function code, on the
receive path that already reads it, and leave the in-flight command
waiting for its reply. Event payloads, such as XID numbers and log
contents, are not decoded.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260930034148.590687-10-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
A GPU interrupt can be lost in the MSI or MSI-X allocation, in the GIN
tree's enables, or in the rearm, and every one of those failures looks
the same: no interrupt arrives, and no register or log line says which
one broke.
Add a probe-time self-test, built under NOVA_CORE_SELFTESTS, that
latches the CPU doorbell vector through the GIN software trigger and
waits for a registered handler to service it. One delivery would pass
with a broken rearm, because the first message-signaled interrupt
arrives whether or not the driver rearms, so the test triggers twice and
waits for the first handler to finish before the second trigger. It runs
after GFW boot and before GSP boot, on a quiesced tree, and fails probe
unless both deliveries arrive, each finds only the doorbell pending, and
the leaf ends clear.
The doorbell has the same vector on every supported GPU, so the test
names it without asking GSP-RM. It allocates the PCI vectors for the
doorbell's subtree and releases them before returning, so under MSI-X
the delivery also exercises that subtree's table entry.
Assisted-by: LLM
Co-developed-by: Joel Fernandes <joelagnelf@nvidia.com>
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260930034148.590687-9-jhubbard@nvidia.com
[acourbot: move relevant bits of previous patch into this one.]
[acourbot: remove reference to self-test section of documentation.]
[acourbot: replace completions with atomic polling code.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
The GPU boots its own firmware, GFW, out of reset, and the driver must
not program the GPU until GFW reports completion.
nova-core waited for GFW inside the Gpu constructor, which also boots
the GSP. Code that has to run after GFW and before GSP boot, such as a
probe-time hardware self-test, had nowhere to go.
Move the wait code into an associated function so it can also be called
by test code that requires it before the Gpu instance is constructed.
Suggested-by: Danilo Krummrich <dakr@kernel.org>
Link: https://lore.kernel.org/all/DLSNXUU1UXIH.JG4L0L5ZFV89@kernel.org/
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260930034148.590687-8-jhubbard@nvidia.com
[acourbot: move code to associated function, move relevant hunks to next
patch.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
Servicing a GIN leaf has a required order: read its pending bits, then
clear them. Clearing a leaf first discards every vector latched in it,
and the hardware keeps no record of what was discarded. The interrupt
never arrives, and no register shows that it was ever pending.
Every subtree enabled at TOP also needs an allocated PCI vector with a
handler registered on it. Under MSI-X each subtree has its own table
entry. Linux masks every entry until a driver requests its IRQ, and a
masked entry sends no message. An enabled subtree whose entry was never
requested raises interrupts that never reach a handler, while the leaf
and TOP registers show them pending and enabled. Under MSI the whole
tree raises a single message, so one entry serves every subtree.
Add the CPU interrupt tree of one PCIe function. The tree allocates the
PCI vectors for the subtrees that it services and hands out the request
for each, so that the chipset is given once and the vectors live as
long as the tree. Reading a leaf yields the handle that clears it, so
that the wrong order does not compile. Building a tree fails if it
names a subtree that the GPU does not implement. A reset disables every
vector, clears every pending bit, rearms delivery, and leaves the
serviced subtrees disabled at TOP, so that a handler registered
afterwards receives no interrupt left over from boot.
Request MSI-X entries 0 through the highest serviced subtree, since an
MSI-X allocation cannot be sparse, and fall back to a single MSI
message, never to INTx.
[jhubbard: reworked on top of the GIN vector types: a leaf read yields
the handle that clears it, enables are guarded, the leaf count and the
rearm come from the interrupt HAL, the drain reads every implemented
leaf rather than descending from TOP, the tree owns its PCI vectors,
and a reset returns the tree to a known state before a handler is
registered]
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Will Pierce <wpierce@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260930034148.590687-7-jhubbard@nvidia.com
[acourbot: keep item that doesn't need to be exported private.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
The GIN CPU interrupt tree differs by GPU family in two ways:
* The size of the tree. Turing, Ampere and Ada implement 8 leaves.
Hopper and later families implement 16.
* The write that rearms delivery. A message-signaled interrupt is
delivered once per edge, and the PCI side delivers nothing more until
the CPU rearms it. Before Hopper, MSI rearms by writing the
end-of-interrupt register in the BAR0 mirror of PCI configuration
space. On Hopper and later, MSI rearms by clearing and then setting
the TOP enables of every serviced subtree, which produces a new edge.
MSI-X rearms the same way on every family, but for the handler's own
subtree only, since each subtree has its own table entry.
Add an interrupt HAL that provides the leaf count and performs the rearm
for each family. Name the interrupt type with two variants, MSI and
MSI-X, since nova-core never allocates the level-triggered INTx that the
kernel's PCI interrupt type also names.
The HAL has no user yet.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Will Pierce <wpierce@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260930034148.590687-6-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
GIN is the GPU's interrupt controller. It latches each interrupt source
in a two-level tree of LEAF registers summarized by TOP, and raises the
PCI interrupt when an enabled vector in an enabled subtree becomes
pending. A message-signaled interrupt is delivered once per edge, and
pre-Hopper MSI rearms delivery by writing the end-of-interrupt register
in the BAR0 mirror of PCI configuration space.
Add the CPU tree registers that receiving GSP interrupts and running the
software-triggered self-test need: the leaf pending and enable arrays,
the TOP enables, and the leaf trigger. Add the end-of-interrupt
register, NV_XVE_CYA_2, alongside them.
Use the NV_VIRTUAL_FUNCTION_PRIV_CPU_INTR_* names on every chipset. The
pre-Hopper headers call the same tree NV_CTRL, but each function reaches
its own tree through this aperture on both families. Declare the leaf
arrays at 16 entries, the widest tree that any supported chipset
implements.
Leave the read-only TOP summary undeclared. A vector that latched while
disabled does not appear in TOP, so nova-core never descends from it and
reads every implemented leaf instead.
Declare each leaf field as a set of vectors within one leaf and each TOP
field as a set of subtrees, so that a set of subtrees cannot be written
to a leaf register, nor the reverse. The trigger register's vector field
is 12 bits wide, wider than any GIN vector, so a vector converts into it
infallibly.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Will Pierce <wpierce@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260930034148.590687-5-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
GIN, the GPU Interrupt and Notification unit, is the GPU's interrupt
controller. Each interrupt source has a GIN vector number, and the
controller latches a pending vector in a two-level tree: one bit of a
LEAF register, summarized two leaves at a time by one bit of the TOP
register. A vector's number fixes its position in that tree:
leaf = vector / 32
bit = vector % 32
subtree = leaf / 2
A tree implements either 8 or 16 leaves, depending on the GPU family.
The leaf count sets both the number of subtrees and the highest vector
that the tree carries.
Without distinct types, a vector, a leaf index, a set of vectors within
one leaf, a subtree and a set of subtrees are all plain integers. A
caller can pass one where another belongs, and a register field can
accept the wrong one.
Add a type for each of those, and for the leaf count. A vector converts
to its own leaf, bit and subtree. Its constructor rejects, at build
time, a number beyond the widest supported tree, and a validation
method rejects, at run time, a number beyond the leaves that the
current tree implements. A leaf count yields the set of subtrees that
it implements.
The module has no user yet.
Suggested-by: Danilo Krummrich <dakr@kernel.org>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260930034148.590687-4-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
This is a high value thing to print, because it acts as kind of a
bookend, showing the beginning of trying to boot up.
Cc: Danilo Krummrich <dakr@kernel.org>
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Link: https://patch.msgid.link/20260929210749.409380-1-jhubbard@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
|
|
Just like in Gpu::new() a single try_pin_init!() block makes the code
more readable.
Besides that, it prepares the code for a proper PCI device enable guard
that we will get soon.
Reviewed-by: Eliot Courtney <ecourtney@nvidia.com>
Acked-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260927144417.1280385-1-dakr@kernel.org
[ Drop useless dev_dbg!() in probe(). - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
|
|
Add the GPU GID to the reported GPU info. This is the 16-byte SHA-1
based identifier GSP-RM reports for the GPU.
Signed-off-by: Alistair Popple <apopple@nvidia.com>
Link: https://patch.msgid.link/20260909064506.910162-14-apopple@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
|
|
Add the short GPU name to the GPU info structure. As with the full name
it is copied from a validated, NUL-terminated string extracted by
nova-core rather than from the raw firmware bytes.
Signed-off-by: Alistair Popple <apopple@nvidia.com>
Link: https://patch.msgid.link/20260909064506.910162-13-apopple@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
|
|
Add the full GPU name to the GPU info structure. The name is copied from
the validated, NUL-terminated string nova-core already extracts from the
GSP static info rather than from the raw firmware bytes.
GpuInfo::new() is deliberately fallible here. A malformed name string
from the GSP is unexpected and is reported to userspace as an error
rather than being silently replaced with an empty name.
Signed-off-by: Alistair Popple <apopple@nvidia.com>
Link: https://patch.msgid.link/20260909064506.910162-12-apopple@nvidia.com
[ Use ..pin_init::zeroed() instead of ..default::Default(). - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
|
|
Currently nova-drm reads the VRAM BAR size directly from the PCIe device
which requires trying to cast the parent device into a PCIe device. This
obviously requires the parent device to actually be a PCIe bus device.
Whilst that is true today it may not always be the case, and there
is no reason to make this assumption now that NovaCoreApi can hold a
reference to the bound PCIe device.
So convert nova-drm to using nova-core to obtain the VRAM_BAR_SIZE
parameter.
Suggested-by: Danilo Krummrich <dakr@kernel.org>
Signed-off-by: Alistair Popple <apopple@nvidia.com>
Link: https://patch.msgid.link/20260909064506.910162-10-apopple@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
|
|
Add a field to the GPU info structure containing the total usable
framebuffer size. The usable framebuffer excludes GSP carveouts and
other protected regions, so it may differ from the BAR size and total
physical VRAM.
Signed-off-by: Alistair Popple <apopple@nvidia.com>
Link: https://patch.msgid.link/20260909064506.910162-9-apopple@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
|
|
Add an extensible info ioctl and use it to report basic GPU information.
One of the first things userspace needs to know about a GPU is its
architecture and which chip it is, so add those as the first fields in
the GPU info result. The chip identifier is opaque to userspace and is
only meaningful when compared against the values of enum
drm_nova_chipid.
The ioctl selects an information type by ID and writes the result through
a sized userspace buffer. The kernel truncates results to the supplied
size, allowing information structures to grow and new logical information
groups to be added without introducing new ioctls. Passing a NULL data
pointer returns the size of the requested information structure instead.
Suggested-by: Danilo Krummrich <dakr@kernel.org>
Signed-off-by: Alistair Popple <apopple@nvidia.com>
Link: https://patch.msgid.link/20260909064506.910162-8-apopple@nvidia.com
[ Avoid 'as' casts (add From<Chipset> for u32, From<Architecture> for
u32 and use IntoSafeCast, IntoSafeCastArch); improve / simplify a few
comments. - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
|
|
The chip identifier uniquely identifies a GPU chip and needs to be
exposed to user-space so it can look up chip specific properties that
cannot be derived from the architecture alone. This adds a public enum
to the userspace headers for each chip ID. Nova-core can then use this
enum to define its chipsets.
The values are opaque to user-space. They may be compared against the
enum to identify a chip but user-space must not assume the values carry
any particular meaning or encoding.
This does create a coupling between nova-drm and nova-core whereby
nova-core depends on the values defined by the user-space API for
nova-drm. However this is entirely appropriate as nova-core must be
bound by the UAPI headers as the enum values are read by nova-core and
passed through to user-space.
Note that nova-core currently refers to the chipid as a chipset. This
isn't ideal given that chipid is the preferred and more accurate term
and is what should be exposed via the uAPI. Therefore a future patch
series will be posted that aligns nova-core internals to using chipid
rather than chipset.
Assisted-by: LLM
Signed-off-by: Alistair Popple <apopple@nvidia.com>
Link: https://patch.msgid.link/20260909064506.910162-6-apopple@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
|
|
The GPU architecture to be exposed to user-space. This adds a public
enum to the userspace headers for each chip architecture. Nova-core can
then use this enum to define its architectures.
This does create a coupling between nova-drm and nova-core whereby
nova-core depends on the values defined by the user-space API for
nova-drm. However this is entirely appropriate as nova-core must be
bound by the UAPI headers as the enum values are read by nova-core and
passed through to user-space.
It also requires a minor change to the bounded_enum! macro to match the
architecture values in an expression context.
Signed-off-by: Alistair Popple <apopple@nvidia.com>
Link: https://patch.msgid.link/20260909064506.910162-5-apopple@nvidia.com
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
|
|
Nova core will be used to export core functionality to other drivers
which will bind to it via auxiliary bus devices. Add a NovaCoreApi type
which drivers can use to call nova-core methods.
Auxiliary bus drivers obtain a NovaCoreApiHandle for a particular GPU
using NovaCoreApi::of(). This takes a reference to a bound auxiliary bus
device and checks that its registration data is a NovaCoreApi. The
NovaCoreApi itself is then accessed by passing a closure to
NovaCoreApiHandle::with().
Closure based access is used because NovaCoreApi cannot be assumed to be
covariant over its lifetime. It references the Gpu, which will carry
locks around types holding device resources, making it invariant.
Covariant sub-fields can still be returned directly from the closure
thanks to the lifetime bound on registration_data_with().
Signed-off-by: Alistair Popple <apopple@nvidia.com>
Link: https://patch.msgid.link/20260909064506.910162-3-apopple@nvidia.com
[ Rebase and use pin_chain() for Gpu::run_selftests(). - Danilo ]
Co-developed-by: Danilo Krummrich <dakr@kernel.org>
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
|
|
BarUser should not use 'gpu as a name for its lifetime; stick to a
generic 'a until we need pin-init selfref for this struct.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260926124405.604707-5-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
|
|
There is no need for a reference count here. BarUserAccess can just
borrow from BarUser.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260926124405.604707-4-dakr@kernel.org
[ Update stale comment in run_self_test(). - Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
|
|
Remove the unnecessary Arc around BarUser and get rid of
Result<impl PinInit<T>> in favor of impl PinInit<T, Error>.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260926124405.604707-3-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
|
|
Make Bar1 an alias for a borrowed pci::Bar in order to match the Bar0
type alias.
Reviewed-by: Alexandre Courbot <acourbot@nvidia.com>
Link: https://patch.msgid.link/20260926124405.604707-2-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
|
|
Sending a command to the GSP ends by writing the GSP doorbell, a BAR0
register, so every send path through the command queue needs the BAR0
mapping. A command too large for one queue element goes out as a first
element followed by continuation records, so the send path has four
functions.
Nova-core passed the mapping down from the caller as an argument, which
was threaded through all four functions, and every caller of a send had
to carry it as well.
Store the mapping in the command queue and drop the argument. The queue
lives exactly as long as the mapping does, since both belong to the
bound device.
Suggested-by: Alexandre Courbot <acourbot@nvidia.com>
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Link: https://patch.msgid.link/20260918010719.1176945-4-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
FSP, the GPU's Foundation Security Processor, receives the driver's
messages as packets of MCTP, the Management Component Transport
Protocol. It requires header version 1 in bits 3:0 of the MCTP transport
header.
Nova-core left those bits at zero, so every Chain of Trust request and
every Product Reconfiguration Control request that it sent carried an
invalid transport header.
Declare the version field and set it. Declare the destination endpoint
ID field as well. The driver leaves that field at zero.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Reviewed-by: Zhi Wang <zhiw@nvidia.com>
Link: https://patch.msgid.link/20260918010719.1176945-3-jhubbard@nvidia.com
[acourbot: explicitly set deid field]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
A TLV firmware file carries the GSP firmware image as a section
extracted from the original ELF, and the section is not itself an ELF.
Before the driver moved to TLV firmware files, it parsed that ELF
itself, and it named the framebuffer range that holds the image for the
ELF. The range kept the name through the move.
Rename the field for the framebuffer range that holds the GSP firmware
image, from elf to fw_image. After this, the following:
$ cd drivers/gpu/nova-core
$ grep -iw ELF
comes up empty. :)
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Link: https://patch.msgid.link/20260913195413.742143-4-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
On Turing through Ada, the booter, a firmware stage that runs on the
GPU's SEC2 falcon, copies the GSP firmware image from system memory into
the framebuffer. It walks the radix3 table to find each page of the
image, and it reads each level of the table by DMA, a whole 4 KiB page
at a time. The last page of a level is only partly filled with entries,
so the bytes past the last entry reach the booter too.
Nova-core allocated each level at the size of its entries and wrote only
the entries, so the rest of the level's last page held whatever the
allocator left there, and the booter read that stale kernel memory.
Allocate each level zeroed and sized to a whole number of 4 KiB pages,
as Open RM does, and write the entries into it.
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Link: https://patch.msgid.link/20260913195413.742143-3-jhubbard@nvidia.com
[acourbot: move chunks from previous commit into this one for
consistency]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
Describe the reusable buffer mapping in module documentation and update
the relevant descriptions.
[acourbot: split the original patch into logical changes.]
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Link: https://patch.msgid.link/20260913195413.742143-2-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
The extracted table builder can map buffers other than the GSP firmware
image. Rename its `fw` field and `fw_vvec` parameter to `data`, updating
the existing documentation references to match.
[acourbot: split the original patch into logical changes.]
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Link: https://patch.msgid.link/20260913195413.742143-2-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
In order to boot the GSP, nova-core loads the GSP firmware image into
system memory, and the GPU copies it into the framebuffer, the GPU's own
memory. The GPU finds the GSP firmware image's pages via "radix3", a
three-level page table of 4 KiB pages that the driver builds. Today the
table exists as fields within the GSP firmware wrapper, which is fine
because there is only one radix3 consumer.
The upcoming r000 series[1], however, introduces a second radix3 caller:
the ucodes firmware image. That is because the GSP has been changed, so
that it automatically fetches the subset of ucodes that it requires,
thus reducing the size of the GSP image. This is a helpful change for
the firmware situation overall, but it will require nova-core to load
and provide ucodes, in much the same way that it was loading and
providing the GSP firmware image.
Therefore, split out radix3 into its own module.
[1] https://lore.kernel.org/all/20260822015448.238214-1-jhubbard@nvidia.com/
[acourbot: keep only the mechanical move, split other changes into
distinct commits]
Assisted-by: LLM
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
Reviewed-by: Timur Tabi <ttabi@nvidia.com>
Link: https://patch.msgid.link/20260913195413.742143-2-jhubbard@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
The PCI default max_seg_size is 64 KiB. nova-core never raises it, so
mapping the GSP firmware image (~60 MiB) as a scatter-gather table
triggers a DMA-API debug warning when contiguous pages coalesce into
segments that exceed the default:
DMA-API: nova-core 0000:00:03.0: mapping sg segment longer than
device claims to support [len=62914560] [max=65536]
CONFIG_DMA_API_DEBUG=y is required to see this warning.
Assisted-by: LLM
Signed-off-by: Matteo Kloiber <kernel@matt3o12.de>
Link: https://patch.msgid.link/20260913210828.125655-2-kernel@matt3o12.de
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
Add self-tests for BAR1 access during driver probe when
CONFIG_NOVA_CORE_SELFTESTS is enabled (default disabled). This results in
testing the Vmm, GPU buddy allocator and BAR1 region all of which should
function correctly for the tests to pass.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
[ecourtney: use existing self-test runner and CONFIG_NOVA_CORE_SELFTESTS]
[ecourtney: update for mutable GpuMm, PRAMIN views and VramAddress raw API]
[ecourtney: restore the conditional dead_code expect]
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260909-mmrebase-v1-16-8dd5d4225d2e@nvidia.com
[acourbot: use vertical import style.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
Add the BAR1 user interface for CPU access to GPU virtual memory through
the BAR1 aperture.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
[ecourtney: map BAR1 in NovaCore, borrow it in Gpu and BarUser, no Devres]
[ecourtney: update for the VramAddress raw API and gsp_resources chipset]
[ecourtney: drop the owned GpuMm, pass it mutably through map and release]
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260909-mmrebase-v1-15-8dd5d4225d2e@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
Add BAR1_SIZE constant and Bar1 type alias for the 256MB BAR1 aperture.
These are prerequisites for BAR1 memory access functionality.
Co-developed-by: Zhi Wang <zhiw@nvidia.com>
Signed-off-by: Zhi Wang <zhiw@nvidia.com>
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
[ecourtney: add the Bar1 lifetime for current PCI BAR ownership]
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260909-mmrebase-v1-14-8dd5d4225d2e@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
Add the page table mapping and unmapping API to the Virtual Memory
Manager, implementing a two-phase prepare/execute model suitable for
use both inside and outside the DMA fence signalling critical path.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
[ecourtney: pass mutable GpuMm, replace window guards with scoped borrows]
[ecourtney: use current raw address helpers, drop stale expects]
[ecourtney: zero page table pages through one PRAMIN view]
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260909-mmrebase-v1-13-8dd5d4225d2e@nvidia.com
[acourbot: use vertical import style.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
Add virtual address range tracking to the VMM using a maple tree
allocator. This enables contiguous virtual address range allocation
for mappings.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260909-mmrebase-v1-12-8dd5d4225d2e@nvidia.com
[acourbot: use vertical import style.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
Add the Virtual Memory Manager (VMM) infrastructure for GPU address
space management. Each Vmm instance manages a single address space
identified by its Page Directory Base (PDB) address, used for Channel,
BAR1 and BAR2 mappings.
Mapping APIs and virtual address range tracking are added in later
commits.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
[ecourtney: pass mutable GpuMm to mapping lookup]
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260909-mmrebase-v1-11-8dd5d4225d2e@nvidia.com
[acourbot: use vertical import style.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
Add the page table walker implementation that traverses the page table
hierarchy for both MMU v2 (5-level) and MMU v3 (6-level) to resolve
virtual addresses to physical addresses or find PTE locations.
Currently only v2 has been tested (nova-core currently boots pre-hopper)
with some initial preparatory work done for v3.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
[ecourtney: take mutable GpuMm and Pramin, not a bound device and window]
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260909-mmrebase-v1-10-8dd5d4225d2e@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
Introduce `MmuConfig`, the trait that ties the entry-operation traits
(`PteOps`, `PdeOps`, `DualPdeOps`) together with the version-specific
constants and helpers.
`MmuV2` and `MmuV3` are zero-sized marker structs that implement
`MmuConfig` for Turing/Ampere/Ada and Hopper/Blackwell respectively.
Dispatch is fully resolved at compile time through these markers, so
version-specific code is selected without runtime overhead and without
wrapper enums.
This enables version-agnostic page-table operations while keeping
version-specific implementation details encapsulated in the `ver2` and
`ver3` modules.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260909-mmrebase-v1-9-8dd5d4225d2e@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
Add page table entry and directory structures for MMU version 3 used by
Hopper and later GPUs. The `Pte`, `Pde`, and `DualPde` types each
implement the `PteOps`, `PdeOps`, and `DualPdeOps` traits introduced
earlier in the series, providing the version-agnostic API used by the
forthcoming page-table walker and mapper.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
[ecourtney: update for the VramAddress raw API, apply rustfmt]
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260909-mmrebase-v1-8-8dd5d4225d2e@nvidia.com
[acourbot: use vertical import style.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
Add page table entry and directory structures for MMU version 2 used by
Hopper and later GPUs. The `Pte`, `Pde`, and `DualPde` types each
implement the `PteOps`, `PdeOps`, and `DualPdeOps` traits introduced
earlier in the series, providing the version-agnostic API used by the
forthcoming page-table walker and mapper.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260909-mmrebase-v1-7-8dd5d4225d2e@nvidia.com
[acourbot: use vertical import style.]
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
|
Introduce a trait for 128-bit Dual Page Directory Entries. The
`read()`/`write()` helpers issue two 64-bit accesses through a
`Pramin` to load/store the 128-bit value.
Signed-off-by: Joel Fernandes <joelagnelf@nvidia.com>
[ecourtney: replace PraminWindow accesses with typed MMIO views]
Signed-off-by: Eliot Courtney <ecourtney@nvidia.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260909-mmrebase-v1-6-8dd5d4225d2e@nvidia.com
Signed-off-by: Alexandre Courbot <acourbot@nvidia.com>
|
