| Age | Commit message (Collapse) | Author |
|---|
|
https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git
|
|
https://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git
|
|
https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux
|
|
Cross-merge networking fixes after downstream PR (net-7.3-rc7).
Conflicts:
include/net/strparser.h
0984ebc631792 ("strparser: make sure __strp_recv isn't running before tearing down the parser")
02fd0a2111374 ("net: strparser: removed the aborted bit and aborts counter")
https://lore.kernel.org/asYfmLsfy-SCxXjr@sirena.co.uk
Adjacent changes:
Documentation/networking/strparser.rst
5f193cdabf25 ("docs: networking: strparser: remove general mode")
0984ebc63179 ("strparser: make sure __strp_recv isn't running before tearing down the parser")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Commit 730ff06d3f5c ("net: mana: Use page pool fragments for RX buffers
instead of full pages to improve memory efficiency.") started handing
out RX buffers with zero headroom so that two buffers fit into one page
at the default MTU.
The MANA TX path, however, stores the per scatter-gather entry DMA
mappings in `struct mana_skb_head` at skb->head, and mana_start_xmit()
therefore calls skb_cow_head(skb, MANA_HEADROOM). The port advertises
this requirement as ndev->needed_headroom = MANA_HEADROOM.
As a result every packet that is received and then forwarded out of a
MANA port fails the skb_cow() in ip_forward() and gets reallocated and
copied by pskb_expand_head(). This is invisible to a plain RX or TX
workload, but it puts a full skb reallocation plus memcpy on the hot
path of every single forwarded packet, which is exactly what a
router/NVA workload does.
Restore the headroom. Note that reserving MANA_HEADROOM (232) is not
enough: ip_forward() asks for LL_RESERVED_SPACE(dev), which rounds
hard_header_len + needed_headroom up to HH_DATA_MOD and is 256 bytes on
ethernet. Use LL_RESERVED_SPACE() directly so the value keeps tracking
both constants. Also, since LL_RESERVED_SPACE() tracks MANA_HEADROOM,
it grows with MAX_SKB_FRAGS and for MAX_SKB_FRAGS >= 19 it is greater
than 256, so we have to account for that by using the headroom the RX
queue actually uses (instead of assuming XDP_PACKET_HEADROOM) and
turning MANA_XDP_MTU_MAX into MANA_XDP_MTU_MAX(ndev) (note that at the
default CONFIG_MAX_SKB_FRAGS=17 they are equivalent).
At the default MTU on a 4K page this means a buffer no longer fits twice
into a page (SKB_DATA_ALIGN(1500 + MANA_RXBUF_PAD + 256) = 2112), so the
frag-vs-single decision is now made by computing the real buffer size
instead of comparing the MTU against PAGE_SIZE / 2. The page_pool
fragment path is still used wherever at least two buffers genuinely fit,
e.g. on 16K and 64K page sizes.
Measured on an Azure VM with a MANA NIC acting as a forwarding NVA (UDP,
1400 byte payload, 4 streams, 8 Gbps offered, only the forwarding
node's kernel differs), 8 runs each, median:
forwarded pps throughput
before 272,830 3.06 Gbps
after 390,560 4.37 Gbps (+43%)
perf on the forwarding node, same workload:
memset_orig __pi_memcpy pskb_expand_head
before 10.07% 3.96% present
after 0.94% 0.64% gone
Cc: stable@vger.kernel.org
Fixes: 730ff06d3f5c ("net: mana: Use page pool fragments for RX buffers instead of full pages to improve memory efficiency.")
Signed-off-by: Hamza Mahfooz <hamzamahfooz@linux.microsoft.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Link: https://patch.msgid.link/20261003013647.2051416-1-hamzamahfooz@linux.microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Add the CDX transport as mana_cdx.ko for device ID 0x00C2, using the shared
gdma_core.ko. Allocate interrupts at probe, limit queues to available
vectors, use 32-bit DMA, and keep CDX auxiliary devices separate from PCI.
Signed-off-by: Manish Awasthi <mawasthi@linux.microsoft.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
Link: https://patch.msgid.link/20260924173054.589291-5-mawasthi@linux.microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Build the shared GDMA and Ethernet code as gdma_core.ko, selected by
MICROSOFT_GDMA_CORE. Build the PCI transport separately as mana.ko and
export the required core symbols.
Preserve the PCI module name, device table, and mana.eth/mana.rdma
auxiliary-device names. Each transport registers its own bus driver.
Signed-off-by: Manish Awasthi <mawasthi@linux.microsoft.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
Link: https://patch.msgid.link/20260924173054.589291-4-mawasthi@linux.microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Move PCI transport code and driver callbacks from gdma_main.c into
gdma_pci.c. Keep shared IRQ bookkeeping in the core and update build
wiring.
Signed-off-by: Manish Awasthi <mawasthi@linux.microsoft.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
Link: https://patch.msgid.link/20260924173054.589291-3-mawasthi@linux.microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Introduce per-device gdma_bus_ops for interrupt allocation, IRQ lookup,
reset, servicing, and bus-specific capabilities. Route the shared GDMA
code through these callbacks and provide the PCI implementation.
Move common device bring-up and teardown into mana_gd_setup() and
mana_gd_cleanup(), including hardware-channel initialization, device
enumeration, interrupt-pool sizing, and servicing workqueue management.
Record the vPort count for transport-specific queue sizing and keep shared
IRQ context bookkeeping in the core.
This prepares the driver for additional bus transports without duplicating
the core or adding configuration checks throughout it.
Signed-off-by: Manish Awasthi <mawasthi@linux.microsoft.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
Link: https://patch.msgid.link/20260924173054.589291-2-mawasthi@linux.microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Preparatory patch to support SRQ in RDMA ionic driver.
Signed-off-by: Leon Romanovsky <leon@kernel.org>
* ionic/ionic-shared: (1790 commits)
net: ionic: Fetch qid allocation and SRQ capability from firmware
Linux 7.3-rc4
...
|
|
Build WCs directly from CQEs and their corresponding published shadow
entries. This avoids iterating over all QPs that share a CQ. Track only
QPs in the error state instead of every QP associated with the CQ.
Keep the explicit GSI send-drain path responsible for adding the QP to
the send error list and invoking its CQ handler. To invoke the handler
for WQEs posted after GSI draining, check the QP state after posting
WQEs. This is required because MANA hardware ignores doorbells for QPs
in the error state.
Introduce a WC-budgeted polling context that caches CQEs representing
multiple WCs.
Before consuming a shadow entry, match the CQE against the low 24 bits
of the posted WQE offset to filter out stale CQEs. Implement WR signaling
to skip WC generation for unsignaled WQEs.
Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Link: https://patch.msgid.link/20260923131155.4055875-6-kotaranov@linux.microsoft.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Track CQ polling credits and ring the doorbell promptly to advance the
hardware CQ tail pointer. The distance between the new and previously
armed tails must remain below four times the CQ length.
Expose CQ arming to mana_ib through mana_gd_wq_ring_doorbell_ext().
RC QPs will also use this helper to ring special doorbell offsets.
MANA hardware ignores duplicate doorbells. Calculate the arming offset
from the previously armed offset and poll_credit to avoid duplicates.
Do not arm the CQ if no CQE was produced, because the CQ remains armed.
Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Link: https://patch.msgid.link/20260923131155.4055875-5-kotaranov@linux.microsoft.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Decouple RQ WQE posting from the QP structure so it can be reused by
other QP types. Batch multiple receive WQEs under a single doorbell.
Introduce GDMA_WR_IB_SGL to pass SGEs directly to GDMA instead of copying
them into a small stack array. Support receive WQEs with num_sge set to
zero.
Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Link: https://patch.msgid.link/20260923131155.4055875-4-kotaranov@linux.microsoft.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Now that arm64 no longer supports a big-endian kernel, simplify some
driver Kconfig entries so that the dependency on a little-endian kernel
no longer needs to be called out explicitly.
Reviewed-by: Michael Kelley <mhklinux@outlook.com> # Hyper-V
Suggested-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Will Deacon <will@kernel.org>
Acked-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
|
|
The hardware provides valid HWC destination vRQ and vRCQ IDs for all PFs
and VFs. Use these IDs unconditionally in mana_hwc_send_request(), as
the PF-specific gate is not required.
Signed-off-by: Manish Awasthi <mawasthi@linux.microsoft.com>
Reviewed-by: Long Li <longli@microsoft.com>
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
Link: https://patch.msgid.link/20260910014427.54272-1-mawasthi@linux.microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Cross-merge networking fixes after downstream PR (net-7.3-rc3).
Conflicts:
drivers/net/dsa/mt7530.c
3c18e3c9a54e ("net: dsa: mt7530: populate lpi_interfaces to fix EEE support")
10d9d8328e8a ("net: dsa: mt7530: replace mt7530_read with regmap_read")
Adjacent changes:
drivers/net/bonding/bond_alb.c
1746ef2e2df2 ("bonding: use skb_cow_head() in bond_do_alb_xmit() and rlb_arp_xmit()")
4cef95f72bbd ("bonding: fix u32 overflow in compute_gap()")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net
Pull networking fixes from Jakub Kicinski:
"Nothing too exciting, usual stream of fixes. Including fixes from
Netfilter, Bluetooth and WPAN.
Current release - new code bugs:
- Bluetooth: hci_sync: fix not setting CE length properly
- eth: enic: match mailbox replies to request numbers
Previous releases - regressions:
- tunnels: drop stale dst when building an ICMP error for PMTUD
- ipv6: null-check fib6_node before accessing in __ip6_del_rt_siblings()
(bug in the rtnl_lock -> RCU conversion)
- eth: bnxt_en:
- fix crashes on Thor2 due to OOB coalescing buffer accesses
- prevent queue stop with deferred completions
Previous releases - always broken:
- eth:
- ice: don't dereference pointers from TP_printk()
- fix OOB writes on ethtool flow rule dump in 3 drivers
- mlx5: fix FEC configuration with RS_544_514_INTERLEAVED_QUAD
- dsa: tag_brcm: legacy FCS: request needed tailroom
Misc:
- net: cap tx_queue_len at S16_MAX to prevent oversized ring alloc
- ipv6: flowlabel: cap duplicate leases per socket"
* tag 'net-7.3-rc3' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net: (164 commits)
selftests: tc-testing: test action batch failure cleanup
net/sched: act_api: release all action references on NEWACTION failure
openvswitch: fix wrong flag value in get_ipv6_ext_hdrs()
ipmr: account multicast table and route memory
net: phy: dp83td510: handle the active-high LED polarity mode
net: macb: initialize PTP state before registering clock
net: hsr: enable promiscuous mode on interlink port with fwd offload
ipv6: fix fib6 walker UAF on seq stop
net: stmmac: fix TX descriptor availability check for TSO traffic
net/rds: fix tcp stream corruption with large pages
net: mana: restore the XDP program pointer when pre-allocation fails
net: phy: dp83867: handle the active-high LED polarity mode
octeontx2-af: fix PF/CGX debugfs PCI bus lookup
net: net_failover: Fix the deadlock in net_failover_slave_name_change()
net: phy: mediatek-ge: disable EEE on the MT7530 PHY
tcp: reject non zerocopy devmem tx
net: ethernet: mtk_eth_soc: populate lpi_interfaces to fix EEE support
net: dsa: mt7530: populate lpi_interfaces to fix EEE support
net: hinic: fix mailbox segment buffer overflow
net: sun4i-emac: fix missing of_node_put() for phy_node
...
|
|
mana_xdp_set() publishes the new program into apc->bpf_prog before it
allocates anything, because mana_pre_alloc_rxbufs() sizes the buffers
from it via mana_get_rxbuf_cfg(). When that allocation fails the
function returns the error directly, skipping the err_dealloc_rxbuffs
label which is the only place that restores the previous pointer.
The attach is reported as failed, so the BPF core drops the reference it
held for the caller and the program can be freed, while apc->bpf_prog
still points at it. The next consumer of mana_xdp_get() - typically
mana_chn_setxdp() from mana_alloc_queues() on the following ifup, or
after a TX timeout reset - then calls bpf_prog_add() on freed memory.
This is reachable from an ordinary "ip link set dev ethX xdp obj ..."
whenever the per-queue RX buffer pre-allocation cannot be satisfied.
Restore the previous program on that error path.
Fixes: 730ff06d3f5c ("net: mana: Use page pool fragments for RX buffers instead of full pages to improve memory efficiency.")
Signed-off-by: Long Li <longli@microsoft.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Link: https://patch.msgid.link/20260904202640.3900685-1-longli@microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
mana_rdma_remove() sets gd->rdma_teardown to stop
mana_rdma_service_handle() from acting on servicing events, but nothing
ever clears it. A hardware service reset (GDMA_EQE_HWC_RESET_REQUEST)
goes through mana_gd_suspend() -> mana_rdma_remove() and mana_gd_resume()
-> mana_rdma_probe(), so from the first reset onwards every
GDMA_EQE_HWC_SOC_SERVICE event returns early and RDMA suspend/resume
servicing is silently dropped for the life of the device.
gd->is_suspended has the same problem: it is set when servicing removes
the adev and is cleared only by a matching resume. A reset while RDMA is
suspended re-adds the adev but leaves is_suspended set, so a later resume
event calls add_adev() on top of a live gd->adev and leaks it. This is
currently masked by the rdma_teardown bug.
Clear both in mana_rdma_probe(). On the reset path mana_rdma_remove()
has closed the gate and drained the service workqueue, so clear
is_suspended first and re-open the gate with smp_store_release(), paired
with smp_load_acquire() in the handler, so the handler cannot observe an
open gate with a stale is_suspended. On the initial probe path the gate
was never closed and both flags are already clear.
This does not order gd->adev, which add_adev() publishes afterwards. A
servicing event arriving in that window is still dropped, as it is in
mainline today on the initial probe path; closing it needs probe and the
handler to be serialized and is left to a separate change.
Fixes: 505cc26bcae0 ("net: mana: Add support for auxiliary device servicing events")
Signed-off-by: Long Li <longli@microsoft.com>
Link: https://patch.msgid.link/20260902175153.3410560-1-longli@microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
This is another run of the Coccinelle script for converting kmalloc()
family of allocations to kmalloc_obj() via the existing rules in
scripts/coccinelle/api/kmalloc_objs.cocci
This catches both the set of kmalloc() uses added since the first
kmalloc_obj() conversions in v7.0 and adds a large group missed in the
first pass due to Coccinelle not interacting well with the cleanup.h
scoped_...() family of macros[1]. I worked around this with spatch's
"--macro-file" argument to a file with all the scoped_...() macros mapped
to Coccinelle's YACFE_ITERATOR[2] as that was the closest viable control
flow indicator I could find.
Build tested allmodconfig on x86, arm64, arm, loongarch, mips, powerpc,
riscv, and s390 with no new warnings.
Link: https://lore.kernel.org/lkml/202609021314.8A9C0B8@keescook/ [1]
Link: https://github.com/coccinelle/coccinelle/blob/master/standard.h [2]
Signed-off-by: Kees Cook <kees+treewide@kernel.org>
|
|
The RX completion queue is sized to hold exactly one CQE per posted RX WQE.
MANA_FENCE_RQ makes hardware post an additional CQE_RX_OBJECT_FENCE after
the packet CQEs. The current sizing reserves no extra slot for it and in
rare cases, CQ has no guaranteed slot for the fence CQE when it is full of
packet CQEs. This can lead to dropping the fence completion while the
driver waits holding RTNL lock throughout the timeout duration.
Reserve one extra CQE slot for CQE_RX_OBJECT_FENCE. mana_gd_alloc_memory()
requires queue_size to be a power-of-two and at least MANA_PAGE_SIZE;
the reservation pushes cq_size past a power-of-two, so round up the CQ size
in mana_create_rxq().
Cc: stable@vger.kernel.org
Fixes: 6cc74443a773 ("net: mana: Add RX fencing")
Signed-off-by: Sahil Chandna <sahilchandna@linux.microsoft.com>
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
Link: https://patch.msgid.link/20260901121837.3503240-1-sahilchandna@linux.microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
mana_bpf() has a trailing 'return ret;' after a switch statement whose
every case (XDP_SETUP_PROG and default) already returns unconditionally.
The statement is therefore unreachable and 'ret' is never used.
Remove both. No functional change intended.
Signed-off-by: Muhammad Falak R Wani <falakreyaz@gmail.com>
Reviewed-by: Hamza Mahfooz <hamzamahfooz@linux.microsoft.com>
Link: https://patch.msgid.link/3d8b67307df6fefdc18743274cef3de4fca0e7de.1788339612.git.falakreyaz@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
mana_gd_query_max_resources() sizes gc->num_msix_usable from resp.max_msix
and the CPU count, but never from the device MSI-X table. On a 1792 vCPU
M-series VM that yields 1793 while the table has 1024 entries, and
mana_gd_setup_remaining_irqs() then walks indices 1..1792, running off the
end of the region mapped by msix_map_region():
BUG: unable to handle page fault for address: ff8e347f8b99800c
RIP: 0010:msix_prepare_msi_desc+0x7a/0x90
RAX: 0000000000004000 RBX: ff4330cb164ea780 RCX: ff8e347f8b998000
Call Trace:
<TASK>
__msi_domain_alloc_irqs+0x13a/0x440
msi_domain_alloc_irq_at+0x149/0x1b0
mana_gd_setup+0x351/0x890
mana_gd_probe+0x274/0x390
</TASK>
RAX is index 1024 * PCI_MSIX_ENTRY_SIZE, one entry past the table.
msi_insert_desc() does range check the index, but only against the MSI
domain hwsize, which matches the table only for devices on an MSI parent
domain. With a global PCI/MSI domain hwsize is MSI_XA_DOMAIN_SIZE, so
nothing bounds the request.
Cap num_msix_usable with pci_msix_vec_count().
Fixes: 755391121038 ("net: mana: Allocate MSI-X vectors dynamically")
Signed-off-by: Long Li <longli@microsoft.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Link: https://patch.msgid.link/20260821183736.733296-1-longli@microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Each GDMA queue ring is one dma_alloc_coherent() of the whole ring size.
Such high-order allocations fail first under memory fragmentation, so
queue setup can fail with memory still free.
The hardware does not need the ring physically contiguous:
mana_gd_create_dma_region() already maps it as a list of MANA_PAGE_SIZE
(4K) device addresses. Only the driver's linear CPU view needs
contiguity, and it goes through mana_gd_ring_ptr() and
mana_gd_ring_contig_avail(); change both to map offsets onto
scattered pages.
Add a fallback in mana_gd_alloc_memory(): data-path queues pass
allow_scatter=true, so when the contiguous allocation fails the ring is
backed by a vector of scattered PAGE_SIZE (order-0) coherent pages,
presenting the same DMA page-list layout to the device. The HW channel
bootstrap keeps allow_scatter=false, and the debugfs ring dumper reads
scattered rings through the same helpers.
Signed-off-by: Aditya Garg <gargaditya@linux.microsoft.com>
Link: https://patch.msgid.link/20260807210002.1695263-3-gargaditya@linux.microsoft.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
|
|
In preparation for backing GDMA queue memory with a vector of
non-contiguous order-0 coherent pages, route CPU access to a queue's
ring buffer through two new helpers: mana_gd_ring_ptr() returns the CPU
address of a byte offset into the ring, and mana_gd_ring_contig_avail()
the number of bytes left before the ring wraps, so a WQ write that runs
past the end of the ring can be split at that point.
Convert the EQ, CQ and work-request paths to use them.
mana_gd_write_sgl() now takes a byte offset rather than a raw pointer,
so mana_gd_post_work_request() derives the SGL position arithmetically.
While queue memory is contiguous both helpers are simple arithmetic on
the ring base and size, so there is no functional change.
Signed-off-by: Aditya Garg <gargaditya@linux.microsoft.com>
Link: https://patch.msgid.link/20260807210002.1695263-2-gargaditya@linux.microsoft.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
|
|
To support up to 8 packets per CQE, update related CQE processing
code and structures.
Update ethtool handlers to set this feature.
Update per queue stat to show the coalesced CQE counters.
This feature is supported on NIC hardware showing the relevant
PF flag.
Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Reviewed-by: Breno Leitao <leitao@debian.org>
Link: https://patch.msgid.link/20260805185404.1052177-1-haiyangz@linux.microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
On some ARM64 platforms with 4K PAGE_SIZE, page_pool fragment
allocation in the RX refill path can cause 15-20% throughput
regression under high connection counts (>16 TCP streams).
Add an ethtool private flag "full-page-rx" that allows the user to
force one RX buffer per page, bypassing the page_pool fragment path.
This restores line-rate (180+ Gbps) performance on affected platforms.
Usage:
ethtool --set-priv-flags eth0 full-page-rx on
There is no behavioral change by default. The flag must be explicitly
enabled by the user or udev rule.
The existing single-buffer-per-page logic for XDP and jumbo frames is
consolidated into a new helper mana_use_single_rxbuf_per_page() which
is now the single decision point for both the automatic and
user-controlled paths.
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
Signed-off-by: Dipayaan Roy <dipayanroy@linux.microsoft.com>
Link: https://patch.msgid.link/20260729063347.3388035-3-dipayanroy@linux.microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Refactor mana_get_strings() and mana_get_sset_count() from if/else to
switch statements in preparation for adding ethtool private flags
support which requires handling ETH_SS_PRIV_FLAGS.
No functional change.
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
Signed-off-by: Dipayaan Roy <dipayanroy@linux.microsoft.com>
Link: https://patch.msgid.link/20260729063347.3388035-2-dipayanroy@linux.microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Cross-merge networking fixes after downstream PR (net-7.2-rc6).
No conflicts.
Adjacent changes:
net/ipv4/route.c
dbc3791e3b24 ("net: do not send ICMP/NDISC Redirects when peer allocation fails")
7804eaa057fe ("ipv4: snapshot dst.dev in ip_rt_send_redirect() and ip_rt_get_source()")
drivers/net/tun.c
23dad2d088df ("tun: no longer rely on RTNL in tun_fill_info()")
c3da92af07ea ("Revert "tun/tap: add ptr_ring consume helper with netdev queue wakeup"")
drivers/net/ethernet/marvell/octeontx2/af/rvu_npc.c
3bd438a58e91 ("octeontx2-af: Block VFs from clobbering special CGX PKIND state")
5ba5611ef946 ("octeontx2-af: reserve 4 PKINDs for skip-size custom use")
drivers/net/wireless/ath/ath12k/core.h
drivers/net/wireless/ath/ath12k/mac.c
drivers/net/wireless/ath/ath12k/peer.c
469d7e6077c1 ("wifi: ath12k: resolve PENDING ML peer ID from MLO_PEER_MAP HTT event")
378e659029d5 ("wifi: ath12k: introduce host_alloc_ml_id hardware parameter")
c42b27336eef ("wifi: ath12k: fix survey indexing across bands")
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
mana_create_rxq() returns a struct mana_rxq pointer and returns NULL on
any failure. The caller, mana_add_rx_queues(), cannot tell what went
wrong and hardcodes the error as -ENOMEM. As a result the actual failure
reported by the lower layers (for example -EPROTO from a failed HW
request) is masked and every RX queue creation failure looks like an
out-of-memory error.
Return an ERR_PTR() encoded error code from mana_create_rxq() on failure
instead of NULL. The caller now propagates the returned error code
directly instead of substituting -ENOMEM.
Fixes: ca9c54d2d6a5 ("net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)")
Signed-off-by: Aditya Garg <gargaditya@linux.microsoft.com>
Reviewed-by: Joe Damato <joe@dama.to>
Link: https://patch.msgid.link/20260727113759.2881500-1-gargaditya@linux.microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Add callback function for the pci_driver / sriov_configure.
It asks the NIC to provide certain number of VFs, or disable
VFs if the request is zero.
Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
Link: https://patch.msgid.link/20260710192735.2921300-1-haiyangz@linux.microsoft.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
|
|
Add a per-port debugfs boolean "tx_timeout_skip_reset" that, when
enabled, makes mana_tx_timeout() log the TX timeout and return without
queueing the per-port detach/attach recovery work.
This is a debug-only aid for bringup and qualification: skipping the
recovery reset keeps the device and queue state intact so a TX timeout
can be correlated with hardware telemetry. The knob defaults to false,
so production recovery behaviour is unchanged.
Signed-off-by: Aditya Garg <gargaditya@linux.microsoft.com>
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
Reviewed-by: Dipayaan Roy <dipayanroy@linux.microsoft.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Link: https://patch.msgid.link/20260710132229.2851441-1-gargaditya@linux.microsoft.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
|
|
Cross-merge networking fixes after downstream PR (net-7.2-rc3).
Conflicts:
tools/testing/selftests/net/lib.sh
dd6a23bac306b ("selftests: net: make busywait timeout clock portable")
895bad9cc4cec ("selftests: net: make busywait timeout clock portable")
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
|
|
Add Static and Dynamic Interrupt Moderation (DIM) support for
Rx and Tx.
Update queue creation procedure with new data struct with the related
settings.
Add functions to collect stat for DIM, and workers to update DIM data
and settings.
Update ethtool handler to get/set the moderation settings from a user.
To avoid detach/re-attach ops, ring DIM doorbell to change settings
at run time.
By default, adaptive-rx/tx (DIM) are enabled if supported by HW.
Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Link: https://patch.msgid.link/20260702220123.815018-1-haiyangz@linux.microsoft.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
|
|
MANA allocates RX buffers from page pool fragments when frag_count is
greater than 1. In that case the buffers remain DMA mapped by page pool
and the RX completion path does not call dma_unmap_single(). As a result,
the implicit sync-for-CPU normally performed by dma_unmap_single() is
missing before the packet data is passed to the networking stack.
This breaks RX on configurations which require explicit DMA syncing, for
example when booted with swiotlb=force.
Fix this by recording the page pool page and DMA sync offset when the RX
buffer is allocated, and syncing the received packet range for CPU access
before handing the RX buffer to the stack.
Fixes: 730ff06d3f5c ("net: mana: Use page pool fragments for RX buffers instead of full pages to improve memory efficiency.")
Cc: stable@vger.kernel.org
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
Signed-off-by: Dexuan Cui <decui@microsoft.com>
Link: https://patch.msgid.link/20260702041237.617719-3-decui@microsoft.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
|
|
Validate the packet length reported in the RX CQE before passing it
to skb processing. The CQE is supplied by the NIC device and should
not be blindly trusted.
Cc: stable@vger.kernel.org
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
Signed-off-by: Dexuan Cui <decui@microsoft.com>
Fixes: ca9c54d2d6a5 ("net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)")
Link: https://patch.msgid.link/20260702041237.617719-2-decui@microsoft.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
|
|
Breno reports following splats on mlx5:
RTNL: assertion failed at net/core/dev.c (2241)
WARNING: net/core/dev.c:2241 at netif_state_change+0xed/0x130, CPU#5: ethtool/1335
RIP: 0010:netif_state_change+0xf9/0x130
Call Trace:
<TASK>
__linkwatch_sync_dev+0xea/0x120
ethtool_op_get_link+0xe/0x20
__ethtool_get_link+0x26/0x40
linkstate_prepare_data+0x51/0x200
ethnl_default_doit+0x213/0x470
genl_family_rcv_msg_doit+0xdd/0x110
Looks like I missed ethtool_op_get_link() trying to sync linkwatch,
which needs rtnl_lock. Not all drivers do this - bnxt doesn't,
it just returns the link state, so add an opt-in bit.
Reported-by: Breno Leitao <leitao@debian.org>
Fixes: 45079e00133e ("net: ethtool: optionally skip rtnl_lock on Netlink path for GET ops")
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Reviewed-by: Breno Leitao <leitao@debian.org>
Acked-by: Harshitha Ramamurthy <hramamurthy@google.com>
Link: https://patch.msgid.link/20260624190439.2521219-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Before the commit 755391121038 ("net: mana: Allocate MSI-X vectors
dynamically"), all the MANA IRQs were assigned statically and together
during early driver load.
After this commit, the IRQ allocation for MANA was done in two phases.
HWC IRQ allocated earlier and then, queue IRQs dynamically added at a
later point. By this time, the IRQ weights on vCPUs can become imbalanced
and if IRQ count is greater than the vCPU count the topology aware IRQ
distribution logic in MANA can cause multiple MANA IRQs to land on the
same vCPUs, while other sibling vCPUs have none (case 1).
On SMP enabled, low-vCPU systems, this becomes a bigger problem as the
softIRQ handling overhead of two IRQs on the same vCPUs becomes much more
than their overheads if they were spread across sibling vCPUs.
In such cases when many parallel TCP connections are tested, the
throughput drops significantly.
Fix the affinity assignment logic, in cases where the IRQ count is greater
than the vCPU count and when IRQs are added dynamically, by utilizing all
the vCPUs irrespective of their NUMA/core bindings (case 2).
The results of setting the affinity and hint to NULL were also studied,
and we observed that, with this logic if there are pre-existing IRQs
allocated on the VM (apart from MANA), during MANA IRQs allocation, it
leads to clustering of the MANA queue IRQs again (case 3).
=======================================================
Case 1: without this patch
=======================================================
4 vcpu(2 cores), 5 MANA IRQs (1 HWC + 4 Queue)
TYPE effective vCPU aff
=======================================================
IRQ0: HWC 0
IRQ1: mana_q1 0
IRQ2: mana_q2 2
IRQ3: mana_q3 0
IRQ4: mana_q4 3
%soft on each vCPU(mpstat -P ALL 1) on receiver
vCPU 0 1 2 3
=======================================================
pass 1: 38.85 0.03 24.89 24.65
pass 2: 39.15 0.03 24.57 25.28
pass 3: 40.36 0.03 23.20 23.17
=======================================================
Case 2: with this patch
=======================================================
4 vcpu(2 cores), 5 MANA IRQs (1 HWC + 4 Queue)
TYPE effective vCPU aff
=======================================================
IRQ0: HWC 0
IRQ1: mana_q1 0
IRQ2: mana_q2 1
IRQ3: mana_q3 2
IRQ4: mana_q4 3
%soft on each vCPU(mpstat -P ALL 1) on receiver
vCPU 0 1 2 3
=======================================================
pass 1: 15.42 15.85 14.99 14.51
pass 2: 15.53 15.94 15.81 15.93
pass 3: 16.41 16.35 16.40 16.36
=======================================================
Case 3: with affinity set to NULL
=======================================================
4 vCPU(2 cores), 5 MANA IRQs (1 HWC + 4 Queue)
TYPE effective vCPU aff
=======================================================
IRQ0: HWC 0
IRQ1: mana_q1 2
IRQ2: mana_q2 3
IRQ3: mana_q3 2
IRQ4: mana_q4 3
=======================================================
Throughput Impact(in Gbps, same env)
=======================================================
TCP conn with patch w/o patch aff NULL
20480 15.65 7.73 5.25
10240 15.63 8.93 5.77
8192 15.64 9.69 7.16
6144 15.64 13.16 9.33
4096 15.69 15.75 13.50
2048 15.69 15.83 13.61
1024 15.71 15.28 13.60
Fixes: 755391121038 ("net: mana: Allocate MSI-X vectors dynamically")
Cc: stable@vger.kernel.org
Co-developed-by: Erni Sri Satya Vennela <ernis@linux.microsoft.com>
Signed-off-by: Erni Sri Satya Vennela <ernis@linux.microsoft.com>
Signed-off-by: Shradha Gupta <shradhagupta@linux.microsoft.com>
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Reviewed-by: Yury Norov <ynorov@nvidia.com>
Link: https://patch.msgid.link/20260624072138.1632849-1-shradhagupta@linux.microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Commit d7709812e13d ("net: mana: hardening: Validate adapter_mtu from
MANA_QUERY_DEV_CONFIG") rejected any adapter_mtu value smaller than
ETH_MIN_MTU + ETH_HLEN, including 0, returning -EPROTO and failing
mana_probe().
Some older PF firmware versions still in the field report
adapter_mtu as 0 in the MANA_QUERY_DEV_CONFIG response. With the
hardening check in place, the MANA VF driver now fails to load on
those hosts, breaking networking entirely for guests.
MANA hardware always supports the standard Ethernet MTU. Treat a
reported adapter_mtu of 0 as "the PF did not advertise a value" and
fall back to ETH_FRAME_LEN, the same value used for the pre-V2
message version path. Only jumbo frames remain unavailable until
the PF reports a valid MTU.
Other small-but-nonzero bogus values are still rejected, preserving
the original protection against the unsigned-subtraction wrap that
would otherwise let ndev->max_mtu underflow to a huge value.
Fixes: d7709812e13d ("net: mana: hardening: Validate adapter_mtu from MANA_QUERY_DEV_CONFIG")
Signed-off-by: Erni Sri Satya Vennela <ernis@linux.microsoft.com>
Reviewed-by: Simon Horman <horms@kernel.org>
Link: https://patch.msgid.link/20260619055348.467224-1-ernis@linux.microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Merge in late fixes in preparation for the net-next PR.
Conflicts:
net/tls/tls_sw.c
406e8a651a7b ("net: skmsg: preserve sg.copy across SG transforms")
79511603a65b ("tls: remove dead sockmap (psock) handling from the SW path")
drivers/net/ethernet/microsoft/mana/mana_en.c
f8fd56977eeea ("net: mana: guard TX wq object destroy with INVALID_MANA_HANDLE check")
d07efe5a6e641 ("net: mana: Use per-queue allocation for tx_qp to reduce allocation size")
https://lore.kernel.org/ajAPXu-C_PuTgV-a@sirena.org.uk
No adjacent changes.
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
mana_create_txq() has several error paths (after mana_alloc_queues() or
mana_create_wq_obj() failure) where tx_qp[i].tx_object stays as the
INVALID_MANA_HANDLE sentinel set at allocation. mana_destroy_txq() then
unconditionally calls mana_destroy_wq_obj() with (u64)-1, which firmware
rejects and logs an error.
Mirror the RX-side pattern in mana_destroy_rxq() and skip the destroy
when the handle is still INVALID_MANA_HANDLE.
Fixes: ca9c54d2d6a5 ("net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)")
Signed-off-by: Aditya Garg <gargaditya@linux.microsoft.com>
Reviewed-by: Dipayaan Roy <dipayanroy@linux.microsoft.com>
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
Link: https://patch.msgid.link/20260608101345.2267320-3-gargaditya@linux.microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
mana_gd_create_mana_wq_cq() leaves queue->id as 0 (from kzalloc_obj())
until mana_create_wq_obj() assigns the firmware-returned id. If creation
fails before that, cleanup calls mana_gd_destroy_cq() with id 0, NULLing
gc->cq_table[0] and silently breaking whichever real CQ owns that slot.
Initialize queue->id to INVALID_QUEUE_ID right after allocation, matching
mana_gd_create_eq(). The existing (id >= max_num_cqs) guard then
short-circuits cleanly.
Fixes: ca9c54d2d6a5 ("net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)")
Signed-off-by: Aditya Garg <gargaditya@linux.microsoft.com>
Reviewed-by: Dipayaan Roy <dipayanroy@linux.microsoft.com>
Reviewed-by: Haiyang Zhang <haiyangz@microsoft.com>
Link: https://patch.msgid.link/20260608101345.2267320-2-gargaditya@linux.microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
mana_query_link_cfg() sends an HWC command to firmware on every call,
but the link speed and QoS values it returns only change when the
driver explicitly calls mana_set_bw_clamp(). This function is called
not only by userspace via ethtool get_link_ksettings, but also
periodically by hv_netvsc through netvsc_get_link_ksettings and by
the sysfs speed_show attribute via dev_attr_show, resulting in
unnecessary HWC traffic every few minutes.
Add a link_cfg_error field to mana_port_context to cache the query
result. The field uses three states: 1 (not yet queried, initial
value set during mana_probe_port), 0 (success, speed/max_speed are
valid), or a negative errno for permanent errors like -EOPNOTSUPP
when the hardware does not support the command. Transient errors and
qos_unconfigured responses are not cached so that subsequent calls
will retry.
MANA is ops-locked because it implements net_shaper_ops, so the core
already takes netdev_lock() around all ethtool_ops and net_shaper_ops
entry points. Reuse that lock to serialize mana_query_link_cfg() and
mana_set_bw_clamp(). This prevents a concurrent mana_set_bw_clamp()
from racing with an in-flight query and publishing stale pre-clamp
speed/max_speed.
Invalidate the cache inside mana_set_bw_clamp() on success, so all
current and future callers that change the link configuration
automatically trigger a fresh query on the next mana_query_link_cfg()
call. Also reset link_cfg_error during resume in mana_probe() under
netdev_lock(), so that any query already in flight cannot later
store 0 and silently overwrite the post-resume invalidation.
Signed-off-by: Erni Sri Satya Vennela <ernis@linux.microsoft.com>
Link: https://patch.msgid.link/20260606133301.2180073-1-ernis@linux.microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Update the device id table to include the new device id 0x00C1.
This device's BAR layout is similar to VF's, update the function,
mana_gd_init_registers(), accordingly.
Signed-off-by: Haiyang Zhang <haiyangz@microsoft.com>
Link: https://patch.msgid.link/20260605212302.2135499-1-haiyangz@linux.microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Use GIC functions to create a dedicated interrupt context or acquire a
shared interrupt context for each EQ when setting up a vPort.
The caller now owns the GIC reference across the EQ create/destroy
lifecycle: mana_create_eq() calls mana_gd_get_gic() before creating
each EQ and mana_destroy_eq() calls mana_gd_put_gic() after destroying
it. The msix_index invalidation is moved from mana_gd_deregister_irq()
to the mana_gd_create_eq() error path so that mana_destroy_eq() can
read the index before teardown.
Signed-off-by: Long Li <longli@microsoft.com>
Link: https://patch.msgid.link/20260605005717.2059954-6-longli@microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Replace the GDMA global interrupt setup code with the new GIC allocation
and release functions for managing interrupt contexts.
This changes the per-queue interrupt names in /proc/interrupts from
mana_q0, mana_q1, ... to mana_msi1, mana_msi2, ... to reflect the
MSI-X index rather than a zero-based queue number. The HWC interrupt
name (mana_hwc) is unchanged.
Signed-off-by: Long Li <longli@microsoft.com>
Link: https://patch.msgid.link/20260605005717.2059954-5-longli@microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
To allow Ethernet EQs to use dedicated or shared MSI-X vectors and RDMA
EQs to share the same MSI-X, introduce a GIC (GDMA IRQ Context) with
reference counting. This allows the driver to create an interrupt context
on an assigned or unassigned MSI-X vector and share it across multiple
EQ consumers.
Signed-off-by: Long Li <longli@microsoft.com>
Link: https://patch.msgid.link/20260605005717.2059954-4-longli@microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
When querying the device, adjust the max number of queues to allow
dedicated MSI-X vectors for each vPort. The per-vPort queue count is
clamped towards MANA_DEF_NUM_QUEUES but will not exceed the hardware
maximum reported by the device.
MSI-X sharing among vPorts is enabled when there are not enough MSI-X
vectors for dedicated allocation, or when the platform does not support
dynamic MSI-X allocation (in which case all vectors are pre-allocated
at probe time and sharing is always used). The msi_sharing flag is
reset at the top of mana_gd_query_max_resources() so it is recomputed
from current hardware state on each probe or resume cycle.
Clamp apc->max_queues to gc->max_num_queues_vport in mana_init_port()
so that on resume, if max_num_queues_vport has decreased due to fewer
MSI-X vectors, num_queues is reduced accordingly before EQ allocation.
A device reporting zero ports now results in a fatal probe error since
the per-vPort MSI-X math requires at least one port.
Rename mana_query_device_cfg() to mana_gd_query_device_cfg() as it is
used at GDMA device probe time for querying device capabilities.
Signed-off-by: Long Li <longli@microsoft.com>
Link: https://patch.msgid.link/20260605005717.2059954-3-longli@microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
To prepare for assigning vPorts to dedicated MSI-X vectors, remove EQ
sharing among the vPorts and create dedicated EQs for each vPort.
Move the EQ definition from struct mana_context to struct mana_port_context
and update related support functions. Export mana_create_eq() and
mana_destroy_eq() for use by the MANA RDMA driver.
RSS QPs now take a vport reference via pd->vport_use_count to ensure
EQs outlive all QP consumers. The vport must already be configured by
a raw QP before an RSS QP can be created. EQs are only destroyed when
the last QP (raw or RSS) on the PD releases its reference.
Restrict each vport to a single RSS QP. The hardware only supports one
steering configuration (indirection table / hash key) per vport, and
mana_disable_vport_rx() on QP destroy disables RX globally for the
vport. Previously, creating a second RSS QP would silently overwrite
the first QP's steering config and destroy would blackhole all traffic.
This is now explicitly rejected with -EBUSY. Existing applications
(DPDK being the primary RDMA consumer) always create one RSS QP per
vport, so no real-world flows are affected.
Reject cross-port PD sharing for both raw and RSS QPs. Since EQs and
vport configuration are per-port, a PD is bound to the port used by
its first raw QP. Subsequent QPs on the same PD must use the same
port or the creation fails with -EINVAL. Previously this was silently
broken: with shared EQs it appeared to work, but with per-vPort EQs
a cross-port PD would cause wrong-port EQ teardown and corruption.
DPDK creates one PD per port so no existing flows are affected.
Serialize mana_set_channels() and the async per-port queue reset
handler against RDMA vport configuration to prevent RDMA from claiming
the vport during the detach/attach window. A channel_changing flag is
set under apc->vport_mutex before detach and checked by
mana_cfg_vport() when called from the RDMA path, blocking RDMA from
grabbing the vport during the entire window. When the port is down
and RDMA already holds the vport, the channel change is rejected with
-EBUSY.
Signed-off-by: Long Li <longli@microsoft.com>
Link: https://patch.msgid.link/20260605005717.2059954-2-longli@microsoft.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
|
Make ethtool not take rtnl_lock for SET commands when operation
is performed on an ops-locked driver. cfg/cfg_pending are now
ops-locked, since only ethtool modifies them.
Some SET driver callbacks will still need rtnl_lock, most notably
those which may end up calling netdev_update_features() or the qdisc
layer (via netif_set_real_num_tx_queues()). Let drivers selectively
opt back into the rtnl_lock with a new bitfield in ops.
We need two helpers since Netlink and ioctl cmds have different
values. Keep the helpers side by side in common.h to make sure
they get updated together, even tho they will only get called
from ioctl.c and netlink.c.
SET commands which don't use ethnl_default_set_doit() are converted
by subsequent commits.
Reviewed-by: Eric Dumazet <edumazet@google.com>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Reviewed-by: Jacob Keller <jacob.e.keller@intel.com>
Link: https://patch.msgid.link/20260605002912.3456868-6-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
|
