1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
|
.. SPDX-License-Identifier: GPL-2.0
==================================
Netmem Support for Network Drivers
==================================
This document outlines the requirements for network drivers to support netmem,
an abstract memory type that enables features like device memory TCP. By
supporting netmem, drivers can work with various underlying memory types
with little to no modification.
Benefits of Netmem :
* Flexibility: Netmem can be backed by different memory types (e.g., struct
page, DMA-buf), allowing drivers to support various use cases such as device
memory TCP.
* Future-proof: Drivers with netmem support are ready for upcoming
features that rely on it.
* Simplified Development: Drivers interact with a consistent API,
regardless of the underlying memory implementation.
Design Principles
=================
Memory providers (or the default ``page_pool`` allocator) allocate underlying
memory (``struct net_iov`` or ``struct page``), cast it to ``netmem_ref``, and
supply it to ``page_pool``. The ``page_pool``, drivers, and networking stack
operate on ``netmem_ref`` as the abstract type. Existing ``page_pool`` APIs
that allocate or free ``struct page`` are legacy compatibility wrappers for
drivers that do not yet support ``netmem_ref``. Code that is not yet
``netmem``-aware should be converted to ``netmem_ref`` unless it will never
need to support ``netmem``.
1. **Operate on netmem_ref, do not downcast**: ``page_pool``, drivers, and the
core networking stack should deal with ``netmem_ref`` rather than
``struct net_iov`` or ``struct page``. Downcasting ``netmem_ref`` to
``struct net_iov`` or ``struct page`` is not allowed unless a code path
strictly cannot function without knowing the underlying memory type (for
example, ``kmap_local_page()``). In those cases, to keep call sites simple,
add a ``netmem`` helper that performs the operation on behalf of the caller,
cleanly handles all ``net_iov`` and ``page`` cases, and returns an error if
the ``netmem`` type cannot support the requested operation.
2. **Decouple memory providers from net_iov**: Memory providers are not
architecturally limited to ``struct net_iov``; a memory provider that returns
``struct page``-backed ``netmem_ref``\ s to upper layers is allowed. Today,
in-tree memory providers only supply ``struct net_iov`` and some existing
code still reflects that limitation, but new code must not assume that using
a memory provider implies ``net_iov`` memory and should, as much as possible,
generalize existing limitations to match the design principles.
3. **Decouple net_iov from unreadability**: ``struct net_iov`` is flexible and
has no inherent restrictions; it may represent either CPU-readable or
unreadable memory. Today, in-tree ``net_iov`` implementations are unreadable
by the CPU (``netmem_address()`` returns ``NULL``) and some existing code
still reflects that limitation, but new code must not assume ``net_iov``
implies unreadable memory (check readability via ``netmem_address()`` or
``skb_frags_readable()`` instead) and should, as much as possible, generalize
existing limitations to match the design principles.
4. **Delegate complexity to the lowest layer**: Each layer must respect its
abstraction boundary. ``page_pool`` must not implement per-memory-provider
custom logic in its main code; instead, it delegates provider-specific
handling to ``struct memory_provider_ops``. Similarly, core networking code
should avoid per-``netmem``-type branching and instead delegate operations
to ``netmem`` helpers that handle the underlying memory type.
5. **Do not mix skb fragment memory sources**: All ``frags[]`` in an
``sk_buff`` must either be ``struct page``-backed or belong to the same
memory provider instance (for example, the same ``devmem`` binding or
``io_uring`` area). Mixing ``struct page`` and memory-provider fragments,
or mixing fragments from different memory providers within a single
``sk_buff``, is not allowed (including when coalescing ``sk_buff``\ s).
Driver RX Requirements
======================
1. The driver must support page_pool.
2. The driver must support the tcp-data-split ethtool option.
3. The driver must use the page_pool netmem APIs for payload memory. The netmem
APIs currently 1-to-1 correspond with page APIs. Conversion to netmem should
be achievable by switching the page APIs to netmem APIs and tracking memory
via netmem_refs in the driver rather than struct page * :
- page_pool_alloc -> page_pool_alloc_netmem
- page_pool_get_dma_addr -> page_pool_get_dma_addr_netmem
- page_pool_put_page -> page_pool_put_netmem
Not all page APIs have netmem equivalents at the moment. If your driver
relies on a missing netmem API, feel free to add and propose to netdev@, or
reach out to the maintainers and/or almasrymina@google.com for help adding
the netmem API.
4. The driver must use the following PP_FLAGS:
- PP_FLAG_DMA_MAP: netmem is not dma-mappable by the driver. The driver
must delegate the dma mapping to the page_pool, which knows when
dma-mapping is (or is not) appropriate.
- PP_FLAG_DMA_SYNC_DEV: netmem dma addr is not necessarily dma-syncable
by the driver. The driver must delegate the dma syncing to the page_pool,
which knows when dma-syncing is (or is not) appropriate.
- PP_FLAG_ALLOW_UNREADABLE_NETMEM. The driver must specify this flag iff
tcp-data-split is enabled.
5. The driver must not assume the netmem is readable and/or backed by pages.
The netmem returned by the page_pool may be unreadable, in which case
netmem_address() will return NULL. The driver must correctly handle
unreadable netmem, i.e. don't attempt to handle its contents when
netmem_address() is NULL.
Ideally, drivers should not have to check the underlying netmem type via
helpers like netmem_is_net_iov() or convert the netmem to any of its
underlying types via netmem_to_page() or netmem_to_net_iov(). In most cases,
netmem or page_pool helpers that abstract this complexity are provided
(and more can be added).
6. The driver must use page_pool_dma_sync_netmem_for_cpu() in lieu of
dma_sync_single_range_for_cpu(). For some memory providers, dma_syncing for
CPU will be done by the page_pool, for others (particularly dmabuf memory
provider), dma syncing for CPU is the responsibility of the userspace using
dmabuf APIs. The driver must delegate the entire dma-syncing operation to
the page_pool which will do it correctly.
7. Avoid implementing driver-specific recycling on top of the page_pool. Drivers
cannot hold onto a struct page to do their own recycling as the netmem may
not be backed by a struct page. However, you may hold onto a page_pool
reference with page_pool_fragment_netmem() or page_pool_ref_netmem() for
that purpose, but be mindful that some netmem types might have longer
circulation times, such as when userspace holds a reference in zerocopy
scenarios.
Driver TX Requirements
======================
1. The Driver must not pass the netmem dma_addr to any of the dma-mapping APIs
directly. This is because netmem dma_addrs may come from a source like
dma-buf that is not compatible with the dma-mapping APIs.
Helpers like netmem_dma_unmap_page_attrs() & netmem_dma_unmap_addr_set()
should be used in lieu of dma_unmap_page[_attrs](), dma_unmap_addr_set().
The netmem variants will handle netmem dma_addrs correctly regardless of the
source, delegating to the dma-mapping APIs when appropriate.
Not all dma-mapping APIs have netmem equivalents at the moment. If your
driver relies on a missing netmem API, feel free to add and propose to
netdev@, or reach out to the maintainers and/or almasrymina@google.com for
help adding the netmem API.
2. Driver should declare support by setting `netdev->netmem_tx` to the
appropriate mode:
- `NETMEM_TX_DMA`: for physical devices that perform DMA.
- `NETMEM_TX_NO_DMA`: for virtual or passthrough devices that do
not DMA, but still support handling of netmem-backed skbs.
|
