| Age | Commit message (Collapse) | Author |
|---|
|
Rename swiotlb_size_or_default() to swiotlb_default_pool_size() to make
clear that it returns the current size of the default SWIOTLB pool.
This includes changes made by command-line options or sizing heuristics,
not just the compiled-in default.
No functional change.
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
Link: https://lore.kernel.org/r/20260924060756.1325156-2-aneesh.kumar@kernel.org
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
|
|
Auto-boot for an always-on remote processor registers an asynchronous
request_firmware_nowait() whose callback, rproc_auto_boot_callback(),
calls rproc_boot() first and only releases the fetched image at the
very end of the callback. rproc_boot() does not take this image as an
argument: it fetches the very same image again with a synchronous
request_firmware(), so every auto-boot reads the firmware image twice
and allocates the buffer twice; on kernels with the sysfs fallback
enabled, the uevent round-trip is repeated as well.
The asynchronous request exists only so that rproc_add() does not
block on the filesystem read; the image it fetches is never used.
The core already defers rproc_boot() to a worker for detached
processors (attach_work). Reuse that worker for offline processors
too and drop the asynchronous firmware request: rproc_boot() fetches
the image exactly once and dispatches between a firmware boot and
an attach based on the processor state. The work is renamed to
boot_work to match its widened role.
commit 400e64df6b23 ("remoteproc: add framework for controlling remote
processors") noted back in 2011 that "we must wait until it completes
before we try to unregister the device". rproc_del() now does exactly
that: it waits for the boot work with cancel_work_sync(), which also
closes the theoretical window in which a pending attach work could
outlive the rproc instance.
The changed fallback behaviour only matters for legacy
configurations. udev dropped its userspace firmware loader back in
2014, as recorded in
Documentation/driver-api/firmware/fallback-mechanisms.rst, and the
kernel has documented since commit 02c399306826 ("firmware_loader:
enhance Kconfig documentation over FW_LOADER") that "Linux no longer
relies on or uses a fallback mechanism in userspace". Auto-boot now
uses the same synchronous fallback semantics as every other explicit
boot source (sysfs, cdev).
Signed-off-by: Yonghao Zhang <hyz3367@gmail.com>
Link: https://lore.kernel.org/r/20261001081056.38293-1-hyz3367@gmail.com
Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
|
|
A ring created with IORING_SETUP_R_DISABLED is exempt from its register
restrictions until it is enabled. This lets the creator register the
restrictions and then enable the ring with IORING_REGISTER_ENABLE_RINGS,
whether or not that opcode is in the allowed set.
Since commit ed82f35b926b ("io_uring: allow registration of per-task
restrictions"), a ring also inherits restrictions from the task that
creates it, and the same exemption applies to those. A task whose
register opcodes are restricted can therefore create a disabled ring,
issue any register opcode on it, and then enable it.
Mark rings whose restrictions were inherited from the task, and keep
checking register opcodes for them while the ring is disabled. Enabling
such a ring now requires IORING_REGISTER_ENABLE_RINGS to be in the
task's allowed set. Per-ring restrictions registered on a disabled ring
work as before.
Fixes: ed82f35b926b ("io_uring: allow registration of per-task restrictions")
Cc: stable@vger.kernel.org
Assisted-by: Claude:claude-opus-5-5
Signed-off-by: Eugene Gvozdetsky <gvozdet@gmail.com>
Link: https://patch.msgid.link/20261009-iouring-task-restrict-fix-v1-1-a49daf55a12c@gmail.com
[axboe: amend with inheritance flag fixup]
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Merge the immutable branch dt into next, to allow the updated DT bindings
to be tested together with the pmdomain changes that are targeted for the
next release.
Signed-off-by: Ulf Hansson <ulfh@kernel.org>
|
|
This file has been left in an unfinished state just defining
the root power domain for the U8500 SoC. Fix it up by adding
the actual existing power domains in this SoC.
The PRCMU code and old regulator driver is mentioning some
*_RET domains, this means "retention" and is a state in the
domain and not a domain of its own.
Acked-by: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>
Signed-off-by: Linus Walleij <linusw@kernel.org>
Signed-off-by: Ulf Hansson <ulfh@kernel.org>
|
|
Keep the internal Ux500 PM domain count local to the only driver
that uses it.
Do not expose this implementation detail through the DT binding header.
Assisted-by: LLM
Suggested-by: Krzysztof Kozlowski <krzk@kernel.org>
Acked-by: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>
Signed-off-by: Linus Walleij <linusw@kernel.org>
Signed-off-by: Ulf Hansson <ulfh@kernel.org>
|
|
Now we have cfg80211_mutex everywhere along with wiphy mutex
for per-device bits, so don't need RTNL any more. Remove it
from the regulatory code.
Also, wiphy_apply_custom_regulatory() no longer needs the RTNL,
and get_wiphy_regdom() can no longer rely on it.
Link: https://patch.msgid.link/20261006120655.df1d3d28cfa2.I637dbd328886b52bed511fb798172a2d57521263@changeid
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
|
|
freq_reg_info() returns a pointer into the current regdomain,
so we need RTNL, cfg80211_mutex or RCU, but I want to remove
the RTNL, so just use RCU here.
Also document that this is required, we didn't say when the
function could be used at all.
Link: https://patch.msgid.link/20261006120655.c6fce71ad991.I07ae7ed37bc8ff6d75c87e19d77e4c5379a8e167@changeid
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
|
|
The DFS state of channels and the CAC state of the wdev is
protected by the wiphy mutex, so the radar and CAC events
must be reported by drivers with the wiphy mutex held. In
mac80211 we do this, but some drivers don't yet:
- mwifiex/nxpwifi have an event handling worker,
- qtnfmac also does it from event processing, and
- mt7915 and mt7996 don't acquire a mutex for background
radar events.
Document the requirement.
Link: https://patch.msgid.link/20261006120655.6fba1ce72650.I57002ac3e57d2ac4613eb0cb0e6d17bedccf0cbd@changeid
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
|
|
wdev_running() uses netif_running() for netdevs, which
can only be used under RTNL in a completely race-free
way.
Track the state for all wdevs via the existing netdev
notifier so we can check the state under wiphy mutex
in later changes.
Link: https://patch.msgid.link/20261006120655.7ed32b42c297.I9ac8bb8e7cd07c962116949aa0c56aab4e87839b@changeid
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
|
|
reg_query_regdb_wmm() is only used by iwlwifi, but even when
it was introduced was already broken, as it reads the global
db without any locking (drivers can't really guarantee any).
Certainly iwlwifi doesn't since it calls it for FW messages.
Therefore, it can race against regdb updates. The function
uses a local pointer, but doesn't help.
Use RCU to protect the regdb pointer to fix that, and update
the documentation to mention this (and do some cleanups).
Fixes: 19d3577e35e0 ("cfg80211: Add API to allow querying regdb for wmm_rule")
Link: https://patch.msgid.link/20261006120655.7c261d237cae.If21cf0af3b0d7e8c62c45ece5dc654b30c9d7757@changeid
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
|
|
Struct field cross-references in kernel-doc use @field, not %field (which
is for constants and #defines). Fix eight instances, while also correcting
%elems -> @elem naming in cfg80211_mbssid_elems and cfg80211_rnr_elems.
Assisted-by: LLM
Signed-off-by: Jeff Johnson <jeff.johnson@oss.qualcomm.com>
Link: https://patch.msgid.link/20261007-cfg80211-kdoc-v1-1-157b358b291f@oss.qualcomm.com
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
|
|
Do not expose INTERNAL pads on UAPI as neither the documentation nor
v4l2-compliance changes to support MEDIA_PAD_FL_INTERNAL pads have been
merged yet.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Hans Verkuil <hverkuil+cisco@kernel.org>
Signed-off-by: Hans Verkuil <hverkuil+cisco@kernel.org>
|
|
Move the following functions and declarations from the public header
include/linux/dmaengine.h into the driver-private header
drivers/dma/dmaengine.h, since they are only used by DMA engine drivers
and not by DMA engine consumers:
- dma_async_device_register()
- dmaenginem_async_device_register()
- dma_async_device_unregister()
- dma_async_device_channel_register()
- dma_async_device_channel_unregister()
- is_slave_direction()
- dmaengine_get_direction_text()
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20261001143955.2195063-1-Frank.Li@oss.nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
Add a dmaengine_desc_set_callback() inline helper that sets the
callback, callback_result, and callback_param fields of a
dma_async_tx_descriptor together in a single call.
Using this helper instead of open-coding the three field assignments
ensures all three related fields are always updated consistently and
makes call sites more readable.
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20261002195410.213610-1-Frank.Li@oss.nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
Add dmaengine_get_provider_device() API and kernel doc for
dmaengine_get_dma_device() to describe the difference.
dmaengine_get_dma_device() is used for DMA mapping because some DMA
engines provide per-channel iommu mapping.
dmaengine_get_provider_device() is used to find a DMA engine channel
by a filter function.
In most cases they return the same value, but they have different
semantic meanings.
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20261002202341.253152-1-Frank.Li@oss.nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
Add a new inline helper dmaengine_get_max_xor() that returns the maximum
number of XOR source buffers supported by the DMA channel's device.
Returns 0 if the channel is NULL or does not support XOR operations.
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20261005-dmaengine_prep_dma_pq-v3-5-a4a241f26fa7@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
Add two new inline helpers to query DMA channel capabilities without
directly accessing the internal dma_device pointer:
dmaengine_get_cap_mask(chan) - returns a const pointer to the
capability mask of the DMA device backing the channel.
dmaengine_has_cap(chan, tx_type) - returns true if the channel
supports the given transaction type.
These complement the existing dma_has_cap() macro which operates on
a dma_cap_mask_t directly.
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20261005-dmaengine_prep_dma_pq-v3-4-a4a241f26fa7@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
Add dmaengine_get_copy_align(), dmaengine_get_xor_align(),
dmaengine_get_pq_align(), and dmaengine_get_fill_align() inline
helpers to retrieve the alignment requirement of a DMA channel as
a power-of-2 exponent.
These complement the existing dmaengine_is_*_aligned() test helpers
by providing direct access to the alignment value, which is useful
when callers need to propagate or compare alignment constraints.
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20261005-dmaengine_prep_dma_pq-v3-3-a4a241f26fa7@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
DMA engine consumers typically hold a struct dma_chan pointer and should
not need to reach into struct dma_device directly to check alignment
requirements. Add dmaengine_is_copy_aligned(), dmaengine_is_xor_aligned(),
dmaengine_is_pq_aligned() and dmaengine_is_fill_aligned() as channel-based
counterparts to the existing is_dma_*_aligned() family.
Each wrapper calls dmaengine_check_align() directly with the alignment
value from chan->device, keeping struct dma_device opaque to consumers.
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20261005-dmaengine_prep_dma_pq-v3-2-a4a241f26fa7@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
Add dmaengine_prep_dma_(pq|pq_val|interrupt|xor)() as an inline
wrapper around the device_prep_dma_(pq|pq_val|interrupt|xor)() callback,
consistent with how other dmaengine prep helpers like
dmaengine_prep_dma_memcpy() are structured.
Provide a clean public API that hides the direct callback access and add
proper NULL-checks on chan, chan->device, and the callback pointer.
Assisted-by: LLM
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Link: https://patch.msgid.link/20261005-dmaengine_prep_dma_pq-v3-1-a4a241f26fa7@nxp.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
|
|
https://gitlab.freedesktop.org/drm/i915/kernel into drm-next
[airlied: add xe fixup]
- Vswing / Pre-emphasis Override (Michal)
- Enable AS SDP Skip Frames (Uma)
- Handle negative configuration index on DP link caps for robustness (Luca)
- Perform full wedge on display reset (Ville)
- PSR related revert and fix (Nemesa)
- Preparation for enabling SDR plane color pipeline (Chaitanya, Pranay)
- An eDP quirk for HP Pavilion Plus (Ankit)
- DP MST fixes (Imre)
- General display fixes and small improvements (Jani, Lalit, Vinod, Sebastian)
- Introduce PME (Power Management Event) support on display hotplug (Vinod)
- Force eDP sink awake on dpcd init (Arun)
- Stop using the configurable fence timeout and other cleanups (Jani)
- Add support for Common SDP Transmission Line (Ankit, Arun)
- Map AUX power domains to DC_off (Suraj)
- Handle VSC SDP revision 7 in unpack (Juasheem)
- Disable VRR DC balance by default (Mitul)
- Switch to modern drm_dp_dpcd* accessors (Jani)
- Lg DPCD access errors (Suraj)
- Fixes to make i915 work well with PREEMPT_RT (Maarten, Sebastian)
- Update the CMN_SDP_TL in fastset path (Ankit)
- Use a shared gtt_view_types.h header (Jani)
Signed-off-by: Dave Airlie <airlied@redhat.com>
From: Rodrigo Vivi <rodrigo.vivi@intel.com>
Link: https://patch.msgid.link/asAD6RRGxlbIPdhz@intel.com
|
|
Add page-selective IOTLB flush using HVCALL_FLUSH_DEVICE_DOMAIN_LIST.
This hypercall accepts a list of (page_number, page_mask_shift) entries,
enabling finer-grained IOTLB invalidation compared to the domain-wide
HVCALL_FLUSH_DEVICE_DOMAIN used by hv_iommu_flush_iotlb_all().
hv_iommu_calc_flush_range() computes the smallest power-of-two aligned
range that covers the target IOVA region, producing a single flush
descriptor. This may over-flush when the range is not naturally aligned,
matching the approach used by Intel VT-d PSI. If the page-selective
flush fails, the code falls back to a full domain flush.
Signed-off-by: Easwar Hariharan <easwar.hariharan@linux.microsoft.com>
Signed-off-by: Yu Zhang <zhangyu1@linux.microsoft.com>
Reviewed-by: Jacob Pan <jacob.pan@linux.microsoft.com>
Reviewed-by: Michael Kelley <mhklinux@outlook.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
Hyper-V identifies each PCI pass-thru device by a logical device ID in
its hypercall interface. This ID consists of a per-bus prefix, derived
from the VMBus device instance GUID, combined with the PCI function
number of the endpoint device.
Add a registry in hv_common.c that maps a PCI domain number to its
logical device ID prefix. The vPCI bus driver (pci-hyperv) registers the
prefix when a bus is probed and unregisters it when the bus is removed.
Consumers such as the para-virtualized IOMMU driver look up the prefix
by PCI domain number and combine it with the function number to form the
complete logical device ID for hypercalls.
Use rhashtable for the sparse exact-match mapping. Lookups copy the
prefix while holding the RCU read lock, and removal defers freeing the
entry until existing readers have completed.
The prefix construction is shared via hv_build_logical_dev_id_prefix() so
that pci-hyperv's interrupt retargeting path and the registry use exactly
the same byte layout. It is derived on demand from the constant hv_device
instance GUID rather than cached in struct hv_pcibus_device, which is
private to the pci-hyperv module; this keeps the interface narrow and
avoids depending on pci-hyperv internals.
Co-developed-by: Easwar Hariharan <easwar.hariharan@linux.microsoft.com>
Signed-off-by: Easwar Hariharan <easwar.hariharan@linux.microsoft.com>
Signed-off-by: Yu Zhang <zhangyu1@linux.microsoft.com>
Reviewed-by: Michael Kelley <mhklinux@outlook.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
Hyper-V guest IOMMU is a para-virtualized IOMMU based on hypercalls.
Introduce the hypercalls used by the child partition to interact with
this facility.
These hypercalls fall into below categories:
- Detection and capability: HVCALL_GET_IOMMU_CAPABILITIES is used to
detect the existence and capabilities of the guest IOMMU.
- Device management: HVCALL_GET_LOGICAL_DEVICE_PROPERTY is used to
check whether an endpoint device is managed by the guest IOMMU.
- Domain management: A set of hypercalls is provided to handle the
creation, configuration, and deletion of guest domains, as well as
the attachment/detachment of endpoint devices to/from those domains.
- IOTLB flushing: HVCALL_FLUSH_DEVICE_DOMAIN is used to ask Hyper-V
for a domain-selective IOTLB flush (which in its handler may flush
the device TLB as well).
Signed-off-by: Wei Liu <wei.liu@kernel.org>
Co-developed-by: Easwar Hariharan <easwar.hariharan@linux.microsoft.com>
Signed-off-by: Easwar Hariharan <easwar.hariharan@linux.microsoft.com>
Co-developed-by: Yu Zhang <zhangyu1@linux.microsoft.com>
Signed-off-by: Yu Zhang <zhangyu1@linux.microsoft.com>
Reviewed-by: Jacob Pan <jacob.pan@linux.microsoft.com>
Reviewed-by: Michael Kelley <mhklinux@outlook.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
The GCC and Clang compilers provide the __counted_by_ptr attribute,
which is used by KASAN and compiler bounds-checking to detect
out-of-bounds accesses to pointer fields.
In "struct fsl_mc_io", the "portal_virt_addr" pointer points to the MC
command portal virtual address. The size of this allocated portal in
bytes is tracked by the "portal_size" field within the same structure.
Annotate the "portal_virt_addr" pointer field with
"__counted_by_ptr(portal_size)" to enable compiler bounds-checking and
harden against potential out-of-bounds accesses.
Cc: codemender-patching+linux@google.com
Assisted-by: LLM
Signed-off-by: Bill Wendling <morbo@google.com>
Reviewed-by: Ioana Ciornei <ioana.ciornei@nxp.com>
Reviewed-by: Gustavo A. R. Silva <gustavoars@kernel.org>
Link: https://lore.kernel.org/r/20260922223538.2287676-1-morbo@google.com
Signed-off-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
|
|
Fix typos in comments, reported by scripts/checkpatch.pl using the
misspelling list in scripts/spelling.txt. Only touches comments, no code
changes.
Assisted-by: Cursor:claude-opus-5
Signed-off-by: Hemanth Selam <hemanth.selam@gmail.com>
Reviewed-by: Frediano Ziglio <freddy77@gmail.com>
Signed-off-by: Juergen Gross <jgross@suse.com>
Message-ID: <20260907063531.23981-1-hemanth.selam@gmail.com>
|
|
The <xen/interface/hvm/hvm_vcpu.h> header appears to have been unused
ever since it was added to the kernel in commit cee2cfb7d18d ("xen/pvh:
Import PVH-related Xen public interfaces"). Remove it.
Signed-off-by: Ethan Nelson-Moore <enelsonmoore@gmail.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
Signed-off-by: Juergen Gross <jgross@suse.com>
Message-ID: <20260919074650.453007-1-enelsonmoore@gmail.com>
|
|
Most of iommu-dma's own internals were privatised a while ago, but
there are still a couple of DMA-domain-specific helpers provided
by the core code. These now definitely have no reason to be exposed
publicly, so move their declarations into the private header.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
Linux 7.3-rc6
This was requested by Thomas to have a new base to start some branch from
Signed-off-by: Dave Airlie <airlied@redhat.com>
|
|
folio->private != NULL indicates a folio carries private data, replacing
PG_private. All PG_private users are converted. Remove PG_private and
reserve the space as PG_folio for future use. Unused PG_private functions
are removed too.
Link: https://lore.kernel.org/20260920-remove-pg_private-v5-17-bb68b6a21869@nvidia.com
Signed-off-by: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Assisted-by: LLM
Cc: Baoquan He <baoquan.he@linux.dev>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
Cc: Pratyush Yadav <pratyush@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org>
Cc: Jan Kara <jack@suse.cz>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Dave Young <ruirui.yang@linux.dev>
Cc: Shuah Khan <skhan@linuxfoundation.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: "Liam R. Howlett" <liam@infradead.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
|
|
PG_private and PagePrivate() are no longer used. Adjust related comments
and documentations to refer to page/folio->private instead.
hugetlbfs_reserv.rst is outdated and left unchanged. It should be
rewritten.
Link: https://lore.kernel.org/20260920-remove-pg_private-v5-16-bb68b6a21869@nvidia.com
Signed-off-by: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Assisted-by: LLM
Cc: Ilya Dryomov <idryomov@gmail.com>
Cc: Alex Markuze <amarkuze@redhat.com>
Cc: Viacheslav Dubeyko <slava@dubeyko.com>
Cc: Trond Myklebust <trondmy@kernel.org>
Cc: Anna Schumaker <anna@kernel.org>
Cc: Richard Weinberger <richard@nod.at>
Cc: Zhihao Cheng <chengzhihao1@huawei.com>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: "Liam R. Howlett" <liam@infradead.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Michal Hocko <mhocko@suse.com>
|
|
Convert the page to a folio and use folio_buffers() instead. This does
introduce one extra call to compound_head(), but will simplify a later
conversion of md-bitmap to use folios instead of pages.
Link: https://lore.kernel.org/20260920-remove-pg_private-v5-15-bb68b6a21869@nvidia.com
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: David Hildenbrand (Arm) <david@kernel.org>
|
|
Remove the last user of alloc_page_buffers(). Use folio_alloc_buffers()
instead, since alloc_page_buffers() is a wrap over it. Although the pages
used in md-bitmap are not folios, as they are not mapped into userspace
nor enter the page cache, but they still have buffer heads attached.
Cleaning up the code to not use buffer heads is future work.
[ziy@nvidia.com: reword commit message]
Link: https://lore.kernel.org/20260920-remove-pg_private-v5-13-bb68b6a21869@nvidia.com
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: David Hildenbrand (Arm) <david@kernel.org>
|
|
They are no-ops now. Remove them.
Link: https://lore.kernel.org/20260920-remove-pg_private-v5-11-bb68b6a21869@nvidia.com
Signed-off-by: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Assisted-by: LLM
Cc: Trond Myklebust <trondmy@kernel.org>
Cc: Anna Schumaker <anna@kernel.org>
Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org>
Cc: Jan Kara <jack@suse.cz>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Joshua Hahn <joshua.hahnjy@gmail.com>
Cc: Rakie Kim <rakie.kim@sk.com>
Cc: Byungchul Park <byungchul@sk.com>
Cc: Gregory Price <gourry@gourry.net>
Cc: Ying Huang <ying.huang@linux.alibaba.com>
Cc: Alistair Popple <apopple@nvidia.com>
|
|
After the changes of the prior commits, page/folio->private != NULL is now
equivalent to checking PG_private.
Stop checking PG_private on pages and folios and use page/folio->private
instead, except swapcache and hugetlb folios, because the former uses a
field (swp_entry_t swap) overlapping with ->private and the latter sets
its flags in ->private. Exclude swapcache and hugetlb when the code is
meant to check PG_private only. PG_swapcache and folio->swap.val cannot
be set/clear as a whole, so excluding swapcache with
folio_test_swapcache() is not reliable. Instead, use
folio_test_swapbacked(), since PG_swapbacked is stable when a folio is
added to/removed from swapcache. Add a helper,
folio_has_attached_private(), for this check.
folio_test_private() and PagePrivate() now read folio/page->private
plainly instead of an atomic read of PG_private bit, so KCSAN complains
about possible data races. Annotate them with data_race().
folio_expected_ref_count() can be called without the folio lock, so
annotate folio->mapping with data_race() while at it.
folio_set/clear_private() and Set/ClearPagePrivate() become no-ops.
PG_private is no longer checked at page free time. They will be removed in
an upcoming commit.
Remove KPF_PRIVATE since PG_private is no longer used.
Link: https://lore.kernel.org/20260920-remove-pg_private-v5-10-bb68b6a21869@nvidia.com
Signed-off-by: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Assisted-by: LLM
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org>
Cc: Jan Kara <jack@suse.cz>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: "Liam R. Howlett" <liam@infradead.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Nico Pache <nico.pache@linux.dev>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Usama Arif <usama.arif@linux.dev>
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: Joshua Hahn <joshua.hahnjy@gmail.com>
Cc: Rakie Kim <rakie.kim@sk.com>
Cc: Byungchul Park <byungchul@sk.com>
Cc: Gregory Price <gourry@gourry.net>
Cc: Ying Huang <ying.huang@linux.alibaba.com>
Cc: Alistair Popple <apopple@nvidia.com>
Cc: Qi Zheng <qi.zheng@linux.dev>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Kairui Song <kasong@tencent.com>
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Cc: Wei Xu <weixugc@google.com>
|
|
erofs needs to traverse readahead folios in reverse order to achieve
maximum performance by
1. reading all folios from readahead_folio();
2. storing the prior folio pointer in folio->private;
3. traverse from the last folio to the first one.
Add readahead_folio_last() to achieve the same function without using
folio->private. __readahead_advance() helper shares readahead_control
adjustment code among __readahead_folio(), readahead_folio_last(), and
__readahead_batch() by checking new private member, _forward, of
readahead_control.
It prepares for a future commit that replaces PG_private checks with
!folio->private checks. After switching the checks, erofs's use of
folio->private without bumping folio refcount can cause unexpected
outcomes, e.g., in filemap_release_folio(), try_to_free_buffers() becomes
reachable.
No functional change intended.
Link: https://lore.kernel.org/20260920-remove-pg_private-v5-8-bb68b6a21869@nvidia.com
Signed-off-by: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Assisted-by: LLM
Cc: Yue Hu <zbestahu@gmail.com>
Cc: Jeffle Xu <jefflexu@linux.alibaba.com>
Cc: Sandeep Dhavale <dhavale@google.com>
Cc: Hongbo Li <hongbohbli@tencent.com>
Cc: Chunhai Guo <guochunhai@vivo.com>
Cc: Gao Xiang <xiang@kernel.org>
Cc: Chao Yu <chao@kernel.org>
Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org>
Cc: Jan Kara <jack@suse.cz>
|
|
DEFINE_MTREE() uses MTREE_INIT(), which initializes the tree's embedded
lock with a static spinlock initializer. If the tree is an automatic
local object, lockdep rejects its address as a non-static class key and
disables locking validation on the first lock acquisition.
Apply ASSERT_STATIC_STORAGE() to DEFINE_MTREE() to catch automatic local
definitions at compile time. For example:
void example(void)
{
DEFINE_MTREE(mt);
mtree_destroy(&mt);
}
GCC reports:
error: initializer element is not constant
name##_storage_check = &(name)
^
note: in expansion of macro 'ASSERT_STATIC_STORAGE'
ASSERT_STATIC_STORAGE(name)
note: in expansion of macro 'DEFINE_MTREE'
DEFINE_MTREE(mt);
File-scope definitions and static local trees remain valid. Automatic
local trees must instead use mt_init() or mt_init_flags(). Direct uses of
MTREE_INIT() and MTREE_INIT_EXT() are unchanged.
Replace the local DEFINE_MTREE() in the interval-tree span test with a
plain declaration; the test already initializes the tree with
mt_init_flags() before use. Convert the three local Maple Trees in the
userspace radix-tree tests to mt_init().
Validated file-scope and static local definitions with a kernel object
build, and confirmed that an automatic local definition fails to compile.
The Maple Tree test and region allocation benchmark objects also build
with lockdep enabled.
Link: https://lore.kernel.org/20260911221444.1523311-4-ynorov@nvidia.com
Signed-off-by: Yury Norov <ynorov@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Alice Ryhl <aliceryhl@google.com>
Cc: Andrew Ballance <andrewjballance@gmail.com>
Cc: Christopher Li <sparse@chrisli.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
|
|
DEFINE_IDA() uses IDA_INIT(), which initializes the embedded XArray lock
with a static spinlock initializer. For an automatic local IDA, lockdep
cannot use the lock address as a persistent class key and reports "INFO:
trying to register non-static key" before disabling itself.
Apply ASSERT_STATIC_STORAGE() to DEFINE_IDA() so that this misuse is
rejected at compile time. For example:
void example(void)
{
DEFINE_IDA(ida);
ida_destroy(&ida);
}
GCC reports:
error: initializer element is not constant
name##_storage_check = &(name)
^
note: in expansion of macro 'ASSERT_STATIC_STORAGE'
ASSERT_STATIC_STORAGE(name)
note: in expansion of macro 'DEFINE_IDA'
DEFINE_IDA(ida);
File-scope definitions and static DEFINE_IDA() within a function remain
valid. Automatic local IDAs must instead be initialized with ida_init().
Direct uses of IDA_INIT() are not covered by this declaration check.
Convert the five automatic local IDAs in the userspace radix-tree tests to
ida_init() so they satisfy the new requirement.
Validated file-scope and static local definitions with a kernel object
build, and confirmed that an automatic local definition fails to compile.
Link: https://lore.kernel.org/20260911221444.1523311-3-ynorov@nvidia.com
Signed-off-by: Yury Norov <ynorov@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Alice Ryhl <aliceryhl@google.com>
Cc: Andrew Ballance <andrewjballance@gmail.com>
Cc: Christopher Li <sparse@chrisli.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
|
|
Patch series "Catch automatic storage in IDA and Maple Tree definitions".
A 0day report [1] from the region allocation benchmark exposed a lockdep
initialization bug: a stack-local Maple Tree used MTREE_INIT(), whose
embedded lock has a static initializer. On the first allocation, lockdep
rejected the lock address as a non-static class key and disabled locking
validation. The IDA benchmark had the same issue, masked because it ran
after Maple Tree had already disabled lockdep.
The fix [2] switches the test to using mt_init_flags() and ida_init().
This series adds a compile-time check to the related DEFINE_IDA() and
DEFINE_MTREE() declaration macros to catch the same class of mistake
earlier.
Patch 1 introduces ASSERT_STATIC_STORAGE(). It declares an unused static
pointer initialized with the object's address, requiring that address to
be a valid static initializer. Patches 2 and 3 apply the helper to IDA
and Maple Tree definitions, respectively.
The helper is mirrored in the tools compiler header. The existing
automatic local IDAs and Maple Trees in the userspace radix-tree tests are
converted to runtime initialization. The interval-tree span test keeps
its existing mt_init_flags() call and uses a plain Maple Tree declaration.
For example, an automatic local definition:
void example(void)
{
DEFINE_IDA(ida);
ida_destroy(&ida);
}
now produces:
error: initializer element is not constant
note: in expansion of macro 'ASSERT_STATIC_STORAGE'
note: in expansion of macro 'DEFINE_IDA'
File-scope definitions and static local definitions remain valid.
Automatic local objects should use ida_init(), mt_init(), or
mt_init_flags().
The check is limited to declaration macros. Direct uses of IDA_INIT(),
MTREE_INIT(), and MTREE_INIT_EXT() remain unchanged. The helper cannot be
inserted directly into those initializer expressions because it expands to
a declaration.
Validated by GCC and Clang checks accepting static storage and rejecting
automatic storage The userspace IDR/IDA and Maple Tree test are passed as
well.
This patch (of 3):
Static lock initializers rely on a persistent object address when lockdep
assigns a lock-class key. Using such an initializer for an automatic
local object can compile successfully but disable lockdep on the first
lock acquisition.
Add ASSERT_STATIC_STORAGE() for declaration macros that require static
storage duration. It declares an unused static pointer initialized with
the object's address. An automatic local object's address is not a valid
static initializer, so the compiler rejects it.
Mirror the helper in tools/include/linux/compiler.h because the userspace
radix-tree tests include the kernel IDA and Maple Tree headers with the
tools compiler definitions.
For example:
void example(void)
{
int object;
ASSERT_STATIC_STORAGE(object);
}
GCC reports:
error: initializer element is not constant
name##_storage_check = &(name)
^
note: in expansion of macro 'ASSERT_STATIC_STORAGE'
ASSERT_STATIC_STORAGE(object);
File-scope objects and static local objects remain valid. The helper
takes an object identifier and must be used as a declaration after that
object has been declared.
Link: https://lore.kernel.org/all/20260911155244.1406122-1-ynorov@nvidia.com/
Link: https://lore.kernel.org/20260911221444.1523311-2-ynorov@nvidia.com
Link: https://download.01.org/0day-ci/archive/20260910/202609101106.771b567e-lkp@intel.com/ [1]
Link: https://lore.kernel.org/all/20260911155244.1406122-1-ynorov@nvidia.com/ [2]
Signed-off-by: Yury Norov <ynorov@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Alice Ryhl <aliceryhl@google.com>
Cc: Andrew Ballance <andrewjballance@gmail.com>
Cc: Christopher Li <sparse@chrisli.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
|
|
damon_probe_hits_wsum() is being called only in aggregation time. In
future, it could also be used by DAMOS. In this case, since DAMOS uses
its own apply_interval, it could be called in sampling time. Then using
the not yet fully aggregated probe_hits could result in suboptimum
outcomes. Extend damon_probe_hits_wsum() to get the weighted sum based on
moving sum to prepare the DAMOS usage.
Link: https://lore.kernel.org/20260911135510.96914-3-sj@kernel.org
Signed-off-by: SJ Park <sj@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
|
|
Patch series "mm/damon: introduce probe_hits_wsum DAMOS core filter", v2.
DAMON can do flexible data attributes monitoring. The classical data
access monitoring can also be done using the attributes monitoring. The
access event is just one of the data attributes that DAMON supports.
Users do monitoring to make some actions based on it. DAMOS is a feature
for automating that. However, DAMOS cannot utilize the data attributes
monitoring results. It is still Data "Access" Monitoring-based Operation
Schemes. It requires users to set the target "access" pattern.
DAMOS core filter is effectively the same as the target access pattern.
It is just a more generalized and flexible way of describing the operation
action target region. Introduce a new DAMOS core filter type,
probe_hits_wsum. It specifies the filter target based on a range of the
probe hits weighted sum. Using this, users can apply DAMOS actions to
regions of specific data attributes pattern.
Note that the classic target access pattern still works. Hence the target
nr_accesses range should still be properly configured. The new filter
would be used in only data attributes-only mode. In the mode, classic
access monitoring is just turned off, and therefore nr_accesses of regions
are always zero. Users could simply set the target nr_accesses range to
include the zero nr_Accesses regions.
Patches Sequence
================
Patch 1 updates the DAMON kernel API for the new filter type. Patch 2
extends damon_probe_hits_wsum() to do the calculation based on moving sum.
Patch 3 implements the filter type in the core layer. Patch 4 refactors
DAMON sysfs interface internal data structure for efficient reuse of data
structure for the probe hits weighted sum range user inputs. Patch 5
updates DAMON sysfs interface to support the new filter type. Patches 6
and 7 update design and usage documents for the new filter type,
respectively.
This patch (of 7):
Update DAMON kernel API to introduce new DAMOS core filter type,
PROBE_HITS_WSUM. It will allow API callers to describe the DAMOS action
target regions based on their probe_hits weighted sum. For describing the
filtering target weighted sum range, add two type-dependent union fields
to damos_filter.
[sj@kernel.org: clarify DAMOS_FILTER_TYPE_PROBE_HITS_WSUM behavior]
Link: https://lore.kernel.org/20260911142522.98013-1-sj@kernel.org
Link: https://lore.kernel.org/20260911135510.96914-1-sj@kernel.org
Link: https://lore.kernel.org/20260911135510.96914-2-sj@kernel.org
Signed-off-by: SJ Park <sj@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
|
|
The device DAX vmemmap population still reserves one extra tail vmemmap
page after the head page.
Drop that extra reservation and let the shared tail page cover all tail
vmemmap pages after the head page, so DAX follows the same reservation
model as HugeTLB.
This reduces the reserved vmemmap pages for optimized DAX mappings to one
and removes the now-unneeded first-tail population from the generic and
powerpc paths to simplify the code as well.
Link: https://lore.kernel.org/20260930140627.57431-11-songmuchun@bytedance.com
Signed-off-by: Muchun Song <muchun.song@linux.dev>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Qi Zheng <qi.zheng@linux.dev>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Christophe Leroy <chleroy@kernel.org>
Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Lance Yang <lance.yang@linux.dev>
|
|
The powerpc radix compound vmemmap population path still finds a reusable
tail page by walking the vmemmap page tables.
Switch it to the common vmemmap_shared_tail_page() helper instead, so it
can use the shared vmemmap page directly to simplify the code.
This removes the powerpc-specific tail-page lookup and its fallback path
and aligns the device DAX vmemmap optimization path with HugeTLB.
[muchun.song@linux.dev: use try_get_page()]
Link: https://lore.kernel.org/20260930152826.76084-1-songmuchun@bytedance.com
Link: https://lore.kernel.org/20260930140627.57431-10-songmuchun@bytedance.com
Signed-off-by: Muchun Song <muchun.song@linux.dev>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Qi Zheng <qi.zheng@linux.dev>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Christophe Leroy <chleroy@kernel.org>
Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Lance Yang <lance.yang@linux.dev>
|
|
The vmemmap optimization helpers currently live in mm/sparse.h, which is
an internal MM header. That works for MM code, but prevents powerpc from
using the same interfaces without including a private header.
Move the declarations and inline helpers to vmemmap-optimization.h. This
is a preparatory change for powerpc, which has its own vmemmap
optimization implementation and needs to use the common vmemmap
optimization interfaces from architecture code.
Link: https://lore.kernel.org/20260930140627.57431-9-songmuchun@bytedance.com
Signed-off-by: Muchun Song <muchun.song@linux.dev>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Qi Zheng <qi.zheng@linux.dev>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Christophe Leroy <chleroy@kernel.org>
Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Lance Yang <lance.yang@linux.dev>
|
|
HugeTLB vmemmap optimization now uses per-zone shared tail vmemmap pages.
Device DAX has not been switched to that mechanism yet.
Switch device DAX to vmemmap_shared_tail_page() as well. This aligns DAX
with HugeTLB by using the common per-zone shared tail vmemmap page.
The optimization is enabled only for DEV-DAX through pgmap->vmemmap_shift,
which supplies the compound page order recorded in section metadata before
vmemmap population. Unlike FS-DAX, DEV-DAX does not modify tail struct
pages, so sharing them is safe.
Since the shared tail page can now back ZONE_DEVICE vmemmap mappings,
initialize its entries with PG_reserved for device zones. Also skip
poisoning vmemmap-optimizable sections while their struct pages may be
shared.
Each PTE mapping the shared device DAX tail page takes a page reference.
A sufficiently large range could therefore cycle the reference count back
to zero if population were allowed to continue after it became
non-positive. Use try_get_page() so further mappings fail at that point.
The section population error path tears down mappings created for the
failed section, while the warning makes this currently impractical limit
visible.
[muchun.song@linux.dev: set PG_reserved before preparing the compound tail]
Link: https://lore.kernel.org/20260930150748.1134516-1-songmuchun@bytedance.com
Link: https://lore.kernel.org/20260930140627.57431-8-songmuchun@bytedance.com
Signed-off-by: Muchun Song <muchun.song@linux.dev>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Qi Zheng <qi.zheng@linux.dev>
Cc: David Hildenbrand <david@kernel.org>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Christophe Leroy <chleroy@kernel.org>
Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Lance Yang <lance.yang@linux.dev>
|
|
The section-based vmemmap optimization infrastructure is guarded by
CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP, but it can also be used by
ZONE_DEVICE users that set dev_pagemap::vmemmap_shift. Introduce
CONFIG_VMEMMAP_OPTIMIZATION as a common config for the shared
infrastructure.
Select the new option from HUGETLB_PAGE_OPTIMIZE_VMEMMAP and from
ZONE_DEVICE when the architecture opts in to DAX vmemmap optimization, and
use it to guard the generic sparse-vmemmap state and helpers.
Link: https://lore.kernel.org/20260930140627.57431-4-songmuchun@bytedance.com
Signed-off-by: Muchun Song <muchun.song@linux.dev>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Qi Zheng <qi.zheng@linux.dev>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Christophe Leroy <chleroy@kernel.org>
Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Lance Yang <lance.yang@linux.dev>
|
|
Commit 622026e87c40 ("mm/hugetlb: remove fake head pages") added the
per-zone vmemmap_tails array. Its size depends on MAX_FOLIO_ORDER, which
had been moved to mmzone.h in preparation for the array.
PUD_ORDER is defined by linux/pgtable.h, which cannot be included from
mmzone.h without creating an include cycle. It was therefore open-coded
as PUD_SHIFT - PAGE_SHIFT.
This removed the dependency on PUD_ORDER, but not the underlying
dependency on architecture page-table definitions. PUD_SHIFT is generally
provided by architecture page-table headers, which are not guaranteed to
have been included when mmzone.h is parsed.
The dependency remained hidden because vmemmap_tails was originally
guarded by CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP. Under that condition,
MAX_FOLIO_ORDER resolves to either MAX_PAGE_ORDER or the fixed HugeTLB
limit, rather than the PUD_SHIFT-based definition.
Device DAX, however, does not require CONFIG_HUGETLB_PAGE. When it is
converted to use section-based vmemmap optimization, MAX_FOLIO_ORDER can
resolve to PUD_SHIFT - PAGE_SHIFT while it is being used to size
vmemmap_tails. This would make struct zone depend on architecture
page-table definitions being available when mmzone.h is parsed.
Replace the embedded array with a pointer and allocate it on first use.
This moves the order-count evaluation into sparse-vmemmap.c, after the
architecture page-table definitions are available, and removes the
dependency from mmzone.h.
Removing the compile-time array also removes the original reason for
keeping MAX_FOLIO_ORDER and the vmemmap optimization sizing definitions in
mmzone.h. Follow-up cleanups can place each definition in the header
owned by its respective subsystem.
Link: https://lore.kernel.org/20260930140627.57431-3-songmuchun@bytedance.com
Signed-off-by: Muchun Song <muchun.song@linux.dev>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: Oscar Salvador <osalvador@suse.de>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Qi Zheng <qi.zheng@linux.dev>
Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Christophe Leroy <chleroy@kernel.org>
Cc: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Lance Yang <lance.yang@linux.dev>
|
|
Patch series "mm/damon: introduce pgidle_set probe filter type".
Users can do effective access monitoring with DAMON, using set_pgidle
probe prep operation and pgidle_unset allowing probe filter. In the
setup, high probe_hits means the region was accessed frequently. Zero
probe_hits means the region was not accessed. The filter, however,
matches only the memory that can show the idleness. Regions of zero
probe_hits may contain memory that was just unable to show the idleness.
On virtual address space based monitoring, for example, non-present pages
are reported as cold. It is truly cold, but if the purpose is to make
some operations like reclaiming or compressing the data, this is not
really useful. Introduce a new probe filter type, pgidle_set, that
confirms the page is marked as idle. Using it instead of pgidle_unset,
users can find cold pages that can effectively be processed.
Patch 1 introduces the pgidle_set probe filter type to DAMON kernel API.
Patches 2 and 3 add support for it on physical and virtual address space
operation sets, respectively. Patch 4 adds support for it on the DAMON
sysfs interface. Finally, patch 5 updates the documentation for the new
filter type.
This patch (of 5):
DAMON_FILTER_TYPE_PGIDLE_UNSET matches only memory that was able to
confirm if it is marked as idle. If the memory cannot be marked as idle,
it simply doesn't match. Hence, finding memory that was marked as idle
with only DAMON_FILTER_TYPE_PGIDLE_UNSET is impossible. Introduce a new
filter for the purpose, DAMON_FILTER_TYPE_PGIDLE_SET.
Link: https://lore.kernel.org/20260910142234.171562-1-sj@kernel.org
Link: https://lore.kernel.org/20260910142234.171562-2-sj@kernel.org
Signed-off-by: SJ Park <sj@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
|
|
For MGLRU, memory.min/low is not honored during global proactive reclaim
(writing to the root memory.reclaim) and global direct reclaim, because
these paths shrink memcgs using stale protection (emin/elow). It can be
reproduced as follows:
# echo 7 > /sys/kernel/mm/lru_gen/enabled
# cd /sys/fs/cgroup
# mkdir -p a/b
# echo 100M > a/memory.min
# echo +memory > a/cgroup.subtree_control
# echo 100M > a/b/memory.min
# echo $$ > a/b/cgroup.procs
# dd if=/dev/zero of=/tmp/testfile bs=1M count=200
# cat a/b/memory.current
222650368
# echo 500M > memory.reclaim
-bash: echo: write error: Resource temporarily unavailable
# cat a/b/memory.current
6070272
memory.min is 100M, yet reclaim drops a/b down to 6M, breaking the
protection. The traditional LRU path is not affected because
shrink_node() calls mem_cgroup_calculate_protection() for each memcg it
visits during a top-down tree walk.
Commit 30d77b7eef01 ("mm/mglru: fix ineffective protection calculation")
moved the protection computation into lru_gen_age_node(), which only runs
for kswapd. Non-kswapd global reclaim reaches shrink_one() through
lru_gen_shrink_node() -> shrink_many() without any protection computation,
so emin/elow are whatever a previous kswapd run left behind - or zero if
kswapd never ran on this node. Relying on a prior kswapd pass is not
correct either: a memcg's emin/elow are derived from its ancestors'
memory.min/low settings and from children_min_usage, both of which change
over time, so emin/elow go stale even after kswapd has run and must be
recomputed at the point of reclaim.
Introduce mem_cgroup_calculate_protection_path() which computes emin/elow
along the root-to-target path only, by iterating through the cgroup
ancestors array top-down. This avoids the full tree traversal that would
be needed with mem_cgroup_calculate_protection(), limiting the cost to
O(depth) per memcg - typically 3-5 levels.
Call it from shrink_one() for the non-kswapd path so that each memcg about
to be shrunk has correct protection values.
Link: https://lore.kernel.org/20260907025445.1836238-3-ridong.chen@linux.dev
Fixes: e4dde56cd208 ("mm: multi-gen LRU: per-node lru_gen_folio lists")
Signed-off-by: Ridong Chen <chenridong@xiaomi.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Barry Song <baohua@kernel.org>
Reviewed-by: Johannes Weiner <hannes@cmpxchg.org>
Assisted-by: Claude:claude-opus-4-8
Cc: Axel Rasmussen <axelrasmussen@google.com>
Cc: Chris Down <chris@chrisdown.name>
Cc: David Hildenbrand <david@kernel.org>
Cc: Kairui Song <kasong@tencent.com>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Muchun Song <muchun.song@linux.dev>
Cc: Roman Gushchin <roman.gushchin@linux.dev>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Tejun Heo <tj@kernel.org>
Cc: Wei Xu <weixugc@google.com>
Cc: Yuanchu Xie <yuanchu@google.com>
Cc: Yu Zhao <yuzhao@google.com>
Cc: <stable@vger.kernel.org>
|
