| Age | Commit message (Collapse) | Author |
|---|
|
f2fs_unlock_cache() and f2fs_end_cache_writeback() take a temporary
reference on the entry around clear_and_wake_up_bit(), so that the entry
is not freed before wake_up_bit() returns. f2fs_destroy_cache() waits
for the LOCKED and WRITEBACK bits and then expects the refcount to be 1,
but the bit waits can return as soon as the bit is cleared, before the
I/O completion drops the temporary reference. When the completion runs
in process context and is preempted there (e.g. dm-flakey in
generic/311), umount hits:
kernel BUG at fs/f2fs/cache.c:567!
f2fs_destroy_cache+0x260/0x268
f2fs_put_super+0x1fc/0x428
Count the temporary references in units of F2FS_CACHE_PIN_BIAS, like
GUP_PIN_COUNTING_BIAS for folios, and check only the regular references
in f2fs_destroy_cache(). A leaked regular reference is still caught
there, without waiting for the completion.
The pinned entry is freed by whichever of f2fs_cache_put() and
f2fs_cache_unpin() drops the refcount to zero, as before. The shrinker
still skips an entry while it is pinned, since the refcount is then
neither 1 nor below 3.
Fixes: 6e392158cf54 ("f2fs: cache: pin cached block in f2fs_end_cache_writeback()")
Fixes: 61ba87b33e87 ("f2fs: cache: pin cached block in f2fs_unlock_cache()")
Signed-off-by: Daeho Jeong <daehojeong@google.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
|
|
This patch introduces cache_wb_dirty_threshold and cache_wb_total_threshold
sysfs nodes under /sys/fs/f2fs/<disk>/, and wakes up the f2fs_writeback
kthread when dirty meta/node caches exceed cache_wb_dirty_threshold and
total cached entries exceed cache_wb_total_threshold; a zero threshold
disables the corresponding check. By default, cache_wb_dirty_threshold is
8192 and cache_wb_total_threshold is 0.
If both thresholds are zero, f2fs_writeback only runs periodically, unless
checkpoint is disabled, as nothing else will flush dirty caches then.
f2fs_writeback won't be woken up unless there are enough dirty caches for
f2fs_write_{meta,node}_caches() to gather. And if thresholds are exceeded
but one round fails to write any caches, e.g. due to cp_error, freezing,
frozen fs, or lock contention, threshold based wakeup is throttled until
a later round makes progress, in order to avoid busy-looping. To detect
progress, introduce a new parameter for f2fs_write_{meta,node}_caches()
and f2fs_writeback_node_caches() to return the number of caches submitted
for writeback.
Thanks Daeho and Wenjie for help and suggestion on this patch.
Suggested-by: Daeho Jeong <daehojeong@google.com>
Suggested-by: Wenjie Qi <qiwenjie@xiaomi.com>
Signed-off-by: Chao Yu <chao@kernel.org>
Reviewed-by: Wenjie Qi <qiwenjie@xiaomi.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
|
|
This patch introduces META_BLOCK and NODE_BLOCK memory types to monitor
meta and node cache usage against available free memory, and integrates
them into f2fs_balance_fs_bg().
In f2fs_balance_fs_bg(), shrink 128 clean cached blocks of each types when
memory is insufficient (cache memory usage is larger than 50% of low memory).
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
|
|
During asynchronous read readahead (e.g., in f2fs_ra_node_cache()), the
caller drops its own reference via f2fs_put_cache(entry, false) immediately
after issuing the read bio, leaving only the radix tree holding an active
reference during I/O flight.
In f2fs_unlock_cache(), clear_and_wake_up_bit() executes two steps:
1. clear_bit_unlock(F2FS_BLOCK_LOCKED, &entry->state): clears the bit.
2. wake_up_bit(&entry->state, F2FS_BLOCK_LOCKED): hashes &entry->state
to find the waitqueue and wake sleeping waiters.
Once step 1 clears the bit, a concurrent waiter in f2fs_lock_cache()
(such as in f2fs_truncate_cache() or f2fs_drop_cache()) is immediately
unblocked. The waiter can acquire the lock, delete the entry from the
radix tree, drop the remaining reference, and kfree() the entry before
step 2 completes. This causes wake_up_bit() to dereference freed memory:
CPU 0 (Read I/O completion) CPU 1 (f2fs_drop_cache / Truncation)
- f2fs_cache_read_end_io()
- f2fs_unlock_cache(entry)
- clear_and_wake_up_bit()
- clear_bit_unlock(LOCKED)
: bit is cleared!
- f2fs_lock_cache(entry)
: acquires lock!
- f2fs_truncate_locked_cache(entry)
- radix_tree_delete(&cache->root, ...)
- entry->cache = NULL;
- atomic_dec(&entry->refcount);
- f2fs_unlock_cache(entry);
- f2fs_put_cache(entry, false);
- atomic_dec_and_test(&refcount) == 0
- f2fs_do_free_cache(entry)
- kfree(entry->data);
- kfree(entry); <--- FREED!
- smp_mb__after_atomic()
- wake_up_bit(&entry->state, ...)
: Dereferences &entry->state on freed entry! (UAF)
Like commit ("f2fs: cache: pin cached block in f2fs_end_cache_writeback()"),
acquire a temporary reference via f2fs_cache_get() before clear_and_wake_up_bit()
and release it with f2fs_cache_put() once wake_up_bit() completes.
This fixes commit 399410a90ca7 ("f2fs: cache: implement metadata cache")
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
|
|
Like page cache folios, writeback does not hold an active reference on
the cached block during I/O flight, relying instead on truncation or
shrinker callers to wait on F2FS_BLOCK_WRITEBACK before releasing the entry.
However, in f2fs_end_cache_writeback(), clear_and_wake_up_bit() executes
two distinct steps:
1. clear_bit_unlock(F2FS_BLOCK_WRITEBACK, &entry->state): clears the bit.
2. wake_up_bit(&entry->state, F2FS_BLOCK_WRITEBACK): hashes &entry->state
to look up the waitqueue and wakes waiting tasks.
Once step 1 clears the bit, a concurrent waiter in f2fs_do_truncate_cache()
or f2fs_do_shrink_cache() is immediately unblocked. The waiter can proceed
to delete the entry from the radix tree, drop the final reference, and
kfree() the entry before step 2 finishes, causing wake_up_bit() to access
freed memory:
CPU 0 (I/O completion) CPU 1 (Truncation / Shrinker)
- f2fs_cache_write_end_io()
- f2fs_end_cache_writeback(entry)
- clear_and_wake_up_bit()
- clear_bit_unlock(WRITEBACK)
: bit is cleared!
- f2fs_do_truncate_cache(entry)
- f2fs_cache_wait_writeback(entry)
: sees WRITEBACK cleared!
- radix_tree_delete(&cache->root, ...)
- f2fs_put_cache(entry, true)
- atomic_dec_and_test(&refcount) == 0
- f2fs_do_free_cache(entry)
- kfree(entry->data);
- kfree(entry); <--- FREED!
- smp_mb__after_atomic()
- wake_up_bit(&entry->state, ...)
: Dereferences &entry->state on freed entry! (UAF)
Mirror the logic in folio_end_writeback() by acquiring a temporary reference
via f2fs_cache_get() before clear_and_wake_up_bit() and releasing it with
f2fs_cache_put() once wake_up_bit() completes. This guarantees the entry
can not be freed until wake_up_bit() has finished.
This fixes commit 399410a90ca7 ("f2fs: cache: implement metadata cache").
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
|
|
The count does not indicate the number of pages anymore.
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
|
|
This patch introduces ftrace tracepoints to observe and profile metadata
cache operations:
- trace_f2fs_cache_set_dirty to trace marking a cached block dirty
- trace_f2fs_write_cache to trace single block writeback submission
- trace_f2fs_write_caches to tracesbatch writeback and sync sessions
Reviewed-by: Zhiguo Niu <zhiguo.niu@unisoc.com>
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
|
|
This patch adds fault injection support for metadata cache allocations
to improve error-path test coverage.
- it integrates entry allocations with FAULT_KMALLOC
Reviewed-by: Zhiguo Niu <zhiguo.niu@unisoc.com>
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
|
|
This patch introduces compress_blocks in f2fs_sb_info structure, initializes
and destroys the compress cache during filesystem mount and unmount.
It adds an ino union field in struct f2fs_cached_block (sharing space
with writeback linkage for zero memory overhead) to track per-inode
cached blocks, and registers compress cache into the memory shrinker.
Reviewed-by: Zhiguo Niu <zhiguo.niu@unisoc.com>
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
|
|
This patch migrates F2FS node block caching from the fake VFS inode
page cache (sbi->node_inode) to node cache (sbi->node_blocks).
It updates node related helpers to use node cache APIs and reference to
struct f2fs_cached_block, and removes sbi->node_inode, especially, unifies
inline and regular dentry block handling via struct f2fs_dentry_block_ref.
Reviewed-by: Zhiguo Niu <zhiguo.niu@unisoc.com>
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
|
|
This patch introduces node_blocks in f2fs_sb_info structure, initializes
and destroys the node cache during filesystem mount and unmount.
It also introduces helper wrappers for node cache operation, and registers
node cache into the memory shrinker and writeback kthread.
Reviewed-by: Zhiguo Niu <zhiguo.niu@unisoc.com>
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
|
|
This patch migrates F2FS meta block caching from the fake VFS inode
page cache (sbi->meta_inode) to meta cache (sbi->meta_blocks).
It converts CP, SIT, NAT, SSA, recovery, and GC metadata I/O paths to
operate on struct f2fs_cached_block instead of folio, and removes
sbi->meta_inode.
Reviewed-by: Zhiguo Niu <zhiguo.niu@unisoc.com>
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
|
|
This patch introduces a background writeback kthread (f2fs_writeback-x:y)
to periodically flush dirty metadata cache entries with a default
interval of 5 seconds.
It manages thread lifecycle across mount, unmount, and remount (rw/ro)
transitions.
It introduces a sysfs entry /sys/fs/f2fs/<disk>/cache_wb_interval to control
writeback interval.
Reviewed-by: Zhiguo Niu <zhiguo.niu@unisoc.com>
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
|
|
This patch integrates the metadata cache into the F2FS memory shrinker
subsystem to reclaim clean, unreferenced cached blocks under memory
pressure.
It implements f2fs_shrink_cache() using a 3-phase cache reclamin method:
1. isolate clean entries from lru list
2. truncate from radix tree under lock
3. splice un-reclaimed entries back
And hooks the new interface into f2fs_shrink_count() and f2fs_shrink_scan().
Reviewed-by: Zhiguo Niu <zhiguo.niu@unisoc.com>
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
|
|
This patch introduces the core metadata block caching infrastructure to
manage f2fs metadata independently of the page cache.
It implements:
- core cache APIs: get, create, put, drop, backed by a radix tree and
a single global LRU list.
- support multiple status of cached block: LOCKED, UPTODATE, DIRTY,
WRITEBACK, INLINE.
- internal bio based read/write helpers with adjacent block vector merging.
Reviewed-by: Zhiguo Niu <zhiguo.niu@unisoc.com>
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
|
