summaryrefslogtreecommitdiff
path: root/net/sunrpc/svc.c
AgeCommit message (Collapse)Author
2023-10-16SUNRPC: Remove BUG_ON call sitesChuck Lever
There is no need to take down the whole system for these assertions. I'd rather not attempt a heroic save here, as some bug has occurred that has left the transport data structures in an unknown state. Just warn and then leak the left-over resources. Acked-by: Christian Brauner <brauner@kernel.org> Reviewed-by: NeilBrown <neilb@suse.de> Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-10-16SUNRPC: change the back-channel queue to lwqNeilBrown
This removes the need to store and update back-links in the list. It also remove the need for the _bh version of spin_lock(). Signed-off-by: NeilBrown <neilb@suse.de> Cc: Trond Myklebust <trond.myklebust@hammerspace.com> Cc: Anna Schumaker <Anna.Schumaker@Netapp.com> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-10-16SUNRPC: discard sp_lockNeilBrown
sp_lock is now only used to protect sp_all_threads. This isn't needed as sp_all_threads is only manipulated through svc_set_num_threads(), which is already serialized. Read-acccess only requires rcu_read_lock(). So no more locking is needed. Signed-off-by: NeilBrown <neilb@suse.de> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-10-16SUNRPC: change sp_nrthreads to atomic_tNeilBrown
Using an atomic_t avoids the need to take a spinlock (which can soon be removed). Choosing a thread to kill needs to be careful as we cannot set the "die now" bit atomically with the test on the count. Instead we temporarily increase the count. Signed-off-by: NeilBrown <neilb@suse.de> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-10-16SUNRPC: use lwq for sp_sockets - renamed to sp_xprtsNeilBrown
lwq avoids using back pointers in lists, and uses less locking. This introduces a new spinlock, but the other one will be removed in a future patch. For svc_clean_up_xprts(), we now dequeue the entire queue, walk it to remove and process the xprts that need cleaning up, then re-enqueue the remaining queue. Signed-off-by: NeilBrown <neilb@suse.de> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-10-16SUNRPC: only have one thread waking up at a timeNeilBrown
Currently if several items of work become available in quick succession, that number of threads (if available) will be woken. By the time some of them wake up another thread that was already cache-warm might have come along and completed the work. Anecdotal evidence suggests as many as 15% of wakes find nothing to do once they get to the point of looking. This patch changes svc_pool_wake_idle_thread() to wake the first thread on the queue but NOT remove it. Subsequent calls will wake the same thread. Once that thread starts it will dequeue itself and after dequeueing some work to do, it will wake the next thread if there is more work ready. This results in a more orderly increase in the number of busy threads. As a bonus, this allows us to reduce locking around the idle queue. svc_pool_wake_idle_thread() no longer needs to take a lock (beyond rcu_read_lock()) as it doesn't manipulate the queue, it just looks at the first item. The thread itself can avoid locking by using the new llist_del_first_this() interface. This will safely remove the thread itself if it is the head. If it isn't the head, it will do nothing. If multiple threads call this concurrently only one will succeed. The others will do nothing, so no corruption can result. If a thread wakes up and finds that it cannot dequeue itself that means either - that it wasn't woken because it was the head of the queue. Maybe the freezer woke it. In that case it can go back to sleep (after trying to freeze of course). - some other thread found there was nothing to do very recently, and placed itself on the head of the queue in front of this thread. It must check again after placing itself there, so it can be deemed to be responsible for any pending work, and this thread can go back to sleep until woken. No code ever tests for busy threads any more. Only each thread itself cares if it is busy. So svc_thread_busy() is no longer needed. Signed-off-by: NeilBrown <neilb@suse.de> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-10-16SUNRPC: change service idle list to be an llistNeilBrown
With an llist we don't need to take a lock to add a thread to the list, though we still need a lock to remove it. That will go in the next patch. Unlike double-linked lists, a thread cannot reliably remove itself from the list. Only the first thread can be removed, and that can change asynchronously. So some care is needed. We already check if there is pending work to do, so we are unlikely to add ourselves to the idle list and then want to remove ourselves again. If we DO find something needs to be done after adding ourselves to the list, we simply wake up the first thread on the list. If that was us, we successfully removed ourselves and can continue. If it was some other thread, they will do the work that needs to be done. We can safely sleep until woken. We also remove the test on freezing() from rqst_should_sleep(). Instead we set TASK_FREEZABLE before scheduling. This makes is safe to schedule() when a freeze is pending. As we now loop waiting to be removed from the idle queue, this is a cleaner way to handle freezing. Signed-off-by: NeilBrown <neilb@suse.de> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-10-16SUNRPC: discard SP_CONGESTEDNeilBrown
We can tell if a pool is congested by checking if the idle list is empty. We don't need a separate flag. Signed-off-by: NeilBrown <neilb@suse.de> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-10-16SUNRPC: add list of idle threadsNeilBrown
Rather than searching a list of threads to find an idle one, having a list of idle threads allows an idle thread to be found immediately. This adds some spin_lock calls which is not ideal, but as the hold-time is tiny it is still faster than searching a list. A future patch will remove them using llist.h. This involves some subtlety and so is left to a separate patch. This removes the need for the RQ_BUSY flag. The rqst is "busy" precisely when it is not on the "idle" list. Signed-off-by: NeilBrown <neilb@suse.de> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-10-16SUNRPC: change how svc threads are asked to exit.NeilBrown
svc threads are currently stopped using kthread_stop(). This requires identifying a specific thread. However we don't care which thread stops, just as long as one does. So instead, set a flag in the svc_pool to say that a thread needs to die, and have each thread check this flag instead of calling kthread_should_stop(). The first thread to find and clear this flag then moves towards exiting. This removes an explicit dependency on sp_all_threads which will make a future patch simpler. Signed-off-by: NeilBrown <neilb@suse.de> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-10-16SUNRPC: integrate back-channel processing with svc_recv()NeilBrown
Using svc_recv() for (NFSv4.1) back-channel handling means we have just one mechanism for waking threads. Also change kthread_freezable_should_stop() in nfs4_callback_svc() to kthread_should_stop() as used elsewhere. kthread_freezable_should_stop() effectively adds a try_to_freeze() call, and svc_recv() already contains that at an appropriate place. Signed-off-by: NeilBrown <neilb@suse.de> Cc: Trond Myklebust <trond.myklebust@hammerspace.com> Cc: Anna Schumaker <Anna.Schumaker@Netapp.com> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-10-16SUNRPC: Clean up bc_svc_process()Chuck Lever
The test robot complained that, in some build configurations, the @error variable in bc_svc_process's only caller is set but never used. This happens because dprintk() is the only consumer of that value. - Remove the dprintk() call sites in favor of the svc_process tracepoint - The @error variable and the return value of bc_svc_process() are now unused, so get rid of them. - The @serv parameter is set to rqstp->rq_serv by the only caller, and bc_svc_process() then uses it only to set rqstp->rq_serv. It can be removed. - Rename bc_svc_process() according to the convention that globally-visible RPC server functions have names that begin with "svc_"; and because it is globally-visible, give it a proper kdoc comment. Reported-by: kernel test robot <lkp@intel.com> Closes: https://lore.kernel.org/oe-kbuild-all/202308121314.HA8Rq2XG-lkp@intel.com/ Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-08-31Merge tag 'nfs-for-6.6-1' of git://git.linux-nfs.org/projects/anna/linux-nfsLinus Torvalds
Pull NFS client updates from Anna Schumaker: "New Features: - Enable the NFS v4.2 READ_PLUS operation by default Stable Fixes: - NFSv4/pnfs: minor fix for cleanup path in nfs4_get_device_info - NFS: Fix a potential data corruption Bugfixes: - Fix various READ_PLUS issues including: - smatch warnings - xdr size calculations - scratch buffer handling - 32bit / highmem xdr page handling - Fix checkpatch errors in file.c - Fix redundant readdir request after an EOF - Fix handling of COPY ERR_OFFLOAD_NO_REQ - Fix assignment of xprtdata.cred Cleanups: - Remove unused xprtrdma function declarations - Clean up an integer overflow check to avoid a warning - Clean up #includes in dns_resolve.c - Clean up nfs4_get_device_info so we don't pass a NULL pointer to __free_page() - Clean up sunrpc TCP socket timeout configuration - Guard against READDIR loops when entry names are too long - Use EXCHID4_FLAG_USE_PNFS_DS for DS servers" * tag 'nfs-for-6.6-1' of git://git.linux-nfs.org/projects/anna/linux-nfs: (22 commits) pNFS: Fix assignment of xprtdata.cred NFSv4.2: fix handling of COPY ERR_OFFLOAD_NO_REQ NFS: Guard against READDIR loop when entry names exceed MAXNAMELEN NFSv4.1: use EXCHGID4_FLAG_USE_PNFS_DS for DS server NFS/pNFS: Set the connect timeout for the pNFS flexfiles driver SUNRPC: Don't override connect timeouts in rpc_clnt_add_xprt() SUNRPC: Allow specification of TCP client connect timeout at setup SUNRPC: Refactor and simplify connect timeout SUNRPC: Set the TCP_SYNCNT to match the socket timeout NFS: Fix a potential data corruption nfs: fix redundant readdir request after get eof nfs/blocklayout: Use the passed in gfp flags filemap: Fix errors in file.c NFSv4/pnfs: minor fix for cleanup path in nfs4_get_device_info NFS: Move common includes outside ifdef SUNRPC: clean up integer overflow check xprtrdma: Remove unused function declaration rpcrdma_bc_post_recv() NFS: Enable the READ_PLUS operation by default SUNRPC: kmap() the xdr pages during decode NFSv4.2: Rework scratch handling for READ_PLUS (again) ...
2023-08-29SUNRPC: Remove return value of svc_pool_wake_idle_thread()NeilBrown
The returned value is not used (any more), so don't return it. Signed-off-by: NeilBrown <neilb@suse.de> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-08-29SUNRPC: Clean up svc_set_num_threadsChuck Lever
Document the API contract and remove stale or obvious comments. Reviewed-by: Jeff Layton <jlayton@redhat.com> Reviewed-by: NeilBrown <neilb@suse.de> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-08-29SUNRPC: Count ingress RPC messages per svc_poolChuck Lever
svc_xprt_enqueue() can be costly, since it involves selecting and waking up a process. More than one enqueue is done per incoming RPC. For example, svc_data_ready() enqueues, and so does svc_xprt_receive(). Also, if an RPC message requires more than one call to ->recvfrom() to receive it fully, each one of those calls does an enqueue. To get a sense of the average number of transport enqueue operations needed to process an incoming RPC message, re-use the "packets" pool stat. Track the number of complete RPC messages processed by each thread pool. Reviewed-by: Jeff Layton <jlayton@redhat.com> Reviewed-by: NeilBrown <neilb@suse.de> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-08-29SUNRPC: Deduplicate thread wake-up codeChuck Lever
Refactor: Extract the loop that finds an idle service thread from svc_xprt_enqueue() and svc_wake_up(). Both functions do just about the same thing. Note that svc_wake_up() currently does not hold the RCU read lock while waking the target thread. It indeed should hold the lock, just as svc_xprt_enqueue() does, to ensure the rqstp does not vanish during the wake-up. This patch adds the RCU lock for svc_wake_up(). Note that shrinking the pool thread count is rare, and calls to svc_wake_up() are also quite infrequent. In practice, this race is very unlikely to be hit, so we are not marking the lock fix for stable backport at this time. Reviewed-by: Jeff Layton <jlayton@redhat.com> Reviewed-by: NeilBrown <neilb@suse.de> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-08-29SUNRPC: Add enum svc_auth_statusChuck Lever
In addition to the benefits of using an enum rather than a set of macros, we now have a named type that can improve static type checking of function return values. As part of this change, I removed a stale comment from svcauth.h; the return values from current implementations of the auth_ops::release method are all zero/negative errno, not the SVC_OK enum values as the old comment suggested. Suggested-by: NeilBrown <neilb@suse.de> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-08-29SUNRPC: remove timeout arg from svc_recv()NeilBrown
Most svc threads have no interest in a timeout. nfsd sets it to 1 hour, but this is a wart of no significance. lockd uses the timeout so that it can call nlmsvc_retry_blocked(). It also sometimes calls svc_wake_up() to ensure this is called. So change lockd to be consistent and always use svc_wake_up() to trigger nlmsvc_retry_blocked() - using a timer instead of a timeout to svc_recv(). And change svc_recv() to not take a timeout arg. This makes the sp_threads_timedout counter always zero. Signed-off-by: NeilBrown <neilb@suse.de> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-08-29SUNRPC: call svc_process() from svc_recv().NeilBrown
All callers of svc_recv() go on to call svc_process() on success. Simplify callers by having svc_recv() do that for them. This loses one call to validate_process_creds() in nfsd. That was debugging code added 14 years ago. I don't think we need to keep it. Signed-off-by: NeilBrown <neilb@suse.de> Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-08-23SUNRPC: kmap() the xdr pages during decodeAnna Schumaker
If the pages are in HIGHMEM then we need to make sure they're mapped before trying to read data off of them, otherwise we could end up with a NULL pointer dereference. The downside to this is that we need an extra cleanup step at the end of decode to kunmap() the last page. I introduced an xdr_finish_decode() function to do this. Right now this function only calls the unmap_current_page() function, but other generic cleanup steps could be added in the future if we come across anything else. Reported-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org> Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
2023-06-28Merge tag 'mm-stable-2023-06-24-19-15' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm Pull mm updates from Andrew Morton: - Yosry Ahmed brought back some cgroup v1 stats in OOM logs - Yosry has also eliminated cgroup's atomic rstat flushing - Nhat Pham adds the new cachestat() syscall. It provides userspace with the ability to query pagecache status - a similar concept to mincore() but more powerful and with improved usability - Mel Gorman provides more optimizations for compaction, reducing the prevalence of page rescanning - Lorenzo Stoakes has done some maintanance work on the get_user_pages() interface - Liam Howlett continues with cleanups and maintenance work to the maple tree code. Peng Zhang also does some work on maple tree - Johannes Weiner has done some cleanup work on the compaction code - David Hildenbrand has contributed additional selftests for get_user_pages() - Thomas Gleixner has contributed some maintenance and optimization work for the vmalloc code - Baolin Wang has provided some compaction cleanups, - SeongJae Park continues maintenance work on the DAMON code - Huang Ying has done some maintenance on the swap code's usage of device refcounting - Christoph Hellwig has some cleanups for the filemap/directio code - Ryan Roberts provides two patch series which yield some rationalization of the kernel's access to pte entries - use the provided APIs rather than open-coding accesses - Lorenzo Stoakes has some fixes to the interaction between pagecache and directio access to file mappings - John Hubbard has a series of fixes to the MM selftesting code - ZhangPeng continues the folio conversion campaign - Hugh Dickins has been working on the pagetable handling code, mainly with a view to reducing the load on the mmap_lock - Catalin Marinas has reduced the arm64 kmalloc() minimum alignment from 128 to 8 - Domenico Cerasuolo has improved the zswap reclaim mechanism by reorganizing the LRU management - Matthew Wilcox provides some fixups to make gfs2 work better with the buffer_head code - Vishal Moola also has done some folio conversion work - Matthew Wilcox has removed the remnants of the pagevec code - their functionality is migrated over to struct folio_batch * tag 'mm-stable-2023-06-24-19-15' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (380 commits) mm/hugetlb: remove hugetlb_set_page_subpool() mm: nommu: correct the range of mmap_sem_read_lock in task_mem() hugetlb: revert use of page_cache_next_miss() Revert "page cache: fix page_cache_next/prev_miss off by one" mm/vmscan: fix root proactive reclaim unthrottling unbalanced node mm: memcg: rename and document global_reclaim() mm: kill [add|del]_page_to_lru_list() mm: compaction: convert to use a folio in isolate_migratepages_block() mm: zswap: fix double invalidate with exclusive loads mm: remove unnecessary pagevec includes mm: remove references to pagevec mm: rename invalidate_mapping_pagevec to mapping_try_invalidate mm: remove struct pagevec net: convert sunrpc from pagevec to folio_batch i915: convert i915_gpu_error to use a folio_batch pagevec: rename fbatch_count() mm: remove check_move_unevictable_pages() drm: convert drm_gem_put_pages() to use a folio_batch i915: convert shmem_sg_free_table() to use a folio_batch scatterlist: add sg_set_folio() ...
2023-06-23net: convert sunrpc from pagevec to folio_batchMatthew Wilcox (Oracle)
Remove the last usage of pagevecs. There is a slight change here; we now free the folio_batch as soon as it fills up instead of freeing the folio_batch when we try to add a page to a full batch. This should have no effect in practice. Link: https://lkml.kernel.org/r/20230621164557.3510324-10-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Acked-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2023-06-17SUNRPC: Address RCU warning in net/sunrpc/svc.cChuck Lever
$ make C=1 W=1 net/sunrpc/svc.o make[1]: Entering directory 'linux/obj/manet.1015granger.net' GEN Makefile CALL linux/server-development/scripts/checksyscalls.sh DESCEND objtool INSTALL libsubcmd_headers DESCEND bpf/resolve_btfids INSTALL libsubcmd_headers CC [M] net/sunrpc/svc.o CHECK linux/server-development/net/sunrpc/svc.c linux/server-development/net/sunrpc/svc.c:1225:9: warning: incorrect type in argument 1 (different address spaces) linux/server-development/net/sunrpc/svc.c:1225:9: expected struct spinlock [usertype] *lock linux/server-development/net/sunrpc/svc.c:1225:9: got struct spinlock [noderef] __rcu * linux/server-development/net/sunrpc/svc.c:1227:40: warning: incorrect type in argument 1 (different address spaces) linux/server-development/net/sunrpc/svc.c:1227:40: expected struct spinlock [usertype] *lock linux/server-development/net/sunrpc/svc.c:1227:40: got struct spinlock [noderef] __rcu * make[1]: Leaving directory 'linux/obj/manet.1015granger.net' Warning introduced by commit 913292c97d75 ("sched.h: Annotate sighand_struct with __rcu"). Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-06-17SUNRPC: Use sysfs_emit in place of strlcpy/sprintfAzeem Shaikh
Part of an effort to remove strlcpy() tree-wide [1]. Direct replacement is safe here since the getter in kernel_params_ops handles -errno return [2]. [1] https://github.com/KSPP/linux/issues/89 [2] https://elixir.bootlin.com/linux/v6.4-rc6/source/include/linux/moduleparam.h#L52 Signed-off-by: Azeem Shaikh <azeemshaikh38@gmail.com> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-06-05SUNRPC: Use __alloc_bulk_pages() in svc_init_buffer()Chuck Lever
Clean up: Use the bulk page allocator when filling a server thread's buffer page array. Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-05-14SUNRPC: Fix trace_svc_register() call siteChuck Lever
The trace event recorded incorrect values for the registered family, protocol, and port because the arguments are in the wrong order. Fixes: b4af59328c25 ("SUNRPC: Trace server-side rpcbind registration events") Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-05-03SUNRPC: Fix encoding of accepted but unsuccessful RPC repliesChuck Lever
Jiri Slaby says: > I bisected to this ... as it breaks nfs3-only servers in 6.3. > I.e. /etc/nfs.conf containing: > [nfsd] > vers4=no > > The client sees: > mount("10.0.2.15:/tmp", "/mnt", "nfs", 0, "vers=4.2,addr=10.0.2.15,clientad"...) = -1 EIO (Input/output error) > write(2, "mount.nfs: mount system call fai"..., 45 > mount.nfs: mount system call failed for /mnt > > And the kernel says: > nfs4_discover_server_trunking unhandled error -5. Exiting with error EIO Reported-by: Jiri Slaby <jirislaby@kernel.org> Link: https://bugzilla.suse.com/show_bug.cgi?id=1210995 Fixes: 4bcf0343e8a6 ("SUNRPC: Set rq_accept_statp inside ->accept methods") Tested-by: Jiri Slaby <jirislaby@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-04-27SUNRPC: Be even lazier about releasing pagesChuck Lever
A single RPC transaction that touches only a couple of pages means rq_pvec will not be even close to full in svc_xpt_release(). This is a common case. Instead, just leave the pages in rq_pvec until it is completely full. This improves the efficiency of the batch release mechanism on workloads that involve small RPC messages. The rq_pvec is also fully emptied just before thread exit. Reviewed-by: Calum Mackay <calum.mackay@oracle.com> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-04-26SUNRPC: Convert svc_xprt_release() to the release_pages() APIChuck Lever
Instead of invoking put_page() one-at-a-time, pass the "response" portion of rq_pages directly to release_pages() to reduce the number of times each nfsd thread invokes a page allocator API. Since svc_xprt_release() is not invoked while a client is waiting for an RPC Reply, this is not expected to directly impact mean request latencies on a lightly or moderately loaded server. However as workload intensity increases, I expect somewhat better scalability: the same number of server threads should be able to handle more work. Reviewed-by: Calum Mackay <calum.mackay@oracle.com> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-04-26SUNRPC: Relocate svc_free_res_pages()Chuck Lever
Clean-up: There doesn't seem to be a reason why this function is stuck in a header. One thing it prevents is the convenient addition of tracing. Moving it to a source file also makes the rq_respages clean-up logic easier to find. Reviewed-by: Calum Mackay <calum.mackay@oracle.com> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-04-26SUNRPC: Ignore return value of ->xpo_sendtoChuck Lever
Clean up: All callers of svc_process() ignore its return value, so svc_process() can safely be converted to return void. Ditto for svc_send(). The return value of ->xpo_sendto() is now used only as part of a trace event. Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-04-26NFSD: Watch for rq_pages bounds checking errors in nfsd_splice_actor()Chuck Lever
There have been several bugs over the years where the NFSD splice actor has attempted to write outside the rq_pages array. This is a "should never happen" condition, but if for some reason the pipe splice actor should attempt to walk past the end of rq_pages, it needs to terminate the READ operation to prevent corruption of the pointer addresses in the fields just beyond the array. A server crash is thus prevented. Since the code is not behaving, the READ operation returns -EIO to the client. None of the READ payload data can be trusted if the splice actor isn't operating as expected. Suggested-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Reviewed-by: Jeff Layton <jlayton@kernel.org>
2023-03-08SUNRPC: Fix a server shutdown leakBenjamin Coddington
Fix a race where kthread_stop() may prevent the threadfn from ever getting called. If that happens the svc_rqst will not be cleaned up. Fixes: ed6473ddc704 ("NFSv4: Fix callback server shutdown") Signed-off-by: Benjamin Coddington <bcodding@redhat.com> Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-02-20SUNRPC: Replace pool stats with per-CPU variablesChuck Lever
Eliminate the use of bus-locked operations in svc_xprt_enqueue(), which is a hot path. Replace them with per-cpu variables to reduce cross-CPU memory bus traffic. Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-02-20SUNRPC: Use per-CPU counters to tally server RPC countsChuck Lever
- Improves counting accuracy - Reduces cross-CPU memory traffic Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-02-20SUNRPC: Set rq_accept_statp inside ->accept methodsChuck Lever
To navigate around the space that svcauth_gss_accept() reserves for the RPC payload body length and sequence number fields, svcauth_gss_release() does a little dance with the reply's accept_stat, moving the accept_stat value in the response buffer down by two words. Instead, let's have the ->accept() methods each set the proper final location of the accept_stat to avoid having to move things. Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-02-20SUNRPC: Refactor RPC server dispatch methodChuck Lever
Currently, svcauth_gss_accept() pre-reserves response buffer space for the RPC payload length and GSS sequence number before returning to the dispatcher, which then adds the header's accept_stat field. The problem is the accept_stat field is supposed to go before the length and seq_num fields. So svcauth_gss_release() has to relocate the accept_stat value (see svcauth_gss_prepare_to_wrap()). To enable these fields to be added to the response buffer in the correct (final) order, the pointer to the accept_stat has to be made available to svcauth_gss_accept() so that it can set it before reserving space for the length and seq_num fields. As a first step, move the pointer to the location of the accept_stat field into struct svc_rqst. Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-02-20SUNRPC: Final clean-up of svc_process_common()Chuck Lever
The @resv parameter is no longer used. Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-02-20SUNRPC: Convert RPC Reply header encoding to use xdr_streamChuck Lever
The main part of RPC header encoding and the formation of error responses are now done using the xdr_stream helpers. Bounds checking before each XDR data item is encoded makes the server's encoding path safer against accidental buffer overflows. Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-02-20SUNRPC: Hoist init_encode out of svc_authenticate()Chuck Lever
Now that each ->accept method has been converted, the svcxdr_init_encode() calls can be hoisted back up into the generic RPC server code. Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-02-20SUNRPC: Move svcxdr_init_encode() into ->accept methodsChuck Lever
Refactor: So that the overhaul of each ->accept method can be done in separate smaller patches, temporarily move the svcxdr_init_encode() call into those methods. Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-02-20SUNRPC: Push svcxdr_init_encode() into svc_process_common()Chuck Lever
Now that all vs_dispatch functions invoke svcxdr_init_encode(), it is common code and can be pushed down into the generic RPC server. Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-02-20SUNRPC: Remove the rpc_stat variable in svc_process_common()Chuck Lever
There's no RPC header field called rpc_stat; more precisely, the variable appears to be recording an accept_stat value. But it looks like we don't need to preserve this value at all, actually, so simply remove the variable. Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-02-20SUNRPC: Hoist svcxdr_init_decode() into svc_process()Chuck Lever
Now the entire RPC Call header parsing path is handled via struct xdr_stream-based decoders. Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-02-20SUNRPC: Remove svc_process_common's argv parameterChuck Lever
Clean up: With xdr_stream decoding, the @argv parameter is no longer used. Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-02-20SUNRPC: Decode most of RPC header with xdr_streamChuck Lever
Done as part of hardening the server-side RPC header decoding path. Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-02-20SUNRPC: Eliminate unneeded variableChuck Lever
Clean up: Saving the RPC program number in two places is unnecessary. Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-02-20SUNRPC: Re-order construction of the first reply fieldsChuck Lever
Clean up: Group these together for legibility. Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2023-02-20SUNRPC: Hoist init_decode out of svc_authenticate()Chuck Lever
Now that each ->accept method has been converted to use xdr_stream, the svcxdr_init_decode() calls can be hoisted back up into the generic RPC server code. The dprintk in svc_authenticate() is removed, since trace_svc_authenticate() reports the same information. Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>