ipc: sem: fix used_sems overflow in newary()

newary() checks the namespace-wide semaphore limit using a signed
addition of used_sems and nsems.  If that addition overflows, the
comparison can fail to reject a request which exceeds sc_semmns.

The default semaphore limits cannot trigger this overflow.  It requires
raised limits and substantial existing allocations.  sem_alloc()
separately bounds each array's allocation size, so a single enormous
request is insufficient to demonstrate a successful quota bypass.

With enough memory, semget() can succeed instead of returning ENOSPC,
allowing allocations beyond the namespace semaphore quota and
increasing memory pressure.  The wrapped used_sems count can also cause
SEM_INFO to report a negative semaphore usage count.

Use check_add_overflow() before comparing the new total against
sc_semmns, rejecting overflow with ENOSPC.  The existing writer lock
covers both the check and the later accounting update.

Link: https://lore.kernel.org/849fb1fdecd1cc241fd5b032602dbffa90f9dd93.1778477179.git.caoruide123@gmail.com
Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2")
Signed-off-by: Ruide Cao <caoruide123@gmail.com>
Signed-off-by: Ren Wei <n05ec@lzu.edu.cn>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reported-by: Yuan Tan <yuantan098@gmail.com>
Reported-by: Yifan Wu <yifanwucs@gmail.com>
Reported-by: Juefei Pu <tomapufckgml@gmail.com>
Reported-by: Xin Liu <bird@lzu.edu.cn>
Tested-by: Ren Wei <enjou1224z@gmail.com>
Acked-by: Davidlohr Bueso <dave@stgolabs.net>
Cc: David Hildenbrand <david@kernel.org>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Kees Cook <kees@kernel.org>
Cc: Simon Schuster <schuster.simon@siemens-energy.com>
Cc: Manfred Spraul <manfred@colorfullife.com>
Cc: <stable@vger.kernel.org>

0 files changed, 0 insertions, 0 deletions