CVE-2025-39756

Published Sep 11, 2025

Last updated 4 days ago

Overview

Description
In the Linux kernel, the following vulnerability has been resolved: fs: Prevent file descriptor table allocations exceeding INT_MAX When sysctl_nr_open is set to a very high value (for example, 1073741816 as set by systemd), processes attempting to use file descriptors near the limit can trigger massive memory allocation attempts that exceed INT_MAX, resulting in a WARNING in mm/slub.c: WARNING: CPU: 0 PID: 44 at mm/slub.c:5027 __kvmalloc_node_noprof+0x21a/0x288 This happens because kvmalloc_array() and kvmalloc() check if the requested size exceeds INT_MAX and emit a warning when the allocation is not flagged with __GFP_NOWARN. Specifically, when nr_open is set to 1073741816 (0x3ffffff8) and a process calls dup2(oldfd, 1073741880), the kernel attempts to allocate: - File descriptor array: 1073741880 * 8 bytes = 8,589,935,040 bytes - Multiple bitmaps: ~400MB - Total allocation size: > 8GB (exceeding INT_MAX = 2,147,483,647) Reproducer: 1. Set /proc/sys/fs/nr_open to 1073741816: # echo 1073741816 > /proc/sys/fs/nr_open 2. Run a program that uses a high file descriptor: #include <unistd.h> #include <sys/resource.h> int main() { struct rlimit rlim = {1073741824, 1073741824}; setrlimit(RLIMIT_NOFILE, &rlim); dup2(2, 1073741880); // Triggers the warning return 0; } 3. Observe WARNING in dmesg at mm/slub.c:5027 systemd commit a8b627a introduced automatic bumping of fs.nr_open to the maximum possible value. The rationale was that systems with memory control groups (memcg) no longer need separate file descriptor limits since memory is properly accounted. However, this change overlooked that: 1. The kernel's allocation functions still enforce INT_MAX as a maximum size regardless of memcg accounting 2. Programs and tests that legitimately test file descriptor limits can inadvertently trigger massive allocations 3. The resulting allocations (>8GB) are impractical and will always fail systemd's algorithm starts with INT_MAX and keeps halving the value until the kernel accepts it. On most systems, this results in nr_open being set to 1073741816 (0x3ffffff8), which is just under 1GB of file descriptors. While processes rarely use file descriptors near this limit in normal operation, certain selftests (like tools/testing/selftests/core/unshare_test.c) and programs that test file descriptor limits can trigger this issue. Fix this by adding a check in alloc_fdtable() to ensure the requested allocation size does not exceed INT_MAX. This causes the operation to fail with -EMFILE instead of triggering a kernel warning and avoids the impractical >8GB memory allocation request.
Source
416baaa9-dc9f-4396-8d5f-8c081fb06d67
NVD status
Modified
Products
linux_kernel, debian_linux

Risk scores

CVSS 3.1

Type
Primary
Base score
5.5
Impact score
3.6
Exploitability score
1.8
Vector string
CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H
Severity
MEDIUM

Weaknesses

nvd@nist.gov
CWE-401

Social media

Hype score
Not currently trending

Configurations

  1. In the Linux kernel, the following vulnerability has been resolved: cgroup: Avoid iteration of dying tasks with zero refcount The commit 260fbcb92bbea ("cgroup: Move dying_tasks cleanup from cgroup_task_release() to cgroup_task_free()") extended the lifetime of tasks on the dying_tasks list. The iterators have provision to go through dying_tasks because of dying threadgroup leaders or explicit CSS_TASK_ITER_WITH_DEAD, however, it was expected that such tasks can obtain a new reference (that is possible before cgroup_task_release()/put_task_struct_rcu_user()). The tasks after cgroup_task_release() and before cgroup_task_free() are subject to race when they may or may not have ->usage count > 0. The race window is between css_task_iter_next() invocations when css_set_lock is released and we may arrive at a new ->task_pos. The iterator should not attempt to resurrect tasks whose ->usage count dropped to zero. (When that happens, __put_task_struct_rcu_cb() is already imminent and the returned task_struct would could be used after free.) As for the fix, we cannot simply check the signal->live count of a task on the dying list because that won't distinguish regular zombies waiting to be reaped from RCU remnant tasks that are going to be free'd. Therefore add an extra check to rule out ->usage==0 tasks from any iteration. The repeat: loop in css_task_iter_advance() doesn't consider ->usage count, so add a new loop to css_task_iter_next() to skip de-used tasks on the dying_list. Rough illustration of the possible race R (reader of cgroup.procs) T (thread) L (group leader) --------------------------------- -------------------------------- -------------------------------- L exits, signal->live > 0 cgroup_task_dead(L) css_set_skip_task_iters() // skips only cset->tasks list_add_tail(&L->cg_list, &cset->dying_tasks) css_task_iter_next() take css_set_lock css_task_iter_advance() leader && signal->live != 0 => it->task_pos = &L->cg_list release css_set_lock T exits --signal->live == 0 cgroup_task_dead(T) // css_set_lock release_task(T) cgroup_task_release(T) release_task(L) // zap_leader cgroup_task_release(L) put_task_struct_rcu_user(L) ...RCU... put_task_struct(L) L->usage = 0 /* L still on dying_tasks */ ...RCU... __put_task_struct(L) css_task_iter_next() // another iteration take css_set_lock it->task_pos = &L->cg_list get_task_struct(L) => addition on 0 drop css_set_lock cgroup_task_free(L) css_set_skip_task_iters() // dying skip comes too late free_task(L) cgroup_procs_show() task_pid_vnr(L)•CVE-2026-98163