- Description
- In the Linux kernel, the following vulnerability has been resolved:
x86/mm: Eliminate window where TLB flushes may be inadvertently skipped
tl;dr: There is a window in the mm switching code where the new CR3 is
set and the CPU should be getting TLB flushes for the new mm. But
should_flush_tlb() has a bug and suppresses the flush. Fix it by
widening the window where should_flush_tlb() sends an IPI.
Long Version:
=== History ===
There were a few things leading up to this.
First, updating mm_cpumask() was observed to be too expensive, so it was
made lazier. But being lazy caused too many unnecessary IPIs to CPUs
due to the now-lazy mm_cpumask(). So code was added to cull
mm_cpumask() periodically[2]. But that culling was a bit too aggressive
and skipped sending TLB flushes to CPUs that need them. So here we are
again.
=== Problem ===
The too-aggressive code in should_flush_tlb() strikes in this window:
// Turn on IPIs for this CPU/mm combination, but only
// if should_flush_tlb() agrees:
cpumask_set_cpu(cpu, mm_cpumask(next));
next_tlb_gen = atomic64_read(&next->context.tlb_gen);
choose_new_asid(next, next_tlb_gen, &new_asid, &need_flush);
load_new_mm_cr3(need_flush);
// ^ After 'need_flush' is set to false, IPIs *MUST*
// be sent to this CPU and not be ignored.
this_cpu_write(cpu_tlbstate.loaded_mm, next);
// ^ Not until this point does should_flush_tlb()
// become true!
should_flush_tlb() will suppress TLB flushes between load_new_mm_cr3()
and writing to 'loaded_mm', which is a window where they should not be
suppressed. Whoops.
=== Solution ===
Thankfully, the fuzzy "just about to write CR3" window is already marked
with loaded_mm==LOADED_MM_SWITCHING. Simply checking for that state in
should_flush_tlb() is sufficient to ensure that the CPU is targeted with
an IPI.
This will cause more TLB flush IPIs. But the window is relatively small
and I do not expect this to cause any kind of measurable performance
impact.
Update the comment where LOADED_MM_SWITCHING is written since it grew
yet another user.
Peter Z also raised a concern that should_flush_tlb() might not observe
'loaded_mm' and 'is_lazy' in the same order that switch_mm_irqs_off()
writes them. Add a barrier to ensure that they are observed in the
order they are written.
- Source
- 416baaa9-dc9f-4396-8d5f-8c081fb06d67
- NVD status
- Analyzed
- Products
- linux_kernel, debian_linux
[
{
"nodes": [
{
"cpeMatch": [
{
"criteria": "cpe:2.3:o:linux:linux_kernel:*:*:*:*:*:*:*:*",
"matchCriteriaId": "D2AFD248-20FC-485D-A1CC-13D0E8DCC81A",
"versionEndExcluding": "5.15.183",
"versionStartIncluding": "5.15.179",
"vulnerable": true
},
{
"criteria": "cpe:2.3:o:linux:linux_kernel:*:*:*:*:*:*:*:*",
"matchCriteriaId": "4C6814B6-DBF3-43D3-8C83-AB2A7B017FC6",
"versionEndExcluding": "6.1.139",
"versionStartIncluding": "6.1.129",
"vulnerable": true
},
{
"criteria": "cpe:2.3:o:linux:linux_kernel:*:*:*:*:*:*:*:*",
"matchCriteriaId": "7CC8774F-8BD4-4950-8F1C-0AD7E981084D",
"versionEndExcluding": "6.6.91",
"versionStartIncluding": "6.6.79",
"vulnerable": true
},
{
"criteria": "cpe:2.3:o:linux:linux_kernel:*:*:*:*:*:*:*:*",
"matchCriteriaId": "AC82108A-29E5-4E88-93EC-856824C6788E",
"versionEndExcluding": "6.12.29",
"versionStartIncluding": "6.12.16",
"vulnerable": true
},
{
"criteria": "cpe:2.3:o:linux:linux_kernel:*:*:*:*:*:*:*:*",
"matchCriteriaId": "8869956D-8A8C-40D1-91E5-1709F4A31C01",
"versionEndExcluding": "6.14.7",
"versionStartIncluding": "6.13.4",
"vulnerable": true
},
{
"criteria": "cpe:2.3:o:linux:linux_kernel:6.15:rc1:*:*:*:*:*:*",
"matchCriteriaId": "8D465631-2980-487A-8E65-40AE2B9F8ED1",
"vulnerable": true
},
{
"criteria": "cpe:2.3:o:linux:linux_kernel:6.15:rc2:*:*:*:*:*:*",
"matchCriteriaId": "4C9D071F-B28E-46EC-AC61-22B913390211",
"vulnerable": true
},
{
"criteria": "cpe:2.3:o:linux:linux_kernel:6.15:rc3:*:*:*:*:*:*",
"matchCriteriaId": "13FC0DDE-E513-465E-9E81-515702D49B74",
"vulnerable": true
},
{
"criteria": "cpe:2.3:o:linux:linux_kernel:6.15:rc4:*:*:*:*:*:*",
"matchCriteriaId": "8C7B5B0E-4EEB-48F5-B4CF-0935A7633845",
"vulnerable": true
},
{
"criteria": "cpe:2.3:o:linux:linux_kernel:6.15:rc5:*:*:*:*:*:*",
"matchCriteriaId": "2D240580-3048-49B2-9E27-F115A9DF8224",
"vulnerable": true
}
],
"negate": false,
"operator": "OR"
}
]
},
{
"nodes": [
{
"cpeMatch": [
{
"criteria": "cpe:2.3:o:debian:debian_linux:11.0:*:*:*:*:*:*:*",
"matchCriteriaId": "FA6FEEC2-9F11-4643-8827-749718254FED",
"vulnerable": true
}
],
"negate": false,
"operator": "OR"
}
]
}
]