11fcc3d64SPawan Gupta========================================= 21fcc3d64SPawan GuptaProcessor MMIO Stale Data Vulnerabilities 31fcc3d64SPawan Gupta========================================= 41fcc3d64SPawan Gupta 51fcc3d64SPawan GuptaProcessor MMIO Stale Data Vulnerabilities are a class of memory-mapped I/O 61fcc3d64SPawan Gupta(MMIO) vulnerabilities that can expose data. The sequences of operations for 71fcc3d64SPawan Guptaexposing data range from simple to very complex. Because most of the 81fcc3d64SPawan Guptavulnerabilities require the attacker to have access to MMIO, many environments 91fcc3d64SPawan Guptaare not affected. System environments using virtualization where MMIO access is 101fcc3d64SPawan Guptaprovided to untrusted guests may need mitigation. These vulnerabilities are 111fcc3d64SPawan Guptanot transient execution attacks. However, these vulnerabilities may propagate 121fcc3d64SPawan Guptastale data into core fill buffers where the data can subsequently be inferred 131fcc3d64SPawan Guptaby an unmitigated transient execution attack. Mitigation for these 141fcc3d64SPawan Guptavulnerabilities includes a combination of microcode update and software 151fcc3d64SPawan Guptachanges, depending on the platform and usage model. Some of these mitigations 161fcc3d64SPawan Guptaare similar to those used to mitigate Microarchitectural Data Sampling (MDS) or 171fcc3d64SPawan Guptathose used to mitigate Special Register Buffer Data Sampling (SRBDS). 181fcc3d64SPawan Gupta 191fcc3d64SPawan GuptaData Propagators 201fcc3d64SPawan Gupta================ 211fcc3d64SPawan GuptaPropagators are operations that result in stale data being copied or moved from 221fcc3d64SPawan Guptaone microarchitectural buffer or register to another. Processor MMIO Stale Data 231fcc3d64SPawan GuptaVulnerabilities are operations that may result in stale data being directly 241fcc3d64SPawan Guptaread into an architectural, software-visible state or sampled from a buffer or 251fcc3d64SPawan Guptaregister. 261fcc3d64SPawan Gupta 271fcc3d64SPawan GuptaFill Buffer Stale Data Propagator (FBSDP) 281fcc3d64SPawan Gupta----------------------------------------- 291fcc3d64SPawan GuptaStale data may propagate from fill buffers (FB) into the non-coherent portion 301fcc3d64SPawan Guptaof the uncore on some non-coherent writes. Fill buffer propagation by itself 311fcc3d64SPawan Guptadoes not make stale data architecturally visible. Stale data must be propagated 321fcc3d64SPawan Guptato a location where it is subject to reading or sampling. 331fcc3d64SPawan Gupta 341fcc3d64SPawan GuptaSideband Stale Data Propagator (SSDP) 351fcc3d64SPawan Gupta------------------------------------- 361fcc3d64SPawan GuptaThe sideband stale data propagator (SSDP) is limited to the client (including 371fcc3d64SPawan GuptaIntel Xeon server E3) uncore implementation. The sideband response buffer is 381fcc3d64SPawan Guptashared by all client cores. For non-coherent reads that go to sideband 391fcc3d64SPawan Guptadestinations, the uncore logic returns 64 bytes of data to the core, including 401fcc3d64SPawan Guptaboth requested data and unrequested stale data, from a transaction buffer and 411fcc3d64SPawan Guptathe sideband response buffer. As a result, stale data from the sideband 421fcc3d64SPawan Guptaresponse and transaction buffers may now reside in a core fill buffer. 431fcc3d64SPawan Gupta 441fcc3d64SPawan GuptaPrimary Stale Data Propagator (PSDP) 451fcc3d64SPawan Gupta------------------------------------ 461fcc3d64SPawan GuptaThe primary stale data propagator (PSDP) is limited to the client (including 471fcc3d64SPawan GuptaIntel Xeon server E3) uncore implementation. Similar to the sideband response 481fcc3d64SPawan Guptabuffer, the primary response buffer is shared by all client cores. For some 491fcc3d64SPawan Guptaprocessors, MMIO primary reads will return 64 bytes of data to the core fill 501fcc3d64SPawan Guptabuffer including both requested data and unrequested stale data. This is 511fcc3d64SPawan Guptasimilar to the sideband stale data propagator. 521fcc3d64SPawan Gupta 531fcc3d64SPawan GuptaVulnerabilities 541fcc3d64SPawan Gupta=============== 551fcc3d64SPawan GuptaDevice Register Partial Write (DRPW) (CVE-2022-21166) 561fcc3d64SPawan Gupta----------------------------------------------------- 571fcc3d64SPawan GuptaSome endpoint MMIO registers incorrectly handle writes that are smaller than 581fcc3d64SPawan Guptathe register size. Instead of aborting the write or only copying the correct 591fcc3d64SPawan Guptasubset of bytes (for example, 2 bytes for a 2-byte write), more bytes than 601fcc3d64SPawan Guptaspecified by the write transaction may be written to the register. On 611fcc3d64SPawan Guptaprocessors affected by FBSDP, this may expose stale data from the fill buffers 621fcc3d64SPawan Guptaof the core that created the write transaction. 631fcc3d64SPawan Gupta 641fcc3d64SPawan GuptaShared Buffers Data Sampling (SBDS) (CVE-2022-21125) 651fcc3d64SPawan Gupta---------------------------------------------------- 661fcc3d64SPawan GuptaAfter propagators may have moved data around the uncore and copied stale data 671fcc3d64SPawan Guptainto client core fill buffers, processors affected by MFBDS can leak data from 681fcc3d64SPawan Guptathe fill buffer. It is limited to the client (including Intel Xeon server E3) 691fcc3d64SPawan Guptauncore implementation. 701fcc3d64SPawan Gupta 711fcc3d64SPawan GuptaShared Buffers Data Read (SBDR) (CVE-2022-21123) 721fcc3d64SPawan Gupta------------------------------------------------ 731fcc3d64SPawan GuptaIt is similar to Shared Buffer Data Sampling (SBDS) except that the data is 741fcc3d64SPawan Guptadirectly read into the architectural software-visible state. It is limited to 751fcc3d64SPawan Guptathe client (including Intel Xeon server E3) uncore implementation. 761fcc3d64SPawan Gupta 771fcc3d64SPawan GuptaAffected Processors 781fcc3d64SPawan Gupta=================== 791fcc3d64SPawan GuptaNot all the CPUs are affected by all the variants. For instance, most 801fcc3d64SPawan Guptaprocessors for the server market (excluding Intel Xeon E3 processors) are 811fcc3d64SPawan Guptaimpacted by only Device Register Partial Write (DRPW). 821fcc3d64SPawan Gupta 831fcc3d64SPawan GuptaBelow is the list of affected Intel processors [#f1]_: 841fcc3d64SPawan Gupta 851fcc3d64SPawan Gupta =================== ============ ========= 861fcc3d64SPawan Gupta Common name Family_Model Steppings 871fcc3d64SPawan Gupta =================== ============ ========= 881fcc3d64SPawan Gupta HASWELL_X 06_3FH 2,4 891fcc3d64SPawan Gupta SKYLAKE_L 06_4EH 3 901fcc3d64SPawan Gupta BROADWELL_X 06_4FH All 911fcc3d64SPawan Gupta SKYLAKE_X 06_55H 3,4,6,7,11 921fcc3d64SPawan Gupta BROADWELL_D 06_56H 3,4,5 931fcc3d64SPawan Gupta SKYLAKE 06_5EH 3 941fcc3d64SPawan Gupta ICELAKE_X 06_6AH 4,5,6 951fcc3d64SPawan Gupta ICELAKE_D 06_6CH 1 961fcc3d64SPawan Gupta ICELAKE_L 06_7EH 5 971fcc3d64SPawan Gupta ATOM_TREMONT_D 06_86H All 981fcc3d64SPawan Gupta LAKEFIELD 06_8AH 1 991fcc3d64SPawan Gupta KABYLAKE_L 06_8EH 9 to 12 1001fcc3d64SPawan Gupta ATOM_TREMONT 06_96H 1 1011fcc3d64SPawan Gupta ATOM_TREMONT_L 06_9CH 0 1021fcc3d64SPawan Gupta KABYLAKE 06_9EH 9 to 13 1031fcc3d64SPawan Gupta COMETLAKE 06_A5H 2,3,5 1041fcc3d64SPawan Gupta COMETLAKE_L 06_A6H 0,1 1051fcc3d64SPawan Gupta ROCKETLAKE 06_A7H 1 1061fcc3d64SPawan Gupta =================== ============ ========= 1071fcc3d64SPawan Gupta 1081fcc3d64SPawan GuptaIf a CPU is in the affected processor list, but not affected by a variant, it 1091fcc3d64SPawan Guptais indicated by new bits in MSR IA32_ARCH_CAPABILITIES. As described in a later 1101fcc3d64SPawan Guptasection, mitigation largely remains the same for all the variants, i.e. to 1111fcc3d64SPawan Guptaclear the CPU fill buffers via VERW instruction. 1121fcc3d64SPawan Gupta 1131fcc3d64SPawan GuptaNew bits in MSRs 1141fcc3d64SPawan Gupta================ 1151fcc3d64SPawan GuptaNewer processors and microcode update on existing affected processors added new 1161fcc3d64SPawan Guptabits to IA32_ARCH_CAPABILITIES MSR. These bits can be used to enumerate 1171fcc3d64SPawan Guptaspecific variants of Processor MMIO Stale Data vulnerabilities and mitigation 1181fcc3d64SPawan Guptacapability. 1191fcc3d64SPawan Gupta 1201fcc3d64SPawan GuptaMSR IA32_ARCH_CAPABILITIES 1211fcc3d64SPawan Gupta-------------------------- 1221fcc3d64SPawan GuptaBit 13 - SBDR_SSDP_NO - When set, processor is not affected by either the 1231fcc3d64SPawan Gupta Shared Buffers Data Read (SBDR) vulnerability or the sideband stale 1241fcc3d64SPawan Gupta data propagator (SSDP). 1251fcc3d64SPawan GuptaBit 14 - FBSDP_NO - When set, processor is not affected by the Fill Buffer 1261fcc3d64SPawan Gupta Stale Data Propagator (FBSDP). 1271fcc3d64SPawan GuptaBit 15 - PSDP_NO - When set, processor is not affected by Primary Stale Data 1281fcc3d64SPawan Gupta Propagator (PSDP). 1291fcc3d64SPawan GuptaBit 17 - FB_CLEAR - When set, VERW instruction will overwrite CPU fill buffer 1301fcc3d64SPawan Gupta values as part of MD_CLEAR operations. Processors that do not 1311fcc3d64SPawan Gupta enumerate MDS_NO (meaning they are affected by MDS) but that do 1321fcc3d64SPawan Gupta enumerate support for both L1D_FLUSH and MD_CLEAR implicitly enumerate 1331fcc3d64SPawan Gupta FB_CLEAR as part of their MD_CLEAR support. 1341fcc3d64SPawan GuptaBit 18 - FB_CLEAR_CTRL - Processor supports read and write to MSR 1351fcc3d64SPawan Gupta IA32_MCU_OPT_CTRL[FB_CLEAR_DIS]. On such processors, the FB_CLEAR_DIS 1361fcc3d64SPawan Gupta bit can be set to cause the VERW instruction to not perform the 1371fcc3d64SPawan Gupta FB_CLEAR action. Not all processors that support FB_CLEAR will support 1381fcc3d64SPawan Gupta FB_CLEAR_CTRL. 1391fcc3d64SPawan Gupta 1401fcc3d64SPawan GuptaMSR IA32_MCU_OPT_CTRL 1411fcc3d64SPawan Gupta--------------------- 1421fcc3d64SPawan GuptaBit 3 - FB_CLEAR_DIS - When set, VERW instruction does not perform the FB_CLEAR 1431fcc3d64SPawan Guptaaction. This may be useful to reduce the performance impact of FB_CLEAR in 1441fcc3d64SPawan Guptacases where system software deems it warranted (for example, when performance 1451fcc3d64SPawan Guptais more critical, or the untrusted software has no MMIO access). Note that 1461fcc3d64SPawan GuptaFB_CLEAR_DIS has no impact on enumeration (for example, it does not change 1471fcc3d64SPawan GuptaFB_CLEAR or MD_CLEAR enumeration) and it may not be supported on all processors 1481fcc3d64SPawan Guptathat enumerate FB_CLEAR. 1491fcc3d64SPawan Gupta 1501fcc3d64SPawan GuptaMitigation 1511fcc3d64SPawan Gupta========== 1521fcc3d64SPawan GuptaLike MDS, all variants of Processor MMIO Stale Data vulnerabilities have the 1531fcc3d64SPawan Guptasame mitigation strategy to force the CPU to clear the affected buffers before 1541fcc3d64SPawan Guptaan attacker can extract the secrets. 1551fcc3d64SPawan Gupta 1561fcc3d64SPawan GuptaThis is achieved by using the otherwise unused and obsolete VERW instruction in 1571fcc3d64SPawan Guptacombination with a microcode update. The microcode clears the affected CPU 1581fcc3d64SPawan Guptabuffers when the VERW instruction is executed. 1591fcc3d64SPawan Gupta 1601fcc3d64SPawan GuptaKernel reuses the MDS function to invoke the buffer clearing: 1611fcc3d64SPawan Gupta 1621fcc3d64SPawan Gupta mds_clear_cpu_buffers() 1631fcc3d64SPawan Gupta 1641fcc3d64SPawan GuptaOn MDS affected CPUs, the kernel already invokes CPU buffer clear on 1651fcc3d64SPawan Guptakernel/userspace, hypervisor/guest and C-state (idle) transitions. No 1661fcc3d64SPawan Guptaadditional mitigation is needed on such CPUs. 1671fcc3d64SPawan Gupta 1681fcc3d64SPawan GuptaFor CPUs not affected by MDS or TAA, mitigation is needed only for the attacker 1691fcc3d64SPawan Guptawith MMIO capability. Therefore, VERW is not required for kernel/userspace. For 1701fcc3d64SPawan Guptavirtualization case, VERW is only needed at VMENTER for a guest with MMIO 1711fcc3d64SPawan Guptacapability. 1721fcc3d64SPawan Gupta 1731fcc3d64SPawan GuptaMitigation points 1741fcc3d64SPawan Gupta----------------- 1751fcc3d64SPawan GuptaReturn to user space 1761fcc3d64SPawan Gupta^^^^^^^^^^^^^^^^^^^^ 1771fcc3d64SPawan GuptaSame mitigation as MDS when affected by MDS/TAA, otherwise no mitigation 1781fcc3d64SPawan Guptaneeded. 1791fcc3d64SPawan Gupta 1801fcc3d64SPawan GuptaC-State transition 1811fcc3d64SPawan Gupta^^^^^^^^^^^^^^^^^^ 1821fcc3d64SPawan GuptaControl register writes by CPU during C-state transition can propagate data 1831fcc3d64SPawan Guptafrom fill buffer to uncore buffers. Execute VERW before C-state transition to 1841fcc3d64SPawan Guptaclear CPU fill buffers. 1851fcc3d64SPawan Gupta 1861fcc3d64SPawan GuptaGuest entry point 1871fcc3d64SPawan Gupta^^^^^^^^^^^^^^^^^ 1881fcc3d64SPawan GuptaSame mitigation as MDS when processor is also affected by MDS/TAA, otherwise 1891fcc3d64SPawan Guptaexecute VERW at VMENTER only for MMIO capable guests. On CPUs not affected by 1901fcc3d64SPawan GuptaMDS/TAA, guest without MMIO access cannot extract secrets using Processor MMIO 1911fcc3d64SPawan GuptaStale Data vulnerabilities, so there is no need to execute VERW for such guests. 1921fcc3d64SPawan Gupta 1931fcc3d64SPawan GuptaMitigation control on the kernel command line 1941fcc3d64SPawan Gupta--------------------------------------------- 1951fcc3d64SPawan GuptaThe kernel command line allows to control the Processor MMIO Stale Data 1961fcc3d64SPawan Guptamitigations at boot time with the option "mmio_stale_data=". The valid 1971fcc3d64SPawan Guptaarguments for this option are: 1981fcc3d64SPawan Gupta 1991fcc3d64SPawan Gupta ========== ================================================================= 2001fcc3d64SPawan Gupta full If the CPU is vulnerable, enable mitigation; CPU buffer clearing 2011fcc3d64SPawan Gupta on exit to userspace and when entering a VM. Idle transitions are 2021fcc3d64SPawan Gupta protected as well. It does not automatically disable SMT. 2031fcc3d64SPawan Gupta full,nosmt Same as full, with SMT disabled on vulnerable CPUs. This is the 2041fcc3d64SPawan Gupta complete mitigation. 2051fcc3d64SPawan Gupta off Disables mitigation completely. 2061fcc3d64SPawan Gupta ========== ================================================================= 2071fcc3d64SPawan Gupta 2081fcc3d64SPawan GuptaIf the CPU is affected and mmio_stale_data=off is not supplied on the kernel 2091fcc3d64SPawan Guptacommand line, then the kernel selects the appropriate mitigation. 2101fcc3d64SPawan Gupta 2111fcc3d64SPawan GuptaMitigation status information 2121fcc3d64SPawan Gupta----------------------------- 2131fcc3d64SPawan GuptaThe Linux kernel provides a sysfs interface to enumerate the current 2141fcc3d64SPawan Guptavulnerability status of the system: whether the system is vulnerable, and 2151fcc3d64SPawan Guptawhich mitigations are active. The relevant sysfs file is: 2161fcc3d64SPawan Gupta 2171fcc3d64SPawan Gupta /sys/devices/system/cpu/vulnerabilities/mmio_stale_data 2181fcc3d64SPawan Gupta 2191fcc3d64SPawan GuptaThe possible values in this file are: 2201fcc3d64SPawan Gupta 2211fcc3d64SPawan Gupta .. list-table:: 2221fcc3d64SPawan Gupta 2231fcc3d64SPawan Gupta * - 'Not affected' 2241fcc3d64SPawan Gupta - The processor is not vulnerable 2251fcc3d64SPawan Gupta * - 'Vulnerable' 2261fcc3d64SPawan Gupta - The processor is vulnerable, but no mitigation enabled 2271fcc3d64SPawan Gupta * - 'Vulnerable: Clear CPU buffers attempted, no microcode' 2281fcc3d64SPawan Gupta - The processor is vulnerable, but microcode is not updated. The 2291fcc3d64SPawan Gupta mitigation is enabled on a best effort basis. 2301fcc3d64SPawan Gupta * - 'Mitigation: Clear CPU buffers' 2311fcc3d64SPawan Gupta - The processor is vulnerable and the CPU buffer clearing mitigation is 2321fcc3d64SPawan Gupta enabled. 23375fa6c73SPawan Gupta * - 'Unknown: No mitigations' 23475fa6c73SPawan Gupta - The processor vulnerability status is unknown because it is 23575fa6c73SPawan Gupta out of Servicing period. Mitigation is not attempted. 23675fa6c73SPawan Gupta 23775fa6c73SPawan GuptaDefinitions: 23875fa6c73SPawan Gupta------------ 23975fa6c73SPawan Gupta 24075fa6c73SPawan GuptaServicing period: The process of providing functional and security updates to 24175fa6c73SPawan GuptaIntel processors or platforms, utilizing the Intel Platform Update (IPU) 24275fa6c73SPawan Guptaprocess or other similar mechanisms. 24375fa6c73SPawan Gupta 24475fa6c73SPawan GuptaEnd of Servicing Updates (ESU): ESU is the date at which Intel will no 24575fa6c73SPawan Guptalonger provide Servicing, such as through IPU or other similar update 24675fa6c73SPawan Guptaprocesses. ESU dates will typically be aligned to end of quarter. 2471fcc3d64SPawan Gupta 2481fcc3d64SPawan GuptaIf the processor is vulnerable then the following information is appended to 2491fcc3d64SPawan Guptathe above information: 2501fcc3d64SPawan Gupta 2511fcc3d64SPawan Gupta ======================== =========================================== 2521fcc3d64SPawan Gupta 'SMT vulnerable' SMT is enabled 2531fcc3d64SPawan Gupta 'SMT disabled' SMT is disabled 2541fcc3d64SPawan Gupta 'SMT Host state unknown' Kernel runs in a VM, Host SMT state unknown 2551fcc3d64SPawan Gupta ======================== =========================================== 2561fcc3d64SPawan Gupta 2571fcc3d64SPawan GuptaReferences 2581fcc3d64SPawan Gupta---------- 2591fcc3d64SPawan Gupta.. [#f1] Affected Processors 2601fcc3d64SPawan Gupta https://www.intel.com/content/www/us/en/developer/topic-technology/software-security-guidance/processors-affected-consolidated-product-cpu-model.html 261