xref: /openbmc/qemu/docs/system/arm/cpu-features.rst (revision 399e5e71)
1Arm CPU Features
2================
3
4CPU features are optional features that a CPU of supporting type may
5choose to implement or not.  In QEMU, optional CPU features have
6corresponding boolean CPU proprieties that, when enabled, indicate
7that the feature is implemented, and, conversely, when disabled,
8indicate that it is not implemented. An example of an Arm CPU feature
9is the Performance Monitoring Unit (PMU).  CPU types such as the
10Cortex-A15 and the Cortex-A57, which respectively implement Arm
11architecture reference manuals ARMv7-A and ARMv8-A, may both optionally
12implement PMUs.  For example, if a user wants to use a Cortex-A15 without
13a PMU, then the ``-cpu`` parameter should contain ``pmu=off`` on the QEMU
14command line, i.e. ``-cpu cortex-a15,pmu=off``.
15
16As not all CPU types support all optional CPU features, then whether or
17not a CPU property exists depends on the CPU type.  For example, CPUs
18that implement the ARMv8-A architecture reference manual may optionally
19support the AArch32 CPU feature, which may be enabled by disabling the
20``aarch64`` CPU property.  A CPU type such as the Cortex-A15, which does
21not implement ARMv8-A, will not have the ``aarch64`` CPU property.
22
23QEMU's support may be limited for some CPU features, only partially
24supporting the feature or only supporting the feature under certain
25configurations.  For example, the ``aarch64`` CPU feature, which, when
26disabled, enables the optional AArch32 CPU feature, is only supported
27when using the KVM accelerator and when running on a host CPU type that
28supports the feature.  While ``aarch64`` currently only works with KVM,
29it could work with TCG.  CPU features that are specific to KVM are
30prefixed with "kvm-" and are described in "KVM VCPU Features".
31
32CPU Feature Probing
33===================
34
35Determining which CPU features are available and functional for a given
36CPU type is possible with the ``query-cpu-model-expansion`` QMP command.
37Below are some examples where ``scripts/qmp/qmp-shell`` (see the top comment
38block in the script for usage) is used to issue the QMP commands.
39
401. Determine which CPU features are available for the ``max`` CPU type
41   (Note, we started QEMU with qemu-system-aarch64, so ``max`` is
42   implementing the ARMv8-A reference manual in this case)::
43
44      (QEMU) query-cpu-model-expansion type=full model={"name":"max"}
45      { "return": {
46        "model": { "name": "max", "props": {
47        "sve1664": true, "pmu": true, "sve1792": true, "sve1920": true,
48        "sve128": true, "aarch64": true, "sve1024": true, "sve": true,
49        "sve640": true, "sve768": true, "sve1408": true, "sve256": true,
50        "sve1152": true, "sve512": true, "sve384": true, "sve1536": true,
51        "sve896": true, "sve1280": true, "sve2048": true
52      }}}}
53
54We see that the ``max`` CPU type has the ``pmu``, ``aarch64``, ``sve``, and many
55``sve<N>`` CPU features.  We also see that all the CPU features are
56enabled, as they are all ``true``.  (The ``sve<N>`` CPU features are all
57optional SVE vector lengths (see "SVE CPU Properties").  While with TCG
58all SVE vector lengths can be supported, when KVM is in use it's more
59likely that only a few lengths will be supported, if SVE is supported at
60all.)
61
62(2) Let's try to disable the PMU::
63
64      (QEMU) query-cpu-model-expansion type=full model={"name":"max","props":{"pmu":false}}
65      { "return": {
66        "model": { "name": "max", "props": {
67        "sve1664": true, "pmu": false, "sve1792": true, "sve1920": true,
68        "sve128": true, "aarch64": true, "sve1024": true, "sve": true,
69        "sve640": true, "sve768": true, "sve1408": true, "sve256": true,
70        "sve1152": true, "sve512": true, "sve384": true, "sve1536": true,
71        "sve896": true, "sve1280": true, "sve2048": true
72      }}}}
73
74We see it worked, as ``pmu`` is now ``false``.
75
76(3) Let's try to disable ``aarch64``, which enables the AArch32 CPU feature::
77
78      (QEMU) query-cpu-model-expansion type=full model={"name":"max","props":{"aarch64":false}}
79      {"error": {
80       "class": "GenericError", "desc":
81       "'aarch64' feature cannot be disabled unless KVM is enabled and 32-bit EL1 is supported"
82      }}
83
84It looks like this feature is limited to a configuration we do not
85currently have.
86
87(4) Let's disable ``sve`` and see what happens to all the optional SVE
88    vector lengths::
89
90      (QEMU) query-cpu-model-expansion type=full model={"name":"max","props":{"sve":false}}
91      { "return": {
92        "model": { "name": "max", "props": {
93        "sve1664": false, "pmu": true, "sve1792": false, "sve1920": false,
94        "sve128": false, "aarch64": true, "sve1024": false, "sve": false,
95        "sve640": false, "sve768": false, "sve1408": false, "sve256": false,
96        "sve1152": false, "sve512": false, "sve384": false, "sve1536": false,
97        "sve896": false, "sve1280": false, "sve2048": false
98      }}}}
99
100As expected they are now all ``false``.
101
102(5) Let's try probing CPU features for the Cortex-A15 CPU type::
103
104      (QEMU) query-cpu-model-expansion type=full model={"name":"cortex-a15"}
105      {"return": {"model": {"name": "cortex-a15", "props": {"pmu": true}}}}
106
107Only the ``pmu`` CPU feature is available.
108
109A note about CPU feature dependencies
110-------------------------------------
111
112It's possible for features to have dependencies on other features. I.e.
113it may be possible to change one feature at a time without error, but
114when attempting to change all features at once an error could occur
115depending on the order they are processed.  It's also possible changing
116all at once doesn't generate an error, because a feature's dependencies
117are satisfied with other features, but the same feature cannot be changed
118independently without error.  For these reasons callers should always
119attempt to make their desired changes all at once in order to ensure the
120collection is valid.
121
122A note about CPU models and KVM
123-------------------------------
124
125Named CPU models generally do not work with KVM.  There are a few cases
126that do work, e.g. using the named CPU model ``cortex-a57`` with KVM on a
127seattle host, but mostly if KVM is enabled the ``host`` CPU type must be
128used.  This means the guest is provided all the same CPU features as the
129host CPU type has.  And, for this reason, the ``host`` CPU type should
130enable all CPU features that the host has by default.  Indeed it's even
131a bit strange to allow disabling CPU features that the host has when using
132the ``host`` CPU type, but in the absence of CPU models it's the best we can
133do if we want to launch guests without all the host's CPU features enabled.
134
135Enabling KVM also affects the ``query-cpu-model-expansion`` QMP command.  The
136affect is not only limited to specific features, as pointed out in example
137(3) of "CPU Feature Probing", but also to which CPU types may be expanded.
138When KVM is enabled, only the ``max``, ``host``, and current CPU type may be
139expanded.  This restriction is necessary as it's not possible to know all
140CPU types that may work with KVM, but it does impose a small risk of users
141experiencing unexpected errors.  For example on a seattle, as mentioned
142above, the ``cortex-a57`` CPU type is also valid when KVM is enabled.
143Therefore a user could use the ``host`` CPU type for the current type, but
144then attempt to query ``cortex-a57``, however that query will fail with our
145restrictions.  This shouldn't be an issue though as management layers and
146users have been preferring the ``host`` CPU type for use with KVM for quite
147some time.  Additionally, if the KVM-enabled QEMU instance running on a
148seattle host is using the ``cortex-a57`` CPU type, then querying ``cortex-a57``
149will work.
150
151Using CPU Features
152==================
153
154After determining which CPU features are available and supported for a
155given CPU type, then they may be selectively enabled or disabled on the
156QEMU command line with that CPU type::
157
158  $ qemu-system-aarch64 -M virt -cpu max,pmu=off,sve=on,sve128=on,sve256=on
159
160The example above disables the PMU and enables the first two SVE vector
161lengths for the ``max`` CPU type.  Note, the ``sve=on`` isn't actually
162necessary, because, as we observed above with our probe of the ``max`` CPU
163type, ``sve`` is already on by default.  Also, based on our probe of
164defaults, it would seem we need to disable many SVE vector lengths, rather
165than only enabling the two we want.  This isn't the case, because, as
166disabling many SVE vector lengths would be quite verbose, the ``sve<N>`` CPU
167properties have special semantics (see "SVE CPU Property Parsing
168Semantics").
169
170KVM VCPU Features
171=================
172
173KVM VCPU features are CPU features that are specific to KVM, such as
174paravirt features or features that enable CPU virtualization extensions.
175The features' CPU properties are only available when KVM is enabled and
176are named with the prefix "kvm-".  KVM VCPU features may be probed,
177enabled, and disabled in the same way as other CPU features.  Below is
178the list of KVM VCPU features and their descriptions.
179
180``kvm-no-adjvtime``
181  By default kvm-no-adjvtime is disabled.  This means that by default
182  the virtual time adjustment is enabled (vtime is not *not* adjusted).
183
184  When virtual time adjustment is enabled each time the VM transitions
185  back to running state the VCPU's virtual counter is updated to
186  ensure stopped time is not counted.  This avoids time jumps
187  surprising guest OSes and applications, as long as they use the
188  virtual counter for timekeeping.  However it has the side effect of
189  the virtual and physical counters diverging.  All timekeeping based
190  on the virtual counter will appear to lag behind any timekeeping
191  that does not subtract VM stopped time.  The guest may resynchronize
192  its virtual counter with other time sources as needed.
193
194  Enable kvm-no-adjvtime to disable virtual time adjustment, also
195  restoring the legacy (pre-5.0) behavior.
196
197``kvm-steal-time``
198  Since v5.2, kvm-steal-time is enabled by default when KVM is
199  enabled, the feature is supported, and the guest is 64-bit.
200
201  When kvm-steal-time is enabled a 64-bit guest can account for time
202  its CPUs were not running due to the host not scheduling the
203  corresponding VCPU threads.  The accounting statistics may influence
204  the guest scheduler behavior and/or be exposed to the guest
205  userspace.
206
207TCG VCPU Features
208=================
209
210TCG VCPU features are CPU features that are specific to TCG.
211Below is the list of TCG VCPU features and their descriptions.
212
213``pauth``
214  Enable or disable ``FEAT_Pauth`` entirely.
215
216``pauth-impdef``
217  When ``pauth`` is enabled, select the QEMU implementation defined algorithm.
218
219``pauth-qarma3``
220  When ``pauth`` is enabled, select the architected QARMA3 algorithm.
221
222Without either ``pauth-impdef`` or ``pauth-qarma3`` enabled,
223the architected QARMA5 algorithm is used.  The architected QARMA5
224and QARMA3 algorithms have good cryptographic properties, but can
225be quite slow to emulate.  The impdef algorithm used by QEMU is
226non-cryptographic but significantly faster.
227
228SVE CPU Properties
229==================
230
231There are two types of SVE CPU properties: ``sve`` and ``sve<N>``.  The first
232is used to enable or disable the entire SVE feature, just as the ``pmu``
233CPU property completely enables or disables the PMU.  The second type
234is used to enable or disable specific vector lengths, where ``N`` is the
235number of bits of the length.  The ``sve<N>`` CPU properties have special
236dependencies and constraints, see "SVE CPU Property Dependencies and
237Constraints" below.  Additionally, as we want all supported vector lengths
238to be enabled by default, then, in order to avoid overly verbose command
239lines (command lines full of ``sve<N>=off``, for all ``N`` not wanted), we
240provide the parsing semantics listed in "SVE CPU Property Parsing
241Semantics".
242
243SVE CPU Property Dependencies and Constraints
244---------------------------------------------
245
246  1) At least one vector length must be enabled when ``sve`` is enabled.
247
248  2) If a vector length ``N`` is enabled, then, when KVM is enabled, all
249     smaller, host supported vector lengths must also be enabled.  If
250     KVM is not enabled, then only all the smaller, power-of-two vector
251     lengths must be enabled.  E.g. with KVM if the host supports all
252     vector lengths up to 512-bits (128, 256, 384, 512), then if ``sve512``
253     is enabled, the 128-bit vector length, 256-bit vector length, and
254     384-bit vector length must also be enabled. Without KVM, the 384-bit
255     vector length would not be required.
256
257  3) If KVM is enabled then only vector lengths that the host CPU type
258     support may be enabled.  If SVE is not supported by the host, then
259     no ``sve*`` properties may be enabled.
260
261SVE CPU Property Parsing Semantics
262----------------------------------
263
264  1) If SVE is disabled (``sve=off``), then which SVE vector lengths
265     are enabled or disabled is irrelevant to the guest, as the entire
266     SVE feature is disabled and that disables all vector lengths for
267     the guest.  However QEMU will still track any ``sve<N>`` CPU
268     properties provided by the user.  If later an ``sve=on`` is provided,
269     then the guest will get only the enabled lengths.  If no ``sve=on``
270     is provided and there are explicitly enabled vector lengths, then
271     an error is generated.
272
273  2) If SVE is enabled (``sve=on``), but no ``sve<N>`` CPU properties are
274     provided, then all supported vector lengths are enabled, which when
275     KVM is not in use means including the non-power-of-two lengths, and,
276     when KVM is in use, it means all vector lengths supported by the host
277     processor.
278
279  3) If SVE is enabled, then an error is generated when attempting to
280     disable the last enabled vector length (see constraint (1) of "SVE
281     CPU Property Dependencies and Constraints").
282
283  4) If one or more vector lengths have been explicitly enabled and at
284     least one of the dependency lengths of the maximum enabled length
285     has been explicitly disabled, then an error is generated (see
286     constraint (2) of "SVE CPU Property Dependencies and Constraints").
287
288  5) When KVM is enabled, if the host does not support SVE, then an error
289     is generated when attempting to enable any ``sve*`` properties (see
290     constraint (3) of "SVE CPU Property Dependencies and Constraints").
291
292  6) When KVM is enabled, if the host does support SVE, then an error is
293     generated when attempting to enable any vector lengths not supported
294     by the host (see constraint (3) of "SVE CPU Property Dependencies and
295     Constraints").
296
297  7) If one or more ``sve<N>`` CPU properties are set ``off``, but no ``sve<N>``,
298     CPU properties are set ``on``, then the specified vector lengths are
299     disabled but the default for any unspecified lengths remains enabled.
300     When KVM is not enabled, disabling a power-of-two vector length also
301     disables all vector lengths larger than the power-of-two length.
302     When KVM is enabled, then disabling any supported vector length also
303     disables all larger vector lengths (see constraint (2) of "SVE CPU
304     Property Dependencies and Constraints").
305
306  8) If one or more ``sve<N>`` CPU properties are set to ``on``, then they
307     are enabled and all unspecified lengths default to disabled, except
308     for the required lengths per constraint (2) of "SVE CPU Property
309     Dependencies and Constraints", which will even be auto-enabled if
310     they were not explicitly enabled.
311
312  9) If SVE was disabled (``sve=off``), allowing all vector lengths to be
313     explicitly disabled (i.e. avoiding the error specified in (3) of
314     "SVE CPU Property Parsing Semantics"), then if later an ``sve=on`` is
315     provided an error will be generated.  To avoid this error, one must
316     enable at least one vector length prior to enabling SVE.
317
318SVE CPU Property Examples
319-------------------------
320
321  1) Disable SVE::
322
323     $ qemu-system-aarch64 -M virt -cpu max,sve=off
324
325  2) Implicitly enable all vector lengths for the ``max`` CPU type::
326
327     $ qemu-system-aarch64 -M virt -cpu max
328
329  3) When KVM is enabled, implicitly enable all host CPU supported vector
330     lengths with the ``host`` CPU type::
331
332     $ qemu-system-aarch64 -M virt,accel=kvm -cpu host
333
334  4) Only enable the 128-bit vector length::
335
336     $ qemu-system-aarch64 -M virt -cpu max,sve128=on
337
338  5) Disable the 512-bit vector length and all larger vector lengths,
339     since 512 is a power-of-two.  This results in all the smaller,
340     uninitialized lengths (128, 256, and 384) defaulting to enabled::
341
342     $ qemu-system-aarch64 -M virt -cpu max,sve512=off
343
344  6) Enable the 128-bit, 256-bit, and 512-bit vector lengths::
345
346     $ qemu-system-aarch64 -M virt -cpu max,sve128=on,sve256=on,sve512=on
347
348  7) The same as (6), but since the 128-bit and 256-bit vector
349     lengths are required for the 512-bit vector length to be enabled,
350     then allow them to be auto-enabled::
351
352     $ qemu-system-aarch64 -M virt -cpu max,sve512=on
353
354  8) Do the same as (7), but by first disabling SVE and then re-enabling it::
355
356     $ qemu-system-aarch64 -M virt -cpu max,sve=off,sve512=on,sve=on
357
358  9) Force errors regarding the last vector length::
359
360     $ qemu-system-aarch64 -M virt -cpu max,sve128=off
361     $ qemu-system-aarch64 -M virt -cpu max,sve=off,sve128=off,sve=on
362
363SVE CPU Property Recommendations
364--------------------------------
365
366The examples in "SVE CPU Property Examples" exhibit many ways to select
367vector lengths which developers may find useful in order to avoid overly
368verbose command lines.  However, the recommended way to select vector
369lengths is to explicitly enable each desired length.  Therefore only
370example's (1), (4), and (6) exhibit recommended uses of the properties.
371
372SME CPU Property Examples
373-------------------------
374
375  1) Disable SME::
376
377     $ qemu-system-aarch64 -M virt -cpu max,sme=off
378
379  2) Implicitly enable all vector lengths for the ``max`` CPU type::
380
381     $ qemu-system-aarch64 -M virt -cpu max
382
383  3) Only enable the 256-bit vector length::
384
385     $ qemu-system-aarch64 -M virt -cpu max,sme256=on
386
387  3) Enable the 256-bit and 1024-bit vector lengths::
388
389     $ qemu-system-aarch64 -M virt -cpu max,sme256=on,sme1024=on
390
391  4) Disable the 512-bit vector length.  This results in all the other
392     lengths supported by ``max`` defaulting to enabled
393     (128, 256, 1024 and 2048)::
394
395     $ qemu-system-aarch64 -M virt -cpu max,sve512=off
396
397SVE User-mode Default Vector Length Property
398--------------------------------------------
399
400For qemu-aarch64, the cpu property ``sve-default-vector-length=N`` is
401defined to mirror the Linux kernel parameter file
402``/proc/sys/abi/sve_default_vector_length``.  The default length, ``N``,
403is in units of bytes and must be between 16 and 8192.
404If not specified, the default vector length is 64.
405
406If the default length is larger than the maximum vector length enabled,
407the actual vector length will be reduced.  Note that the maximum vector
408length supported by QEMU is 256.
409
410If this property is set to ``-1`` then the default vector length
411is set to the maximum possible length.
412
413SME CPU Properties
414==================
415
416The SME CPU properties are much like the SVE properties: ``sme`` is
417used to enable or disable the entire SME feature, and ``sme<N>`` is
418used to enable or disable specific vector lengths.  Finally,
419``sme_fa64`` is used to enable or disable ``FEAT_SME_FA64``, which
420allows execution of the "full a64" instruction set while Streaming
421SVE mode is enabled.
422
423SME is not supported by KVM at this time.
424
425At least one vector length must be enabled when ``sme`` is enabled,
426and all vector lengths must be powers of 2.  The maximum vector
427length supported by qemu is 2048 bits.  Otherwise, there are no
428additional constraints on the set of vector lengths supported by SME.
429
430SME User-mode Default Vector Length Property
431--------------------------------------------
432
433For qemu-aarch64, the cpu property ``sme-default-vector-length=N`` is
434defined to mirror the Linux kernel parameter file
435``/proc/sys/abi/sme_default_vector_length``.  The default length, ``N``,
436is in units of bytes and must be between 16 and 8192.
437If not specified, the default vector length is 32.
438
439As with ``sve-default-vector-length``, if the default length is larger
440than the maximum vector length enabled, the actual vector length will
441be reduced.  If this property is set to ``-1`` then the default vector
442length is set to the maximum possible length.
443
444RME CPU Properties
445==================
446
447The status of RME support with QEMU is experimental.  At this time we
448only support RME within the CPU proper, not within the SMMU or GIC.
449The feature is enabled by the CPU property ``x-rme``, with the ``x-``
450prefix present as a reminder of the experimental status, and defaults off.
451
452The method for enabling RME will change in some future QEMU release
453without notice or backward compatibility.
454
455RME Level 0 GPT Size Property
456-----------------------------
457
458To aid firmware developers in testing different possible CPU
459configurations, ``x-l0gptsz=S`` may be used to specify the value
460to encode into ``GPCCR_EL3.L0GPTSZ``, a read-only field that
461specifies the size of the Level 0 Granule Protection Table.
462Legal values for ``S`` are 30, 34, 36, and 39; the default is 30.
463
464As with ``x-rme``, the ``x-l0gptsz`` property may be renamed or
465removed in some future QEMU release.
466