system/s390x/vfio-ap.rst

1 Adjunct Processor (AP) Device
9 The IBM Adjunct Processor (AP) Cryptographic Facility is comprised
10 of three AP instructions and from 1 to 256 PCIe cryptographic adapter cards.
11 These AP devices provide cryptographic functions to all CPUs assigned to a
14 On s390x, AP adapter cards are exposed via the AP bus. This document
18 AP Architectural Overview
24 * AP adapter
26   An AP adapter is an IBM Z adapter card that can perform cryptographic
31   determined by machine model. When installed, an AP adapter is accessed by
32   AP instructions executed by any CPU.
34 * AP domain
37   a set of hardware registers for processing AP instructions. An adapter can
42     * Usage domains are domains that can be accessed directly to process AP
45     * Control domains are domains that are accessed indirectly by AP
49 * AP Queue
51   An AP queue is the means by which an AP command-request message is sent to an
52   AP usage domain inside a specific AP. An AP queue is identified by a tuple
53   comprised of an AP adapter ID (APID) and an AP queue index (APQI). The
55   forms an AP Queue Number (APQN) uniquely identifying an AP queue. AP
56   instructions include a field containing the APQN to identify the AP queue to
57   which the AP command-request message is to be sent for processing.
59 * AP Instructions:
61   There are three AP instructions:
63   * NQAP: to enqueue an AP command-request message to a queue
64   * DQAP: to dequeue an AP command-reply message from a queue
67   AP instructions identify the domain that is targeted to process the AP
68   command; this must be one of the usage domains. An AP command may modify a
82 * The AP Mask (APM) field is a bit mask that identifies the AP adapters assigned
87 * The AP Queue Mask (AQM) field is a bit mask identifying the AP usage domains
89   corresponds to  an AP queue index (APQI) from 0-255. If a bit is set, the
92 * The AP Domain Mask field is a bit mask that identifies the AP control domains
94   changed by an AP command-request message sent to a usage domain from the
96   0-255. If a bit is set, the corresponding domain can be modified by an AP
99 If you recall from the description of an AP Queue, AP instructions include
100 an APQN to identify the AP adapter and AP queue to which an AP command-request
159 AP Matrix Configuration on Linux Host
163 the AP resources configured for the LPAR. The LPAR's AP matrix is
165 linux system is started, the AP bus will detect the AP devices assigned to the
178   is AP adapter number xx (in hex)
183 For example, if AP adapters 5 and 6 and domains 4, 71 (0x47), 171 (0xab) and
200 A set of default device drivers are also created to control each type of AP
206   ...... [cex2aqueue]       for AP queues served by Crypto Express 2/3
210   ...... [cex4queue]        for AP queues served by Crypto Express 4/5/6
213   ...... [pcixccqueue]      for AP queues served by Crypto Express 2/3
216 Binding AP devices to device drivers
220 range as 'usable by the default AP queue device drivers' or 'not usable by the
227 The ``apmask`` is a 256-bit mask that identifies a set of AP adapter IDs
230 0-255. If a bit is set, the APID is marked as usable only by the default AP
234 The ``aqmask`` is a 256-bit mask that identifies a set of AP queue indexes
237 0-255. If a bit is set, the APQI is marked as usable only by the default AP
250 The APQN of each AP queue device assigned to the linux host is checked by the
251 AP bus against the set of APQNs derived from the cross product of APIDs
252 and APQIs marked as usable only by the default AP queue device drivers. If a
253 match is detected,  only the default AP queue device drivers will be probed;
257 AP queue device drivers. There are two ways the default masks can be changed:
317 Configuring an AP matrix for a linux guest
320 The sysfs interfaces for configuring an AP matrix for a guest are built on the
321 VFIO mediated device framework. To configure an AP matrix for a guest, a
335 A mediated AP matrix device is created by writing a UUID to the attribute file
346 When a mediated AP matrix device is created, a sysfs directory named after
359 matrix device's sysfs directory to configure an AP matrix for the
379    To assign an AP adapter to the mediated matrix device, its APID is written
410    To unassign an AP adapter, its APID is written to the ``unassign_adapter``
471 Notes: No changes to the AP matrix will be allowed while a guest using
475 Starting a Linux Guest Configured with an AP Matrix
498 1. Have direct access to the APQNs derived from the cross product of the AP
500    fields of the guests's CRYCB respectively. These APQNs identify the AP queues
501    that are valid for use by the guest; meaning, AP commands can be sent by the
504 2. Have authorization to process AP commands to change a control domain
505    identified in the ADM field of the guest's CRYCB. The AP command must be sent
510 Three CPU model features are available for controlling guest access to AP
513 1. AP facilities feature
515    The AP facilities feature indicates that AP facilities are installed on the
516    guest. This feature will be exposed for use only if the AP facilities
529         indicates whether AP facilities are installed (on) or not
531         is ``ap=on``. AP facilities must be installed on the guest if a
537    The QCI facility is used by the AP bus running on the guest to query the
538    configuration of the AP facilities. This facility will be available
555         If QCI is installed (``apqci=on``) but AP facilities are not
558         installed if the AP facilities are not; this is considered
562         greater than 15 will not be detected by the AP bus
567    The APFT facility is used by the AP bus running on the guest to test the
568    AP facilities available for a given AP queue. This facility will be available
587         If APFT is installed (``apft=on``) but AP facilities are not
590         installed if the AP facilities are not; this is considered
593         It also makes no sense to turn APFT off because the AP bus
596         for guest usage, no AP devices can be made accessible to a
612 device to which AP resources to be used by the guest have been assigned.
615 Note that on Linux guests, the AP devices will be created in the
616 ``/sys/bus/ap/devices`` directory when the AP bus subsequently performs its periodic
617 scan, so there may be a short delay before the AP devices are accessible on the
624 * The CPU model features for controlling guest access to AP facilities are not
640 On a Linux guest, the AP devices will be removed from the ``/sys/bus/ap/devices``
641 directory on the guest when the AP bus subsequently performs its periodic scan,
642 so there may be a short delay before the AP devices are no longer accessible by
649 Example: Configure AP Matrices for Three Linux Guests
653 access to AP facilities. For this example, we will show how to configure
712            select S390 AP IOMMU Support
717         -> VFIO support for AP devices
719 2. Secure the AP queues to be used by the three guests so that the host can not
720    access them. To secure the AP queues 05.0004, 05.0047, 05.00ab, 05.00ff,
729    This will result in AP queues 05.0004, 05.0047, 05.00ab, 05.00ff, 06.0004,
732    to the AP queue devices bound to it::
752    The administrator, therefore, must take care to secure only AP queues that
753    can be bound to the vfio_ap device driver. The device type for a given AP
762 3. Create the mediated devices needed to configure the AP matrixes for the
906   assigned lest the host be given access to the private data of the AP queue
909 * Dynamically assigning AP resources to or unassigning AP resources from a
910   mediated matrix device - see `Configuring an AP matrix for a linux guest`_
913 * Live guest migration is not supported for guests using AP devices. If a guest
914   is using AP devices, the vfio-ap device configured for the guest must be