1.. SPDX-License-Identifier: GPL-2.0
2.. _ultravisor:
3
4============================
5Protected Execution Facility
6============================
7
8.. contents::
9    :depth: 3
10
11Protected Execution Facility
12############################
13
14    Protected Execution Facility (PEF) is an architectural change for
15    POWER 9 that enables Secure Virtual Machines (SVMs). DD2.3 chips
16    (PVR=0x004e1203) or greater will be PEF-capable. A new ISA release
17    will include the PEF RFC02487 changes.
18
19    When enabled, PEF adds a new higher privileged mode, called Ultravisor
20    mode, to POWER architecture. Along with the new mode there is new
21    firmware called the Protected Execution Ultravisor (or Ultravisor
22    for short). Ultravisor mode is the highest privileged mode in POWER
23    architecture.
24
25	+------------------+
26	| Privilege States |
27	+==================+
28	|  Problem         |
29	+------------------+
30	|  Supervisor      |
31	+------------------+
32	|  Hypervisor      |
33	+------------------+
34	|  Ultravisor      |
35	+------------------+
36
37    PEF protects SVMs from the hypervisor, privileged users, and other
38    VMs in the system. SVMs are protected while at rest and can only be
39    executed by an authorized machine. All virtual machines utilize
40    hypervisor services. The Ultravisor filters calls between the SVMs
41    and the hypervisor to assure that information does not accidentally
42    leak. All hypercalls except H_RANDOM are reflected to the hypervisor.
43    H_RANDOM is not reflected to prevent the hypervisor from influencing
44    random values in the SVM.
45
46    To support this there is a refactoring of the ownership of resources
47    in the CPU. Some of the resources which were previously hypervisor
48    privileged are now ultravisor privileged.
49
50Hardware
51========
52
53    The hardware changes include the following:
54
55    * There is a new bit in the MSR that determines whether the current
56      process is running in secure mode, MSR(S) bit 41. MSR(S)=1, process
57      is in secure mode, MSR(s)=0 process is in normal mode.
58
59    * The MSR(S) bit can only be set by the Ultravisor.
60
61    * HRFID cannot be used to set the MSR(S) bit. If the hypervisor needs
62      to return to a SVM it must use an ultracall. It can determine if
63      the VM it is returning to is secure.
64
65    * There is a new Ultravisor privileged register, SMFCTRL, which has an
66      enable/disable bit SMFCTRL(E).
67
68    * The privilege of a process is now determined by three MSR bits,
69      MSR(S, HV, PR). In each of the tables below the modes are listed
70      from least privilege to highest privilege. The higher privilege
71      modes can access all the resources of the lower privilege modes.
72
73      **Secure Mode MSR Settings**
74
75      +---+---+---+---------------+
76      | S | HV| PR|Privilege      |
77      +===+===+===+===============+
78      | 1 | 0 | 1 | Problem       |
79      +---+---+---+---------------+
80      | 1 | 0 | 0 | Privileged(OS)|
81      +---+---+---+---------------+
82      | 1 | 1 | 0 | Ultravisor    |
83      +---+---+---+---------------+
84      | 1 | 1 | 1 | Reserved      |
85      +---+---+---+---------------+
86
87      **Normal Mode MSR Settings**
88
89      +---+---+---+---------------+
90      | S | HV| PR|Privilege      |
91      +===+===+===+===============+
92      | 0 | 0 | 1 | Problem       |
93      +---+---+---+---------------+
94      | 0 | 0 | 0 | Privileged(OS)|
95      +---+---+---+---------------+
96      | 0 | 1 | 0 | Hypervisor    |
97      +---+---+---+---------------+
98      | 0 | 1 | 1 | Problem (Host)|
99      +---+---+---+---------------+
100
101    * Memory is partitioned into secure and normal memory. Only processes
102      that are running in secure mode can access secure memory.
103
104    * The hardware does not allow anything that is not running secure to
105      access secure memory. This means that the Hypervisor cannot access
106      the memory of the SVM without using an ultracall (asking the
107      Ultravisor). The Ultravisor will only allow the hypervisor to see
108      the SVM memory encrypted.
109
110    * I/O systems are not allowed to directly address secure memory. This
111      limits the SVMs to virtual I/O only.
112
113    * The architecture allows the SVM to share pages of memory with the
114      hypervisor that are not protected with encryption. However, this
115      sharing must be initiated by the SVM.
116
117    * When a process is running in secure mode all hypercalls
118      (syscall lev=1) go to the Ultravisor.
119
120    * When a process is in secure mode all interrupts go to the
121      Ultravisor.
122
123    * The following resources have become Ultravisor privileged and
124      require an Ultravisor interface to manipulate:
125
126      * Processor configurations registers (SCOMs).
127
128      * Stop state information.
129
130      * The debug registers CIABR, DAWR, and DAWRX when SMFCTRL(D) is set.
131        If SMFCTRL(D) is not set they do not work in secure mode. When set,
132        reading and writing requires an Ultravisor call, otherwise that
133        will cause a Hypervisor Emulation Assistance interrupt.
134
135      * PTCR and partition table entries (partition table is in secure
136        memory). An attempt to write to PTCR will cause a Hypervisor
137        Emulation Assitance interrupt.
138
139      * LDBAR (LD Base Address Register) and IMC (In-Memory Collection)
140        non-architected registers. An attempt to write to them will cause a
141        Hypervisor Emulation Assistance interrupt.
142
143      * Paging for an SVM, sharing of memory with Hypervisor for an SVM.
144        (Including Virtual Processor Area (VPA) and virtual I/O).
145
146
147Software/Microcode
148==================
149
150    The software changes include:
151
152    * SVMs are created from normal VM using (open source) tooling supplied
153      by IBM.
154
155    * All SVMs start as normal VMs and utilize an ultracall, UV_ESM
156      (Enter Secure Mode), to make the transition.
157
158    * When the UV_ESM ultracall is made the Ultravisor copies the VM into
159      secure memory, decrypts the verification information, and checks the
160      integrity of the SVM. If the integrity check passes the Ultravisor
161      passes control in secure mode.
162
163    * The verification information includes the pass phrase for the
164      encrypted disk associated with the SVM. This pass phrase is given
165      to the SVM when requested.
166
167    * The Ultravisor is not involved in protecting the encrypted disk of
168      the SVM while at rest.
169
170    * For external interrupts the Ultravisor saves the state of the SVM,
171      and reflects the interrupt to the hypervisor for processing.
172      For hypercalls, the Ultravisor inserts neutral state into all
173      registers not needed for the hypercall then reflects the call to
174      the hypervisor for processing. The H_RANDOM hypercall is performed
175      by the Ultravisor and not reflected.
176
177    * For virtual I/O to work bounce buffering must be done.
178
179    * The Ultravisor uses AES (IAPM) for protection of SVM memory. IAPM
180      is a mode of AES that provides integrity and secrecy concurrently.
181
182    * The movement of data between normal and secure pages is coordinated
183      with the Ultravisor by a new HMM plug-in in the Hypervisor.
184
185    The Ultravisor offers new services to the hypervisor and SVMs. These
186    are accessed through ultracalls.
187
188Terminology
189===========
190
191    * Hypercalls: special system calls used to request services from
192      Hypervisor.
193
194    * Normal memory: Memory that is accessible to Hypervisor.
195
196    * Normal page: Page backed by normal memory and available to
197      Hypervisor.
198
199    * Shared page: A page backed by normal memory and available to both
200      the Hypervisor/QEMU and the SVM (i.e page has mappings in SVM and
201      Hypervisor/QEMU).
202
203    * Secure memory: Memory that is accessible only to Ultravisor and
204      SVMs.
205
206    * Secure page: Page backed by secure memory and only available to
207      Ultravisor and SVM.
208
209    * SVM: Secure Virtual Machine.
210
211    * Ultracalls: special system calls used to request services from
212      Ultravisor.
213
214
215Ultravisor calls API
216####################
217
218    This section describes Ultravisor calls (ultracalls) needed to
219    support Secure Virtual Machines (SVM)s and Paravirtualized KVM. The
220    ultracalls allow the SVMs and Hypervisor to request services from the
221    Ultravisor such as accessing a register or memory region that can only
222    be accessed when running in Ultravisor-privileged mode.
223
224    The specific service needed from an ultracall is specified in register
225    R3 (the first parameter to the ultracall). Other parameters to the
226    ultracall, if any, are specified in registers R4 through R12.
227
228    Return value of all ultracalls is in register R3. Other output values
229    from the ultracall, if any, are returned in registers R4 through R12.
230    The only exception to this register usage is the ``UV_RETURN``
231    ultracall described below.
232
233    Each ultracall returns specific error codes, applicable in the context
234    of the ultracall. However, like with the PowerPC Architecture Platform
235    Reference (PAPR), if no specific error code is defined for a
236    particular situation, then the ultracall will fallback to an erroneous
237    parameter-position based code. i.e U_PARAMETER, U_P2, U_P3 etc
238    depending on the ultracall parameter that may have caused the error.
239
240    Some ultracalls involve transferring a page of data between Ultravisor
241    and Hypervisor.  Secure pages that are transferred from secure memory
242    to normal memory may be encrypted using dynamically generated keys.
243    When the secure pages are transferred back to secure memory, they may
244    be decrypted using the same dynamically generated keys. Generation and
245    management of these keys will be covered in a separate document.
246
247    For now this only covers ultracalls currently implemented and being
248    used by Hypervisor and SVMs but others can be added here when it
249    makes sense.
250
251    The full specification for all hypercalls/ultracalls will eventually
252    be made available in the public/OpenPower version of the PAPR
253    specification.
254
255    .. note::
256
257        If PEF is not enabled, the ultracalls will be redirected to the
258        Hypervisor which must handle/fail the calls.
259
260Ultracalls used by Hypervisor
261=============================
262
263    This section describes the virtual memory management ultracalls used
264    by the Hypervisor to manage SVMs.
265
266UV_PAGE_OUT
267-----------
268
269    Encrypt and move the contents of a page from secure memory to normal
270    memory.
271
272Syntax
273~~~~~~
274
275.. code-block:: c
276
277	uint64_t ultracall(const uint64_t UV_PAGE_OUT,
278		uint16_t lpid,		/* LPAR ID */
279		uint64_t dest_ra,	/* real address of destination page */
280		uint64_t src_gpa,	/* source guest-physical-address */
281		uint8_t  flags,		/* flags */
282		uint64_t order)		/* page size order */
283
284Return values
285~~~~~~~~~~~~~
286
287    One of the following values:
288
289	* U_SUCCESS	on success.
290	* U_PARAMETER	if ``lpid`` is invalid.
291	* U_P2 		if ``dest_ra`` is invalid.
292	* U_P3		if the ``src_gpa`` address is invalid.
293	* U_P4		if any bit in the ``flags`` is unrecognized
294	* U_P5		if the ``order`` parameter is unsupported.
295	* U_FUNCTION	if functionality is not supported.
296	* U_BUSY	if page cannot be currently paged-out.
297
298Description
299~~~~~~~~~~~
300
301    Encrypt the contents of a secure-page and make it available to
302    Hypervisor in a normal page.
303
304    By default, the source page is unmapped from the SVM's partition-
305    scoped page table. But the Hypervisor can provide a hint to the
306    Ultravisor to retain the page mapping by setting the ``UV_SNAPSHOT``
307    flag in ``flags`` parameter.
308
309    If the source page is already a shared page the call returns
310    U_SUCCESS, without doing anything.
311
312Use cases
313~~~~~~~~~
314
315    #. QEMU attempts to access an address belonging to the SVM but the
316       page frame for that address is not mapped into QEMU's address
317       space. In this case, the Hypervisor will allocate a page frame,
318       map it into QEMU's address space and issue the ``UV_PAGE_OUT``
319       call to retrieve the encrypted contents of the page.
320
321    #. When Ultravisor runs low on secure memory and it needs to page-out
322       an LRU page. In this case, Ultravisor will issue the
323       ``H_SVM_PAGE_OUT`` hypercall to the Hypervisor. The Hypervisor will
324       then allocate a normal page and issue the ``UV_PAGE_OUT`` ultracall
325       and the Ultravisor will encrypt and move the contents of the secure
326       page into the normal page.
327
328    #. When Hypervisor accesses SVM data, the Hypervisor requests the
329       Ultravisor to transfer the corresponding page into a insecure page,
330       which the Hypervisor can access. The data in the normal page will
331       be encrypted though.
332
333UV_PAGE_IN
334----------
335
336    Move the contents of a page from normal memory to secure memory.
337
338Syntax
339~~~~~~
340
341.. code-block:: c
342
343	uint64_t ultracall(const uint64_t UV_PAGE_IN,
344		uint16_t lpid,		/* the LPAR ID */
345		uint64_t src_ra,	/* source real address of page */
346		uint64_t dest_gpa,	/* destination guest physical address */
347		uint64_t flags,		/* flags */
348		uint64_t order)		/* page size order */
349
350Return values
351~~~~~~~~~~~~~
352
353    One of the following values:
354
355	* U_SUCCESS	on success.
356	* U_BUSY	if page cannot be currently paged-in.
357	* U_FUNCTION	if functionality is not supported
358	* U_PARAMETER	if ``lpid`` is invalid.
359	* U_P2 		if ``src_ra`` is invalid.
360	* U_P3		if the ``dest_gpa`` address is invalid.
361	* U_P4		if any bit in the ``flags`` is unrecognized
362	* U_P5		if the ``order`` parameter is unsupported.
363
364Description
365~~~~~~~~~~~
366
367    Move the contents of the page identified by ``src_ra`` from normal
368    memory to secure memory and map it to the guest physical address
369    ``dest_gpa``.
370
371    If `dest_gpa` refers to a shared address, map the page into the
372    partition-scoped page-table of the SVM.  If `dest_gpa` is not shared,
373    copy the contents of the page into the corresponding secure page.
374    Depending on the context, decrypt the page before being copied.
375
376    The caller provides the attributes of the page through the ``flags``
377    parameter. Valid values for ``flags`` are:
378
379	* CACHE_INHIBITED
380	* CACHE_ENABLED
381	* WRITE_PROTECTION
382
383    The Hypervisor must pin the page in memory before making
384    ``UV_PAGE_IN`` ultracall.
385
386Use cases
387~~~~~~~~~
388
389    #. When a normal VM switches to secure mode, all its pages residing
390       in normal memory, are moved into secure memory.
391
392    #. When an SVM requests to share a page with Hypervisor the Hypervisor
393       allocates a page and informs the Ultravisor.
394
395    #. When an SVM accesses a secure page that has been paged-out,
396       Ultravisor invokes the Hypervisor to locate the page. After
397       locating the page, the Hypervisor uses UV_PAGE_IN to make the
398       page available to Ultravisor.
399
400UV_PAGE_INVAL
401-------------
402
403    Invalidate the Ultravisor mapping of a page.
404
405Syntax
406~~~~~~
407
408.. code-block:: c
409
410	uint64_t ultracall(const uint64_t UV_PAGE_INVAL,
411		uint16_t lpid,		/* the LPAR ID */
412		uint64_t guest_pa,	/* destination guest-physical-address */
413		uint64_t order)		/* page size order */
414
415Return values
416~~~~~~~~~~~~~
417
418    One of the following values:
419
420	* U_SUCCESS	on success.
421	* U_PARAMETER	if ``lpid`` is invalid.
422	* U_P2 		if ``guest_pa`` is invalid (or corresponds to a secure
423                        page mapping).
424	* U_P3		if the ``order`` is invalid.
425	* U_FUNCTION	if functionality is not supported.
426	* U_BUSY	if page cannot be currently invalidated.
427
428Description
429~~~~~~~~~~~
430
431    This ultracall informs Ultravisor that the page mapping in Hypervisor
432    corresponding to the given guest physical address has been invalidated
433    and that the Ultravisor should not access the page. If the specified
434    ``guest_pa`` corresponds to a secure page, Ultravisor will ignore the
435    attempt to invalidate the page and return U_P2.
436
437Use cases
438~~~~~~~~~
439
440    #. When a shared page is unmapped from the QEMU's page table, possibly
441       because it is paged-out to disk, Ultravisor needs to know that the
442       page should not be accessed from its side too.
443
444
445UV_WRITE_PATE
446-------------
447
448    Validate and write the partition table entry (PATE) for a given
449    partition.
450
451Syntax
452~~~~~~
453
454.. code-block:: c
455
456	uint64_t ultracall(const uint64_t UV_WRITE_PATE,
457		uint32_t lpid,		/* the LPAR ID */
458		uint64_t dw0		/* the first double word to write */
459		uint64_t dw1)		/* the second double word to write */
460
461Return values
462~~~~~~~~~~~~~
463
464    One of the following values:
465
466	* U_SUCCESS	on success.
467	* U_BUSY	if PATE cannot be currently written to.
468	* U_FUNCTION	if functionality is not supported.
469	* U_PARAMETER	if ``lpid`` is invalid.
470	* U_P2 		if ``dw0`` is invalid.
471	* U_P3		if the ``dw1`` address is invalid.
472	* U_PERMISSION	if the Hypervisor is attempting to change the PATE
473			of a secure virtual machine or if called from a
474			context other than Hypervisor.
475
476Description
477~~~~~~~~~~~
478
479    Validate and write a LPID and its partition-table-entry for the given
480    LPID.  If the LPID is already allocated and initialized, this call
481    results in changing the partition table entry.
482
483Use cases
484~~~~~~~~~
485
486    #. The Partition table resides in Secure memory and its entries,
487       called PATE (Partition Table Entries), point to the partition-
488       scoped page tables for the Hypervisor as well as each of the
489       virtual machines (both secure and normal). The Hypervisor
490       operates in partition 0 and its partition-scoped page tables
491       reside in normal memory.
492
493    #. This ultracall allows the Hypervisor to register the partition-
494       scoped and process-scoped page table entries for the Hypervisor
495       and other partitions (virtual machines) with the Ultravisor.
496
497    #. If the value of the PATE for an existing partition (VM) changes,
498       the TLB cache for the partition is flushed.
499
500    #. The Hypervisor is responsible for allocating LPID. The LPID and
501       its PATE entry are registered together.  The Hypervisor manages
502       the PATE entries for a normal VM and can change the PATE entry
503       anytime. Ultravisor manages the PATE entries for an SVM and
504       Hypervisor is not allowed to modify them.
505
506UV_RETURN
507---------
508
509    Return control from the Hypervisor back to the Ultravisor after
510    processing an hypercall or interrupt that was forwarded (aka
511    *reflected*) to the Hypervisor.
512
513Syntax
514~~~~~~
515
516.. code-block:: c
517
518	uint64_t ultracall(const uint64_t UV_RETURN)
519
520Return values
521~~~~~~~~~~~~~
522
523     This call never returns to Hypervisor on success.  It returns
524     U_INVALID if ultracall is not made from a Hypervisor context.
525
526Description
527~~~~~~~~~~~
528
529    When an SVM makes an hypercall or incurs some other exception, the
530    Ultravisor usually forwards (aka *reflects*) the exceptions to the
531    Hypervisor.  After processing the exception, Hypervisor uses the
532    ``UV_RETURN`` ultracall to return control back to the SVM.
533
534    The expected register state on entry to this ultracall is:
535
536    * Non-volatile registers are restored to their original values.
537    * If returning from an hypercall, register R0 contains the return
538      value (**unlike other ultracalls**) and, registers R4 through R12
539      contain any output values of the hypercall.
540    * R3 contains the ultracall number, i.e UV_RETURN.
541    * If returning with a synthesized interrupt, R2 contains the
542      synthesized interrupt number.
543
544Use cases
545~~~~~~~~~
546
547    #. Ultravisor relies on the Hypervisor to provide several services to
548       the SVM such as processing hypercall and other exceptions. After
549       processing the exception, Hypervisor uses UV_RETURN to return
550       control back to the Ultravisor.
551
552    #. Hypervisor has to use this ultracall to return control to the SVM.
553
554
555UV_REGISTER_MEM_SLOT
556--------------------
557
558    Register an SVM address-range with specified properties.
559
560Syntax
561~~~~~~
562
563.. code-block:: c
564
565	uint64_t ultracall(const uint64_t UV_REGISTER_MEM_SLOT,
566		uint64_t lpid,		/* LPAR ID of the SVM */
567		uint64_t start_gpa,	/* start guest physical address */
568		uint64_t size,		/* size of address range in bytes */
569		uint64_t flags		/* reserved for future expansion */
570		uint16_t slotid)	/* slot identifier */
571
572Return values
573~~~~~~~~~~~~~
574
575    One of the following values:
576
577	* U_SUCCESS	on success.
578	* U_PARAMETER	if ``lpid`` is invalid.
579	* U_P2 		if ``start_gpa`` is invalid.
580	* U_P3		if ``size`` is invalid.
581	* U_P4		if any bit in the ``flags`` is unrecognized.
582	* U_P5		if the ``slotid`` parameter is unsupported.
583	* U_PERMISSION	if called from context other than Hypervisor.
584	* U_FUNCTION	if functionality is not supported.
585
586
587Description
588~~~~~~~~~~~
589
590    Register a memory range for an SVM.  The memory range starts at the
591    guest physical address ``start_gpa`` and is ``size`` bytes long.
592
593Use cases
594~~~~~~~~~
595
596
597    #. When a virtual machine goes secure, all the memory slots managed by
598       the Hypervisor move into secure memory. The Hypervisor iterates
599       through each of memory slots, and registers the slot with
600       Ultravisor.  Hypervisor may discard some slots such as those used
601       for firmware (SLOF).
602
603    #. When new memory is hot-plugged, a new memory slot gets registered.
604
605
606UV_UNREGISTER_MEM_SLOT
607----------------------
608
609    Unregister an SVM address-range that was previously registered using
610    UV_REGISTER_MEM_SLOT.
611
612Syntax
613~~~~~~
614
615.. code-block:: c
616
617	uint64_t ultracall(const uint64_t UV_UNREGISTER_MEM_SLOT,
618		uint64_t lpid,		/* LPAR ID of the SVM */
619		uint64_t slotid)	/* reservation slotid */
620
621Return values
622~~~~~~~~~~~~~
623
624    One of the following values:
625
626	* U_SUCCESS	on success.
627	* U_FUNCTION	if functionality is not supported.
628	* U_PARAMETER	if ``lpid`` is invalid.
629	* U_P2 		if ``slotid`` is invalid.
630	* U_PERMISSION	if called from context other than Hypervisor.
631
632Description
633~~~~~~~~~~~
634
635    Release the memory slot identified by ``slotid`` and free any
636    resources allocated towards the reservation.
637
638Use cases
639~~~~~~~~~
640
641    #. Memory hot-remove.
642
643
644UV_SVM_TERMINATE
645----------------
646
647    Terminate an SVM and release its resources.
648
649Syntax
650~~~~~~
651
652.. code-block:: c
653
654	uint64_t ultracall(const uint64_t UV_SVM_TERMINATE,
655		uint64_t lpid,		/* LPAR ID of the SVM */)
656
657Return values
658~~~~~~~~~~~~~
659
660    One of the following values:
661
662	* U_SUCCESS	on success.
663	* U_FUNCTION	if functionality is not supported.
664	* U_PARAMETER	if ``lpid`` is invalid.
665	* U_INVALID	if VM is not secure.
666	* U_PERMISSION  if not called from a Hypervisor context.
667
668Description
669~~~~~~~~~~~
670
671    Terminate an SVM and release all its resources.
672
673Use cases
674~~~~~~~~~
675
676    #. Called by Hypervisor when terminating an SVM.
677
678
679Ultracalls used by SVM
680======================
681
682UV_SHARE_PAGE
683-------------
684
685    Share a set of guest physical pages with the Hypervisor.
686
687Syntax
688~~~~~~
689
690.. code-block:: c
691
692	uint64_t ultracall(const uint64_t UV_SHARE_PAGE,
693		uint64_t gfn,	/* guest page frame number */
694		uint64_t num)	/* number of pages of size PAGE_SIZE */
695
696Return values
697~~~~~~~~~~~~~
698
699    One of the following values:
700
701	* U_SUCCESS	on success.
702	* U_FUNCTION	if functionality is not supported.
703	* U_INVALID	if the VM is not secure.
704	* U_PARAMETER	if ``gfn`` is invalid.
705	* U_P2 		if ``num`` is invalid.
706
707Description
708~~~~~~~~~~~
709
710    Share the ``num`` pages starting at guest physical frame number ``gfn``
711    with the Hypervisor. Assume page size is PAGE_SIZE bytes. Zero the
712    pages before returning.
713
714    If the address is already backed by a secure page, unmap the page and
715    back it with an insecure page, with the help of the Hypervisor. If it
716    is not backed by any page yet, mark the PTE as insecure and back it
717    with an insecure page when the address is accessed. If it is already
718    backed by an insecure page, zero the page and return.
719
720Use cases
721~~~~~~~~~
722
723    #. The Hypervisor cannot access the SVM pages since they are backed by
724       secure pages. Hence an SVM must explicitly request Ultravisor for
725       pages it can share with Hypervisor.
726
727    #. Shared pages are needed to support virtio and Virtual Processor Area
728       (VPA) in SVMs.
729
730
731UV_UNSHARE_PAGE
732---------------
733
734    Restore a shared SVM page to its initial state.
735
736Syntax
737~~~~~~
738
739.. code-block:: c
740
741	uint64_t ultracall(const uint64_t UV_UNSHARE_PAGE,
742		uint64_t gfn,	/* guest page frame number */
743		uint73 num)	/* number of pages of size PAGE_SIZE*/
744
745Return values
746~~~~~~~~~~~~~
747
748    One of the following values:
749
750	* U_SUCCESS	on success.
751	* U_FUNCTION	if functionality is not supported.
752	* U_INVALID	if VM is not secure.
753	* U_PARAMETER	if ``gfn`` is invalid.
754	* U_P2 		if ``num`` is invalid.
755
756Description
757~~~~~~~~~~~
758
759    Stop sharing ``num`` pages starting at ``gfn`` with the Hypervisor.
760    Assume that the page size is PAGE_SIZE. Zero the pages before
761    returning.
762
763    If the address is already backed by an insecure page, unmap the page
764    and back it with a secure page. Inform the Hypervisor to release
765    reference to its shared page. If the address is not backed by a page
766    yet, mark the PTE as secure and back it with a secure page when that
767    address is accessed. If it is already backed by an secure page zero
768    the page and return.
769
770Use cases
771~~~~~~~~~
772
773    #. The SVM may decide to unshare a page from the Hypervisor.
774
775
776UV_UNSHARE_ALL_PAGES
777--------------------
778
779    Unshare all pages the SVM has shared with Hypervisor.
780
781Syntax
782~~~~~~
783
784.. code-block:: c
785
786	uint64_t ultracall(const uint64_t UV_UNSHARE_ALL_PAGES)
787
788Return values
789~~~~~~~~~~~~~
790
791    One of the following values:
792
793	* U_SUCCESS	on success.
794	* U_FUNCTION	if functionality is not supported.
795	* U_INVAL	if VM is not secure.
796
797Description
798~~~~~~~~~~~
799
800    Unshare all shared pages from the Hypervisor. All unshared pages are
801    zeroed on return. Only pages explicitly shared by the SVM with the
802    Hypervisor (using UV_SHARE_PAGE ultracall) are unshared. Ultravisor
803    may internally share some pages with the Hypervisor without explicit
804    request from the SVM.  These pages will not be unshared by this
805    ultracall.
806
807Use cases
808~~~~~~~~~
809
810    #. This call is needed when ``kexec`` is used to boot a different
811       kernel. It may also be needed during SVM reset.
812
813UV_ESM
814------
815
816    Secure the virtual machine (*enter secure mode*).
817
818Syntax
819~~~~~~
820
821.. code-block:: c
822
823	uint64_t ultracall(const uint64_t UV_ESM,
824		uint64_t esm_blob_addr,	/* location of the ESM blob */
825		unint64_t fdt)		/* Flattened device tree */
826
827Return values
828~~~~~~~~~~~~~
829
830    One of the following values:
831
832	* U_SUCCESS	on success (including if VM is already secure).
833	* U_FUNCTION	if functionality is not supported.
834	* U_INVALID	if VM is not secure.
835	* U_PARAMETER	if ``esm_blob_addr`` is invalid.
836	* U_P2 		if ``fdt`` is invalid.
837	* U_PERMISSION	if any integrity checks fail.
838	* U_RETRY	insufficient memory to create SVM.
839	* U_NO_KEY	symmetric key unavailable.
840
841Description
842~~~~~~~~~~~
843
844    Secure the virtual machine. On successful completion, return
845    control to the virtual machine at the address specified in the
846    ESM blob.
847
848Use cases
849~~~~~~~~~
850
851    #. A normal virtual machine can choose to switch to a secure mode.
852
853Hypervisor Calls API
854####################
855
856    This document describes the Hypervisor calls (hypercalls) that are
857    needed to support the Ultravisor. Hypercalls are services provided by
858    the Hypervisor to virtual machines and Ultravisor.
859
860    Register usage for these hypercalls is identical to that of the other
861    hypercalls defined in the Power Architecture Platform Reference (PAPR)
862    document.  i.e on input, register R3 identifies the specific service
863    that is being requested and registers R4 through R11 contain
864    additional parameters to the hypercall, if any. On output, register
865    R3 contains the return value and registers R4 through R9 contain any
866    other output values from the hypercall.
867
868    This document only covers hypercalls currently implemented/planned
869    for Ultravisor usage but others can be added here when it makes sense.
870
871    The full specification for all hypercalls/ultracalls will eventually
872    be made available in the public/OpenPower version of the PAPR
873    specification.
874
875Hypervisor calls to support Ultravisor
876======================================
877
878    Following are the set of hypercalls needed to support Ultravisor.
879
880H_SVM_INIT_START
881----------------
882
883    Begin the process of converting a normal virtual machine into an SVM.
884
885Syntax
886~~~~~~
887
888.. code-block:: c
889
890	uint64_t hypercall(const uint64_t H_SVM_INIT_START)
891
892Return values
893~~~~~~~~~~~~~
894
895    One of the following values:
896
897	* H_SUCCESS	 on success.
898
899Description
900~~~~~~~~~~~
901
902    Initiate the process of securing a virtual machine. This involves
903    coordinating with the Ultravisor, using ultracalls, to allocate
904    resources in the Ultravisor for the new SVM, transferring the VM's
905    pages from normal to secure memory etc. When the process is
906    completed, Ultravisor issues the H_SVM_INIT_DONE hypercall.
907
908Use cases
909~~~~~~~~~
910
911     #. Ultravisor uses this hypercall to inform Hypervisor that a VM
912        has initiated the process of switching to secure mode.
913
914
915H_SVM_INIT_DONE
916---------------
917
918    Complete the process of securing an SVM.
919
920Syntax
921~~~~~~
922
923.. code-block:: c
924
925	uint64_t hypercall(const uint64_t H_SVM_INIT_DONE)
926
927Return values
928~~~~~~~~~~~~~
929
930    One of the following values:
931
932	* H_SUCCESS 		on success.
933	* H_UNSUPPORTED		if called from the wrong context (e.g.
934				from an SVM or before an H_SVM_INIT_START
935				hypercall).
936
937Description
938~~~~~~~~~~~
939
940    Complete the process of securing a virtual machine. This call must
941    be made after a prior call to ``H_SVM_INIT_START`` hypercall.
942
943Use cases
944~~~~~~~~~
945
946    On successfully securing a virtual machine, the Ultravisor informs
947    Hypervisor about it. Hypervisor can use this call to finish setting
948    up its internal state for this virtual machine.
949
950
951H_SVM_PAGE_IN
952-------------
953
954    Move the contents of a page from normal memory to secure memory.
955
956Syntax
957~~~~~~
958
959.. code-block:: c
960
961	uint64_t hypercall(const uint64_t H_SVM_PAGE_IN,
962		uint64_t guest_pa,	/* guest-physical-address */
963		uint64_t flags,		/* flags */
964		uint64_t order)		/* page size order */
965
966Return values
967~~~~~~~~~~~~~
968
969    One of the following values:
970
971	* H_SUCCESS	on success.
972	* H_PARAMETER	if ``guest_pa`` is invalid.
973	* H_P2		if ``flags`` is invalid.
974	* H_P3		if ``order`` of page is invalid.
975
976Description
977~~~~~~~~~~~
978
979    Retrieve the content of the page, belonging to the VM at the specified
980    guest physical address.
981
982    Only valid value(s) in ``flags`` are:
983
984        * H_PAGE_IN_SHARED which indicates that the page is to be shared
985	  with the Ultravisor.
986
987	* H_PAGE_IN_NONSHARED indicates that the UV is not anymore
988          interested in the page. Applicable if the page is a shared page.
989
990    The ``order`` parameter must correspond to the configured page size.
991
992Use cases
993~~~~~~~~~
994
995    #. When a normal VM becomes a secure VM (using the UV_ESM ultracall),
996       the Ultravisor uses this hypercall to move contents of each page of
997       the VM from normal memory to secure memory.
998
999    #. Ultravisor uses this hypercall to ask Hypervisor to provide a page
1000       in normal memory that can be shared between the SVM and Hypervisor.
1001
1002    #. Ultravisor uses this hypercall to page-in a paged-out page. This
1003       can happen when the SVM touches a paged-out page.
1004
1005    #. If SVM wants to disable sharing of pages with Hypervisor, it can
1006       inform Ultravisor to do so. Ultravisor will then use this hypercall
1007       and inform Hypervisor that it has released access to the normal
1008       page.
1009
1010H_SVM_PAGE_OUT
1011---------------
1012
1013    Move the contents of the page to normal memory.
1014
1015Syntax
1016~~~~~~
1017
1018.. code-block:: c
1019
1020	uint64_t hypercall(const uint64_t H_SVM_PAGE_OUT,
1021		uint64_t guest_pa,	/* guest-physical-address */
1022		uint64_t flags,		/* flags (currently none) */
1023		uint64_t order)		/* page size order */
1024
1025Return values
1026~~~~~~~~~~~~~
1027
1028    One of the following values:
1029
1030	* H_SUCCESS	on success.
1031	* H_PARAMETER	if ``guest_pa`` is invalid.
1032	* H_P2		if ``flags`` is invalid.
1033	* H_P3		if ``order`` is invalid.
1034
1035Description
1036~~~~~~~~~~~
1037
1038    Move the contents of the page identified by ``guest_pa`` to normal
1039    memory.
1040
1041    Currently ``flags`` is unused and must be set to 0. The ``order``
1042    parameter must correspond to the configured page size.
1043
1044Use cases
1045~~~~~~~~~
1046
1047    #. If Ultravisor is running low on secure pages, it can move the
1048       contents of some secure pages, into normal pages using this
1049       hypercall. The content will be encrypted.
1050
1051References
1052##########
1053
1054- `Supporting Protected Computing on IBM Power Architecture <https://developer.ibm.com/articles/l-support-protected-computing/>`_
1055