1*daec8d40SPaolo Bonzini.. SPDX-License-Identifier: GPL-2.0 2*daec8d40SPaolo Bonzini 3*daec8d40SPaolo Bonzini========================================= 4*daec8d40SPaolo Bonzinis390 (IBM Z) Ultravisor and Protected VMs 5*daec8d40SPaolo Bonzini========================================= 6*daec8d40SPaolo Bonzini 7*daec8d40SPaolo BonziniSummary 8*daec8d40SPaolo Bonzini------- 9*daec8d40SPaolo BonziniProtected virtual machines (PVM) are KVM VMs that do not allow KVM to 10*daec8d40SPaolo Bonziniaccess VM state like guest memory or guest registers. Instead, the 11*daec8d40SPaolo BonziniPVMs are mostly managed by a new entity called Ultravisor (UV). The UV 12*daec8d40SPaolo Bonziniprovides an API that can be used by PVMs and KVM to request management 13*daec8d40SPaolo Bonziniactions. 14*daec8d40SPaolo Bonzini 15*daec8d40SPaolo BonziniEach guest starts in non-protected mode and then may make a request to 16*daec8d40SPaolo Bonzinitransition into protected mode. On transition, KVM registers the guest 17*daec8d40SPaolo Bonziniand its VCPUs with the Ultravisor and prepares everything for running 18*daec8d40SPaolo Bonziniit. 19*daec8d40SPaolo Bonzini 20*daec8d40SPaolo BonziniThe Ultravisor will secure and decrypt the guest's boot memory 21*daec8d40SPaolo Bonzini(i.e. kernel/initrd). It will safeguard state changes like VCPU 22*daec8d40SPaolo Bonzinistarts/stops and injected interrupts while the guest is running. 23*daec8d40SPaolo Bonzini 24*daec8d40SPaolo BonziniAs access to the guest's state, such as the SIE state description, is 25*daec8d40SPaolo Bonzininormally needed to be able to run a VM, some changes have been made in 26*daec8d40SPaolo Bonzinithe behavior of the SIE instruction. A new format 4 state description 27*daec8d40SPaolo Bonzinihas been introduced, where some fields have different meanings for a 28*daec8d40SPaolo BonziniPVM. SIE exits are minimized as much as possible to improve speed and 29*daec8d40SPaolo Bonzinireduce exposed guest state. 30*daec8d40SPaolo Bonzini 31*daec8d40SPaolo Bonzini 32*daec8d40SPaolo BonziniInterrupt injection 33*daec8d40SPaolo Bonzini------------------- 34*daec8d40SPaolo BonziniInterrupt injection is safeguarded by the Ultravisor. As KVM doesn't 35*daec8d40SPaolo Bonzinihave access to the VCPUs' lowcores, injection is handled via the 36*daec8d40SPaolo Bonziniformat 4 state description. 37*daec8d40SPaolo Bonzini 38*daec8d40SPaolo BonziniMachine check, external, IO and restart interruptions each can be 39*daec8d40SPaolo Bonziniinjected on SIE entry via a bit in the interrupt injection control 40*daec8d40SPaolo Bonzinifield (offset 0x54). If the guest cpu is not enabled for the interrupt 41*daec8d40SPaolo Bonziniat the time of injection, a validity interception is recognized. The 42*daec8d40SPaolo Bonziniformat 4 state description contains fields in the interception data 43*daec8d40SPaolo Bonziniblock where data associated with the interrupt can be transported. 44*daec8d40SPaolo Bonzini 45*daec8d40SPaolo BonziniProgram and Service Call exceptions have another layer of 46*daec8d40SPaolo Bonzinisafeguarding; they can only be injected for instructions that have 47*daec8d40SPaolo Bonzinibeen intercepted into KVM. The exceptions need to be a valid outcome 48*daec8d40SPaolo Bonziniof an instruction emulation by KVM, e.g. we can never inject a 49*daec8d40SPaolo Bonziniaddressing exception as they are reported by SIE since KVM has no 50*daec8d40SPaolo Bonziniaccess to the guest memory. 51*daec8d40SPaolo Bonzini 52*daec8d40SPaolo Bonzini 53*daec8d40SPaolo BonziniMask notification interceptions 54*daec8d40SPaolo Bonzini------------------------------- 55*daec8d40SPaolo BonziniKVM cannot intercept lctl(g) and lpsw(e) anymore in order to be 56*daec8d40SPaolo Bonzininotified when a PVM enables a certain class of interrupt. As a 57*daec8d40SPaolo Bonzinireplacement, two new interception codes have been introduced: One 58*daec8d40SPaolo Bonziniindicating that the contents of CRs 0, 6, or 14 have been changed, 59*daec8d40SPaolo Bonziniindicating different interruption subclasses; and one indicating that 60*daec8d40SPaolo BonziniPSW bit 13 has been changed, indicating that a machine check 61*daec8d40SPaolo Bonziniintervention was requested and those are now enabled. 62*daec8d40SPaolo Bonzini 63*daec8d40SPaolo BonziniInstruction emulation 64*daec8d40SPaolo Bonzini--------------------- 65*daec8d40SPaolo BonziniWith the format 4 state description for PVMs, the SIE instruction already 66*daec8d40SPaolo Bonziniinterprets more instructions than it does with format 2. It is not able 67*daec8d40SPaolo Bonzinito interpret every instruction, but needs to hand some tasks to KVM; 68*daec8d40SPaolo Bonzinitherefore, the SIE and the ultravisor safeguard emulation inputs and outputs. 69*daec8d40SPaolo Bonzini 70*daec8d40SPaolo BonziniThe control structures associated with SIE provide the Secure 71*daec8d40SPaolo BonziniInstruction Data Area (SIDA), the Interception Parameters (IP) and the 72*daec8d40SPaolo BonziniSecure Interception General Register Save Area. Guest GRs and most of 73*daec8d40SPaolo Bonzinithe instruction data, such as I/O data structures, are filtered. 74*daec8d40SPaolo BonziniInstruction data is copied to and from the SIDA when needed. Guest 75*daec8d40SPaolo BonziniGRs are put into / retrieved from the Secure Interception General 76*daec8d40SPaolo BonziniRegister Save Area. 77*daec8d40SPaolo Bonzini 78*daec8d40SPaolo BonziniOnly GR values needed to emulate an instruction will be copied into this 79*daec8d40SPaolo Bonzinisave area and the real register numbers will be hidden. 80*daec8d40SPaolo Bonzini 81*daec8d40SPaolo BonziniThe Interception Parameters state description field still contains 82*daec8d40SPaolo Bonzinithe bytes of the instruction text, but with pre-set register values 83*daec8d40SPaolo Bonziniinstead of the actual ones. I.e. each instruction always uses the same 84*daec8d40SPaolo Bonziniinstruction text, in order not to leak guest instruction text. 85*daec8d40SPaolo BonziniThis also implies that the register content that a guest had in r<n> 86*daec8d40SPaolo Bonzinimay be in r<m> from the hypervisor's point of view. 87*daec8d40SPaolo Bonzini 88*daec8d40SPaolo BonziniThe Secure Instruction Data Area contains instruction storage 89*daec8d40SPaolo Bonzinidata. Instruction data, i.e. data being referenced by an instruction 90*daec8d40SPaolo Bonzinilike the SCCB for sclp, is moved via the SIDA. When an instruction is 91*daec8d40SPaolo Bonziniintercepted, the SIE will only allow data and program interrupts for 92*daec8d40SPaolo Bonzinithis instruction to be moved to the guest via the two data areas 93*daec8d40SPaolo Bonzinidiscussed before. Other data is either ignored or results in validity 94*daec8d40SPaolo Bonziniinterceptions. 95*daec8d40SPaolo Bonzini 96*daec8d40SPaolo Bonzini 97*daec8d40SPaolo BonziniInstruction emulation interceptions 98*daec8d40SPaolo Bonzini----------------------------------- 99*daec8d40SPaolo BonziniThere are two types of SIE secure instruction intercepts: the normal 100*daec8d40SPaolo Bonziniand the notification type. Normal secure instruction intercepts will 101*daec8d40SPaolo Bonzinimake the guest pending for instruction completion of the intercepted 102*daec8d40SPaolo Bonziniinstruction type, i.e. on SIE entry it is attempted to complete 103*daec8d40SPaolo Bonziniemulation of the instruction with the data provided by KVM. That might 104*daec8d40SPaolo Bonzinibe a program exception or instruction completion. 105*daec8d40SPaolo Bonzini 106*daec8d40SPaolo BonziniThe notification type intercepts inform KVM about guest environment 107*daec8d40SPaolo Bonzinichanges due to guest instruction interpretation. Such an interception 108*daec8d40SPaolo Bonziniis recognized, for example, for the store prefix instruction to provide 109*daec8d40SPaolo Bonzinithe new lowcore location. On SIE reentry, any KVM data in the data areas 110*daec8d40SPaolo Bonziniis ignored and execution continues as if the guest instruction had 111*daec8d40SPaolo Bonzinicompleted. For that reason KVM is not allowed to inject a program 112*daec8d40SPaolo Bonziniinterrupt. 113*daec8d40SPaolo Bonzini 114*daec8d40SPaolo BonziniLinks 115*daec8d40SPaolo Bonzini----- 116*daec8d40SPaolo Bonzini`KVM Forum 2019 presentation <https://static.sched.com/hosted_files/kvmforum2019/3b/ibm_protected_vms_s390x.pdf>`_ 117