1*dc142431SXiaoyao LiIntel Trusted Domain eXtension (TDX) 2*dc142431SXiaoyao Li==================================== 3*dc142431SXiaoyao Li 4*dc142431SXiaoyao LiIntel Trusted Domain eXtensions (TDX) refers to an Intel technology that extends 5*dc142431SXiaoyao LiVirtual Machine Extensions (VMX) and Multi-Key Total Memory Encryption (MKTME) 6*dc142431SXiaoyao Liwith a new kind of virtual machine guest called a Trust Domain (TD). A TD runs 7*dc142431SXiaoyao Liin a CPU mode that is designed to protect the confidentiality of its memory 8*dc142431SXiaoyao Licontents and its CPU state from any other software, including the hosting 9*dc142431SXiaoyao LiVirtual Machine Monitor (VMM), unless explicitly shared by the TD itself. 10*dc142431SXiaoyao Li 11*dc142431SXiaoyao LiPrerequisites 12*dc142431SXiaoyao Li------------- 13*dc142431SXiaoyao Li 14*dc142431SXiaoyao LiTo run TD, the physical machine needs to have TDX module loaded and initialized 15*dc142431SXiaoyao Liwhile KVM hypervisor has TDX support and has TDX enabled. If those requirements 16*dc142431SXiaoyao Liare met, the ``KVM_CAP_VM_TYPES`` will report the support of ``KVM_X86_TDX_VM``. 17*dc142431SXiaoyao Li 18*dc142431SXiaoyao LiTrust Domain Virtual Firmware (TDVF) 19*dc142431SXiaoyao Li~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 20*dc142431SXiaoyao Li 21*dc142431SXiaoyao LiTrust Domain Virtual Firmware (TDVF) is required to provide TD services to boot 22*dc142431SXiaoyao LiTD Guest OS. TDVF needs to be copied to guest private memory and measured before 23*dc142431SXiaoyao Lithe TD boots. 24*dc142431SXiaoyao Li 25*dc142431SXiaoyao LiKVM vcpu ioctl ``KVM_TDX_INIT_MEM_REGION`` can be used to populate the TDVF 26*dc142431SXiaoyao Licontent into its private memory. 27*dc142431SXiaoyao Li 28*dc142431SXiaoyao LiSince TDX doesn't support readonly memslot, TDVF cannot be mapped as pflash 29*dc142431SXiaoyao Lidevice and it actually works as RAM. "-bios" option is chosen to load TDVF. 30*dc142431SXiaoyao Li 31*dc142431SXiaoyao LiOVMF is the opensource firmware that implements the TDVF support. Thus the 32*dc142431SXiaoyao Licommand line to specify and load TDVF is ``-bios OVMF.fd`` 33*dc142431SXiaoyao Li 34*dc142431SXiaoyao LiFeature Configuration 35*dc142431SXiaoyao Li--------------------- 36*dc142431SXiaoyao Li 37*dc142431SXiaoyao LiUnlike non-TDX VM, the CPU features (enumerated by CPU or MSR) of a TD are not 38*dc142431SXiaoyao Liunder full control of VMM. VMM can only configure part of features of a TD on 39*dc142431SXiaoyao Li``KVM_TDX_INIT_VM`` command of VM scope ``MEMORY_ENCRYPT_OP`` ioctl. 40*dc142431SXiaoyao Li 41*dc142431SXiaoyao LiThe configurable features have three types: 42*dc142431SXiaoyao Li 43*dc142431SXiaoyao Li- Attributes: 44*dc142431SXiaoyao Li - PKS (bit 30) controls whether Supervisor Protection Keys is exposed to TD, 45*dc142431SXiaoyao Li which determines related CPUID bit and CR4 bit; 46*dc142431SXiaoyao Li - PERFMON (bit 63) controls whether PMU is exposed to TD. 47*dc142431SXiaoyao Li 48*dc142431SXiaoyao Li- XSAVE related features (XFAM): 49*dc142431SXiaoyao Li XFAM is a 64b mask, which has the same format as XCR0 or IA32_XSS MSR. It 50*dc142431SXiaoyao Li determines the set of extended features available for use by the guest TD. 51*dc142431SXiaoyao Li 52*dc142431SXiaoyao Li- CPUID features: 53*dc142431SXiaoyao Li Only some bits of some CPUID leaves are directly configurable by VMM. 54*dc142431SXiaoyao Li 55*dc142431SXiaoyao LiWhat features can be configured is reported via TDX capabilities. 56*dc142431SXiaoyao Li 57*dc142431SXiaoyao LiTDX capabilities 58*dc142431SXiaoyao Li~~~~~~~~~~~~~~~~ 59*dc142431SXiaoyao Li 60*dc142431SXiaoyao LiThe VM scope ``MEMORY_ENCRYPT_OP`` ioctl provides command ``KVM_TDX_CAPABILITIES`` 61*dc142431SXiaoyao Lito get the TDX capabilities from KVM. It returns a data structure of 62*dc142431SXiaoyao Li``struct kvm_tdx_capabilities``, which tells the supported configuration of 63*dc142431SXiaoyao Liattributes, XFAM and CPUIDs. 64*dc142431SXiaoyao Li 65*dc142431SXiaoyao LiTD attributes 66*dc142431SXiaoyao Li~~~~~~~~~~~~~ 67*dc142431SXiaoyao Li 68*dc142431SXiaoyao LiQEMU supports configuring raw 64-bit TD attributes directly via "attributes" 69*dc142431SXiaoyao Liproperty of "tdx-guest" object. Note, it's users' responsibility to provide a 70*dc142431SXiaoyao Livalid value because some bits may not supported by current QEMU or KVM yet. 71*dc142431SXiaoyao Li 72*dc142431SXiaoyao LiQEMU also supports the configuration of individual attribute bits that are 73*dc142431SXiaoyao Lisupported by it, via properties of "tdx-guest" object. 74*dc142431SXiaoyao LiE.g., "sept-ve-disable" (bit 28). 75*dc142431SXiaoyao Li 76*dc142431SXiaoyao LiMSR based features 77*dc142431SXiaoyao Li~~~~~~~~~~~~~~~~~~ 78*dc142431SXiaoyao Li 79*dc142431SXiaoyao LiCurrent KVM doesn't support MSR based feature (e.g., MSR_IA32_ARCH_CAPABILITIES) 80*dc142431SXiaoyao Liconfiguration for TDX, and it's a future work to enable it in QEMU when KVM adds 81*dc142431SXiaoyao Lisupport of it. 82*dc142431SXiaoyao Li 83*dc142431SXiaoyao LiFeature check 84*dc142431SXiaoyao Li~~~~~~~~~~~~~ 85*dc142431SXiaoyao Li 86*dc142431SXiaoyao LiQEMU checks if the final (CPU) features, determined by given cpu model and 87*dc142431SXiaoyao Liexplicit feature adjustment of "+featureA/-featureB", can be supported or not. 88*dc142431SXiaoyao LiIt can produce feature not supported warning like 89*dc142431SXiaoyao Li 90*dc142431SXiaoyao Li "warning: host doesn't support requested feature: CPUID.07H:EBX.intel-pt [bit 25]" 91*dc142431SXiaoyao Li 92*dc142431SXiaoyao LiIt can also produce warning like 93*dc142431SXiaoyao Li 94*dc142431SXiaoyao Li "warning: TDX forcibly sets the feature: CPUID.80000007H:EDX.invtsc [bit 8]" 95*dc142431SXiaoyao Li 96*dc142431SXiaoyao Liif the fixed-1 feature is requested to be disabled explicitly. This is newly 97*dc142431SXiaoyao Liadded to QEMU for TDX because TDX has fixed-1 features that are forcibly enabled 98*dc142431SXiaoyao Liby TDX module and VMM cannot disable them. 99*dc142431SXiaoyao Li 100*dc142431SXiaoyao LiLaunching a TD (TDX VM) 101*dc142431SXiaoyao Li----------------------- 102*dc142431SXiaoyao Li 103*dc142431SXiaoyao LiTo launch a TD, the necessary command line options are tdx-guest object and 104*dc142431SXiaoyao Lisplit kernel-irqchip, as below: 105*dc142431SXiaoyao Li 106*dc142431SXiaoyao Li.. parsed-literal:: 107*dc142431SXiaoyao Li 108*dc142431SXiaoyao Li |qemu_system_x86| \\ 109*dc142431SXiaoyao Li -accel kvm \\ 110*dc142431SXiaoyao Li -cpu host \\ 111*dc142431SXiaoyao Li -object tdx-guest,id=tdx0 \\ 112*dc142431SXiaoyao Li -machine ...,confidential-guest-support=tdx0 \\ 113*dc142431SXiaoyao Li -bios OVMF.fd \\ 114*dc142431SXiaoyao Li 115*dc142431SXiaoyao LiRestrictions 116*dc142431SXiaoyao Li------------ 117*dc142431SXiaoyao Li 118*dc142431SXiaoyao Li - kernel-irqchip must be split; 119*dc142431SXiaoyao Li 120*dc142431SXiaoyao Li This is set by default for TDX guest if kernel-irqchip is left on its default 121*dc142431SXiaoyao Li 'auto' setting. 122*dc142431SXiaoyao Li 123*dc142431SXiaoyao Li - No readonly support for private memory; 124*dc142431SXiaoyao Li 125*dc142431SXiaoyao Li - No SMM support: SMM support requires manipulating the guest register states 126*dc142431SXiaoyao Li which is not allowed; 127*dc142431SXiaoyao Li 128*dc142431SXiaoyao LiDebugging 129*dc142431SXiaoyao Li--------- 130*dc142431SXiaoyao Li 131*dc142431SXiaoyao LiBit 0 of TD attributes, is DEBUG bit, which decides if the TD runs in off-TD 132*dc142431SXiaoyao Lidebug mode. When in off-TD debug mode, TD's VCPU state and private memory are 133*dc142431SXiaoyao Liaccessible via given SEAMCALLs. This requires KVM to expose APIs to invoke those 134*dc142431SXiaoyao LiSEAMCALLs and corresonponding QEMU change. 135*dc142431SXiaoyao Li 136*dc142431SXiaoyao LiIt's targeted as future work. 137*dc142431SXiaoyao Li 138*dc142431SXiaoyao LiTD attestation 139*dc142431SXiaoyao Li-------------- 140*dc142431SXiaoyao Li 141*dc142431SXiaoyao LiIn TD guest, the attestation process is used to verify the TDX guest 142*dc142431SXiaoyao Litrustworthiness to other entities before provisioning secrets to the guest. 143*dc142431SXiaoyao Li 144*dc142431SXiaoyao LiTD attestation is initiated first by calling TDG.MR.REPORT inside TD to get the 145*dc142431SXiaoyao LiREPORT. Then the REPORT data needs to be converted into a remotely verifiable 146*dc142431SXiaoyao LiQuote by SGX Quoting Enclave (QE). 147*dc142431SXiaoyao Li 148*dc142431SXiaoyao LiIt's a future work in QEMU to add support of TD attestation since it lacks 149*dc142431SXiaoyao Lisupport in current KVM. 150*dc142431SXiaoyao Li 151*dc142431SXiaoyao LiLive Migration 152*dc142431SXiaoyao Li-------------- 153*dc142431SXiaoyao Li 154*dc142431SXiaoyao LiFuture work. 155*dc142431SXiaoyao Li 156*dc142431SXiaoyao LiReferences 157*dc142431SXiaoyao Li---------- 158*dc142431SXiaoyao Li 159*dc142431SXiaoyao Li- `TDX Homepage <https://www.intel.com/content/www/us/en/developer/articles/technical/intel-trust-domain-extensions.html>`__ 160*dc142431SXiaoyao Li 161*dc142431SXiaoyao Li- `SGX QE <https://github.com/intel/SGXDataCenterAttestationPrimitives/tree/master/QuoteGeneration>`__ 162