1'nitro-enclave' virtual machine (``nitro-enclave``)
2===================================================
3
4``nitro-enclave`` is a machine type which emulates an *AWS nitro enclave*
5virtual machine. `AWS nitro enclaves`_ is an Amazon EC2 feature that allows
6creating isolated execution environments, called enclaves, from Amazon EC2
7instances which are used for processing highly sensitive data. Enclaves have
8no persistent storage and no external networking. The enclave VMs are based
9on Firecracker microvm with a vhost-vsock device for communication with the
10parent EC2 instance that spawned it and a Nitro Secure Module (NSM) device
11for cryptographic attestation. The parent instance VM always has CID 3 while
12the enclave VM gets a dynamic CID. Enclaves use an EIF (`Enclave Image Format`_)
13file which contains the necessary kernel, cmdline and ramdisk(s) to boot.
14
15In QEMU, ``nitro-enclave`` is a machine type based on ``microvm`` similar to how
16AWS nitro enclaves are based on `Firecracker`_ microvm. This is useful for
17local testing of EIF files using QEMU instead of running real AWS Nitro Enclaves
18which can be difficult for debugging due to its roots in security. The vsock
19device emulation is done using vhost-user-vsock which means another process that
20can do the userspace emulation, like `vhost-device-vsock`_ from rust-vmm crate,
21must be run alongside nitro-enclave for the vsock communication to work.
22
23``libcbor`` and ``gnutls`` are required dependencies for nitro-enclave machine
24support to be added when building QEMU from source.
25
26.. _AWS nitro enclaves: https://docs.aws.amazon.com/enclaves/latest/user/nitro-enclave.html
27.. _Enclave Image Format: https://github.com/aws/aws-nitro-enclaves-image-format
28.. _vhost-device-vsock: https://github.com/rust-vmm/vhost-device/tree/main/vhost-device-vsock
29.. _Firecracker: https://firecracker-microvm.github.io
30
31Using the nitro-enclave machine type
32------------------------------------
33
34Machine-specific options
35~~~~~~~~~~~~~~~~~~~~~~~~
36
37It supports the following machine-specific options:
38
39- nitro-enclave.vsock=string (required) (Id of the chardev from '-chardev' option that vhost-user-vsock device will use)
40- nitro-enclave.id=string (optional) (Set enclave identifier)
41- nitro-enclave.parent-role=string (optional) (Set parent instance IAM role ARN)
42- nitro-enclave.parent-id=string (optional) (Set parent instance identifier)
43
44
45Running a nitro-enclave VM
46~~~~~~~~~~~~~~~~~~~~~~~~~~
47
48First, run `vhost-device-vsock`__ (or a similar tool that supports vhost-user-vsock).
49The forward-cid option below with value 1 forwards all connections from the enclave
50VM to the host machine and the forward-listen (port numbers separated by '+') is used
51for forwarding connections from the host machine to the enclave VM.
52
53__ https://github.com/rust-vmm/vhost-device/tree/main/vhost-device-vsock#using-the-vsock-backend
54
55  $ vhost-device-vsock \
56     --vm guest-cid=4,forward-cid=1,forward-listen=9001+9002,socket=/tmp/vhost4.socket
57
58Now run the necessary applications on the host machine so that the nitro-enclave VM
59applications' vsock communication works. For example, the nitro-enclave VM's init
60process connects to CID 3 and sends a single byte hello heartbeat (0xB7) to let the
61parent VM know that it booted expecting a heartbeat (0xB7) response. So you must run
62a AF_VSOCK server on the host machine that listens on port 9000 and sends the heartbeat
63after it receives the heartbeat for enclave VM to boot successfully. You should run all
64the applications on the host machine that would typically be running in the parent EC2
65VM for successful communication with the enclave VM.
66
67Then run the nitro-enclave VM using the following command where ``hello.eif`` is
68an EIF file you would use to spawn a real AWS nitro enclave virtual machine:
69
70  $ qemu-system-x86_64 -M nitro-enclave,vsock=c,id=hello-world \
71     -kernel hello-world.eif -nographic -m 4G --enable-kvm -cpu host \
72     -chardev socket,id=c,path=/tmp/vhost4.socket
73
74In this example, the nitro-enclave VM has CID 4. If there are applications that
75connect to the enclave VM, run them on the host machine after enclave VM starts.
76You need to modify the applications to connect to CID 1 (instead of the enclave
77VM's CID) and use the forward-listen (e.g., 9001+9002) option of vhost-device-vsock
78to forward the ports they connect to.
79