1============= 2TEE subsystem 3============= 4 5This document describes the TEE subsystem in Linux. 6 7A TEE (Trusted Execution Environment) is a trusted OS running in some 8secure environment, for example, TrustZone on ARM CPUs, or a separate 9secure co-processor etc. A TEE driver handles the details needed to 10communicate with the TEE. 11 12This subsystem deals with: 13 14- Registration of TEE drivers 15 16- Managing shared memory between Linux and the TEE 17 18- Providing a generic API to the TEE 19 20The TEE interface 21================= 22 23include/uapi/linux/tee.h defines the generic interface to a TEE. 24 25User space (the client) connects to the driver by opening /dev/tee[0-9]* or 26/dev/teepriv[0-9]*. 27 28- TEE_IOC_SHM_ALLOC allocates shared memory and returns a file descriptor 29 which user space can mmap. When user space doesn't need the file 30 descriptor any more, it should be closed. When shared memory isn't needed 31 any longer it should be unmapped with munmap() to allow the reuse of 32 memory. 33 34- TEE_IOC_VERSION lets user space know which TEE this driver handles and 35 its capabilities. 36 37- TEE_IOC_OPEN_SESSION opens a new session to a Trusted Application. 38 39- TEE_IOC_INVOKE invokes a function in a Trusted Application. 40 41- TEE_IOC_CANCEL may cancel an ongoing TEE_IOC_OPEN_SESSION or TEE_IOC_INVOKE. 42 43- TEE_IOC_CLOSE_SESSION closes a session to a Trusted Application. 44 45There are two classes of clients, normal clients and supplicants. The latter is 46a helper process for the TEE to access resources in Linux, for example file 47system access. A normal client opens /dev/tee[0-9]* and a supplicant opens 48/dev/teepriv[0-9]. 49 50Much of the communication between clients and the TEE is opaque to the 51driver. The main job for the driver is to receive requests from the 52clients, forward them to the TEE and send back the results. In the case of 53supplicants the communication goes in the other direction, the TEE sends 54requests to the supplicant which then sends back the result. 55 56The TEE kernel interface 57======================== 58 59Kernel provides a TEE bus infrastructure where a Trusted Application is 60represented as a device identified via Universally Unique Identifier (UUID) and 61client drivers register a table of supported device UUIDs. 62 63TEE bus infrastructure registers following APIs: 64 65match(): 66 iterates over the client driver UUID table to find a corresponding 67 match for device UUID. If a match is found, then this particular device is 68 probed via corresponding probe API registered by the client driver. This 69 process happens whenever a device or a client driver is registered with TEE 70 bus. 71 72uevent(): 73 notifies user-space (udev) whenever a new device is registered on 74 TEE bus for auto-loading of modularized client drivers. 75 76TEE bus device enumeration is specific to underlying TEE implementation, so it 77is left open for TEE drivers to provide corresponding implementation. 78 79Then TEE client driver can talk to a matched Trusted Application using APIs 80listed in include/linux/tee_drv.h. 81 82TEE client driver example 83------------------------- 84 85Suppose a TEE client driver needs to communicate with a Trusted Application 86having UUID: ``ac6a4085-0e82-4c33-bf98-8eb8e118b6c2``, so driver registration 87snippet would look like:: 88 89 static const struct tee_client_device_id client_id_table[] = { 90 {UUID_INIT(0xac6a4085, 0x0e82, 0x4c33, 91 0xbf, 0x98, 0x8e, 0xb8, 0xe1, 0x18, 0xb6, 0xc2)}, 92 {} 93 }; 94 95 MODULE_DEVICE_TABLE(tee, client_id_table); 96 97 static struct tee_client_driver client_driver = { 98 .id_table = client_id_table, 99 .driver = { 100 .name = DRIVER_NAME, 101 .bus = &tee_bus_type, 102 .probe = client_probe, 103 .remove = client_remove, 104 }, 105 }; 106 107 static int __init client_init(void) 108 { 109 return driver_register(&client_driver.driver); 110 } 111 112 static void __exit client_exit(void) 113 { 114 driver_unregister(&client_driver.driver); 115 } 116 117 module_init(client_init); 118 module_exit(client_exit); 119 120OP-TEE driver 121============= 122 123The OP-TEE driver handles OP-TEE [1] based TEEs. Currently it is only the ARM 124TrustZone based OP-TEE solution that is supported. 125 126Lowest level of communication with OP-TEE builds on ARM SMC Calling 127Convention (SMCCC) [2], which is the foundation for OP-TEE's SMC interface 128[3] used internally by the driver. Stacked on top of that is OP-TEE Message 129Protocol [4]. 130 131OP-TEE SMC interface provides the basic functions required by SMCCC and some 132additional functions specific for OP-TEE. The most interesting functions are: 133 134- OPTEE_SMC_FUNCID_CALLS_UID (part of SMCCC) returns the version information 135 which is then returned by TEE_IOC_VERSION 136 137- OPTEE_SMC_CALL_GET_OS_UUID returns the particular OP-TEE implementation, used 138 to tell, for instance, a TrustZone OP-TEE apart from an OP-TEE running on a 139 separate secure co-processor. 140 141- OPTEE_SMC_CALL_WITH_ARG drives the OP-TEE message protocol 142 143- OPTEE_SMC_GET_SHM_CONFIG lets the driver and OP-TEE agree on which memory 144 range to used for shared memory between Linux and OP-TEE. 145 146The GlobalPlatform TEE Client API [5] is implemented on top of the generic 147TEE API. 148 149Picture of the relationship between the different components in the 150OP-TEE architecture:: 151 152 User space Kernel Secure world 153 ~~~~~~~~~~ ~~~~~~ ~~~~~~~~~~~~ 154 +--------+ +-------------+ 155 | Client | | Trusted | 156 +--------+ | Application | 157 /\ +-------------+ 158 || +----------+ /\ 159 || |tee- | || 160 || |supplicant| \/ 161 || +----------+ +-------------+ 162 \/ /\ | TEE Internal| 163 +-------+ || | API | 164 + TEE | || +--------+--------+ +-------------+ 165 | Client| || | TEE | OP-TEE | | OP-TEE | 166 | API | \/ | subsys | driver | | Trusted OS | 167 +-------+----------------+----+-------+----+-----------+-------------+ 168 | Generic TEE API | | OP-TEE MSG | 169 | IOCTL (TEE_IOC_*) | | SMCCC (OPTEE_SMC_CALL_*) | 170 +-----------------------------+ +------------------------------+ 171 172RPC (Remote Procedure Call) are requests from secure world to kernel driver 173or tee-supplicant. An RPC is identified by a special range of SMCCC return 174values from OPTEE_SMC_CALL_WITH_ARG. RPC messages which are intended for the 175kernel are handled by the kernel driver. Other RPC messages will be forwarded to 176tee-supplicant without further involvement of the driver, except switching 177shared memory buffer representation. 178 179OP-TEE device enumeration 180------------------------- 181 182OP-TEE provides a pseudo Trusted Application: drivers/tee/optee/device.c in 183order to support device enumeration. In other words, OP-TEE driver invokes this 184application to retrieve a list of Trusted Applications which can be registered 185as devices on the TEE bus. 186 187OP-TEE notifications 188-------------------- 189 190There are two kinds of notifications that secure world can use to make 191normal world aware of some event. 192 1931. Synchronous notifications delivered with ``OPTEE_RPC_CMD_NOTIFICATION`` 194 using the ``OPTEE_RPC_NOTIFICATION_SEND`` parameter. 1952. Asynchronous notifications delivered with a combination of a non-secure 196 edge-triggered interrupt and a fast call from the non-secure interrupt 197 handler. 198 199Synchronous notifications are limited by depending on RPC for delivery, 200this is only usable when secure world is entered with a yielding call via 201``OPTEE_SMC_CALL_WITH_ARG``. This excludes such notifications from secure 202world interrupt handlers. 203 204An asynchronous notification is delivered via a non-secure edge-triggered 205interrupt to an interrupt handler registered in the OP-TEE driver. The 206actual notification value are retrieved with the fast call 207``OPTEE_SMC_GET_ASYNC_NOTIF_VALUE``. Note that one interrupt can represent 208multiple notifications. 209 210One notification value ``OPTEE_SMC_ASYNC_NOTIF_VALUE_DO_BOTTOM_HALF`` has a 211special meaning. When this value is received it means that normal world is 212supposed to make a yielding call ``OPTEE_MSG_CMD_DO_BOTTOM_HALF``. This 213call is done from the thread assisting the interrupt handler. This is a 214building block for OP-TEE OS in secure world to implement the top half and 215bottom half style of device drivers. 216 217OPTEE_INSECURE_LOAD_IMAGE Kconfig option 218---------------------------------------- 219 220The OPTEE_INSECURE_LOAD_IMAGE Kconfig option enables the ability to load the 221BL32 OP-TEE image from the kernel after the kernel boots, rather than loading 222it from the firmware before the kernel boots. This also requires enabling the 223corresponding option in Trusted Firmware for Arm. The Trusted Firmware for Arm 224documentation [8] explains the security threat associated with enabling this as 225well as mitigations at the firmware and platform level. 226 227There are additional attack vectors/mitigations for the kernel that should be 228addressed when using this option. 229 2301. Boot chain security. 231 232 * Attack vector: Replace the OP-TEE OS image in the rootfs to gain control of 233 the system. 234 235 * Mitigation: There must be boot chain security that verifies the kernel and 236 rootfs, otherwise an attacker can modify the loaded OP-TEE binary by 237 modifying it in the rootfs. 238 2392. Alternate boot modes. 240 241 * Attack vector: Using an alternate boot mode (i.e. recovery mode), the 242 OP-TEE driver isn't loaded, leaving the SMC hole open. 243 244 * Mitigation: If there are alternate methods of booting the device, such as a 245 recovery mode, it should be ensured that the same mitigations are applied 246 in that mode. 247 2483. Attacks prior to SMC invocation. 249 250 * Attack vector: Code that is executed prior to issuing the SMC call to load 251 OP-TEE can be exploited to then load an alternate OS image. 252 253 * Mitigation: The OP-TEE driver must be loaded before any potential attack 254 vectors are opened up. This should include mounting of any modifiable 255 filesystems, opening of network ports or communicating with external 256 devices (e.g. USB). 257 2584. Blocking SMC call to load OP-TEE. 259 260 * Attack vector: Prevent the driver from being probed, so the SMC call to 261 load OP-TEE isn't executed when desired, leaving it open to being executed 262 later and loading a modified OS. 263 264 * Mitigation: It is recommended to build the OP-TEE driver as builtin driver 265 rather than as a module to prevent exploits that may cause the module to 266 not be loaded. 267 268AMD-TEE driver 269============== 270 271The AMD-TEE driver handles the communication with AMD's TEE environment. The 272TEE environment is provided by AMD Secure Processor. 273 274The AMD Secure Processor (formerly called Platform Security Processor or PSP) 275is a dedicated processor that features ARM TrustZone technology, along with a 276software-based Trusted Execution Environment (TEE) designed to enable 277third-party Trusted Applications. This feature is currently enabled only for 278APUs. 279 280The following picture shows a high level overview of AMD-TEE:: 281 282 | 283 x86 | 284 | 285 User space (Kernel space) | AMD Secure Processor (PSP) 286 ~~~~~~~~~~ ~~~~~~~~~~~~~~ | ~~~~~~~~~~~~~~~~~~~~~~~~~~ 287 | 288 +--------+ | +-------------+ 289 | Client | | | Trusted | 290 +--------+ | | Application | 291 /\ | +-------------+ 292 || | /\ 293 || | || 294 || | \/ 295 || | +----------+ 296 || | | TEE | 297 || | | Internal | 298 \/ | | API | 299 +---------+ +-----------+---------+ +----------+ 300 | TEE | | TEE | AMD-TEE | | AMD-TEE | 301 | Client | | subsystem | driver | | Trusted | 302 | API | | | | | OS | 303 +---------+-----------+----+------+---------+---------+----------+ 304 | Generic TEE API | | ASP | Mailbox | 305 | IOCTL (TEE_IOC_*) | | driver | Register Protocol | 306 +--------------------------+ +---------+--------------------+ 307 308At the lowest level (in x86), the AMD Secure Processor (ASP) driver uses the 309CPU to PSP mailbox register to submit commands to the PSP. The format of the 310command buffer is opaque to the ASP driver. It's role is to submit commands to 311the secure processor and return results to AMD-TEE driver. The interface 312between AMD-TEE driver and AMD Secure Processor driver can be found in [6]. 313 314The AMD-TEE driver packages the command buffer payload for processing in TEE. 315The command buffer format for the different TEE commands can be found in [7]. 316 317The TEE commands supported by AMD-TEE Trusted OS are: 318 319* TEE_CMD_ID_LOAD_TA - loads a Trusted Application (TA) binary into 320 TEE environment. 321* TEE_CMD_ID_UNLOAD_TA - unloads TA binary from TEE environment. 322* TEE_CMD_ID_OPEN_SESSION - opens a session with a loaded TA. 323* TEE_CMD_ID_CLOSE_SESSION - closes session with loaded TA 324* TEE_CMD_ID_INVOKE_CMD - invokes a command with loaded TA 325* TEE_CMD_ID_MAP_SHARED_MEM - maps shared memory 326* TEE_CMD_ID_UNMAP_SHARED_MEM - unmaps shared memory 327 328AMD-TEE Trusted OS is the firmware running on AMD Secure Processor. 329 330The AMD-TEE driver registers itself with TEE subsystem and implements the 331following driver function callbacks: 332 333* get_version - returns the driver implementation id and capability. 334* open - sets up the driver context data structure. 335* release - frees up driver resources. 336* open_session - loads the TA binary and opens session with loaded TA. 337* close_session - closes session with loaded TA and unloads it. 338* invoke_func - invokes a command with loaded TA. 339 340cancel_req driver callback is not supported by AMD-TEE. 341 342The GlobalPlatform TEE Client API [5] can be used by the user space (client) to 343talk to AMD's TEE. AMD's TEE provides a secure environment for loading, opening 344a session, invoking commands and closing session with TA. 345 346References 347========== 348 349[1] https://github.com/OP-TEE/optee_os 350 351[2] http://infocenter.arm.com/help/topic/com.arm.doc.den0028a/index.html 352 353[3] drivers/tee/optee/optee_smc.h 354 355[4] drivers/tee/optee/optee_msg.h 356 357[5] http://www.globalplatform.org/specificationsdevice.asp look for 358 "TEE Client API Specification v1.0" and click download. 359 360[6] include/linux/psp-tee.h 361 362[7] drivers/tee/amdtee/amdtee_if.h 363 364[8] https://trustedfirmware-a.readthedocs.io/en/latest/threat_model/threat_model.html 365