1============== 2The memory API 3============== 4 5The memory API models the memory and I/O buses and controllers of a QEMU 6machine. It attempts to allow modelling of: 7 8- ordinary RAM 9- memory-mapped I/O (MMIO) 10- memory controllers that can dynamically reroute physical memory regions 11 to different destinations 12 13The memory model provides support for 14 15- tracking RAM changes by the guest 16- setting up coalesced memory for kvm 17- setting up ioeventfd regions for kvm 18 19Memory is modelled as an acyclic graph of MemoryRegion objects. Sinks 20(leaves) are RAM and MMIO regions, while other nodes represent 21buses, memory controllers, and memory regions that have been rerouted. 22 23In addition to MemoryRegion objects, the memory API provides AddressSpace 24objects for every root and possibly for intermediate MemoryRegions too. 25These represent memory as seen from the CPU or a device's viewpoint. 26 27Types of regions 28---------------- 29 30There are multiple types of memory regions (all represented by a single C type 31MemoryRegion): 32 33- RAM: a RAM region is simply a range of host memory that can be made available 34 to the guest. 35 You typically initialize these with memory_region_init_ram(). Some special 36 purposes require the variants memory_region_init_resizeable_ram(), 37 memory_region_init_ram_from_file(), or memory_region_init_ram_ptr(). 38 39- MMIO: a range of guest memory that is implemented by host callbacks; 40 each read or write causes a callback to be called on the host. 41 You initialize these with memory_region_init_io(), passing it a 42 MemoryRegionOps structure describing the callbacks. 43 44- ROM: a ROM memory region works like RAM for reads (directly accessing 45 a region of host memory), and forbids writes. You initialize these with 46 memory_region_init_rom(). 47 48- ROM device: a ROM device memory region works like RAM for reads 49 (directly accessing a region of host memory), but like MMIO for 50 writes (invoking a callback). You initialize these with 51 memory_region_init_rom_device(). 52 53- IOMMU region: an IOMMU region translates addresses of accesses made to it 54 and forwards them to some other target memory region. As the name suggests, 55 these are only needed for modelling an IOMMU, not for simple devices. 56 You initialize these with memory_region_init_iommu(). 57 58- container: a container simply includes other memory regions, each at 59 a different offset. Containers are useful for grouping several regions 60 into one unit. For example, a PCI BAR may be composed of a RAM region 61 and an MMIO region. 62 63 A container's subregions are usually non-overlapping. In some cases it is 64 useful to have overlapping regions; for example a memory controller that 65 can overlay a subregion of RAM with MMIO or ROM, or a PCI controller 66 that does not prevent card from claiming overlapping BARs. 67 68 You initialize a pure container with memory_region_init(). 69 70- alias: a subsection of another region. Aliases allow a region to be 71 split apart into discontiguous regions. Examples of uses are memory 72 banks used when the guest address space is smaller than the amount 73 of RAM addressed, or a memory controller that splits main memory to 74 expose a "PCI hole". You can also create aliases to avoid trying to 75 add the original region to multiple parents via 76 `memory_region_add_subregion`. 77 78 Aliases may point to any type of region, including other aliases, 79 but an alias may not point back to itself, directly or indirectly. 80 You initialize these with memory_region_init_alias(). 81 82- reservation region: a reservation region is primarily for debugging. 83 It claims I/O space that is not supposed to be handled by QEMU itself. 84 The typical use is to track parts of the address space which will be 85 handled by the host kernel when KVM is enabled. You initialize these 86 by passing a NULL callback parameter to memory_region_init_io(). 87 88It is valid to add subregions to a region which is not a pure container 89(that is, to an MMIO, RAM or ROM region). This means that the region 90will act like a container, except that any addresses within the container's 91region which are not claimed by any subregion are handled by the 92container itself (ie by its MMIO callbacks or RAM backing). However 93it is generally possible to achieve the same effect with a pure container 94one of whose subregions is a low priority "background" region covering 95the whole address range; this is often clearer and is preferred. 96Subregions cannot be added to an alias region. 97 98Migration 99--------- 100 101Where the memory region is backed by host memory (RAM, ROM and 102ROM device memory region types), this host memory needs to be 103copied to the destination on migration. These APIs which allocate 104the host memory for you will also register the memory so it is 105migrated: 106 107- memory_region_init_ram() 108- memory_region_init_rom() 109- memory_region_init_rom_device() 110 111For most devices and boards this is the correct thing. If you 112have a special case where you need to manage the migration of 113the backing memory yourself, you can call the functions: 114 115- memory_region_init_ram_nomigrate() 116- memory_region_init_rom_nomigrate() 117- memory_region_init_rom_device_nomigrate() 118 119which only initialize the MemoryRegion and leave handling 120migration to the caller. 121 122The functions: 123 124- memory_region_init_resizeable_ram() 125- memory_region_init_ram_from_file() 126- memory_region_init_ram_from_fd() 127- memory_region_init_ram_ptr() 128- memory_region_init_ram_device_ptr() 129 130are for special cases only, and so they do not automatically 131register the backing memory for migration; the caller must 132manage migration if necessary. 133 134Region names 135------------ 136 137Regions are assigned names by the constructor. For most regions these are 138only used for debugging purposes, but RAM regions also use the name to identify 139live migration sections. This means that RAM region names need to have ABI 140stability. 141 142Region lifecycle 143---------------- 144 145A region is created by one of the memory_region_init*() functions and 146attached to an object, which acts as its owner or parent. QEMU ensures 147that the owner object remains alive as long as the region is visible to 148the guest, or as long as the region is in use by a virtual CPU or another 149device. For example, the owner object will not die between an 150address_space_map operation and the corresponding address_space_unmap. 151 152After creation, a region can be added to an address space or a 153container with memory_region_add_subregion(), and removed using 154memory_region_del_subregion(). 155 156Various region attributes (read-only, dirty logging, coalesced mmio, 157ioeventfd) can be changed during the region lifecycle. They take effect 158as soon as the region is made visible. This can be immediately, later, 159or never. 160 161Destruction of a memory region happens automatically when the owner 162object dies. 163 164If however the memory region is part of a dynamically allocated data 165structure, you should call object_unparent() to destroy the memory region 166before the data structure is freed. For an example see VFIOMSIXInfo 167and VFIOQuirk in hw/vfio/pci.c. 168 169You must not destroy a memory region as long as it may be in use by a 170device or CPU. In order to do this, as a general rule do not create or 171destroy memory regions dynamically during a device's lifetime, and only 172call object_unparent() in the memory region owner's instance_finalize 173callback. The dynamically allocated data structure that contains the 174memory region then should obviously be freed in the instance_finalize 175callback as well. 176 177If you break this rule, the following situation can happen: 178 179- the memory region's owner had a reference taken via memory_region_ref 180 (for example by address_space_map) 181 182- the region is unparented, and has no owner anymore 183 184- when address_space_unmap is called, the reference to the memory region's 185 owner is leaked. 186 187 188There is an exception to the above rule: it is okay to call 189object_unparent at any time for an alias or a container region. It is 190therefore also okay to create or destroy alias and container regions 191dynamically during a device's lifetime. 192 193This exceptional usage is valid because aliases and containers only help 194QEMU building the guest's memory map; they are never accessed directly. 195memory_region_ref and memory_region_unref are never called on aliases 196or containers, and the above situation then cannot happen. Exploiting 197this exception is rarely necessary, and therefore it is discouraged, 198but nevertheless it is used in a few places. 199 200For regions that "have no owner" (NULL is passed at creation time), the 201machine object is actually used as the owner. Since instance_finalize is 202never called for the machine object, you must never call object_unparent 203on regions that have no owner, unless they are aliases or containers. 204 205 206Overlapping regions and priority 207-------------------------------- 208Usually, regions may not overlap each other; a memory address decodes into 209exactly one target. In some cases it is useful to allow regions to overlap, 210and sometimes to control which of an overlapping regions is visible to the 211guest. This is done with memory_region_add_subregion_overlap(), which 212allows the region to overlap any other region in the same container, and 213specifies a priority that allows the core to decide which of two regions at 214the same address are visible (highest wins). 215Priority values are signed, and the default value is zero. This means that 216you can use memory_region_add_subregion_overlap() both to specify a region 217that must sit 'above' any others (with a positive priority) and also a 218background region that sits 'below' others (with a negative priority). 219 220If the higher priority region in an overlap is a container or alias, then 221the lower priority region will appear in any "holes" that the higher priority 222region has left by not mapping subregions to that area of its address range. 223(This applies recursively -- if the subregions are themselves containers or 224aliases that leave holes then the lower priority region will appear in these 225holes too.) 226 227For example, suppose we have a container A of size 0x8000 with two subregions 228B and C. B is a container mapped at 0x2000, size 0x4000, priority 2; C is 229an MMIO region mapped at 0x0, size 0x6000, priority 1. B currently has two 230of its own subregions: D of size 0x1000 at offset 0 and E of size 0x1000 at 231offset 0x2000. As a diagram:: 232 233 0 1000 2000 3000 4000 5000 6000 7000 8000 234 |------|------|------|------|------|------|------|------| 235 A: [ ] 236 C: [CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC] 237 B: [ ] 238 D: [DDDDD] 239 E: [EEEEE] 240 241The regions that will be seen within this address range then are:: 242 243 [CCCCCCCCCCCC][DDDDD][CCCCC][EEEEE][CCCCC] 244 245Since B has higher priority than C, its subregions appear in the flat map 246even where they overlap with C. In ranges where B has not mapped anything 247C's region appears. 248 249If B had provided its own MMIO operations (ie it was not a pure container) 250then these would be used for any addresses in its range not handled by 251D or E, and the result would be:: 252 253 [CCCCCCCCCCCC][DDDDD][BBBBB][EEEEE][BBBBB] 254 255Priority values are local to a container, because the priorities of two 256regions are only compared when they are both children of the same container. 257This means that the device in charge of the container (typically modelling 258a bus or a memory controller) can use them to manage the interaction of 259its child regions without any side effects on other parts of the system. 260In the example above, the priorities of D and E are unimportant because 261they do not overlap each other. It is the relative priority of B and C 262that causes D and E to appear on top of C: D and E's priorities are never 263compared against the priority of C. 264 265Visibility 266---------- 267The memory core uses the following rules to select a memory region when the 268guest accesses an address: 269 270- all direct subregions of the root region are matched against the address, in 271 descending priority order 272 273 - if the address lies outside the region offset/size, the subregion is 274 discarded 275 - if the subregion is a leaf (RAM or MMIO), the search terminates, returning 276 this leaf region 277 - if the subregion is a container, the same algorithm is used within the 278 subregion (after the address is adjusted by the subregion offset) 279 - if the subregion is an alias, the search is continued at the alias target 280 (after the address is adjusted by the subregion offset and alias offset) 281 - if a recursive search within a container or alias subregion does not 282 find a match (because of a "hole" in the container's coverage of its 283 address range), then if this is a container with its own MMIO or RAM 284 backing the search terminates, returning the container itself. Otherwise 285 we continue with the next subregion in priority order 286 287- if none of the subregions match the address then the search terminates 288 with no match found 289 290Example memory map 291------------------ 292 293:: 294 295 system_memory: container@0-2^48-1 296 | 297 +---- lomem: alias@0-0xdfffffff ---> #ram (0-0xdfffffff) 298 | 299 +---- himem: alias@0x100000000-0x11fffffff ---> #ram (0xe0000000-0xffffffff) 300 | 301 +---- vga-window: alias@0xa0000-0xbffff ---> #pci (0xa0000-0xbffff) 302 | (prio 1) 303 | 304 +---- pci-hole: alias@0xe0000000-0xffffffff ---> #pci (0xe0000000-0xffffffff) 305 306 pci (0-2^32-1) 307 | 308 +--- vga-area: container@0xa0000-0xbffff 309 | | 310 | +--- alias@0x00000-0x7fff ---> #vram (0x010000-0x017fff) 311 | | 312 | +--- alias@0x08000-0xffff ---> #vram (0x020000-0x027fff) 313 | 314 +---- vram: ram@0xe1000000-0xe1ffffff 315 | 316 +---- vga-mmio: mmio@0xe2000000-0xe200ffff 317 318 ram: ram@0x00000000-0xffffffff 319 320This is a (simplified) PC memory map. The 4GB RAM block is mapped into the 321system address space via two aliases: "lomem" is a 1:1 mapping of the first 3223.5GB; "himem" maps the last 0.5GB at address 4GB. This leaves 0.5GB for the 323so-called PCI hole, that allows a 32-bit PCI bus to exist in a system with 3244GB of memory. 325 326The memory controller diverts addresses in the range 640K-768K to the PCI 327address space. This is modelled using the "vga-window" alias, mapped at a 328higher priority so it obscures the RAM at the same addresses. The vga window 329can be removed by programming the memory controller; this is modelled by 330removing the alias and exposing the RAM underneath. 331 332The pci address space is not a direct child of the system address space, since 333we only want parts of it to be visible (we accomplish this using aliases). 334It has two subregions: vga-area models the legacy vga window and is occupied 335by two 32K memory banks pointing at two sections of the framebuffer. 336In addition the vram is mapped as a BAR at address e1000000, and an additional 337BAR containing MMIO registers is mapped after it. 338 339Note that if the guest maps a BAR outside the PCI hole, it would not be 340visible as the pci-hole alias clips it to a 0.5GB range. 341 342MMIO Operations 343--------------- 344 345MMIO regions are provided with ->read() and ->write() callbacks, 346which are sufficient for most devices. Some devices change behaviour 347based on the attributes used for the memory transaction, or need 348to be able to respond that the access should provoke a bus error 349rather than completing successfully; those devices can use the 350->read_with_attrs() and ->write_with_attrs() callbacks instead. 351 352In addition various constraints can be supplied to control how these 353callbacks are called: 354 355- .valid.min_access_size, .valid.max_access_size define the access sizes 356 (in bytes) which the device accepts; accesses outside this range will 357 have device and bus specific behaviour (ignored, or machine check) 358- .valid.unaligned specifies that the *device being modelled* supports 359 unaligned accesses; if false, unaligned accesses will invoke the 360 appropriate bus or CPU specific behaviour. 361- .impl.min_access_size, .impl.max_access_size define the access sizes 362 (in bytes) supported by the *implementation*; other access sizes will be 363 emulated using the ones available. For example a 4-byte write will be 364 emulated using four 1-byte writes, if .impl.max_access_size = 1. 365- .impl.unaligned specifies that the *implementation* supports unaligned 366 accesses; if false, unaligned accesses will be emulated by two aligned 367 accesses. 368 369API Reference 370------------- 371 372.. kernel-doc:: include/exec/memory.h 373