1==================== 2High Memory Handling 3==================== 4 5By: Peter Zijlstra <a.p.zijlstra@chello.nl> 6 7.. contents:: :local: 8 9What Is High Memory? 10==================== 11 12High memory (highmem) is used when the size of physical memory approaches or 13exceeds the maximum size of virtual memory. At that point it becomes 14impossible for the kernel to keep all of the available physical memory mapped 15at all times. This means the kernel needs to start using temporary mappings of 16the pieces of physical memory that it wants to access. 17 18The part of (physical) memory not covered by a permanent mapping is what we 19refer to as 'highmem'. There are various architecture dependent constraints on 20where exactly that border lies. 21 22In the i386 arch, for example, we choose to map the kernel into every process's 23VM space so that we don't have to pay the full TLB invalidation costs for 24kernel entry/exit. This means the available virtual memory space (4GiB on 25i386) has to be divided between user and kernel space. 26 27The traditional split for architectures using this approach is 3:1, 3GiB for 28userspace and the top 1GiB for kernel space:: 29 30 +--------+ 0xffffffff 31 | Kernel | 32 +--------+ 0xc0000000 33 | | 34 | User | 35 | | 36 +--------+ 0x00000000 37 38This means that the kernel can at most map 1GiB of physical memory at any one 39time, but because we need virtual address space for other things - including 40temporary maps to access the rest of the physical memory - the actual direct 41map will typically be less (usually around ~896MiB). 42 43Other architectures that have mm context tagged TLBs can have separate kernel 44and user maps. Some hardware (like some ARMs), however, have limited virtual 45space when they use mm context tags. 46 47 48Temporary Virtual Mappings 49========================== 50 51The kernel contains several ways of creating temporary mappings. The following 52list shows them in order of preference of use. 53 54* kmap_local_page(). This function is used to require short term mappings. 55 It can be invoked from any context (including interrupts) but the mappings 56 can only be used in the context which acquired them. 57 58 This function should be preferred, where feasible, over all the others. 59 60 These mappings are thread-local and CPU-local, meaning that the mapping 61 can only be accessed from within this thread and the thread is bound to the 62 CPU while the mapping is active. Although preemption is never disabled by 63 this function, the CPU can not be unplugged from the system via 64 CPU-hotplug until the mapping is disposed. 65 66 It's valid to take pagefaults in a local kmap region, unless the context 67 in which the local mapping is acquired does not allow it for other reasons. 68 69 As said, pagefaults and preemption are never disabled. There is no need to 70 disable preemption because, when context switches to a different task, the 71 maps of the outgoing task are saved and those of the incoming one are 72 restored. 73 74 kmap_local_page() always returns a valid virtual address and it is assumed 75 that kunmap_local() will never fail. 76 77 On CONFIG_HIGHMEM=n kernels and for low memory pages this returns the 78 virtual address of the direct mapping. Only real highmem pages are 79 temporarily mapped. Therefore, users may call a plain page_address() 80 for pages which are known to not come from ZONE_HIGHMEM. However, it is 81 always safe to use kmap_local_page() / kunmap_local(). 82 83 While it is significantly faster than kmap(), for the higmem case it 84 comes with restrictions about the pointers validity. Contrary to kmap() 85 mappings, the local mappings are only valid in the context of the caller 86 and cannot be handed to other contexts. This implies that users must 87 be absolutely sure to keep the use of the return address local to the 88 thread which mapped it. 89 90 Most code can be designed to use thread local mappings. User should 91 therefore try to design their code to avoid the use of kmap() by mapping 92 pages in the same thread the address will be used and prefer 93 kmap_local_page(). 94 95 Nesting kmap_local_page() and kmap_atomic() mappings is allowed to a certain 96 extent (up to KMAP_TYPE_NR) but their invocations have to be strictly ordered 97 because the map implementation is stack based. See kmap_local_page() kdocs 98 (included in the "Functions" section) for details on how to manage nested 99 mappings. 100 101* kmap_atomic(). This permits a very short duration mapping of a single 102 page. Since the mapping is restricted to the CPU that issued it, it 103 performs well, but the issuing task is therefore required to stay on that 104 CPU until it has finished, lest some other task displace its mappings. 105 106 kmap_atomic() may also be used by interrupt contexts, since it does not 107 sleep and the callers too may not sleep until after kunmap_atomic() is 108 called. 109 110 Each call of kmap_atomic() in the kernel creates a non-preemptible section 111 and disable pagefaults. This could be a source of unwanted latency. Therefore 112 users should prefer kmap_local_page() instead of kmap_atomic(). 113 114 It is assumed that k[un]map_atomic() won't fail. 115 116* kmap(). This should be used to make short duration mapping of a single 117 page with no restrictions on preemption or migration. It comes with an 118 overhead as mapping space is restricted and protected by a global lock 119 for synchronization. When mapping is no longer needed, the address that 120 the page was mapped to must be released with kunmap(). 121 122 Mapping changes must be propagated across all the CPUs. kmap() also 123 requires global TLB invalidation when the kmap's pool wraps and it might 124 block when the mapping space is fully utilized until a slot becomes 125 available. Therefore, kmap() is only callable from preemptible context. 126 127 All the above work is necessary if a mapping must last for a relatively 128 long time but the bulk of high-memory mappings in the kernel are 129 short-lived and only used in one place. This means that the cost of 130 kmap() is mostly wasted in such cases. kmap() was not intended for long 131 term mappings but it has morphed in that direction and its use is 132 strongly discouraged in newer code and the set of the preceding functions 133 should be preferred. 134 135 On 64-bit systems, calls to kmap_local_page(), kmap_atomic() and kmap() have 136 no real work to do because a 64-bit address space is more than sufficient to 137 address all the physical memory whose pages are permanently mapped. 138 139* vmap(). This can be used to make a long duration mapping of multiple 140 physical pages into a contiguous virtual space. It needs global 141 synchronization to unmap. 142 143 144Cost of Temporary Mappings 145========================== 146 147The cost of creating temporary mappings can be quite high. The arch has to 148manipulate the kernel's page tables, the data TLB and/or the MMU's registers. 149 150If CONFIG_HIGHMEM is not set, then the kernel will try and create a mapping 151simply with a bit of arithmetic that will convert the page struct address into 152a pointer to the page contents rather than juggling mappings about. In such a 153case, the unmap operation may be a null operation. 154 155If CONFIG_MMU is not set, then there can be no temporary mappings and no 156highmem. In such a case, the arithmetic approach will also be used. 157 158 159i386 PAE 160======== 161 162The i386 arch, under some circumstances, will permit you to stick up to 64GiB 163of RAM into your 32-bit machine. This has a number of consequences: 164 165* Linux needs a page-frame structure for each page in the system and the 166 pageframes need to live in the permanent mapping, which means: 167 168* you can have 896M/sizeof(struct page) page-frames at most; with struct 169 page being 32-bytes that would end up being something in the order of 112G 170 worth of pages; the kernel, however, needs to store more than just 171 page-frames in that memory... 172 173* PAE makes your page tables larger - which slows the system down as more 174 data has to be accessed to traverse in TLB fills and the like. One 175 advantage is that PAE has more PTE bits and can provide advanced features 176 like NX and PAT. 177 178The general recommendation is that you don't use more than 8GiB on a 32-bit 179machine - although more might work for you and your workload, you're pretty 180much on your own - don't expect kernel developers to really care much if things 181come apart. 182 183 184Functions 185========= 186 187.. kernel-doc:: include/linux/highmem.h 188.. kernel-doc:: include/linux/highmem-internal.h 189