xref: /openbmc/linux/Documentation/mm/highmem.rst (revision 92b64bd0)
1ee65728eSMike Rapoport.. _highmem:
2ee65728eSMike Rapoport
3ee65728eSMike Rapoport====================
4ee65728eSMike RapoportHigh Memory Handling
5ee65728eSMike Rapoport====================
6ee65728eSMike Rapoport
7ee65728eSMike RapoportBy: Peter Zijlstra <a.p.zijlstra@chello.nl>
8ee65728eSMike Rapoport
9ee65728eSMike Rapoport.. contents:: :local:
10ee65728eSMike Rapoport
11ee65728eSMike RapoportWhat Is High Memory?
12ee65728eSMike Rapoport====================
13ee65728eSMike Rapoport
14ee65728eSMike RapoportHigh memory (highmem) is used when the size of physical memory approaches or
15ee65728eSMike Rapoportexceeds the maximum size of virtual memory.  At that point it becomes
16ee65728eSMike Rapoportimpossible for the kernel to keep all of the available physical memory mapped
17ee65728eSMike Rapoportat all times.  This means the kernel needs to start using temporary mappings of
18ee65728eSMike Rapoportthe pieces of physical memory that it wants to access.
19ee65728eSMike Rapoport
20ee65728eSMike RapoportThe part of (physical) memory not covered by a permanent mapping is what we
21ee65728eSMike Rapoportrefer to as 'highmem'.  There are various architecture dependent constraints on
22ee65728eSMike Rapoportwhere exactly that border lies.
23ee65728eSMike Rapoport
24ee65728eSMike RapoportIn the i386 arch, for example, we choose to map the kernel into every process's
25ee65728eSMike RapoportVM space so that we don't have to pay the full TLB invalidation costs for
26ee65728eSMike Rapoportkernel entry/exit.  This means the available virtual memory space (4GiB on
27ee65728eSMike Rapoporti386) has to be divided between user and kernel space.
28ee65728eSMike Rapoport
29ee65728eSMike RapoportThe traditional split for architectures using this approach is 3:1, 3GiB for
30ee65728eSMike Rapoportuserspace and the top 1GiB for kernel space::
31ee65728eSMike Rapoport
32ee65728eSMike Rapoport		+--------+ 0xffffffff
33ee65728eSMike Rapoport		| Kernel |
34ee65728eSMike Rapoport		+--------+ 0xc0000000
35ee65728eSMike Rapoport		|        |
36ee65728eSMike Rapoport		| User   |
37ee65728eSMike Rapoport		|        |
38ee65728eSMike Rapoport		+--------+ 0x00000000
39ee65728eSMike Rapoport
40ee65728eSMike RapoportThis means that the kernel can at most map 1GiB of physical memory at any one
41ee65728eSMike Rapoporttime, but because we need virtual address space for other things - including
42ee65728eSMike Rapoporttemporary maps to access the rest of the physical memory - the actual direct
43ee65728eSMike Rapoportmap will typically be less (usually around ~896MiB).
44ee65728eSMike Rapoport
45ee65728eSMike RapoportOther architectures that have mm context tagged TLBs can have separate kernel
46ee65728eSMike Rapoportand user maps.  Some hardware (like some ARMs), however, have limited virtual
47ee65728eSMike Rapoportspace when they use mm context tags.
48ee65728eSMike Rapoport
49ee65728eSMike Rapoport
50ee65728eSMike RapoportTemporary Virtual Mappings
51ee65728eSMike Rapoport==========================
52ee65728eSMike Rapoport
53ee65728eSMike RapoportThe kernel contains several ways of creating temporary mappings. The following
54ee65728eSMike Rapoportlist shows them in order of preference of use.
55ee65728eSMike Rapoport
56ee65728eSMike Rapoport* kmap_local_page().  This function is used to require short term mappings.
57ee65728eSMike Rapoport  It can be invoked from any context (including interrupts) but the mappings
58ee65728eSMike Rapoport  can only be used in the context which acquired them.
59ee65728eSMike Rapoport
60*92b64bd0SFabio M. De Francesco  This function should always be used, whereas kmap_atomic() and kmap() have
61*92b64bd0SFabio M. De Francesco  been deprecated.
62ee65728eSMike Rapoport
63ee65728eSMike Rapoport  These mappings are thread-local and CPU-local, meaning that the mapping
64a9e9c939SFabio M. De Francesco  can only be accessed from within this thread and the thread is bound to the
65a9e9c939SFabio M. De Francesco  CPU while the mapping is active. Although preemption is never disabled by
66a9e9c939SFabio M. De Francesco  this function, the CPU can not be unplugged from the system via
67a9e9c939SFabio M. De Francesco  CPU-hotplug until the mapping is disposed.
68ee65728eSMike Rapoport
69ee65728eSMike Rapoport  It's valid to take pagefaults in a local kmap region, unless the context
70ee65728eSMike Rapoport  in which the local mapping is acquired does not allow it for other reasons.
71ee65728eSMike Rapoport
72a9e9c939SFabio M. De Francesco  As said, pagefaults and preemption are never disabled. There is no need to
73a9e9c939SFabio M. De Francesco  disable preemption because, when context switches to a different task, the
74a9e9c939SFabio M. De Francesco  maps of the outgoing task are saved and those of the incoming one are
75a9e9c939SFabio M. De Francesco  restored.
76a9e9c939SFabio M. De Francesco
77ee65728eSMike Rapoport  kmap_local_page() always returns a valid virtual address and it is assumed
78ee65728eSMike Rapoport  that kunmap_local() will never fail.
79ee65728eSMike Rapoport
80516ea046SFabio M. De Francesco  On CONFIG_HIGHMEM=n kernels and for low memory pages this returns the
81516ea046SFabio M. De Francesco  virtual address of the direct mapping. Only real highmem pages are
82516ea046SFabio M. De Francesco  temporarily mapped. Therefore, users may call a plain page_address()
83516ea046SFabio M. De Francesco  for pages which are known to not come from ZONE_HIGHMEM. However, it is
84516ea046SFabio M. De Francesco  always safe to use kmap_local_page() / kunmap_local().
85516ea046SFabio M. De Francesco
866b3afe2eSFabio M. De Francesco  While it is significantly faster than kmap(), for the higmem case it
876b3afe2eSFabio M. De Francesco  comes with restrictions about the pointers validity. Contrary to kmap()
886b3afe2eSFabio M. De Francesco  mappings, the local mappings are only valid in the context of the caller
896b3afe2eSFabio M. De Francesco  and cannot be handed to other contexts. This implies that users must
906b3afe2eSFabio M. De Francesco  be absolutely sure to keep the use of the return address local to the
916b3afe2eSFabio M. De Francesco  thread which mapped it.
926b3afe2eSFabio M. De Francesco
9384b86f60SFabio M. De Francesco  Most code can be designed to use thread local mappings. User should
9484b86f60SFabio M. De Francesco  therefore try to design their code to avoid the use of kmap() by mapping
9584b86f60SFabio M. De Francesco  pages in the same thread the address will be used and prefer
9684b86f60SFabio M. De Francesco  kmap_local_page().
9784b86f60SFabio M. De Francesco
98ee65728eSMike Rapoport  Nesting kmap_local_page() and kmap_atomic() mappings is allowed to a certain
99ee65728eSMike Rapoport  extent (up to KMAP_TYPE_NR) but their invocations have to be strictly ordered
100ee65728eSMike Rapoport  because the map implementation is stack based. See kmap_local_page() kdocs
101ee65728eSMike Rapoport  (included in the "Functions" section) for details on how to manage nested
102ee65728eSMike Rapoport  mappings.
103ee65728eSMike Rapoport
104*92b64bd0SFabio M. De Francesco* kmap_atomic(). This function has been deprecated; use kmap_local_page().
105*92b64bd0SFabio M. De Francesco
106*92b64bd0SFabio M. De Francesco  NOTE: Conversions to kmap_local_page() must take care to follow the mapping
107*92b64bd0SFabio M. De Francesco  restrictions imposed on kmap_local_page(). Furthermore, the code between
108*92b64bd0SFabio M. De Francesco  calls to kmap_atomic() and kunmap_atomic() may implicitly depend on the side
109*92b64bd0SFabio M. De Francesco  effects of atomic mappings, i.e. disabling page faults or preemption, or both.
110*92b64bd0SFabio M. De Francesco  In that case, explicit calls to pagefault_disable() or preempt_disable() or
111*92b64bd0SFabio M. De Francesco  both must be made in conjunction with the use of kmap_local_page().
112*92b64bd0SFabio M. De Francesco
113*92b64bd0SFabio M. De Francesco  [Legacy documentation]
114*92b64bd0SFabio M. De Francesco
115*92b64bd0SFabio M. De Francesco  This permits a very short duration mapping of a single page.  Since the
116*92b64bd0SFabio M. De Francesco  mapping is restricted to the CPU that issued it, it performs well, but
117*92b64bd0SFabio M. De Francesco  the issuing task is therefore required to stay on that CPU until it has
118*92b64bd0SFabio M. De Francesco  finished, lest some other task displace its mappings.
119ee65728eSMike Rapoport
120ee65728eSMike Rapoport  kmap_atomic() may also be used by interrupt contexts, since it does not
121ee65728eSMike Rapoport  sleep and the callers too may not sleep until after kunmap_atomic() is
122ee65728eSMike Rapoport  called.
123ee65728eSMike Rapoport
124ee65728eSMike Rapoport  Each call of kmap_atomic() in the kernel creates a non-preemptible section
125ee65728eSMike Rapoport  and disable pagefaults. This could be a source of unwanted latency. Therefore
126ee65728eSMike Rapoport  users should prefer kmap_local_page() instead of kmap_atomic().
127ee65728eSMike Rapoport
128ee65728eSMike Rapoport  It is assumed that k[un]map_atomic() won't fail.
129ee65728eSMike Rapoport
130*92b64bd0SFabio M. De Francesco* kmap(). This function has been deprecated; use kmap_local_page().
131*92b64bd0SFabio M. De Francesco
132*92b64bd0SFabio M. De Francesco  NOTE: Conversions to kmap_local_page() must take care to follow the mapping
133*92b64bd0SFabio M. De Francesco  restrictions imposed on kmap_local_page(). In particular, it is necessary to
134*92b64bd0SFabio M. De Francesco  make sure that the kernel virtual memory pointer is only valid in the thread
135*92b64bd0SFabio M. De Francesco  that obtained it.
136*92b64bd0SFabio M. De Francesco
137*92b64bd0SFabio M. De Francesco  [Legacy documentation]
138*92b64bd0SFabio M. De Francesco
139*92b64bd0SFabio M. De Francesco  This should be used to make short duration mapping of a single page with no
140*92b64bd0SFabio M. De Francesco  restrictions on preemption or migration. It comes with an overhead as mapping
141*92b64bd0SFabio M. De Francesco  space is restricted and protected by a global lock for synchronization. When
142*92b64bd0SFabio M. De Francesco  mapping is no longer needed, the address that the page was mapped to must be
143*92b64bd0SFabio M. De Francesco  released with kunmap().
144ee65728eSMike Rapoport
145ee65728eSMike Rapoport  Mapping changes must be propagated across all the CPUs. kmap() also
146ee65728eSMike Rapoport  requires global TLB invalidation when the kmap's pool wraps and it might
147ee65728eSMike Rapoport  block when the mapping space is fully utilized until a slot becomes
148ee65728eSMike Rapoport  available. Therefore, kmap() is only callable from preemptible context.
149ee65728eSMike Rapoport
150ee65728eSMike Rapoport  All the above work is necessary if a mapping must last for a relatively
151ee65728eSMike Rapoport  long time but the bulk of high-memory mappings in the kernel are
152ee65728eSMike Rapoport  short-lived and only used in one place. This means that the cost of
153ee65728eSMike Rapoport  kmap() is mostly wasted in such cases. kmap() was not intended for long
154ee65728eSMike Rapoport  term mappings but it has morphed in that direction and its use is
155ee65728eSMike Rapoport  strongly discouraged in newer code and the set of the preceding functions
156ee65728eSMike Rapoport  should be preferred.
157ee65728eSMike Rapoport
158ee65728eSMike Rapoport  On 64-bit systems, calls to kmap_local_page(), kmap_atomic() and kmap() have
159ee65728eSMike Rapoport  no real work to do because a 64-bit address space is more than sufficient to
160ee65728eSMike Rapoport  address all the physical memory whose pages are permanently mapped.
161ee65728eSMike Rapoport
162ee65728eSMike Rapoport* vmap().  This can be used to make a long duration mapping of multiple
163ee65728eSMike Rapoport  physical pages into a contiguous virtual space.  It needs global
164ee65728eSMike Rapoport  synchronization to unmap.
165ee65728eSMike Rapoport
166ee65728eSMike Rapoport
167ee65728eSMike RapoportCost of Temporary Mappings
168ee65728eSMike Rapoport==========================
169ee65728eSMike Rapoport
170ee65728eSMike RapoportThe cost of creating temporary mappings can be quite high.  The arch has to
171ee65728eSMike Rapoportmanipulate the kernel's page tables, the data TLB and/or the MMU's registers.
172ee65728eSMike Rapoport
173ee65728eSMike RapoportIf CONFIG_HIGHMEM is not set, then the kernel will try and create a mapping
174ee65728eSMike Rapoportsimply with a bit of arithmetic that will convert the page struct address into
175ee65728eSMike Rapoporta pointer to the page contents rather than juggling mappings about.  In such a
176ee65728eSMike Rapoportcase, the unmap operation may be a null operation.
177ee65728eSMike Rapoport
178ee65728eSMike RapoportIf CONFIG_MMU is not set, then there can be no temporary mappings and no
179ee65728eSMike Rapoporthighmem.  In such a case, the arithmetic approach will also be used.
180ee65728eSMike Rapoport
181ee65728eSMike Rapoport
182ee65728eSMike Rapoporti386 PAE
183ee65728eSMike Rapoport========
184ee65728eSMike Rapoport
185ee65728eSMike RapoportThe i386 arch, under some circumstances, will permit you to stick up to 64GiB
186ee65728eSMike Rapoportof RAM into your 32-bit machine.  This has a number of consequences:
187ee65728eSMike Rapoport
188ee65728eSMike Rapoport* Linux needs a page-frame structure for each page in the system and the
189ee65728eSMike Rapoport  pageframes need to live in the permanent mapping, which means:
190ee65728eSMike Rapoport
191ee65728eSMike Rapoport* you can have 896M/sizeof(struct page) page-frames at most; with struct
192ee65728eSMike Rapoport  page being 32-bytes that would end up being something in the order of 112G
193ee65728eSMike Rapoport  worth of pages; the kernel, however, needs to store more than just
194ee65728eSMike Rapoport  page-frames in that memory...
195ee65728eSMike Rapoport
196ee65728eSMike Rapoport* PAE makes your page tables larger - which slows the system down as more
197ee65728eSMike Rapoport  data has to be accessed to traverse in TLB fills and the like.  One
198ee65728eSMike Rapoport  advantage is that PAE has more PTE bits and can provide advanced features
199ee65728eSMike Rapoport  like NX and PAT.
200ee65728eSMike Rapoport
201ee65728eSMike RapoportThe general recommendation is that you don't use more than 8GiB on a 32-bit
202ee65728eSMike Rapoportmachine - although more might work for you and your workload, you're pretty
203ee65728eSMike Rapoportmuch on your own - don't expect kernel developers to really care much if things
204ee65728eSMike Rapoportcome apart.
205ee65728eSMike Rapoport
206ee65728eSMike Rapoport
207ee65728eSMike RapoportFunctions
208ee65728eSMike Rapoport=========
209ee65728eSMike Rapoport
210ee65728eSMike Rapoport.. kernel-doc:: include/linux/highmem.h
211ee65728eSMike Rapoport.. kernel-doc:: include/linux/highmem-internal.h
212