1.. SPDX-License-Identifier: GPL-2.0
2
3===========================
4Ramfs, rootfs and initramfs
5===========================
6
7October 17, 2005
8
9Rob Landley <rob@landley.net>
10=============================
11
12What is ramfs?
13--------------
14
15Ramfs is a very simple filesystem that exports Linux's disk caching
16mechanisms (the page cache and dentry cache) as a dynamically resizable
17RAM-based filesystem.
18
19Normally all files are cached in memory by Linux.  Pages of data read from
20backing store (usually the block device the filesystem is mounted on) are kept
21around in case it's needed again, but marked as clean (freeable) in case the
22Virtual Memory system needs the memory for something else.  Similarly, data
23written to files is marked clean as soon as it has been written to backing
24store, but kept around for caching purposes until the VM reallocates the
25memory.  A similar mechanism (the dentry cache) greatly speeds up access to
26directories.
27
28With ramfs, there is no backing store.  Files written into ramfs allocate
29dentries and page cache as usual, but there's nowhere to write them to.
30This means the pages are never marked clean, so they can't be freed by the
31VM when it's looking to recycle memory.
32
33The amount of code required to implement ramfs is tiny, because all the
34work is done by the existing Linux caching infrastructure.  Basically,
35you're mounting the disk cache as a filesystem.  Because of this, ramfs is not
36an optional component removable via menuconfig, since there would be negligible
37space savings.
38
39ramfs and ramdisk:
40------------------
41
42The older "ram disk" mechanism created a synthetic block device out of
43an area of RAM and used it as backing store for a filesystem.  This block
44device was of fixed size, so the filesystem mounted on it was of fixed
45size.  Using a ram disk also required unnecessarily copying memory from the
46fake block device into the page cache (and copying changes back out), as well
47as creating and destroying dentries.  Plus it needed a filesystem driver
48(such as ext2) to format and interpret this data.
49
50Compared to ramfs, this wastes memory (and memory bus bandwidth), creates
51unnecessary work for the CPU, and pollutes the CPU caches.  (There are tricks
52to avoid this copying by playing with the page tables, but they're unpleasantly
53complicated and turn out to be about as expensive as the copying anyway.)
54More to the point, all the work ramfs is doing has to happen _anyway_,
55since all file access goes through the page and dentry caches.  The RAM
56disk is simply unnecessary; ramfs is internally much simpler.
57
58Another reason ramdisks are semi-obsolete is that the introduction of
59loopback devices offered a more flexible and convenient way to create
60synthetic block devices, now from files instead of from chunks of memory.
61See losetup (8) for details.
62
63ramfs and tmpfs:
64----------------
65
66One downside of ramfs is you can keep writing data into it until you fill
67up all memory, and the VM can't free it because the VM thinks that files
68should get written to backing store (rather than swap space), but ramfs hasn't
69got any backing store.  Because of this, only root (or a trusted user) should
70be allowed write access to a ramfs mount.
71
72A ramfs derivative called tmpfs was created to add size limits, and the ability
73to write the data to swap space.  Normal users can be allowed write access to
74tmpfs mounts.  See Documentation/filesystems/tmpfs.rst for more information.
75
76What is rootfs?
77---------------
78
79Rootfs is a special instance of ramfs (or tmpfs, if that's enabled), which is
80always present in 2.6 systems.  You can't unmount rootfs for approximately the
81same reason you can't kill the init process; rather than having special code
82to check for and handle an empty list, it's smaller and simpler for the kernel
83to just make sure certain lists can't become empty.
84
85Most systems just mount another filesystem over rootfs and ignore it.  The
86amount of space an empty instance of ramfs takes up is tiny.
87
88If CONFIG_TMPFS is enabled, rootfs will use tmpfs instead of ramfs by
89default.  To force ramfs, add "rootfstype=ramfs" to the kernel command
90line.
91
92What is initramfs?
93------------------
94
95All 2.6 Linux kernels contain a gzipped "cpio" format archive, which is
96extracted into rootfs when the kernel boots up.  After extracting, the kernel
97checks to see if rootfs contains a file "init", and if so it executes it as PID
981.  If found, this init process is responsible for bringing the system the
99rest of the way up, including locating and mounting the real root device (if
100any).  If rootfs does not contain an init program after the embedded cpio
101archive is extracted into it, the kernel will fall through to the older code
102to locate and mount a root partition, then exec some variant of /sbin/init
103out of that.
104
105All this differs from the old initrd in several ways:
106
107  - The old initrd was always a separate file, while the initramfs archive is
108    linked into the linux kernel image.  (The directory ``linux-*/usr`` is
109    devoted to generating this archive during the build.)
110
111  - The old initrd file was a gzipped filesystem image (in some file format,
112    such as ext2, that needed a driver built into the kernel), while the new
113    initramfs archive is a gzipped cpio archive (like tar only simpler,
114    see cpio(1) and Documentation/driver-api/early-userspace/buffer-format.rst).
115    The kernel's cpio extraction code is not only extremely small, it's also
116    __init text and data that can be discarded during the boot process.
117
118  - The program run by the old initrd (which was called /initrd, not /init) did
119    some setup and then returned to the kernel, while the init program from
120    initramfs is not expected to return to the kernel.  (If /init needs to hand
121    off control it can overmount / with a new root device and exec another init
122    program.  See the switch_root utility, below.)
123
124  - When switching another root device, initrd would pivot_root and then
125    umount the ramdisk.  But initramfs is rootfs: you can neither pivot_root
126    rootfs, nor unmount it.  Instead delete everything out of rootfs to
127    free up the space (find -xdev / -exec rm '{}' ';'), overmount rootfs
128    with the new root (cd /newmount; mount --move . /; chroot .), attach
129    stdin/stdout/stderr to the new /dev/console, and exec the new init.
130
131    Since this is a remarkably persnickety process (and involves deleting
132    commands before you can run them), the klibc package introduced a helper
133    program (utils/run_init.c) to do all this for you.  Most other packages
134    (such as busybox) have named this command "switch_root".
135
136Populating initramfs:
137---------------------
138
139The 2.6 kernel build process always creates a gzipped cpio format initramfs
140archive and links it into the resulting kernel binary.  By default, this
141archive is empty (consuming 134 bytes on x86).
142
143The config option CONFIG_INITRAMFS_SOURCE (in General Setup in menuconfig,
144and living in usr/Kconfig) can be used to specify a source for the
145initramfs archive, which will automatically be incorporated into the
146resulting binary.  This option can point to an existing gzipped cpio
147archive, a directory containing files to be archived, or a text file
148specification such as the following example::
149
150  dir /dev 755 0 0
151  nod /dev/console 644 0 0 c 5 1
152  nod /dev/loop0 644 0 0 b 7 0
153  dir /bin 755 1000 1000
154  slink /bin/sh busybox 777 0 0
155  file /bin/busybox initramfs/busybox 755 0 0
156  dir /proc 755 0 0
157  dir /sys 755 0 0
158  dir /mnt 755 0 0
159  file /init initramfs/init.sh 755 0 0
160
161Run "usr/gen_init_cpio" (after the kernel build) to get a usage message
162documenting the above file format.
163
164One advantage of the configuration file is that root access is not required to
165set permissions or create device nodes in the new archive.  (Note that those
166two example "file" entries expect to find files named "init.sh" and "busybox" in
167a directory called "initramfs", under the linux-2.6.* directory.  See
168Documentation/driver-api/early-userspace/early_userspace_support.rst for more details.)
169
170The kernel does not depend on external cpio tools.  If you specify a
171directory instead of a configuration file, the kernel's build infrastructure
172creates a configuration file from that directory (usr/Makefile calls
173usr/gen_initramfs.sh), and proceeds to package up that directory
174using the config file (by feeding it to usr/gen_init_cpio, which is created
175from usr/gen_init_cpio.c).  The kernel's build-time cpio creation code is
176entirely self-contained, and the kernel's boot-time extractor is also
177(obviously) self-contained.
178
179The one thing you might need external cpio utilities installed for is creating
180or extracting your own preprepared cpio files to feed to the kernel build
181(instead of a config file or directory).
182
183The following command line can extract a cpio image (either by the above script
184or by the kernel build) back into its component files::
185
186  cpio -i -d -H newc -F initramfs_data.cpio --no-absolute-filenames
187
188The following shell script can create a prebuilt cpio archive you can
189use in place of the above config file::
190
191  #!/bin/sh
192
193  # Copyright 2006 Rob Landley <rob@landley.net> and TimeSys Corporation.
194  # Licensed under GPL version 2
195
196  if [ $# -ne 2 ]
197  then
198    echo "usage: mkinitramfs directory imagename.cpio.gz"
199    exit 1
200  fi
201
202  if [ -d "$1" ]
203  then
204    echo "creating $2 from $1"
205    (cd "$1"; find . | cpio -o -H newc | gzip) > "$2"
206  else
207    echo "First argument must be a directory"
208    exit 1
209  fi
210
211.. Note::
212
213   The cpio man page contains some bad advice that will break your initramfs
214   archive if you follow it.  It says "A typical way to generate the list
215   of filenames is with the find command; you should give find the -depth
216   option to minimize problems with permissions on directories that are
217   unwritable or not searchable."  Don't do this when creating
218   initramfs.cpio.gz images, it won't work.  The Linux kernel cpio extractor
219   won't create files in a directory that doesn't exist, so the directory
220   entries must go before the files that go in those directories.
221   The above script gets them in the right order.
222
223External initramfs images:
224--------------------------
225
226If the kernel has initrd support enabled, an external cpio.gz archive can also
227be passed into a 2.6 kernel in place of an initrd.  In this case, the kernel
228will autodetect the type (initramfs, not initrd) and extract the external cpio
229archive into rootfs before trying to run /init.
230
231This has the memory efficiency advantages of initramfs (no ramdisk block
232device) but the separate packaging of initrd (which is nice if you have
233non-GPL code you'd like to run from initramfs, without conflating it with
234the GPL licensed Linux kernel binary).
235
236It can also be used to supplement the kernel's built-in initramfs image.  The
237files in the external archive will overwrite any conflicting files in
238the built-in initramfs archive.  Some distributors also prefer to customize
239a single kernel image with task-specific initramfs images, without recompiling.
240
241Contents of initramfs:
242----------------------
243
244An initramfs archive is a complete self-contained root filesystem for Linux.
245If you don't already understand what shared libraries, devices, and paths
246you need to get a minimal root filesystem up and running, here are some
247references:
248
249- https://www.tldp.org/HOWTO/Bootdisk-HOWTO/
250- https://www.tldp.org/HOWTO/From-PowerUp-To-Bash-Prompt-HOWTO.html
251- http://www.linuxfromscratch.org/lfs/view/stable/
252
253The "klibc" package (https://www.kernel.org/pub/linux/libs/klibc) is
254designed to be a tiny C library to statically link early userspace
255code against, along with some related utilities.  It is BSD licensed.
256
257I use uClibc (https://www.uclibc.org) and busybox (https://www.busybox.net)
258myself.  These are LGPL and GPL, respectively.  (A self-contained initramfs
259package is planned for the busybox 1.3 release.)
260
261In theory you could use glibc, but that's not well suited for small embedded
262uses like this.  (A "hello world" program statically linked against glibc is
263over 400k.  With uClibc it's 7k.  Also note that glibc dlopens libnss to do
264name lookups, even when otherwise statically linked.)
265
266A good first step is to get initramfs to run a statically linked "hello world"
267program as init, and test it under an emulator like qemu (www.qemu.org) or
268User Mode Linux, like so::
269
270  cat > hello.c << EOF
271  #include <stdio.h>
272  #include <unistd.h>
273
274  int main(int argc, char *argv[])
275  {
276    printf("Hello world!\n");
277    sleep(999999999);
278  }
279  EOF
280  gcc -static hello.c -o init
281  echo init | cpio -o -H newc | gzip > test.cpio.gz
282  # Testing external initramfs using the initrd loading mechanism.
283  qemu -kernel /boot/vmlinuz -initrd test.cpio.gz /dev/zero
284
285When debugging a normal root filesystem, it's nice to be able to boot with
286"init=/bin/sh".  The initramfs equivalent is "rdinit=/bin/sh", and it's
287just as useful.
288
289Why cpio rather than tar?
290-------------------------
291
292This decision was made back in December, 2001.  The discussion started here:
293
294  http://www.uwsg.iu.edu/hypermail/linux/kernel/0112.2/1538.html
295
296And spawned a second thread (specifically on tar vs cpio), starting here:
297
298  http://www.uwsg.iu.edu/hypermail/linux/kernel/0112.2/1587.html
299
300The quick and dirty summary version (which is no substitute for reading
301the above threads) is:
302
3031) cpio is a standard.  It's decades old (from the AT&T days), and already
304   widely used on Linux (inside RPM, Red Hat's device driver disks).  Here's
305   a Linux Journal article about it from 1996:
306
307      http://www.linuxjournal.com/article/1213
308
309   It's not as popular as tar because the traditional cpio command line tools
310   require _truly_hideous_ command line arguments.  But that says nothing
311   either way about the archive format, and there are alternative tools,
312   such as:
313
314     http://freecode.com/projects/afio
315
3162) The cpio archive format chosen by the kernel is simpler and cleaner (and
317   thus easier to create and parse) than any of the (literally dozens of)
318   various tar archive formats.  The complete initramfs archive format is
319   explained in buffer-format.txt, created in usr/gen_init_cpio.c, and
320   extracted in init/initramfs.c.  All three together come to less than 26k
321   total of human-readable text.
322
3233) The GNU project standardizing on tar is approximately as relevant as
324   Windows standardizing on zip.  Linux is not part of either, and is free
325   to make its own technical decisions.
326
3274) Since this is a kernel internal format, it could easily have been
328   something brand new.  The kernel provides its own tools to create and
329   extract this format anyway.  Using an existing standard was preferable,
330   but not essential.
331
3325) Al Viro made the decision (quote: "tar is ugly as hell and not going to be
333   supported on the kernel side"):
334
335      http://www.uwsg.iu.edu/hypermail/linux/kernel/0112.2/1540.html
336
337   explained his reasoning:
338
339     - http://www.uwsg.iu.edu/hypermail/linux/kernel/0112.2/1550.html
340     - http://www.uwsg.iu.edu/hypermail/linux/kernel/0112.2/1638.html
341
342   and, most importantly, designed and implemented the initramfs code.
343
344Future directions:
345------------------
346
347Today (2.6.16), initramfs is always compiled in, but not always used.  The
348kernel falls back to legacy boot code that is reached only if initramfs does
349not contain an /init program.  The fallback is legacy code, there to ensure a
350smooth transition and allowing early boot functionality to gradually move to
351"early userspace" (I.E. initramfs).
352
353The move to early userspace is necessary because finding and mounting the real
354root device is complex.  Root partitions can span multiple devices (raid or
355separate journal).  They can be out on the network (requiring dhcp, setting a
356specific MAC address, logging into a server, etc).  They can live on removable
357media, with dynamically allocated major/minor numbers and persistent naming
358issues requiring a full udev implementation to sort out.  They can be
359compressed, encrypted, copy-on-write, loopback mounted, strangely partitioned,
360and so on.
361
362This kind of complexity (which inevitably includes policy) is rightly handled
363in userspace.  Both klibc and busybox/uClibc are working on simple initramfs
364packages to drop into a kernel build.
365
366The klibc package has now been accepted into Andrew Morton's 2.6.17-mm tree.
367The kernel's current early boot code (partition detection, etc) will probably
368be migrated into a default initramfs, automatically created and used by the
369kernel build.
370