1.. SPDX-License-Identifier: GPL-2.0
2
3======
4NILFS2
5======
6
7NILFS2 is a log-structured file system (LFS) supporting continuous
8snapshotting.  In addition to versioning capability of the entire file
9system, users can even restore files mistakenly overwritten or
10destroyed just a few seconds ago.  Since NILFS2 can keep consistency
11like conventional LFS, it achieves quick recovery after system
12crashes.
13
14NILFS2 creates a number of checkpoints every few seconds or per
15synchronous write basis (unless there is no change).  Users can select
16significant versions among continuously created checkpoints, and can
17change them into snapshots which will be preserved until they are
18changed back to checkpoints.
19
20There is no limit on the number of snapshots until the volume gets
21full.  Each snapshot is mountable as a read-only file system
22concurrently with its writable mount, and this feature is convenient
23for online backup.
24
25The userland tools are included in nilfs-utils package, which is
26available from the following download page.  At least "mkfs.nilfs2",
27"mount.nilfs2", "umount.nilfs2", and "nilfs_cleanerd" (so called
28cleaner or garbage collector) are required.  Details on the tools are
29described in the man pages included in the package.
30
31:Project web page:    https://nilfs.sourceforge.io/
32:Download page:       https://nilfs.sourceforge.io/en/download.html
33:List info:           http://vger.kernel.org/vger-lists.html#linux-nilfs
34
35Caveats
36=======
37
38Features which NILFS2 does not support yet:
39
40	- atime
41	- extended attributes
42	- POSIX ACLs
43	- quotas
44	- fsck
45	- defragmentation
46
47Mount options
48=============
49
50NILFS2 supports the following mount options:
51(*) == default
52
53======================= =======================================================
54barrier(*)		This enables/disables the use of write barriers.  This
55nobarrier		requires an IO stack which can support barriers, and
56			if nilfs gets an error on a barrier write, it will
57			disable again with a warning.
58errors=continue		Keep going on a filesystem error.
59errors=remount-ro(*)	Remount the filesystem read-only on an error.
60errors=panic		Panic and halt the machine if an error occurs.
61cp=n			Specify the checkpoint-number of the snapshot to be
62			mounted.  Checkpoints and snapshots are listed by lscp
63			user command.  Only the checkpoints marked as snapshot
64			are mountable with this option.  Snapshot is read-only,
65			so a read-only mount option must be specified together.
66order=relaxed(*)	Apply relaxed order semantics that allows modified data
67			blocks to be written to disk without making a
68			checkpoint if no metadata update is going.  This mode
69			is equivalent to the ordered data mode of the ext3
70			filesystem except for the updates on data blocks still
71			conserve atomicity.  This will improve synchronous
72			write performance for overwriting.
73order=strict		Apply strict in-order semantics that preserves sequence
74			of all file operations including overwriting of data
75			blocks.  That means, it is guaranteed that no
76			overtaking of events occurs in the recovered file
77			system after a crash.
78norecovery		Disable recovery of the filesystem on mount.
79			This disables every write access on the device for
80			read-only mounts or snapshots.  This option will fail
81			for r/w mounts on an unclean volume.
82discard			This enables/disables the use of discard/TRIM commands.
83nodiscard(*)		The discard/TRIM commands are sent to the underlying
84			block device when blocks are freed.  This is useful
85			for SSD devices and sparse/thinly-provisioned LUNs.
86======================= =======================================================
87
88Ioctls
89======
90
91There is some NILFS2 specific functionality which can be accessed by applications
92through the system call interfaces. The list of all NILFS2 specific ioctls are
93shown in the table below.
94
95Table of NILFS2 specific ioctls:
96
97 ============================== ===============================================
98 Ioctl			        Description
99 ============================== ===============================================
100 NILFS_IOCTL_CHANGE_CPMODE      Change mode of given checkpoint between
101			        checkpoint and snapshot state. This ioctl is
102			        used in chcp and mkcp utilities.
103
104 NILFS_IOCTL_DELETE_CHECKPOINT  Remove checkpoint from NILFS2 file system.
105			        This ioctl is used in rmcp utility.
106
107 NILFS_IOCTL_GET_CPINFO         Return info about requested checkpoints. This
108			        ioctl is used in lscp utility and by
109			        nilfs_cleanerd daemon.
110
111 NILFS_IOCTL_GET_CPSTAT         Return checkpoints statistics. This ioctl is
112			        used by lscp, rmcp utilities and by
113			        nilfs_cleanerd daemon.
114
115 NILFS_IOCTL_GET_SUINFO         Return segment usage info about requested
116			        segments. This ioctl is used in lssu,
117			        nilfs_resize utilities and by nilfs_cleanerd
118			        daemon.
119
120 NILFS_IOCTL_SET_SUINFO         Modify segment usage info of requested
121				segments. This ioctl is used by
122				nilfs_cleanerd daemon to skip unnecessary
123				cleaning operation of segments and reduce
124				performance penalty or wear of flash device
125				due to redundant move of in-use blocks.
126
127 NILFS_IOCTL_GET_SUSTAT         Return segment usage statistics. This ioctl
128			        is used in lssu, nilfs_resize utilities and
129			        by nilfs_cleanerd daemon.
130
131 NILFS_IOCTL_GET_VINFO          Return information on virtual block addresses.
132			        This ioctl is used by nilfs_cleanerd daemon.
133
134 NILFS_IOCTL_GET_BDESCS         Return information about descriptors of disk
135			        block numbers. This ioctl is used by
136			        nilfs_cleanerd daemon.
137
138 NILFS_IOCTL_CLEAN_SEGMENTS     Do garbage collection operation in the
139			        environment of requested parameters from
140			        userspace. This ioctl is used by
141			        nilfs_cleanerd daemon.
142
143 NILFS_IOCTL_SYNC               Make a checkpoint. This ioctl is used in
144			        mkcp utility.
145
146 NILFS_IOCTL_RESIZE             Resize NILFS2 volume. This ioctl is used
147			        by nilfs_resize utility.
148
149 NILFS_IOCTL_SET_ALLOC_RANGE    Define lower limit of segments in bytes and
150			        upper limit of segments in bytes. This ioctl
151			        is used by nilfs_resize utility.
152 ============================== ===============================================
153
154NILFS2 usage
155============
156
157To use nilfs2 as a local file system, simply::
158
159 # mkfs -t nilfs2 /dev/block_device
160 # mount -t nilfs2 /dev/block_device /dir
161
162This will also invoke the cleaner through the mount helper program
163(mount.nilfs2).
164
165Checkpoints and snapshots are managed by the following commands.
166Their manpages are included in the nilfs-utils package above.
167
168  ====     ===========================================================
169  lscp     list checkpoints or snapshots.
170  mkcp     make a checkpoint or a snapshot.
171  chcp     change an existing checkpoint to a snapshot or vice versa.
172  rmcp     invalidate specified checkpoint(s).
173  ====     ===========================================================
174
175To mount a snapshot::
176
177 # mount -t nilfs2 -r -o cp=<cno> /dev/block_device /snap_dir
178
179where <cno> is the checkpoint number of the snapshot.
180
181To unmount the NILFS2 mount point or snapshot, simply::
182
183 # umount /dir
184
185Then, the cleaner daemon is automatically shut down by the umount
186helper program (umount.nilfs2).
187
188Disk format
189===========
190
191A nilfs2 volume is equally divided into a number of segments except
192for the super block (SB) and segment #0.  A segment is the container
193of logs.  Each log is composed of summary information blocks, payload
194blocks, and an optional super root block (SR)::
195
196   ______________________________________________________
197  | |SB| | Segment | Segment | Segment | ... | Segment | |
198  |_|__|_|____0____|____1____|____2____|_____|____N____|_|
199  0 +1K +4K       +8M       +16M      +24M  +(8MB x N)
200       .             .            (Typical offsets for 4KB-block)
201    .                  .
202  .______________________.
203  | log | log |... | log |
204  |__1__|__2__|____|__m__|
205        .       .
206      .               .
207    .                       .
208  .______________________________.
209  | Summary | Payload blocks  |SR|
210  |_blocks__|_________________|__|
211
212The payload blocks are organized per file, and each file consists of
213data blocks and B-tree node blocks::
214
215    |<---       File-A        --->|<---       File-B        --->|
216   _______________________________________________________________
217    | Data blocks | B-tree blocks | Data blocks | B-tree blocks | ...
218   _|_____________|_______________|_____________|_______________|_
219
220
221Since only the modified blocks are written in the log, it may have
222files without data blocks or B-tree node blocks.
223
224The organization of the blocks is recorded in the summary information
225blocks, which contains a header structure (nilfs_segment_summary), per
226file structures (nilfs_finfo), and per block structures (nilfs_binfo)::
227
228  _________________________________________________________________________
229 | Summary | finfo | binfo | ... | binfo | finfo | binfo | ... | binfo |...
230 |_blocks__|___A___|_(A,1)_|_____|(A,Na)_|___B___|_(B,1)_|_____|(B,Nb)_|___
231
232
233The logs include regular files, directory files, symbolic link files
234and several meta data files.  The mata data files are the files used
235to maintain file system meta data.  The current version of NILFS2 uses
236the following meta data files::
237
238 1) Inode file (ifile)             -- Stores on-disk inodes
239 2) Checkpoint file (cpfile)       -- Stores checkpoints
240 3) Segment usage file (sufile)    -- Stores allocation state of segments
241 4) Data address translation file  -- Maps virtual block numbers to usual
242    (DAT)                             block numbers.  This file serves to
243                                      make on-disk blocks relocatable.
244
245The following figure shows a typical organization of the logs::
246
247  _________________________________________________________________________
248 | Summary | regular file | file  | ... | ifile | cpfile | sufile | DAT |SR|
249 |_blocks__|_or_directory_|_______|_____|_______|________|________|_____|__|
250
251
252To stride over segment boundaries, this sequence of files may be split
253into multiple logs.  The sequence of logs that should be treated as
254logically one log, is delimited with flags marked in the segment
255summary.  The recovery code of nilfs2 looks this boundary information
256to ensure atomicity of updates.
257
258The super root block is inserted for every checkpoints.  It includes
259three special inodes, inodes for the DAT, cpfile, and sufile.  Inodes
260of regular files, directories, symlinks and other special files, are
261included in the ifile.  The inode of ifile itself is included in the
262corresponding checkpoint entry in the cpfile.  Thus, the hierarchy
263among NILFS2 files can be depicted as follows::
264
265  Super block (SB)
266       |
267       v
268  Super root block (the latest cno=xx)
269       |-- DAT
270       |-- sufile
271       `-- cpfile
272              |-- ifile (cno=c1)
273              |-- ifile (cno=c2) ---- file (ino=i1)
274              :        :          |-- file (ino=i2)
275              `-- ifile (cno=xx)  |-- file (ino=i3)
276                                  :        :
277                                  `-- file (ino=yy)
278                                    ( regular file, directory, or symlink )
279
280For detail on the format of each file, please see nilfs2_ondisk.h
281located at include/uapi/linux directory.
282
283There are no patents or other intellectual property that we protect
284with regard to the design of NILFS2.  It is allowed to replicate the
285design in hopes that other operating systems could share (mount, read,
286write, etc.) data stored in this format.
287