1=======================================================
2Configfs - Userspace-driven Kernel Object Configuration
3=======================================================
4
5Joel Becker <joel.becker@oracle.com>
6
7Updated: 31 March 2005
8
9Copyright (c) 2005 Oracle Corporation,
10	Joel Becker <joel.becker@oracle.com>
11
12
13What is configfs?
14=================
15
16configfs is a ram-based filesystem that provides the converse of
17sysfs's functionality.  Where sysfs is a filesystem-based view of
18kernel objects, configfs is a filesystem-based manager of kernel
19objects, or config_items.
20
21With sysfs, an object is created in kernel (for example, when a device
22is discovered) and it is registered with sysfs.  Its attributes then
23appear in sysfs, allowing userspace to read the attributes via
24readdir(3)/read(2).  It may allow some attributes to be modified via
25write(2).  The important point is that the object is created and
26destroyed in kernel, the kernel controls the lifecycle of the sysfs
27representation, and sysfs is merely a window on all this.
28
29A configfs config_item is created via an explicit userspace operation:
30mkdir(2).  It is destroyed via rmdir(2).  The attributes appear at
31mkdir(2) time, and can be read or modified via read(2) and write(2).
32As with sysfs, readdir(3) queries the list of items and/or attributes.
33symlink(2) can be used to group items together.  Unlike sysfs, the
34lifetime of the representation is completely driven by userspace.  The
35kernel modules backing the items must respond to this.
36
37Both sysfs and configfs can and should exist together on the same
38system.  One is not a replacement for the other.
39
40Using configfs
41==============
42
43configfs can be compiled as a module or into the kernel.  You can access
44it by doing::
45
46	mount -t configfs none /config
47
48The configfs tree will be empty unless client modules are also loaded.
49These are modules that register their item types with configfs as
50subsystems.  Once a client subsystem is loaded, it will appear as a
51subdirectory (or more than one) under /config.  Like sysfs, the
52configfs tree is always there, whether mounted on /config or not.
53
54An item is created via mkdir(2).  The item's attributes will also
55appear at this time.  readdir(3) can determine what the attributes are,
56read(2) can query their default values, and write(2) can store new
57values.  Don't mix more than one attribute in one attribute file.
58
59There are two types of configfs attributes:
60
61* Normal attributes, which similar to sysfs attributes, are small ASCII text
62  files, with a maximum size of one page (PAGE_SIZE, 4096 on i386).  Preferably
63  only one value per file should be used, and the same caveats from sysfs apply.
64  Configfs expects write(2) to store the entire buffer at once.  When writing to
65  normal configfs attributes, userspace processes should first read the entire
66  file, modify the portions they wish to change, and then write the entire
67  buffer back.
68
69* Binary attributes, which are somewhat similar to sysfs binary attributes,
70  but with a few slight changes to semantics.  The PAGE_SIZE limitation does not
71  apply, but the whole binary item must fit in single kernel vmalloc'ed buffer.
72  The write(2) calls from user space are buffered, and the attributes'
73  write_bin_attribute method will be invoked on the final close, therefore it is
74  imperative for user-space to check the return code of close(2) in order to
75  verify that the operation finished successfully.
76  To avoid a malicious user OOMing the kernel, there's a per-binary attribute
77  maximum buffer value.
78
79When an item needs to be destroyed, remove it with rmdir(2).  An
80item cannot be destroyed if any other item has a link to it (via
81symlink(2)).  Links can be removed via unlink(2).
82
83Configuring FakeNBD: an Example
84===============================
85
86Imagine there's a Network Block Device (NBD) driver that allows you to
87access remote block devices.  Call it FakeNBD.  FakeNBD uses configfs
88for its configuration.  Obviously, there will be a nice program that
89sysadmins use to configure FakeNBD, but somehow that program has to tell
90the driver about it.  Here's where configfs comes in.
91
92When the FakeNBD driver is loaded, it registers itself with configfs.
93readdir(3) sees this just fine::
94
95	# ls /config
96	fakenbd
97
98A fakenbd connection can be created with mkdir(2).  The name is
99arbitrary, but likely the tool will make some use of the name.  Perhaps
100it is a uuid or a disk name::
101
102	# mkdir /config/fakenbd/disk1
103	# ls /config/fakenbd/disk1
104	target device rw
105
106The target attribute contains the IP address of the server FakeNBD will
107connect to.  The device attribute is the device on the server.
108Predictably, the rw attribute determines whether the connection is
109read-only or read-write::
110
111	# echo 10.0.0.1 > /config/fakenbd/disk1/target
112	# echo /dev/sda1 > /config/fakenbd/disk1/device
113	# echo 1 > /config/fakenbd/disk1/rw
114
115That's it.  That's all there is.  Now the device is configured, via the
116shell no less.
117
118Coding With configfs
119====================
120
121Every object in configfs is a config_item.  A config_item reflects an
122object in the subsystem.  It has attributes that match values on that
123object.  configfs handles the filesystem representation of that object
124and its attributes, allowing the subsystem to ignore all but the
125basic show/store interaction.
126
127Items are created and destroyed inside a config_group.  A group is a
128collection of items that share the same attributes and operations.
129Items are created by mkdir(2) and removed by rmdir(2), but configfs
130handles that.  The group has a set of operations to perform these tasks
131
132A subsystem is the top level of a client module.  During initialization,
133the client module registers the subsystem with configfs, the subsystem
134appears as a directory at the top of the configfs filesystem.  A
135subsystem is also a config_group, and can do everything a config_group
136can.
137
138struct config_item
139==================
140
141::
142
143	struct config_item {
144		char                    *ci_name;
145		char                    ci_namebuf[UOBJ_NAME_LEN];
146		struct kref             ci_kref;
147		struct list_head        ci_entry;
148		struct config_item      *ci_parent;
149		struct config_group     *ci_group;
150		struct config_item_type *ci_type;
151		struct dentry           *ci_dentry;
152	};
153
154	void config_item_init(struct config_item *);
155	void config_item_init_type_name(struct config_item *,
156					const char *name,
157					struct config_item_type *type);
158	struct config_item *config_item_get(struct config_item *);
159	void config_item_put(struct config_item *);
160
161Generally, struct config_item is embedded in a container structure, a
162structure that actually represents what the subsystem is doing.  The
163config_item portion of that structure is how the object interacts with
164configfs.
165
166Whether statically defined in a source file or created by a parent
167config_group, a config_item must have one of the _init() functions
168called on it.  This initializes the reference count and sets up the
169appropriate fields.
170
171All users of a config_item should have a reference on it via
172config_item_get(), and drop the reference when they are done via
173config_item_put().
174
175By itself, a config_item cannot do much more than appear in configfs.
176Usually a subsystem wants the item to display and/or store attributes,
177among other things.  For that, it needs a type.
178
179struct config_item_type
180=======================
181
182::
183
184	struct configfs_item_operations {
185		void (*release)(struct config_item *);
186		int (*allow_link)(struct config_item *src,
187				  struct config_item *target);
188		void (*drop_link)(struct config_item *src,
189				 struct config_item *target);
190	};
191
192	struct config_item_type {
193		struct module                           *ct_owner;
194		struct configfs_item_operations         *ct_item_ops;
195		struct configfs_group_operations        *ct_group_ops;
196		struct configfs_attribute               **ct_attrs;
197		struct configfs_bin_attribute		**ct_bin_attrs;
198	};
199
200The most basic function of a config_item_type is to define what
201operations can be performed on a config_item.  All items that have been
202allocated dynamically will need to provide the ct_item_ops->release()
203method.  This method is called when the config_item's reference count
204reaches zero.
205
206struct configfs_attribute
207=========================
208
209::
210
211	struct configfs_attribute {
212		char                    *ca_name;
213		struct module           *ca_owner;
214		umode_t                  ca_mode;
215		ssize_t (*show)(struct config_item *, char *);
216		ssize_t (*store)(struct config_item *, const char *, size_t);
217	};
218
219When a config_item wants an attribute to appear as a file in the item's
220configfs directory, it must define a configfs_attribute describing it.
221It then adds the attribute to the NULL-terminated array
222config_item_type->ct_attrs.  When the item appears in configfs, the
223attribute file will appear with the configfs_attribute->ca_name
224filename.  configfs_attribute->ca_mode specifies the file permissions.
225
226If an attribute is readable and provides a ->show method, that method will
227be called whenever userspace asks for a read(2) on the attribute.  If an
228attribute is writable and provides a ->store  method, that method will be
229called whenever userspace asks for a write(2) on the attribute.
230
231struct configfs_bin_attribute
232=============================
233
234::
235
236	struct configfs_bin_attribute {
237		struct configfs_attribute	cb_attr;
238		void				*cb_private;
239		size_t				cb_max_size;
240	};
241
242The binary attribute is used when the one needs to use binary blob to
243appear as the contents of a file in the item's configfs directory.
244To do so add the binary attribute to the NULL-terminated array
245config_item_type->ct_bin_attrs, and the item appears in configfs, the
246attribute file will appear with the configfs_bin_attribute->cb_attr.ca_name
247filename.  configfs_bin_attribute->cb_attr.ca_mode specifies the file
248permissions.
249The cb_private member is provided for use by the driver, while the
250cb_max_size member specifies the maximum amount of vmalloc buffer
251to be used.
252
253If binary attribute is readable and the config_item provides a
254ct_item_ops->read_bin_attribute() method, that method will be called
255whenever userspace asks for a read(2) on the attribute.  The converse
256will happen for write(2). The reads/writes are bufferred so only a
257single read/write will occur; the attributes' need not concern itself
258with it.
259
260struct config_group
261===================
262
263A config_item cannot live in a vacuum.  The only way one can be created
264is via mkdir(2) on a config_group.  This will trigger creation of a
265child item::
266
267	struct config_group {
268		struct config_item		cg_item;
269		struct list_head		cg_children;
270		struct configfs_subsystem 	*cg_subsys;
271		struct list_head		default_groups;
272		struct list_head		group_entry;
273	};
274
275	void config_group_init(struct config_group *group);
276	void config_group_init_type_name(struct config_group *group,
277					 const char *name,
278					 struct config_item_type *type);
279
280
281The config_group structure contains a config_item.  Properly configuring
282that item means that a group can behave as an item in its own right.
283However, it can do more: it can create child items or groups.  This is
284accomplished via the group operations specified on the group's
285config_item_type::
286
287	struct configfs_group_operations {
288		struct config_item *(*make_item)(struct config_group *group,
289						 const char *name);
290		struct config_group *(*make_group)(struct config_group *group,
291						   const char *name);
292		int (*commit_item)(struct config_item *item);
293		void (*disconnect_notify)(struct config_group *group,
294					  struct config_item *item);
295		void (*drop_item)(struct config_group *group,
296				  struct config_item *item);
297	};
298
299A group creates child items by providing the
300ct_group_ops->make_item() method.  If provided, this method is called from
301mkdir(2) in the group's directory.  The subsystem allocates a new
302config_item (or more likely, its container structure), initializes it,
303and returns it to configfs.  Configfs will then populate the filesystem
304tree to reflect the new item.
305
306If the subsystem wants the child to be a group itself, the subsystem
307provides ct_group_ops->make_group().  Everything else behaves the same,
308using the group _init() functions on the group.
309
310Finally, when userspace calls rmdir(2) on the item or group,
311ct_group_ops->drop_item() is called.  As a config_group is also a
312config_item, it is not necessary for a separate drop_group() method.
313The subsystem must config_item_put() the reference that was initialized
314upon item allocation.  If a subsystem has no work to do, it may omit
315the ct_group_ops->drop_item() method, and configfs will call
316config_item_put() on the item on behalf of the subsystem.
317
318Important:
319   drop_item() is void, and as such cannot fail.  When rmdir(2)
320   is called, configfs WILL remove the item from the filesystem tree
321   (assuming that it has no children to keep it busy).  The subsystem is
322   responsible for responding to this.  If the subsystem has references to
323   the item in other threads, the memory is safe.  It may take some time
324   for the item to actually disappear from the subsystem's usage.  But it
325   is gone from configfs.
326
327When drop_item() is called, the item's linkage has already been torn
328down.  It no longer has a reference on its parent and has no place in
329the item hierarchy.  If a client needs to do some cleanup before this
330teardown happens, the subsystem can implement the
331ct_group_ops->disconnect_notify() method.  The method is called after
332configfs has removed the item from the filesystem view but before the
333item is removed from its parent group.  Like drop_item(),
334disconnect_notify() is void and cannot fail.  Client subsystems should
335not drop any references here, as they still must do it in drop_item().
336
337A config_group cannot be removed while it still has child items.  This
338is implemented in the configfs rmdir(2) code.  ->drop_item() will not be
339called, as the item has not been dropped.  rmdir(2) will fail, as the
340directory is not empty.
341
342struct configfs_subsystem
343=========================
344
345A subsystem must register itself, usually at module_init time.  This
346tells configfs to make the subsystem appear in the file tree::
347
348	struct configfs_subsystem {
349		struct config_group	su_group;
350		struct mutex		su_mutex;
351	};
352
353	int configfs_register_subsystem(struct configfs_subsystem *subsys);
354	void configfs_unregister_subsystem(struct configfs_subsystem *subsys);
355
356A subsystem consists of a toplevel config_group and a mutex.
357The group is where child config_items are created.  For a subsystem,
358this group is usually defined statically.  Before calling
359configfs_register_subsystem(), the subsystem must have initialized the
360group via the usual group _init() functions, and it must also have
361initialized the mutex.
362
363When the register call returns, the subsystem is live, and it
364will be visible via configfs.  At that point, mkdir(2) can be called and
365the subsystem must be ready for it.
366
367An Example
368==========
369
370The best example of these basic concepts is the simple_children
371subsystem/group and the simple_child item in
372samples/configfs/configfs_sample.c. It shows a trivial object displaying
373and storing an attribute, and a simple group creating and destroying
374these children.
375
376Hierarchy Navigation and the Subsystem Mutex
377============================================
378
379There is an extra bonus that configfs provides.  The config_groups and
380config_items are arranged in a hierarchy due to the fact that they
381appear in a filesystem.  A subsystem is NEVER to touch the filesystem
382parts, but the subsystem might be interested in this hierarchy.  For
383this reason, the hierarchy is mirrored via the config_group->cg_children
384and config_item->ci_parent structure members.
385
386A subsystem can navigate the cg_children list and the ci_parent pointer
387to see the tree created by the subsystem.  This can race with configfs'
388management of the hierarchy, so configfs uses the subsystem mutex to
389protect modifications.  Whenever a subsystem wants to navigate the
390hierarchy, it must do so under the protection of the subsystem
391mutex.
392
393A subsystem will be prevented from acquiring the mutex while a newly
394allocated item has not been linked into this hierarchy.   Similarly, it
395will not be able to acquire the mutex while a dropping item has not
396yet been unlinked.  This means that an item's ci_parent pointer will
397never be NULL while the item is in configfs, and that an item will only
398be in its parent's cg_children list for the same duration.  This allows
399a subsystem to trust ci_parent and cg_children while they hold the
400mutex.
401
402Item Aggregation Via symlink(2)
403===============================
404
405configfs provides a simple group via the group->item parent/child
406relationship.  Often, however, a larger environment requires aggregation
407outside of the parent/child connection.  This is implemented via
408symlink(2).
409
410A config_item may provide the ct_item_ops->allow_link() and
411ct_item_ops->drop_link() methods.  If the ->allow_link() method exists,
412symlink(2) may be called with the config_item as the source of the link.
413These links are only allowed between configfs config_items.  Any
414symlink(2) attempt outside the configfs filesystem will be denied.
415
416When symlink(2) is called, the source config_item's ->allow_link()
417method is called with itself and a target item.  If the source item
418allows linking to target item, it returns 0.  A source item may wish to
419reject a link if it only wants links to a certain type of object (say,
420in its own subsystem).
421
422When unlink(2) is called on the symbolic link, the source item is
423notified via the ->drop_link() method.  Like the ->drop_item() method,
424this is a void function and cannot return failure.  The subsystem is
425responsible for responding to the change.
426
427A config_item cannot be removed while it links to any other item, nor
428can it be removed while an item links to it.  Dangling symlinks are not
429allowed in configfs.
430
431Automatically Created Subgroups
432===============================
433
434A new config_group may want to have two types of child config_items.
435While this could be codified by magic names in ->make_item(), it is much
436more explicit to have a method whereby userspace sees this divergence.
437
438Rather than have a group where some items behave differently than
439others, configfs provides a method whereby one or many subgroups are
440automatically created inside the parent at its creation.  Thus,
441mkdir("parent") results in "parent", "parent/subgroup1", up through
442"parent/subgroupN".  Items of type 1 can now be created in
443"parent/subgroup1", and items of type N can be created in
444"parent/subgroupN".
445
446These automatic subgroups, or default groups, do not preclude other
447children of the parent group.  If ct_group_ops->make_group() exists,
448other child groups can be created on the parent group directly.
449
450A configfs subsystem specifies default groups by adding them using the
451configfs_add_default_group() function to the parent config_group
452structure.  Each added group is populated in the configfs tree at the same
453time as the parent group.  Similarly, they are removed at the same time
454as the parent.  No extra notification is provided.  When a ->drop_item()
455method call notifies the subsystem the parent group is going away, it
456also means every default group child associated with that parent group.
457
458As a consequence of this, default groups cannot be removed directly via
459rmdir(2).  They also are not considered when rmdir(2) on the parent
460group is checking for children.
461
462Dependent Subsystems
463====================
464
465Sometimes other drivers depend on particular configfs items.  For
466example, ocfs2 mounts depend on a heartbeat region item.  If that
467region item is removed with rmdir(2), the ocfs2 mount must BUG or go
468readonly.  Not happy.
469
470configfs provides two additional API calls: configfs_depend_item() and
471configfs_undepend_item().  A client driver can call
472configfs_depend_item() on an existing item to tell configfs that it is
473depended on.  configfs will then return -EBUSY from rmdir(2) for that
474item.  When the item is no longer depended on, the client driver calls
475configfs_undepend_item() on it.
476
477These API cannot be called underneath any configfs callbacks, as
478they will conflict.  They can block and allocate.  A client driver
479probably shouldn't calling them of its own gumption.  Rather it should
480be providing an API that external subsystems call.
481
482How does this work?  Imagine the ocfs2 mount process.  When it mounts,
483it asks for a heartbeat region item.  This is done via a call into the
484heartbeat code.  Inside the heartbeat code, the region item is looked
485up.  Here, the heartbeat code calls configfs_depend_item().  If it
486succeeds, then heartbeat knows the region is safe to give to ocfs2.
487If it fails, it was being torn down anyway, and heartbeat can gracefully
488pass up an error.
489
490Committable Items
491=================
492
493Note:
494     Committable items are currently unimplemented.
495
496Some config_items cannot have a valid initial state.  That is, no
497default values can be specified for the item's attributes such that the
498item can do its work.  Userspace must configure one or more attributes,
499after which the subsystem can start whatever entity this item
500represents.
501
502Consider the FakeNBD device from above.  Without a target address *and*
503a target device, the subsystem has no idea what block device to import.
504The simple example assumes that the subsystem merely waits until all the
505appropriate attributes are configured, and then connects.  This will,
506indeed, work, but now every attribute store must check if the attributes
507are initialized.  Every attribute store must fire off the connection if
508that condition is met.
509
510Far better would be an explicit action notifying the subsystem that the
511config_item is ready to go.  More importantly, an explicit action allows
512the subsystem to provide feedback as to whether the attributes are
513initialized in a way that makes sense.  configfs provides this as
514committable items.
515
516configfs still uses only normal filesystem operations.  An item is
517committed via rename(2).  The item is moved from a directory where it
518can be modified to a directory where it cannot.
519
520Any group that provides the ct_group_ops->commit_item() method has
521committable items.  When this group appears in configfs, mkdir(2) will
522not work directly in the group.  Instead, the group will have two
523subdirectories: "live" and "pending".  The "live" directory does not
524support mkdir(2) or rmdir(2) either.  It only allows rename(2).  The
525"pending" directory does allow mkdir(2) and rmdir(2).  An item is
526created in the "pending" directory.  Its attributes can be modified at
527will.  Userspace commits the item by renaming it into the "live"
528directory.  At this point, the subsystem receives the ->commit_item()
529callback.  If all required attributes are filled to satisfaction, the
530method returns zero and the item is moved to the "live" directory.
531
532As rmdir(2) does not work in the "live" directory, an item must be
533shutdown, or "uncommitted".  Again, this is done via rename(2), this
534time from the "live" directory back to the "pending" one.  The subsystem
535is notified by the ct_group_ops->uncommit_object() method.
536