#
0b17c28a |
| 11-May-2018 |
Vivek Goyal <vgoyal@redhat.com> |
ovl: Treat metacopy dentries as type OVL_PATH_MERGE
Right now OVL_PATH_MERGE is used only for merged directories. But conceptually, a metacopy dentry (backed by a lower data dentry) is a merged ent
ovl: Treat metacopy dentries as type OVL_PATH_MERGE
Right now OVL_PATH_MERGE is used only for merged directories. But conceptually, a metacopy dentry (backed by a lower data dentry) is a merged entity as well.
So mark metacopy dentries as OVL_PATH_MERGE and ovl_rename() makes use of this property later to set redirect on a metacopy file.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
#
4823d49c |
| 11-May-2018 |
Vivek Goyal <vgoyal@redhat.com> |
ovl: Add helper ovl_inode_realdata()
Add an helper to retrieve real data inode associated with overlay inode. This helper will ignore all metacopy inodes and will return only the real inode which ha
ovl: Add helper ovl_inode_realdata()
Add an helper to retrieve real data inode associated with overlay inode. This helper will ignore all metacopy inodes and will return only the real inode which has data.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
#
2664bd08 |
| 11-May-2018 |
Vivek Goyal <vgoyal@redhat.com> |
ovl: Store lower data inode in ovl_inode
Right now ovl_inode stores inode pointer for lower inode. This helps with quickly getting lower inode given overlay inode (ovl_inode_lower()).
Now with met
ovl: Store lower data inode in ovl_inode
Right now ovl_inode stores inode pointer for lower inode. This helps with quickly getting lower inode given overlay inode (ovl_inode_lower()).
Now with metadata only copy-up, we can have metacopy inode in middle layer as well and inode containing data can be different from ->lower. I need to be able to open the real file in ovl_open_realfile() and for that I need to quickly find the lower data inode.
Hence store lower data inode also in ovl_inode. Also provide an helper ovl_inode_lowerdata() to access this field.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
#
67d756c2 |
| 11-May-2018 |
Vivek Goyal <vgoyal@redhat.com> |
ovl: Fix ovl_getattr() to get number of blocks from lower
If an inode has been copied up metadata only, then we need to query the number of blocks from lower and fill up the stat->st_blocks.
We nee
ovl: Fix ovl_getattr() to get number of blocks from lower
If an inode has been copied up metadata only, then we need to query the number of blocks from lower and fill up the stat->st_blocks.
We need to be careful about races where we are doing stat on one cpu and data copy up is taking place on other cpu. We want to return stat->st_blocks either from lower or stable upper and not something in between. Hence, ovl_has_upperdata() is called first to figure out whether block reporting will take place from lower or upper.
We now support metacopy dentries in middle layer. That means number of blocks reporting needs to come from lowest data dentry and this could be different from lower dentry. Hence we end up making a separate vfs_getxattr() call for metacopy dentries to get number of blocks.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
#
647d253f |
| 11-May-2018 |
Vivek Goyal <vgoyal@redhat.com> |
ovl: Add helper ovl_dentry_lowerdata() to get lower data dentry
Now we have the notion of data dentry and metacopy dentry. ovl_dentry_lower() will return uppermost lower dentry, but it could be eith
ovl: Add helper ovl_dentry_lowerdata() to get lower data dentry
Now we have the notion of data dentry and metacopy dentry. ovl_dentry_lower() will return uppermost lower dentry, but it could be either data or metacopy dentry. Now we support metacopy dentries in lower layers so it is possible that lowerstack[0] is metacopy dentry while lowerstack[1] is actual data dentry.
So add an helper which returns lowest most dentry which is supposed to be data dentry.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
#
4f93b426 |
| 11-May-2018 |
Vivek Goyal <vgoyal@redhat.com> |
ovl: Copy up meta inode data from lowest data inode
So far lower could not be a meta inode. So whenever it was time to copy up data of a meta inode, we could copy it up from top most lower dentry.
ovl: Copy up meta inode data from lowest data inode
So far lower could not be a meta inode. So whenever it was time to copy up data of a meta inode, we could copy it up from top most lower dentry.
But now lower itself can be a metacopy inode. That means data copy up needs to take place from a data inode in metacopy inode chain. Find lower data inode in the chain and use that for data copy up.
Introduced a helper called ovl_path_lowerdata() to find the lower data inode chain.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
#
9d3dfea3 |
| 11-May-2018 |
Vivek Goyal <vgoyal@redhat.com> |
ovl: Modify ovl_lookup() and friends to lookup metacopy dentry
This patch modifies ovl_lookup() and friends to lookup metacopy dentries. It also allows for presence of metacopy dentries in lower lay
ovl: Modify ovl_lookup() and friends to lookup metacopy dentry
This patch modifies ovl_lookup() and friends to lookup metacopy dentries. It also allows for presence of metacopy dentries in lower layer.
During lookup, check for presence of OVL_XATTR_METACOPY and if not present, set OVL_UPPERDATA bit in flags.
We don't support metacopy feature with nfs_export. So in nfs_export code, we set OVL_UPPERDATA flag set unconditionally if upper inode exists.
Do not follow metacopy origin if we find a metacopy only inode and metacopy feature is not enabled for that mount. Like redirect, this can have security implications where an attacker could hand craft upper and try to gain access to file on lower which it should not have to begin with.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
#
0c288874 |
| 11-May-2018 |
Vivek Goyal <vgoyal@redhat.com> |
ovl: A new xattr OVL_XATTR_METACOPY for file on upper
Now we will have the capability to have upper inodes which might be only metadata copy up and data is still on lower inode. So add a new xattr
ovl: A new xattr OVL_XATTR_METACOPY for file on upper
Now we will have the capability to have upper inodes which might be only metadata copy up and data is still on lower inode. So add a new xattr OVL_XATTR_METACOPY to distinguish between two cases.
Presence of OVL_XATTR_METACOPY reflects that file has been copied up metadata only and and data will be copied up later from lower origin. So this xattr is set when a metadata copy takes place and cleared when data copy takes place.
We also use a bit in ovl_inode->flags to cache OVL_UPPERDATA which reflects whether ovl inode has data or not (as opposed to metadata only copy up).
If a file is copied up metadata only and later when same file is opened for WRITE, then data copy up takes place. We copy up data, remove METACOPY xattr and then set the UPPERDATA flag in ovl_inode->flags. While all these operations happen with oi->lock held, read side of oi->flags can be lockless. That is another thread on another cpu can check if UPPERDATA flag is set or not.
So this gives us an ordering requirement w.r.t UPPERDATA flag. That is, if another cpu sees UPPERDATA flag set, then it should be guaranteed that effects of data copy up and remove xattr operations are also visible.
For example.
CPU1 CPU2 ovl_open() acquire(oi->lock) ovl_open_maybe_copy_up() ovl_copy_up_data() open_open_need_copy_up() vfs_removexattr() ovl_already_copied_up() ovl_dentry_needs_data_copy_up() ovl_set_flag(OVL_UPPERDATA) ovl_test_flag(OVL_UPPERDATA) release(oi->lock)
Say CPU2 is copying up data and in the end sets UPPERDATA flag. But if CPU1 perceives the effects of setting UPPERDATA flag but not the effects of preceding operations (ex. upper that is not fully copied up), it will be a problem.
Hence this patch introduces smp_wmb() on setting UPPERDATA flag operation and smp_rmb() on UPPERDATA flag test operation.
May be some other lock or barrier is already covering it. But I am not sure what that is and is it obvious enough that we will not break it in future.
So hence trying to be safe here and introducing barriers explicitly for UPPERDATA flag/bit.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
#
2002df85 |
| 11-May-2018 |
Vivek Goyal <vgoyal@redhat.com> |
ovl: Add helper ovl_already_copied_up()
There are couple of places where we need to know if file is already copied up (in lockless manner). Right now its open coded and there are only two condition
ovl: Add helper ovl_already_copied_up()
There are couple of places where we need to know if file is already copied up (in lockless manner). Right now its open coded and there are only two conditions to check. Soon this patch series will introduce another condition to check and Amir wants to introduce one more. So introduce a helper instead to check this so that code is easier to read.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com> Reviewed-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
#
4f357295 |
| 18-Jul-2018 |
Miklos Szeredi <mszeredi@redhat.com> |
ovl: copy up inode flags
On inode creation copy certain inode flags from the underlying real inode to the overlay inode.
This is in preparation for moving overlay functionality out of the VFS.
Sig
ovl: copy up inode flags
On inode creation copy certain inode flags from the underlying real inode to the overlay inode.
This is in preparation for moving overlay functionality out of the VFS.
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
#
d9854c87 |
| 18-Jul-2018 |
Miklos Szeredi <mszeredi@redhat.com> |
ovl: copy up times
Copy up mtime and ctime to overlay inode after times in real object are modified. Be careful not to dirty cachelines when not necessary.
This is in preparation for moving overla
ovl: copy up times
Copy up mtime and ctime to overlay inode after times in real object are modified. Be careful not to dirty cachelines when not necessary.
This is in preparation for moving overlay functionality out of the VFS.
This patch shouldn't have any observable effect.
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
Revision tags: v4.16, v4.15, v4.13.16, v4.14 |
|
#
e487d889 |
| 07-Nov-2017 |
Amir Goldstein <amir73il@gmail.com> |
ovl: constant st_ino for non-samefs with xino
On 64bit systems, when overlay layers are not all on the same fs, but all inode numbers of underlying fs are not using the high bits, use the high bits
ovl: constant st_ino for non-samefs with xino
On 64bit systems, when overlay layers are not all on the same fs, but all inode numbers of underlying fs are not using the high bits, use the high bits to partition the overlay st_ino address space. The high bits hold the fsid (upper fsid is 0). This way overlay inode numbers are unique and all inodes use overlay st_dev. Inode numbers are also persistent for a given layer configuration.
Currently, our only indication for available high ino bits is from a filesystem that supports file handles and uses the default encode_fh() operation, which encodes a 32bit inode number.
Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
#
5148626b |
| 28-Mar-2018 |
Amir Goldstein <amir73il@gmail.com> |
ovl: allocate anon bdev per unique lower fs
Instead of allocating an anonymous bdev per lower layer, allocate one anonymous bdev per every unique lower fs that is different than upper fs.
Every uni
ovl: allocate anon bdev per unique lower fs
Instead of allocating an anonymous bdev per lower layer, allocate one anonymous bdev per every unique lower fs that is different than upper fs.
Every unique lower fs is assigned an fsid > 0 and the number of unique lower fs are stored in ofs->numlowerfs.
The assigned fsid is stored in the lower layer struct and will be used also for inode number multiplexing.
Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
#
da309e8c |
| 08-Nov-2017 |
Amir Goldstein <amir73il@gmail.com> |
ovl: factor out ovl_map_dev_ino() helper
A helper for ovl_getattr() to map the values of st_dev and st_ino according to constant st_ino rules.
Signed-off-by: Amir Goldstein <amir73il@gmail.com> Sig
ovl: factor out ovl_map_dev_ino() helper
A helper for ovl_getattr() to map the values of st_dev and st_ino according to constant st_ino rules.
Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
#
695b46e7 |
| 15-Mar-2018 |
Amir Goldstein <amir73il@gmail.com> |
ovl: set i_ino to the value of st_ino for NFS export
Eddie Horng reported that readdir of an overlayfs directory that was exported via NFSv3 returns entries with d_type set to DT_UNKNOWN. The reason
ovl: set i_ino to the value of st_ino for NFS export
Eddie Horng reported that readdir of an overlayfs directory that was exported via NFSv3 returns entries with d_type set to DT_UNKNOWN. The reason is that while preparing the response for readdirplus, nfsd checks inside encode_entryplus_baggage() that a child dentry's inode number matches the value of d_ino returns by overlayfs readdir iterator.
Because the overlayfs inodes use arbitrary inode numbers that are not correlated with the values of st_ino/d_ino, NFSv3 falls back to not encoding d_type. Although this is an allowed behavior, we can fix it for the case of all overlayfs layers on the same underlying filesystem.
When NFS export is enabled and d_ino is consistent with st_ino (samefs), set the same value also to i_ino in ovl_fill_inode() for all overlayfs inodes, nfsd readdirplus sanity checks will pass. ovl_fill_inode() may be called from ovl_new_inode(), before real inode was created with ino arg 0. In that case, i_ino will be updated to real upper inode i_ino on ovl_inode_init() or ovl_inode_update().
Reported-by: Eddie Horng <eddiehorng.tw@gmail.com> Tested-by: Eddie Horng <eddiehorng.tw@gmail.com> Signed-off-by: Amir Goldstein <amir73il@gmail.com> Fixes: 8383f1748829 ("ovl: wire up NFS export operations") Cc: <stable@vger.kernel.org> #v4.16 Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
#
c62520a8 |
| 14-Jan-2018 |
Amir Goldstein <amir73il@gmail.com> |
ovl: store 'has_upper' and 'opaque' as bit flags
We need to make some room in struct ovl_entry to store information about redirected ancestors for NFS export, so cram two booleans as bit flags.
Sig
ovl: store 'has_upper' and 'opaque' as bit flags
We need to make some room in struct ovl_entry to store information about redirected ancestors for NFS export, so cram two booleans as bit flags.
Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
#
aa3ff3c1 |
| 15-Oct-2017 |
Amir Goldstein <amir73il@gmail.com> |
ovl: copy up of disconnected dentries
With NFS export, some operations on decoded file handles (e.g. open, link, setattr, xattr_set) may call copy up with a disconnected non-dir. In this case, we wi
ovl: copy up of disconnected dentries
With NFS export, some operations on decoded file handles (e.g. open, link, setattr, xattr_set) may call copy up with a disconnected non-dir. In this case, we will copy up lower inode to index dir without linking it to upper dir.
Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
#
e7dd0e71 |
| 24-Oct-2017 |
Amir Goldstein <amir73il@gmail.com> |
ovl: whiteout index when union nlink drops to zero
With NFS export feature enabled, when overlay inode nlink drops to zero, instead of removing the index entry, replace it with a whiteout index entr
ovl: whiteout index when union nlink drops to zero
With NFS export feature enabled, when overlay inode nlink drops to zero, instead of removing the index entry, replace it with a whiteout index entry.
This is needed for NFS export in order to prevent future open by handle from opening the lower file directly.
Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
Revision tags: v4.13.5 |
|
#
89a17556 |
| 25-Sep-2017 |
Amir Goldstein <amir73il@gmail.com> |
ovl: cleanup dir index when dir nlink drops to zero
When non-dir index union nlink drops to zero the non-dir index is cleaned. Do the same for directory type index entries when union directory is re
ovl: cleanup dir index when dir nlink drops to zero
When non-dir index union nlink drops to zero the non-dir index is cleaned. Do the same for directory type index entries when union directory is removed.
Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
#
016b720f |
| 11-Jan-2018 |
Amir Goldstein <amir73il@gmail.com> |
ovl: index directories on copy up for NFS export
With the NFS export feature enabled, all dirs are indexed on copy up. Non-dir files are copied up directly to indexdir and then hardlinked to upper d
ovl: index directories on copy up for NFS export
With the NFS export feature enabled, all dirs are indexed on copy up. Non-dir files are copied up directly to indexdir and then hardlinked to upper dir.
Directories are copied up to indexdir, then an index entry is created in indexdir with 'upper' xattr pointing to the copied up dir and then the copied up dir is moved to upper dir.
Directory index is also used for consistency verification, like detecting multiple redirected dirs to the same lower dir on lookup.
Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
#
fbd2d207 |
| 21-Nov-2017 |
Amir Goldstein <amir73il@gmail.com> |
ovl: index all non-dir on copy up for NFS export
With the NFS export feature enabled, all non-dir are indexed on copy up. The copy up origin inode of an indexed non-dir can be used as a unique ident
ovl: index all non-dir on copy up for NFS export
With the NFS export feature enabled, all non-dir are indexed on copy up. The copy up origin inode of an indexed non-dir can be used as a unique identifier of the overlay object.
The full index is also used for consistency verfication, like detecting multiple non-hardlink uppers with the same 'origin' on lookup.
Directory index on copy up will be implemented by following patch.
Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
#
24b33ee1 |
| 25-Sep-2017 |
Amir Goldstein <amir73il@gmail.com> |
ovl: create ovl_need_index() helper
The helper determines which lower file needs to be indexed on copy up and before nlink changes.
For index=on, the helper evaluates to true for lower hardlinks.
ovl: create ovl_need_index() helper
The helper determines which lower file needs to be indexed on copy up and before nlink changes.
For index=on, the helper evaluates to true for lower hardlinks.
Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
#
f168f109 |
| 19-Jan-2018 |
Amir Goldstein <amir73il@gmail.com> |
ovl: add support for "nfs_export" configuration
Introduce the "nfs_export" config, module and mount options.
The NFS export feature depends on the "index" feature and enables two implicit overlayfs
ovl: add support for "nfs_export" configuration
Introduce the "nfs_export" config, module and mount options.
The NFS export feature depends on the "index" feature and enables two implicit overlayfs features: "index_all" and "verify_lower". The "index_all" feature creates an index on copy up of every file and directory. The "verify_lower" feature uses the full index to detect overlay filesystems inconsistencies on lookup, like redirect from multiple upper dirs to the same lower dir.
NFS export can be enabled for non-upper mount with no index. However, because lower layer redirects cannot be verified with the index, enabling NFS export support on an overlay with no upper layer requires turning off redirect follow (e.g. "redirect_dir=nofollow").
The full index may incur some overhead on mount time, especially when verifying that lower directory file handles are not stale.
NFS export support, full index and consistency verification will be implemented by following patches.
Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|
#
31747eda |
| 14-Jan-2018 |
Amir Goldstein <amir73il@gmail.com> |
ovl: hash directory inodes for fsnotify
fsnotify pins a watched directory inode in cache, but if directory dentry is released, new lookup will allocate a new dentry and a new inode. Directory events
ovl: hash directory inodes for fsnotify
fsnotify pins a watched directory inode in cache, but if directory dentry is released, new lookup will allocate a new dentry and a new inode. Directory events will be notified on the new inode, while fsnotify listener is watching the old pinned inode.
Hash all directory inodes to reuse the pinned inode on lookup. Pure upper dirs are hashes by real upper inode, merge and lower dirs are hashed by real lower inode.
The reference to lower inode was being held by the lower dentry object in the overlay dentry (oe->lowerstack[0]). Releasing the overlay dentry may drop lower inode refcount to zero. Add a refcount on behalf of the overlay inode to prevent that.
As a by-product, hashing directory inodes also detects multiple redirected dirs to the same lower dir and uncovered redirected dir target on and returns -ESTALE on lookup.
The reported issue dates back to initial version of overlayfs, but this patch depends on ovl_inode code that was introduced in kernel v4.13.
Cc: <stable@vger.kernel.org> #v4.13 Reported-by: Niklas Cassel <niklas.cassel@axis.com> Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com> Tested-by: Niklas Cassel <niklas.cassel@axis.com>
show more ...
|
Revision tags: v4.13 |
|
#
b9343632 |
| 24-Jul-2017 |
Chandan Rajendra <chandan@linux.vnet.ibm.com> |
ovl: re-structure overlay lower layers in-memory
Define new structures to represent overlay instance lower layers and overlay merge dir lower layers to make room for storing more per layer informati
ovl: re-structure overlay lower layers in-memory
Define new structures to represent overlay instance lower layers and overlay merge dir lower layers to make room for storing more per layer information in-memory.
Instead of keeping the fs instance lower layers in an array of struct vfsmount, keep them in an array of new struct ovl_layer, that has a pointer to struct vfsmount.
Instead of keeping the dentry lower layers in an array of struct path, keep them in an array of new struct ovl_path, that has a pointer to struct dentry and to struct ovl_layer.
Add a small helper to find the fs layer id that correspopnds to a lower struct ovl_path and use it in ovl_lookup().
[amir: split re-structure from anonymous bdev patch]
Signed-off-by: Chandan Rajendra <chandan@linux.vnet.ibm.com> Signed-off-by: Amir Goldstein <amir73il@gmail.com> Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
show more ...
|