Revision tags: v5.15.45 |
|
#
03ab8e62 |
| 31-May-2022 |
Konstantin Komarov <almaz.alexandrovich@paragon-software.com> |
Merge tag 'v5.18'
Linux 5.18
|
Revision tags: v5.15.44, v5.15.43, v5.15.42 |
|
#
65965d95 |
| 24-May-2022 |
Linus Torvalds <torvalds@linux-foundation.org> |
Merge tag 'erofs-for-5.19-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs
Pull erofs (and fscache) updates from Gao Xiang: "After working on it on the mailing list for more than h
Merge tag 'erofs-for-5.19-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs
Pull erofs (and fscache) updates from Gao Xiang: "After working on it on the mailing list for more than half a year, we finally form 'erofs over fscache' feature into shape. Hopefully it could bring more possibility to the communities.
The story mainly started from a new project what we called "RAFS v6" [1] for Nydus image service almost a year ago, which enhances EROFS to be a new form of one bootstrap (which includes metadata representing the whole fs tree) + several data-deduplicated content addressable blobs (actually treated as multiple devices). Each blob can represent one container image layer but not quite exactly since all new data can be fully existed in the previous blobs so no need to introduce another new blob.
It is actually not a new idea (at least on my side it's much like a simpilied casync [2] for now) and has many benefits over per-file blobs or some other exist ways since typically each RAFS v6 image only has dozens of device blobs instead of thousands of per-file blobs. It's easy to be signed with user keys as a golden image, transfered untouchedly with minimal overhead over the network, kept in some type of storage conveniently, and run with (optional) runtime verification but without involving too many irrelevant features crossing the system beyond EROFS itself. At least it's our final goal and we're keeping working on it. There was also a good summary of this approach from the casync author [3].
Regardless further optimizations, this work is almost done in the previous Linux release cycles. In this round, we'd like to introduce on-demand load for EROFS with the fscache/cachefiles infrastructure, considering the following advantages:
- Introduce new file-based backend to EROFS. Although each image only contains dozens of blobs but in densely-deployed runC host for example, there could still be massive blobs on a machine, which is messy if each blob is treated as a device. In contrast, fscache and cachefiles are really great interfaces for us to make them work.
- Introduce on-demand load to fscache and EROFS. Previously, fscache is mainly used to caching network-likewise filesystems, now it can support on-demand downloading for local fses too with the exact localfs on-disk format. It has many advantages which we're been described in the latest patchset cover letter [4]. In addition to that, most importantly, the cached data is still stored in the original local fs on-disk format so that it's still the one signed with private keys but only could be partially available. Users can fully trust it during running. Later, users can also back up cachefiles easily to another machine.
- More reliable on-demand approach in principle. After data is all available locally, user daemon can be no longer online in some use cases, which helps daemon crash recovery (filesystems can still in service) and hot-upgrade (user daemon can be upgraded more frequently due to new features or protocols introduced.)
- Other format can also be converted to EROFS filesystem format over the internet on the fly with the new on-demand load feature and mounted. That is entirely possible with on-demand load feature as long as such archive format metadata can be fetched in advance like stargz.
In addition, although currently our target user is Nydus image service [5], but laterly, it can be used for other use cases like on-demand system booting, etc. As for the fscache on-demand load feature itself, strictly it can be used for other local fses too. Laterly we could promote most code to the iomap infrastructure and also enhance it in the read-write way if other local fses are interested.
Thanks David Howells for taking so much time and patience on this these months, many thanks with great respect here again! Thanks Jeffle for working on this feature and Xin Yin from Bytedance for asynchronous I/O implementation as well as Zichen Tian, Jia Zhu, and Yan Song for testing, much appeciated. We're also exploring more possibly over fscache cache management over FSDAX for secure containers and working on more improvements and useful features for fscache, cachefiles, and on-demand load.
In addition to "erofs over fscache", NFS export and idmapped mount are also completed in this cycle for container use cases as well.
Summary:
- Add erofs on-demand load support over fscache
- Support NFS export for erofs
- Support idmapped mounts for erofs
- Don't prompt for risk any more when using big pcluster
- Fix buffer copy overflow of ztailpacking feature
- Several minor cleanups"
[1] https://lore.kernel.org/r/20210730194625.93856-1-hsiangkao@linux.alibaba.com [2] https://github.com/systemd/casync [3] http://0pointer.net/blog/casync-a-tool-for-distributing-file-system-images.html [4] https://lore.kernel.org/r/20220509074028.74954-1-jefflexu@linux.alibaba.com [5] https://github.com/dragonflyoss/image-service
* tag 'erofs-for-5.19-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs: (29 commits) erofs: scan devices from device table erofs: change to use asynchronous io for fscache readpage/readahead erofs: add 'fsid' mount option erofs: implement fscache-based data readahead erofs: implement fscache-based data read for inline layout erofs: implement fscache-based data read for non-inline layout erofs: implement fscache-based metadata read erofs: register fscache context for extra data blobs erofs: register fscache context for primary data blob erofs: add erofs_fscache_read_folios() helper erofs: add anonymous inode caching metadata for data blobs erofs: add fscache context helper functions erofs: register fscache volume erofs: add fscache mode check helper erofs: make erofs_map_blocks() generally available cachefiles: document on-demand read mode cachefiles: add tracepoints for on-demand read mode cachefiles: enable on-demand read mode cachefiles: implement on-demand read cachefiles: notify the user daemon when withdrawing cookie ...
show more ...
|
Revision tags: v5.18, v5.15.41, v5.15.40, v5.15.39, v5.15.38, v5.15.37, v5.15.36 |
|
#
c6be2bd0 |
| 25-Apr-2022 |
Jeffle Xu <jefflexu@linux.alibaba.com> |
erofs: register fscache volume
A new fscache based mode is going to be introduced for erofs, in which case on-demand read semantics is implemented through fscache.
As the first step, register fscac
erofs: register fscache volume
A new fscache based mode is going to be introduced for erofs, in which case on-demand read semantics is implemented through fscache.
As the first step, register fscache volume for each erofs filesystem. That means, data blobs can not be shared among erofs filesystems. In the following iteration, we are going to introduce the domain semantics, in which case several erofs filesystems can belong to one domain, and data blobs can be shared among these erofs filesystems of one domain.
Signed-off-by: Jeffle Xu <jefflexu@linux.alibaba.com> Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com> Link: https://lore.kernel.org/r/20220425122143.56815-12-jefflexu@linux.alibaba.com Acked-by: Chao Yu <chao@kernel.org> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
show more ...
|
Revision tags: v5.15.35, v5.15.34, v5.15.33, v5.15.32, v5.15.31, v5.17, v5.15.30, v5.15.29, v5.15.28, v5.15.27, v5.15.26 |
|
#
1136fa0c |
| 01-Mar-2022 |
Dmitry Torokhov <dmitry.torokhov@gmail.com> |
Merge tag 'v5.17-rc4' into for-linus
Merge with mainline to get the Intel ASoC generic helpers header and other changes.
|
Revision tags: v5.15.25, v5.15.24, v5.15.23, v5.15.22, v5.15.21, v5.15.20, v5.15.19, v5.15.18, v5.15.17, v5.4.173, v5.15.16 |
|
#
87a0b2fa |
| 17-Jan-2022 |
Dmitry Torokhov <dmitry.torokhov@gmail.com> |
Merge tag 'v5.16' into next
Sync up with mainline to bring in the latest API changes.
|
Revision tags: v5.15.15 |
|
#
762f99f4 |
| 15-Jan-2022 |
Dmitry Torokhov <dmitry.torokhov@gmail.com> |
Merge branch 'next' into for-linus
Prepare input updates for 5.17 merge window.
|
Revision tags: v5.16 |
|
#
f81483aa |
| 05-Jan-2022 |
Takashi Iwai <tiwai@suse.de> |
Merge branch 'for-next' into for-linus
Pull 5.17 materials.
Signed-off-by: Takashi Iwai <tiwai@suse.de>
|
#
17580470 |
| 17-Dec-2021 |
Thomas Zimmermann <tzimmermann@suse.de> |
Merge drm/drm-next into drm-misc-next-fixes
Backmerging to bring drm-misc-next-fixes up to the latest state for the current release cycle.
Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
|
Revision tags: v5.15.10, v5.15.9, v5.15.8 |
|
#
86329873 |
| 09-Dec-2021 |
Daniel Lezcano <daniel.lezcano@linaro.org> |
Merge branch 'reset/of-get-optional-exclusive' of git://git.pengutronix.de/pza/linux into timers/drivers/next
"Add optional variant of of_reset_control_get_exclusive(). If the requested reset is not
Merge branch 'reset/of-get-optional-exclusive' of git://git.pengutronix.de/pza/linux into timers/drivers/next
"Add optional variant of of_reset_control_get_exclusive(). If the requested reset is not specified in the device tree, this function returns NULL instead of an error."
This dependency is needed for the Generic Timer Module (a.k.a OSTM) support for RZ/G2L.
Signed-off-by: Daniel Lezcano <daniel.lezcano@linaro.org>
show more ...
|
#
5d8dfaa7 |
| 09-Dec-2021 |
Dmitry Torokhov <dmitry.torokhov@gmail.com> |
Merge tag 'v5.15' into next
Sync up with the mainline to get the latest APIs and DT bindings.
|
Revision tags: v5.15.7, v5.15.6, v5.15.5 |
|
#
448cc2fb |
| 22-Nov-2021 |
Jani Nikula <jani.nikula@intel.com> |
Merge drm/drm-next into drm-intel-next
Sync up with drm-next to get v5.16-rc2.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
|
#
8626afb1 |
| 22-Nov-2021 |
Tvrtko Ursulin <tvrtko.ursulin@intel.com> |
Merge drm/drm-next into drm-intel-gt-next
Thomas needs the dma_resv_for_each_fence API for i915/ttm async migration work.
Signed-off-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
|
Revision tags: v5.15.4, v5.15.3 |
|
#
a713ca23 |
| 18-Nov-2021 |
Thomas Zimmermann <tzimmermann@suse.de> |
Merge drm/drm-next into drm-misc-next
Backmerging from drm/drm-next for v5.16-rc1.
Signed-off-by: Thomas Zimmermann <tzimmermann@suse.de>
|
#
467dd91e |
| 16-Nov-2021 |
Maxime Ripard <maxime@cerno.tech> |
Merge drm/drm-fixes into drm-misc-fixes
We need -rc1 to address a breakage in drm/scheduler affecting panfrost.
Signed-off-by: Maxime Ripard <maxime@cerno.tech>
|
Revision tags: v5.15.2 |
|
#
7f9f8792 |
| 06-Nov-2021 |
Arnaldo Carvalho de Melo <acme@redhat.com> |
Merge remote-tracking branch 'torvalds/master' into perf/core
To pick up some tools/perf/ patches that went via tip/perf/core, such as:
tools/perf: Add mem_hops field in perf_mem_data_src structu
Merge remote-tracking branch 'torvalds/master' into perf/core
To pick up some tools/perf/ patches that went via tip/perf/core, such as:
tools/perf: Add mem_hops field in perf_mem_data_src structure
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
show more ...
|
Revision tags: v5.15.1 |
|
#
40e64a88 |
| 02-Nov-2021 |
Petr Mladek <pmladek@suse.com> |
Merge branch 'for-5.16-vsprintf-pgp' into for-linus
|
#
67a135b8 |
| 01-Nov-2021 |
Linus Torvalds <torvalds@linux-foundation.org> |
Merge tag 'erofs-for-5.16-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs
Pull erofs updates from Gao Xiang: "There are some new features available for this cycle. Firstly, EROFS
Merge tag 'erofs-for-5.16-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs
Pull erofs updates from Gao Xiang: "There are some new features available for this cycle. Firstly, EROFS LZMA algorithm support, specifically called MicroLZMA, is available as an option for embedded devices, LiveCDs and/or as the secondary auxiliary compression algorithm besides the primary algorithm in one file.
In order to better support the LZMA fixed-sized output compression, especially for 4KiB pcluster size (which has lowest memory pressure thus useful for memory-sensitive scenarios), Lasse introduced a new LZMA header/container format called MicroLZMA to minimize the original LZMA1 header (for example, we don't need to waste 4-byte dictionary size and another 8-byte uncompressed size, which can be calculated by fs directly, for each pcluster) and enable EROFS fixed-sized output compression.
Note that MicroLZMA can also be later used by other things in addition to EROFS too where wasting minimal amount of space for headers is important and it can be only compiled by enabling XZ_DEC_MICROLZMA. MicroLZMA has been supported by the latest upstream XZ embedded [1] & XZ utils [2], apply the latest related XZ embedded upstream patches by the XZ author Lasse here.
Secondly, multiple device is also supported in this cycle, which is designed for multi-layer container images. By working together with inter-layer data deduplication and compression, we can achieve the next high-performance container image solution. Our team will announce the new Nydus container image service [3] implementation with new RAFS v6 (EROFS-compatible) format in Open Source Summit 2021 China [4] soon.
Besides, the secondary compression head support and readmore decompression strategy are also included in this cycle. There are also some minor bugfixes and cleanups, as always.
Summary:
- support multiple devices for multi-layer container images;
- support the secondary compression head;
- support readmore decompression strategy;
- support new LZMA algorithm (specifically called MicroLZMA);
- some bugfixes & cleanups"
* tag 'erofs-for-5.16-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs: erofs: don't trigger WARN() when decompression fails erofs: get rid of ->lru usage erofs: lzma compression support erofs: rename some generic methods in decompressor lib/xz, lib/decompress_unxz.c: Fix spelling in comments lib/xz: Add MicroLZMA decoder lib/xz: Move s->lzma.len = 0 initialization to lzma_reset() lib/xz: Validate the value before assigning it to an enum variable lib/xz: Avoid overlapping memcpy() with invalid input with in-place decompression erofs: introduce readmore decompression strategy erofs: introduce the secondary compression head erofs: get compression algorithms directly on mapping erofs: add multiple device support erofs: decouple basic mount options from fs_context erofs: remove the fast path of per-CPU buffer decompression
show more ...
|
Revision tags: v5.15, v5.14.14, v5.14.13, v5.14.12 |
|
#
622ceadd |
| 10-Oct-2021 |
Gao Xiang <hsiangkao@linux.alibaba.com> |
erofs: lzma compression support
Add MicroLZMA support in order to maximize compression ratios for specific scenarios. For example, it's useful for low-end embedded boards and as a secondary algorith
erofs: lzma compression support
Add MicroLZMA support in order to maximize compression ratios for specific scenarios. For example, it's useful for low-end embedded boards and as a secondary algorithm in a file for specific access patterns.
MicroLZMA is a new container format for raw LZMA1, which was created by Lasse Collin aiming to minimize old LZMA headers and get rid of unnecessary EOPM (end of payload marker) as well as to enable fixed-sized output compression, especially for 4KiB pclusters.
Similar to LZ4, inplace I/O approach is used to minimize runtime memory footprint when dealing with I/O. Overlapped decompression is handled with 1) bounced buffer for data under processing or 2) extra short-lived pages from the on-stack pagepool which will be shared in the same read request (128KiB for example).
Link: https://lore.kernel.org/r/20211010213145.17462-8-xiang@kernel.org Acked-by: Chao Yu <chao@kernel.org> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
show more ...
|
#
dfeab2e9 |
| 14-Oct-2021 |
Gao Xiang <hsiangkao@linux.alibaba.com> |
erofs: add multiple device support
In order to support multi-layer container images, add multiple device feature to EROFS. Two ways are available to use for now:
- Devices can be mapped into 32-bi
erofs: add multiple device support
In order to support multi-layer container images, add multiple device feature to EROFS. Two ways are available to use for now:
- Devices can be mapped into 32-bit global block address space; - Device ID can be specified with the chunk indexes format.
Note that it assumes no extent would cross device boundary and mkfs should take care of it seriously.
In the future, a dedicated device manager could be introduced then thus extra devices can be automatically scanned by UUID as well.
Link: https://lore.kernel.org/r/20211014081010.43485-1-hsiangkao@linux.alibaba.com Reviewed-by: Chao Yu <chao@kernel.org> Reviewed-by: Liu Bo <bo.liu@linux.alibaba.com> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
show more ...
|
Revision tags: v5.14.11, v5.14.10 |
|
#
e700ac21 |
| 05-Oct-2021 |
Tony Lindgren <tony@atomide.com> |
Merge branch 'pruss-fix' into fixes
Merge in a fix for pruss reset issue caused by enabling pruss for am335x.
|
Revision tags: v5.14.9, v5.14.8, v5.14.7 |
|
#
ffb1e76f |
| 20-Sep-2021 |
Mark Brown <broonie@kernel.org> |
Merge tag 'v5.15-rc2' into spi-5.15
Linux 5.15-rc2
|
Revision tags: v5.14.6, v5.10.67, v5.10.66, v5.14.5 |
|
#
d1b803f4 |
| 15-Sep-2021 |
Rodrigo Vivi <rodrigo.vivi@intel.com> |
Merge drm/drm-next into drm-intel-next
Catch-up on 5.15-rc1 and sync with drm-intel-gt-next to prepare the PXP topic branch.
Signed-off-by: Rodrigo Vivi <rodrigo.vivi@intel.com>
|
#
d5dd580d |
| 15-Sep-2021 |
Joonas Lahtinen <joonas.lahtinen@linux.intel.com> |
Merge drm/drm-next into drm-intel-gt-next
Close the divergence which has caused patches not to apply and have a solid baseline for the PXP patches that Rodrigo will send a topic branch PR for.
Sign
Merge drm/drm-next into drm-intel-gt-next
Close the divergence which has caused patches not to apply and have a solid baseline for the PXP patches that Rodrigo will send a topic branch PR for.
Signed-off-by: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
show more ...
|
Revision tags: v5.14.4, v5.10.65 |
|
#
2f765205 |
| 14-Sep-2021 |
Maxime Ripard <maxime@cerno.tech> |
Merge drm/drm-next into drm-misc-next
Kickstart new drm-misc-next cycle.
Signed-off-by: Maxime Ripard <maxime@cerno.tech>
|
Revision tags: v5.14.3, v5.10.64 |
|
#
c2f4954c |
| 10-Sep-2021 |
Thomas Gleixner <tglx@linutronix.de> |
Merge branch 'linus' into smp/urgent
Ensure that all usage sites of get/put_online_cpus() except for the struggler in drivers/thermal are gone. So the last user and the deprecated inlines can be rem
Merge branch 'linus' into smp/urgent
Ensure that all usage sites of get/put_online_cpus() except for the struggler in drivers/thermal are gone. So the last user and the deprecated inlines can be removed.
show more ...
|