Revision tags: v6.6.25, v6.6.24, v6.6.23, v6.6.16, v6.6.15, v6.6.14, v6.6.13, v6.6.12, v6.6.11, v6.6.10, v6.6.9, v6.6.8, v6.6.7, v6.6.6, v6.6.5 |
|
#
f36d200a |
| 05-Dec-2023 |
Gao Xiang <hsiangkao@linux.alibaba.com> |
erofs: fix lz4 inplace decompression
commit 3c12466b6b7bf1e56f9b32c366a3d83d87afb4de upstream.
Currently EROFS can map another compressed buffer for inplace decompression, that was used to handle t
erofs: fix lz4 inplace decompression
commit 3c12466b6b7bf1e56f9b32c366a3d83d87afb4de upstream.
Currently EROFS can map another compressed buffer for inplace decompression, that was used to handle the cases that some pages of compressed data are actually not in-place I/O.
However, like most simple LZ77 algorithms, LZ4 expects the compressed data is arranged at the end of the decompressed buffer and it explicitly uses memmove() to handle overlapping: __________________________________________________________ |_ direction of decompression --> ____ |_ compressed data _|
Although EROFS arranges compressed data like this, it typically maps two individual virtual buffers so the relative order is uncertain. Previously, it was hardly observed since LZ4 only uses memmove() for short overlapped literals and x86/arm64 memmove implementations seem to completely cover it up and they don't have this issue. Juhyung reported that EROFS data corruption can be found on a new Intel x86 processor. After some analysis, it seems that recent x86 processors with the new FSRM feature expose this issue with "rep movsb".
Let's strictly use the decompressed buffer for lz4 inplace decompression for now. Later, as an useful improvement, we could try to tie up these two buffers together in the correct order.
Reported-and-tested-by: Juhyung Park <qkrwngud825@gmail.com> Closes: https://lore.kernel.org/r/CAD14+f2AVKf8Fa2OO1aAUdDNTDsVzzR6ctU_oJSmTyd6zSYR2Q@mail.gmail.com Fixes: 0ffd71bcc3a0 ("staging: erofs: introduce LZ4 decompression inplace") Fixes: 598162d05080 ("erofs: support decompress big pcluster for lz4 backend") Cc: stable <stable@vger.kernel.org> # 5.4+ Tested-by: Yifan Zhao <zhaoyifan@sjtu.edu.cn> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com> Link: https://lore.kernel.org/r/20231206045534.3920847-1-hsiangkao@linux.alibaba.com Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
show more ...
|
#
823ba1d2 |
| 13-Jan-2024 |
Gao Xiang <hsiangkao@linux.alibaba.com> |
erofs: fix inconsistent per-file compression format
[ Upstream commit 118a8cf504d7dfa519562d000f423ee3ca75d2c4 ]
EROFS can select compression algorithms on a per-file basis, and each per-file compr
erofs: fix inconsistent per-file compression format
[ Upstream commit 118a8cf504d7dfa519562d000f423ee3ca75d2c4 ]
EROFS can select compression algorithms on a per-file basis, and each per-file compression algorithm needs to be marked in the on-disk superblock for initialization.
However, syzkaller can generate inconsistent crafted images that use an unsupported algorithmtype for specific inodes, e.g. use MicroLZMA algorithmtype even it's not set in `sbi->available_compr_algs`. This can lead to an unexpected "BUG: kernel NULL pointer dereference" if the corresponding decompressor isn't built-in.
Fix this by checking against `sbi->available_compr_algs` for each m_algorithmformat request. Incorrect !erofs_sb_has_compr_cfgs preset bitmap is now fixed together since it was harmless previously.
Reported-by: <bugreport@ubisectech.com> Fixes: 8f89926290c4 ("erofs: get compression algorithms directly on mapping") Fixes: 622ceaddb764 ("erofs: lzma compression support") Reviewed-by: Yue Hu <huyue2@coolpad.com> Link: https://lore.kernel.org/r/20240113150602.1471050-1-hsiangkao@linux.alibaba.com Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
show more ...
|
Revision tags: v6.6.4, v6.6.3, v6.6.2, v6.5.11, v6.6.1, v6.5.10, v6.6, v6.5.9 |
|
#
586814ed |
| 22-Oct-2023 |
Gao Xiang <hsiangkao@linux.alibaba.com> |
erofs: simplify compression configuration parser
[ Upstream commit efb4fb02cef3ab410b603c8f0e1c67f61d55f542 ]
Move erofs_load_compr_cfgs() into decompressor.c as well as introduce a callback instea
erofs: simplify compression configuration parser
[ Upstream commit efb4fb02cef3ab410b603c8f0e1c67f61d55f542 ]
Move erofs_load_compr_cfgs() into decompressor.c as well as introduce a callback instead of a hard-coded switch for each algorithm for simplicity.
Reviewed-by: Chao Yu <chao@kernel.org> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com> Link: https://lore.kernel.org/r/20231022130957.11398-1-xiang@kernel.org Stable-dep-of: 118a8cf504d7 ("erofs: fix inconsistent per-file compression format") Signed-off-by: Sasha Levin <sashal@kernel.org>
show more ...
|
Revision tags: v6.5.8, v6.5.7, v6.5.6, v6.5.5, v6.5.4, v6.5.3, v6.5.2, v6.1.51, v6.5.1, v6.1.50, v6.5, v6.1.49, v6.1.48, v6.1.46, v6.1.45 |
|
#
ffa09b3b |
| 10-Aug-2023 |
Gao Xiang <hsiangkao@linux.alibaba.com> |
erofs: DEFLATE compression support
Add DEFLATE compression as the 3rd supported algorithm.
DEFLATE is a popular generic-purpose compression algorithm for quite long time (many advanced formats like
erofs: DEFLATE compression support
Add DEFLATE compression as the 3rd supported algorithm.
DEFLATE is a popular generic-purpose compression algorithm for quite long time (many advanced formats like gzip, zlib, zip, png are all based on that) as Apple documentation written "If you require interoperability with non-Apple devices, use COMPRESSION_ZLIB. [1]".
Due to its popularity, there are several hardware on-market DEFLATE accelerators, such as (s390) DFLTCC, (Intel) IAA/QAT, (HiSilicon) ZIP accelerator, etc. In addition, there are also several high-performence IP cores and even open-source FPGA approches available for DEFLATE. Therefore, it's useful to support DEFLATE compression in order to find a way to utilize these accelerators for asynchronous I/Os and get benefits from these later.
Besides, it's a good choice to trade off between compression ratios and performance compared to LZ4 and LZMA. The DEFLATE core format is simple as well as easy to understand, therefore the code size of its decompressor is small even for the bootloader use cases. The runtime memory consumption is quite limited too (e.g. 32K + ~7K for each zlib stream). As usual, EROFS ourperforms similar approaches too.
Alternatively, DEFLATE could still be used for some specific files since EROFS supports multiple compression algorithms in one image.
[1] https://developer.apple.com/documentation/compression/compression_algorithm Reviewed-by: Chao Yu <chao@kernel.org> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com> Link: https://lore.kernel.org/r/20230810154859.118330-1-hsiangkao@linux.alibaba.com
show more ...
|
Revision tags: v6.1.44, v6.1.43, v6.1.42, v6.1.41, v6.1.40, v6.1.39, v6.1.38, v6.1.37, v6.1.36 |
|
#
c5539762 |
| 27-Jun-2023 |
Gao Xiang <hsiangkao@linux.alibaba.com> |
erofs: simplify z_erofs_transform_plain()
Use memcpy_to_page() instead of open-coding them.
In addition, add a missing flush_dcache_page() even though almost all modern architectures clear `PG_dcac
erofs: simplify z_erofs_transform_plain()
Use memcpy_to_page() instead of open-coding them.
In addition, add a missing flush_dcache_page() even though almost all modern architectures clear `PG_dcache_clean` flag for new file cache pages so that it doesn't change anything in practice.
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com> Reviewed-by: Yue Hu <huyue2@coolpad.com> Reviewed-by: Chao Yu <chao@kernel.org> Link: https://lore.kernel.org/r/20230627161240.331-2-hsiangkao@linux.alibaba.com Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
show more ...
|
#
123ec246 |
| 27-Jun-2023 |
Gao Xiang <hsiangkao@linux.alibaba.com> |
erofs: get rid of the remaining kmap_atomic()
It's unnecessary to use kmap_atomic() compared with kmap_local_page(). In addition, kmap_atomic() is deprecated now.
Signed-off-by: Gao Xiang <hsiangka
erofs: get rid of the remaining kmap_atomic()
It's unnecessary to use kmap_atomic() compared with kmap_local_page(). In addition, kmap_atomic() is deprecated now.
Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com> Reviewed-by: Yue Hu <huyue2@coolpad.com> Reviewed-by: Chao Yu <chao@kernel.org> Link: https://lore.kernel.org/r/20230627161240.331-1-hsiangkao@linux.alibaba.com Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
show more ...
|
Revision tags: v6.4, v6.1.35, v6.1.34, v6.1.33, v6.1.32, v6.1.31, v6.1.30, v6.1.29, v6.1.28, v6.1.27, v6.1.26 |
|
#
597e2953 |
| 26-Apr-2023 |
Yue Hu <huyue2@coolpad.com> |
erofs: fold in z_erofs_decompress()
No need this helper since it's just a simple wrapper for decompress method and only one caller. So, let's fold in directly instead.
Signed-off-by: Yue Hu <huyue
erofs: fold in z_erofs_decompress()
No need this helper since it's just a simple wrapper for decompress method and only one caller. So, let's fold in directly instead.
Signed-off-by: Yue Hu <huyue2@coolpad.com> Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com> Link: https://lore.kernel.org/r/20230426084449.12781-1-zbestahu@gmail.com Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
show more ...
|
Revision tags: v6.3, v6.1.25, v6.1.24, v6.1.23, v6.1.22, v6.1.21, v6.1.20 |
|
#
3acea5fc |
| 13-Mar-2023 |
Jingbo Xu <jefflexu@linux.alibaba.com> |
erofs: avoid hardcoded blocksize for subpage block support
As the first step of converting hardcoded blocksize to that specified in on-disk superblock, convert all call sites of hardcoded blocksize
erofs: avoid hardcoded blocksize for subpage block support
As the first step of converting hardcoded blocksize to that specified in on-disk superblock, convert all call sites of hardcoded blocksize to sb->s_blocksize except for:
1) use sbi->blkszbits instead of sb->s_blocksize in erofs_superblock_csum_verify() since sb->s_blocksize has not been updated with the on-disk blocksize yet when the function is called.
2) use inode->i_blkbits instead of sb->s_blocksize in erofs_bread(), since the inode operated on may be an anonymous inode in fscache mode. Currently the anonymous inode is allocated from an anonymous mount maintained in erofs, while in the near future we may allocate anonymous inodes from a generic API directly and thus have no access to the anonymous inode's i_sb. Thus we keep the block size in i_blkbits for anonymous inodes in fscache mode.
Be noted that this patch only gets rid of the hardcoded blocksize, in preparation for actually setting the on-disk block size in the following patch. The hard limit of constraining the block size to PAGE_SIZE still exists until the next patch.
Signed-off-by: Jingbo Xu <jefflexu@linux.alibaba.com> Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com> Reviewed-by: Yue Hu <huyue2@coolpad.com> Reviewed-by: Chao Yu <chao@kernel.org> Link: https://lore.kernel.org/r/20230313135309.75269-2-jefflexu@linux.alibaba.com [ Gao Xiang: fold a patch to fix incorrect truncated offsets. ] Link: https://lore.kernel.org/r/20230413035734.15457-1-zhujia.zj@bytedance.com Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
show more ...
|
Revision tags: v6.1.19, v6.1.18, v6.1.17, v6.1.16, v6.1.15, v6.1.14, v6.1.13, v6.2, v6.1.12, v6.1.11, v6.1.10, v6.1.9, v6.1.8, v6.1.7, v6.1.6, v6.1.5, v6.0.19, v6.0.18, v6.1.4, v6.1.3, v6.0.17, v6.1.2, v6.0.16, v6.1.1, v6.0.15, v6.0.14, v6.0.13, v6.1, v6.0.12, v6.0.11, v6.0.10, v5.15.80, v6.0.9, v5.15.79, v6.0.8, v5.15.78, v6.0.7, v5.15.77, v5.15.76, v6.0.6, v6.0.5, v5.15.75, v6.0.4, v6.0.3, v6.0.2, v5.15.74, v5.15.73, v6.0.1, v5.15.72, v6.0, v5.15.71, v5.15.70 |
|
#
fdffc091 |
| 22-Sep-2022 |
Yue Hu <huyue2@coolpad.com> |
erofs: support interlaced uncompressed data for compressed files
Currently, uncompressed data is all handled in the shifted way, which means we have to shift the whole on-disk plain pcluster to get
erofs: support interlaced uncompressed data for compressed files
Currently, uncompressed data is all handled in the shifted way, which means we have to shift the whole on-disk plain pcluster to get the logical data. However, since we are also using in-place I/O for uncompressed data, data copy will be reduced a lot if pcluster is recorded in the interlaced way as illustrated below: _______________________________________________________________ | | | |_ tail part |_ head part _| |<- blk0 ->| .. |<- blkn-2 ->|<- blkn-1 ->|
The logical data then becomes: ________________________________________________________ |_ head part _|_ blk0 _| .. |_ blkn-2 _|_ tail part _|
In addition, non-4k plain pclusters are also survived by the interlaced way, which can be used for non-4k lclusters as well.
However, it's almost impossible to de-duplicate uncompressed data in the interlaced way, therefore shifted uncompressed data is still useful.
Signed-off-by: Yue Hu <huyue2@coolpad.com> Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com> Link: https://lore.kernel.org/r/8369112678604fdf4ef796626d59b1fdd0745a53.1663898962.git.huyue2@coolpad.com Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
show more ...
|
Revision tags: v5.15.69, v5.15.68, v5.15.67, v5.15.66, v5.15.65, v5.15.64, v5.15.63, v5.15.62, v5.15.61, v5.15.60, v5.15.59, v5.19, v5.15.58, v5.15.57, v5.15.56 |
|
#
267f2492 |
| 15-Jul-2022 |
Gao Xiang <hsiangkao@linux.alibaba.com> |
erofs: introduce multi-reference pclusters (fully-referenced)
Let's introduce multi-reference pclusters at runtime. In details, if one pcluster is requested by multiple extents at almost the same ti
erofs: introduce multi-reference pclusters (fully-referenced)
Let's introduce multi-reference pclusters at runtime. In details, if one pcluster is requested by multiple extents at almost the same time (even belong to different files), the longest extent will be decompressed as representative and the other extents are actually copied from the longest one in one round.
After this patch, fully-referenced extents can be correctly handled and the full decoding check needs to be bypassed for partial-referenced extents.
Acked-by: Chao Yu <chao@kernel.org> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com> Link: https://lore.kernel.org/r/20220715154203.48093-17-hsiangkao@linux.alibaba.com
show more ...
|
Revision tags: v5.15.55, v5.15.54 |
|
#
448b5a15 |
| 08-Jul-2022 |
Gao Xiang <hsiangkao@linux.alibaba.com> |
erofs: avoid consecutive detection for Highmem memory
Currently, vmap()s are avoided if physical addresses are consecutive for decompressed buffers.
I observed that is very common for 4KiB pcluster
erofs: avoid consecutive detection for Highmem memory
Currently, vmap()s are avoided if physical addresses are consecutive for decompressed buffers.
I observed that is very common for 4KiB pclusters since the numbers of decompressed pages are almost 2 or 3.
However, such detection doesn't work for Highmem pages on 32-bit machines, let's fix it now.
Reported-by: Liu Jinbao <liujinbao1@xiaomi.com> Fixes: 7fc45dbc938a ("staging: erofs: introduce generic decompression backend") Link: https://lore.kernel.org/r/20220708101001.21242-1-hsiangkao@linux.alibaba.com Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
show more ...
|
Revision tags: v5.15.53, v5.15.52, v5.15.51, v5.15.50, v5.15.49, v5.15.48, v5.15.47, v5.15.46, v5.15.45, v5.15.44, v5.15.43, v5.15.42, v5.18, v5.15.41, v5.15.40 |
|
#
dcbe6803 |
| 12-May-2022 |
Gao Xiang <hsiangkao@linux.alibaba.com> |
erofs: fix buffer copy overflow of ztailpacking feature
I got some KASAN report as below:
[ 46.959738] ================================================================== [ 46.960430] BUG: KASAN
erofs: fix buffer copy overflow of ztailpacking feature
I got some KASAN report as below:
[ 46.959738] ================================================================== [ 46.960430] BUG: KASAN: use-after-free in z_erofs_shifted_transform+0x2bd/0x370 [ 46.960430] Read of size 4074 at addr ffff8880300c2f8e by task fssum/188 ... [ 46.960430] Call Trace: [ 46.960430] <TASK> [ 46.960430] dump_stack_lvl+0x41/0x5e [ 46.960430] print_report.cold+0xb2/0x6b7 [ 46.960430] ? z_erofs_shifted_transform+0x2bd/0x370 [ 46.960430] kasan_report+0x8a/0x140 [ 46.960430] ? z_erofs_shifted_transform+0x2bd/0x370 [ 46.960430] kasan_check_range+0x14d/0x1d0 [ 46.960430] memcpy+0x20/0x60 [ 46.960430] z_erofs_shifted_transform+0x2bd/0x370 [ 46.960430] z_erofs_decompress_pcluster+0xaae/0x1080
The root cause is that the tail pcluster won't be a complete filesystem block anymore. So if ztailpacking is used, the second part of an uncompressed tail pcluster may not be ``rq->pageofs_out``.
Fixes: ab749badf9f4 ("erofs: support unaligned data decompression") Fixes: cecf864d3d76 ("erofs: support inline data decompression") Reviewed-by: Yue Hu <huyue2@coolpad.com> Reviewed-by: Chao Yu <chao@kernel.org> Link: https://lore.kernel.org/r/20220512115833.24175-1-hsiangkao@linux.alibaba.com Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
show more ...
|
Revision tags: v5.15.39, v5.15.38, v5.15.37, v5.15.36, v5.15.35, v5.15.34, v5.15.33 |
|
#
1e59af07 |
| 07-Apr-2022 |
Yue Hu <huyue2@coolpad.com> |
erofs: do not prompt for risk any more when using big pcluster
The big pcluster feature has been merged for a year, it has been mostly stable now.
Signed-off-by: Yue Hu <huyue2@coolpad.com> Reviewe
erofs: do not prompt for risk any more when using big pcluster
The big pcluster feature has been merged for a year, it has been mostly stable now.
Signed-off-by: Yue Hu <huyue2@coolpad.com> Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com> Reviewed-by: Chao Yu <chao@kernel.org> Link: https://lore.kernel.org/r/20220407050505.12683-1-huyue2@coolpad.com Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
show more ...
|
Revision tags: v5.15.32, v5.15.31, v5.17, v5.15.30, v5.15.29, v5.15.28, v5.15.27, v5.15.26, v5.15.25, v5.15.24, v5.15.23, v5.15.22, v5.15.21, v5.15.20, v5.15.19, v5.15.18, v5.15.17, v5.4.173, v5.15.16, v5.15.15, v5.16 |
|
#
ab749bad |
| 27-Dec-2021 |
Gao Xiang <hsiangkao@linux.alibaba.com> |
erofs: support unaligned data decompression
Previously, compressed data was assumed as block-aligned. This should be changed due to in-block tail-packing inline data.
Link: https://lore.kernel.org/
erofs: support unaligned data decompression
Previously, compressed data was assumed as block-aligned. This should be changed due to in-block tail-packing inline data.
Link: https://lore.kernel.org/r/20211228054604.114518-4-hsiangkao@linux.alibaba.com Reviewed-by: Chao Yu <chao@kernel.org> Signed-off-by: Yue Hu <huyue2@yulong.com> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
show more ...
|
#
10e5f6e4 |
| 27-Dec-2021 |
Gao Xiang <hsiangkao@linux.alibaba.com> |
erofs: introduce z_erofs_fixup_insize
To prepare for the upcoming ztailpacking feature, introduce z_erofs_fixup_insize() and pageofs_in to wrap up the process to get the exact compressed size via ze
erofs: introduce z_erofs_fixup_insize
To prepare for the upcoming ztailpacking feature, introduce z_erofs_fixup_insize() and pageofs_in to wrap up the process to get the exact compressed size via zero padding.
Link: https://lore.kernel.org/r/20211228054604.114518-3-hsiangkao@linux.alibaba.com Reviewed-by: Chao Yu <chao@kernel.org> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
show more ...
|
#
d67aee76 |
| 27-Dec-2021 |
Gao Xiang <hsiangkao@linux.alibaba.com> |
erofs: tidy up z_erofs_lz4_decompress
To prepare for the upcoming ztailpacking feature and further cleanups, introduce a unique z_erofs_lz4_decompress_ctx to keep the context, including inpages, out
erofs: tidy up z_erofs_lz4_decompress
To prepare for the upcoming ztailpacking feature and further cleanups, introduce a unique z_erofs_lz4_decompress_ctx to keep the context, including inpages, outpages and oend, which are frequently used by the lz4 decompressor.
No logic changes.
Link: https://lore.kernel.org/r/20211228054604.114518-2-hsiangkao@linux.alibaba.com Reviewed-by: Chao Yu <chao@kernel.org> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
show more ...
|
Revision tags: v5.15.10, v5.15.9, v5.15.8, v5.15.7, v5.15.6, v5.15.5, v5.15.4, v5.15.3 |
|
#
7e508f2c |
| 12-Nov-2021 |
Huang Jianan <huangjianan@oppo.com> |
erofs: rename lz4_0pading to zero_padding
Renaming lz4_0padding to zero_padding globally since LZMA and later algorithms also need that.
Link: https://lore.kernel.org/r/20211112160935.19394-1-jnhua
erofs: rename lz4_0pading to zero_padding
Renaming lz4_0padding to zero_padding globally since LZMA and later algorithms also need that.
Link: https://lore.kernel.org/r/20211112160935.19394-1-jnhuang95@gmail.com Reviewed-by: Chao Yu <chao@kernel.org> Signed-off-by: Huang Jianan <huangjianan@oppo.com> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
show more ...
|
Revision tags: v5.15.2, v5.15.1, v5.15 |
|
#
a0961f35 |
| 25-Oct-2021 |
Gao Xiang <hsiangkao@linux.alibaba.com> |
erofs: don't trigger WARN() when decompression fails
syzbot reported a WARNING [1] due to corrupted compressed data.
As Dmitry said, "If this is not a kernel bug, then the code should not use WARN.
erofs: don't trigger WARN() when decompression fails
syzbot reported a WARNING [1] due to corrupted compressed data.
As Dmitry said, "If this is not a kernel bug, then the code should not use WARN. WARN if for kernel bugs and is recognized as such by all testing systems and humans."
[1] https://lore.kernel.org/r/000000000000b3586105cf0ff45e@google.com
Link: https://lore.kernel.org/r/20211025074311.130395-1-hsiangkao@linux.alibaba.com Cc: Dmitry Vyukov <dvyukov@google.com> Reviewed-by: Chao Yu <chao@kernel.org> Reported-by: syzbot+d8aaffc3719597e8cfb4@syzkaller.appspotmail.com Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
show more ...
|
#
eaa9172a |
| 22-Oct-2021 |
Gao Xiang <hsiangkao@linux.alibaba.com> |
erofs: get rid of ->lru usage
Currently, ->lru is a way to arrange non-LRU pages and has some in-kernel users. In order to minimize noticable issues of page reclaim and cache thrashing under high me
erofs: get rid of ->lru usage
Currently, ->lru is a way to arrange non-LRU pages and has some in-kernel users. In order to minimize noticable issues of page reclaim and cache thrashing under high memory presure, limited temporary pages were all chained with ->lru and can be reused during the request. However, it seems that ->lru could be removed when folio is landing.
Let's use page->private to chain temporary pages for now instead and transform EROFS formally after the topic of the folio / file page design is finalized.
Link: https://lore.kernel.org/r/20211022090120.14675-1-hsiangkao@linux.alibaba.com Cc: Matthew Wilcox <willy@infradead.org> Reviewed-by: Kent Overstreet <kent.overstreet@gmail.com> Reviewed-by: Chao Yu <chao@kernel.org> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
show more ...
|
Revision tags: v5.14.14, v5.14.13, v5.14.12 |
|
#
622ceadd |
| 10-Oct-2021 |
Gao Xiang <hsiangkao@linux.alibaba.com> |
erofs: lzma compression support
Add MicroLZMA support in order to maximize compression ratios for specific scenarios. For example, it's useful for low-end embedded boards and as a secondary algorith
erofs: lzma compression support
Add MicroLZMA support in order to maximize compression ratios for specific scenarios. For example, it's useful for low-end embedded boards and as a secondary algorithm in a file for specific access patterns.
MicroLZMA is a new container format for raw LZMA1, which was created by Lasse Collin aiming to minimize old LZMA headers and get rid of unnecessary EOPM (end of payload marker) as well as to enable fixed-sized output compression, especially for 4KiB pclusters.
Similar to LZ4, inplace I/O approach is used to minimize runtime memory footprint when dealing with I/O. Overlapped decompression is handled with 1) bounced buffer for data under processing or 2) extra short-lived pages from the on-stack pagepool which will be shared in the same read request (128KiB for example).
Link: https://lore.kernel.org/r/20211010213145.17462-8-xiang@kernel.org Acked-by: Chao Yu <chao@kernel.org> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
show more ...
|
#
966edfb0 |
| 10-Oct-2021 |
Gao Xiang <hsiangkao@linux.alibaba.com> |
erofs: rename some generic methods in decompressor
Previously, some LZ4 methods were named with `generic'. However, while evaluating the effective LZMA approach, it seems they aren't quite generic a
erofs: rename some generic methods in decompressor
Previously, some LZ4 methods were named with `generic'. However, while evaluating the effective LZMA approach, it seems they aren't quite generic at all (e.g. no need preparing dstpages for most LZMA cases.)
Avoid such naming instead.
Link: https://lore.kernel.org/r/20211010213145.17462-7-xiang@kernel.org Acked-by: Chao Yu <chao@kernel.org> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
show more ...
|
#
5b6e7e12 |
| 14-Oct-2021 |
Yue Hu <huyue2@yulong.com> |
erofs: remove the fast path of per-CPU buffer decompression
As Xiang mentioned, such path has no real impact to our current decompression strategy, remove it directly. Also, update the return value
erofs: remove the fast path of per-CPU buffer decompression
As Xiang mentioned, such path has no real impact to our current decompression strategy, remove it directly. Also, update the return value of z_erofs_lz4_decompress() to 0 if success to keep consistent with LZMA which will return 0 as well for that case.
Link: https://lore.kernel.org/r/20211014065744.1787-1-zbestahu@gmail.com Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com> Signed-off-by: Yue Hu <huyue2@yulong.com> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
show more ...
|
#
1c350a59 |
| 08-Jul-2022 |
Gao Xiang <hsiangkao@linux.alibaba.com> |
erofs: avoid consecutive detection for Highmem memory
[ Upstream commit 448b5a1548d87c246c3d0c3df8480d3c6eb6c11a ]
Currently, vmap()s are avoided if physical addresses are consecutive for decompres
erofs: avoid consecutive detection for Highmem memory
[ Upstream commit 448b5a1548d87c246c3d0c3df8480d3c6eb6c11a ]
Currently, vmap()s are avoided if physical addresses are consecutive for decompressed buffers.
I observed that is very common for 4KiB pclusters since the numbers of decompressed pages are almost 2 or 3.
However, such detection doesn't work for Highmem pages on 32-bit machines, let's fix it now.
Reported-by: Liu Jinbao <liujinbao1@xiaomi.com> Fixes: 7fc45dbc938a ("staging: erofs: introduce generic decompression backend") Link: https://lore.kernel.org/r/20220708101001.21242-1-hsiangkao@linux.alibaba.com Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
show more ...
|
#
ab7ce95a |
| 25-Oct-2021 |
Gao Xiang <hsiangkao@linux.alibaba.com> |
erofs: don't trigger WARN() when decompression fails
[ Upstream commit a0961f351d82d43ab0b845304caa235dfe249ae9 ]
syzbot reported a WARNING [1] due to corrupted compressed data.
As Dmitry said, "I
erofs: don't trigger WARN() when decompression fails
[ Upstream commit a0961f351d82d43ab0b845304caa235dfe249ae9 ]
syzbot reported a WARNING [1] due to corrupted compressed data.
As Dmitry said, "If this is not a kernel bug, then the code should not use WARN. WARN if for kernel bugs and is recognized as such by all testing systems and humans."
[1] https://lore.kernel.org/r/000000000000b3586105cf0ff45e@google.com
Link: https://lore.kernel.org/r/20211025074311.130395-1-hsiangkao@linux.alibaba.com Cc: Dmitry Vyukov <dvyukov@google.com> Reviewed-by: Chao Yu <chao@kernel.org> Reported-by: syzbot+d8aaffc3719597e8cfb4@syzkaller.appspotmail.com Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com> Signed-off-by: Sasha Levin <sashal@kernel.org>
show more ...
|
Revision tags: v5.14.11, v5.14.10, v5.14.9, v5.14.8, v5.14.7, v5.14.6, v5.10.67, v5.10.66, v5.14.5, v5.14.4, v5.10.65, v5.14.3, v5.10.64, v5.14.2, v5.10.63, v5.14.1, v5.10.62, v5.14, v5.10.61, v5.10.60, v5.10.53, v5.10.52, v5.10.51, v5.10.50, v5.10.49, v5.13, v5.10.46, v5.10.43, v5.10.42 |
|
#
c5fcb511 |
| 02-Jun-2021 |
Gao Xiang <hsiangkao@linux.alibaba.com> |
erofs: clean up file headers & footers
- Remove my outdated misleading email address;
- Get rid of all unnecessary trailing newline by accident.
Link: https://lore.kernel.org/r/20210602160634.10
erofs: clean up file headers & footers
- Remove my outdated misleading email address;
- Get rid of all unnecessary trailing newline by accident.
Link: https://lore.kernel.org/r/20210602160634.10757-1-xiang@kernel.org Reviewed-by: Chao Yu <yuchao0@huawei.com> Signed-off-by: Gao Xiang <hsiangkao@linux.alibaba.com>
show more ...
|