Linux 6.4

kbuild: make modules_install copy modules.builtin(.modinfo)

Josh Triplett reports that initramfs-tools needs modules.builtin and
modules.builtin.modinfo to create a working initramfs for a non-modul

kbuild: make modules_install copy modules.builtin(.modinfo)

Josh Triplett reports that initramfs-tools needs modules.builtin and
modules.builtin.modinfo to create a working initramfs for a non-modular
kernel.

If this is a general tooling issue not limited to Debian, I think it
makes sense to change modules_install.

This commit changes the targets as follows when CONFIG_MODULES=n.

In-tree builds:
make modules -> no-op
make modules_install -> install modules.builtin(.modinfo)

External module builds:
make modules -> show error message like before
make modules_install -> show error message like before

Link: https://lore.kernel.org/lkml/36a4014c73a52af27d930d3ca31d362b60f4461c.1686356364.git.josh@joshtriplett.org/
Reported-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
Tested-by: Nicolas Schier <nicolas@fjasle.eu>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Tested-by: Josh Triplett <josh@joshtriplett.org>

show more ...

kbuild: implement CONFIG_TRIM_UNUSED_KSYMS without recursion

When CONFIG_TRIM_UNUSED_KSYMS is enabled, Kbuild recursively traverses
the directory tree to determine which EXPORT_SYMBOL to trim. If an

kbuild: implement CONFIG_TRIM_UNUSED_KSYMS without recursion

When CONFIG_TRIM_UNUSED_KSYMS is enabled, Kbuild recursively traverses
the directory tree to determine which EXPORT_SYMBOL to trim. If an
EXPORT_SYMBOL turns out to be unused by anyone, Kbuild begins the
second traverse, where some source files are recompiled with their
EXPORT_SYMBOL() tuned into a no-op.

Linus stated negative opinions about this slowness in commits:

- 5cf0fd591f2e ("Kbuild: disable TRIM_UNUSED_KSYMS option")
- a555bdd0c58c ("Kbuild: enable TRIM_UNUSED_KSYMS again, with some guarding")

We can do this better now. The final data structures of EXPORT_SYMBOL
are generated by the modpost stage, so modpost can selectively emit
KSYMTAB entries that are really used by modules.

Commit f73edc8951b2 ("kbuild: unify two modpost invocations") is another
ground-work to do this in a one-pass algorithm. With the list of modules,
modpost sets sym->used if it is used by a module. modpost emits KSYMTAB
only for symbols with sym->used==true.

BTW, Nicolas explained why the trimming was implemented with recursion:

https://lore.kernel.org/all/2o2rpn97-79nq-p7s2-nq5-8p83391473r@syhkavp.arg/

Actually, we never achieved that level of optimization where the chain
reaction of trimming comes into play because:

- CONFIG_LTO_CLANG cannot remove any unused symbols
- CONFIG_LD_DEAD_CODE_DATA_ELIMINATION is enabled only for vmlinux,
but not modules

If deeper trimming is required, we need to revisit this, but I guess
that is unlikely to happen.

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

show more ...

Linux 6.4-rc7

Linux 6.4-rc6

Linux 6.4-rc5

Linux 6.4-rc4

kbuild: Enable -fstrict-flex-arrays=3

The -fstrict-flex-arrays=3 option is now available with the release
of GCC 13[1] and Clang 16[2]. This feature instructs the compiler to
treat only C99 flexible

kbuild: Enable -fstrict-flex-arrays=3

The -fstrict-flex-arrays=3 option is now available with the release
of GCC 13[1] and Clang 16[2]. This feature instructs the compiler to
treat only C99 flexible arrays as dynamically sized for the purposes of
object size calculations. In other words, the ancient practice of using
1-element arrays, or the GNU extension of using 0-sized arrays, as a
dynamically sized array is disabled. This allows CONFIG_UBSAN_BOUNDS,
CONFIG_FORTIFY_SOURCE, and other object-size aware features to behave
unambiguously in the face of trailing arrays: only C99 flexible arrays
are considered to be dynamically sized. For yet more detail, see:
https://people.kernel.org/kees/bounded-flexible-arrays-in-c

Enabling this will help track down any outstanding cases of fake
flexible arrays that need attention in kernel code.

[1] https://gcc.gnu.org/onlinedocs/gcc/C-Dialect-Options.html#index-fstrict-flex-arrays
[2] https://clang.llvm.org/docs/ClangCommandLineReference.html#cmdoption-clang-fstrict-flex-arrays

Cc: Masahiro Yamada <masahiroy@kernel.org>
Cc: Nathan Chancellor <nathan@kernel.org>
Cc: Nick Desaulniers <ndesaulniers@google.com>
Cc: Nicolas Schier <nicolas@fjasle.eu>
Cc: linux-kbuild@vger.kernel.org
Co-developed-by: "Gustavo A. R. Silva" <gustavoars@kernel.org>
Signed-off-by: "Gustavo A. R. Silva" <gustavoars@kernel.org>
Signed-off-by: Kees Cook <keescook@chromium.org>

show more ...

Linux 6.4-rc3

Linux 6.4-rc2

Linux 6.4-rc1

Linux 6.3

Makefile: use -z pack-relative-relocs

Commit 27f2a4db76e8 ("Makefile: fix GDB warning with CONFIG_RELR")
added --use-android-relr-tags to fix a GDB warning

BFD: /android0/linux-next/vmlinux: unknow

Makefile: use -z pack-relative-relocs

Commit 27f2a4db76e8 ("Makefile: fix GDB warning with CONFIG_RELR")
added --use-android-relr-tags to fix a GDB warning

BFD: /android0/linux-next/vmlinux: unknown type [0x13] section `.relr.dyn'

The GDB warning has been fixed in version 11.2.

The DT_ANDROID_RELR tag was deprecated since DT_RELR was standardized.
Thus, --use-android-relr-tags should be removed. While making the
change, try -z pack-relative-relocs, which is supported since LLD 15.
Keep supporting --pack-dyn-relocs=relr as well for older LLD versions.
There is no indication of obsolescence for --pack-dyn-relocs=relr.

As of today, GNU ld supports the latter option for x86 and powerpc64
ports and has no intention to support --pack-dyn-relocs=relr. In the
absence of the glibc symbol version GLIBC_ABI_DT_RELR,
--pack-dyn-relocs=relr and -z pack-relative-relocs are identical in
ld.lld.

GNU ld and newer versions of LLD report warnings (instead of errors) for
unknown -z options. Only errors lead to non-zero exit codes. Therefore,
we should test --pack-dyn-relocs=relr before testing
-z pack-relative-relocs.

Link: https://github.com/ClangBuiltLinux/linux/issues/1057
Link: https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=a619b58721f0a03fd91c27670d3e4c2fb0d88f1e
Signed-off-by: Fangrui Song <maskray@google.com>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

show more ...

Linux 6.3-rc7

Linux 6.3-rc6

Linux 6.3-rc5

Linux 6.3-rc4

Linux 6.3-rc3

kbuild: use git-archive for source package creation

Commit 5c3d1d0abb12 ("kbuild: add a tool to list files ignored by git")
added a new tool, scripts/list-gitignored. My intention was to create
sour

kbuild: use git-archive for source package creation

Commit 5c3d1d0abb12 ("kbuild: add a tool to list files ignored by git")
added a new tool, scripts/list-gitignored. My intention was to create
source packages without cleaning the source tree, without relying on git.

Linus strongly objected to it, and suggested using 'git archive' instead.
[1] [2] [3]

This commit goes in that direction - Remove scripts/list-gitignored.c
and rewrites Makefiles and scripts to use 'git archive' for building
Debian and RPM source packages. It also makes 'make perf-tar*-src-pkg'
use 'git archive' again.

Going forward, building source packages is only possible in a git-managed
tree. Building binary packages does not require git.

[1]: https://lore.kernel.org/lkml/CAHk-=wi49sMaC7vY1yMagk7eqLK=1jHeHQ=yZ_k45P=xBccnmA@mail.gmail.com/
[2]: https://lore.kernel.org/lkml/CAHk-=wh5AixGsLeT0qH2oZHKq0FLUTbyTw4qY921L=PwYgoGVw@mail.gmail.com/
[3]: https://lore.kernel.org/lkml/CAHk-=wgM-W6Fu==EoAVCabxyX8eYBz9kNC88-tm9ExRQwA79UQ@mail.gmail.com/

Fixes: 5c3d1d0abb12 ("kbuild: add a tool to list files ignored by git")
Fixes: e0ca16749ac3 ("kbuild: make perf-tar*-src-pkg work without relying on git")
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

show more ...

kbuild: rpm-pkg: move source components to rpmbuild/SOURCES

Prepare to add more files to the source RPM.

Also, fix the build error when KCONFIG_CONFIG is set:
error: Bad file: ./.config: No such

kbuild: rpm-pkg: move source components to rpmbuild/SOURCES

Prepare to add more files to the source RPM.

Also, fix the build error when KCONFIG_CONFIG is set:
error: Bad file: ./.config: No such file or directory

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

show more ...

Makefile: Make kernelrelease target work with M=

That commit required the use of filechk_kernel.release for the
kernelrelease Makefile target. It is currently only being set when
KBUILD_EXTMOD is no

Makefile: Make kernelrelease target work with M=

That commit required the use of filechk_kernel.release for the
kernelrelease Makefile target. It is currently only being set when
KBUILD_EXTMOD is not set. Make sure it is set in that case as well.

Fixes: 1cb86b6c3136 ("kbuild: save overridden KERNELRELEASE in include/config/kernel.release")
Signed-off-by: Tzafrir Cohen <nvidia@cohens.org.il>
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

show more ...

Linux 6.3-rc2

Linux 6.3-rc1

Linux 6.2

kbuild: add a tool to list files ignored by git

In short, the motivation of this commit is to build a source package
without cleaning the source tree.

The deb-pkg and (src)rpm-pkg targets first run

kbuild: add a tool to list files ignored by git

In short, the motivation of this commit is to build a source package
without cleaning the source tree.

The deb-pkg and (src)rpm-pkg targets first run 'make clean' before
creating a source tarball. Otherwise build artifacts such as *.o,
*.a, etc. would be included in the tarball. Yet, the tarball ends up
containing several garbage files since 'make clean' does not clean
everything.

Cleaning the tree every time is annoying since it makes the incremental
build impossible. It is desirable to create a source tarball without
cleaning the tree.

In fact, there are some ways to achieve this.

The easiest solution is 'git archive'. 'make perf-tar*-src-pkg' uses
it, but I do not like it because it works only when the source tree is
managed by git, and all files you want in the tarball must be committed
in advance.

I want to make it work without relying on git. We can do this.

Files that are ignored by git are generated files, so should be excluded
from the source tarball. We can list them out by parsing the .gitignore
files. Of course, .gitignore does not cover all the cases, but it works
well enough.

tar(1) claims to support it:

--exclude-vcs-ignores

Exclude files that match patterns read from VCS-specific ignore files.
Supported files are: .cvsignore, .gitignore, .bzrignore, and .hgignore.

The best scenario would be to use 'tar --exclude-vcs-ignores', but this
option does not work. --exclude-vcs-ignore does not understand any of
the negation (!), preceding slash, following slash, etc.. So, this option
is just useless.

Hence, I wrote this gitignore parser. The previous version [1], written
in Python, was so slow. This version is implemented in C, so it works
much faster.

I imported the code from git (commit: 23c56f7bd5f1), so we get the same
result.

This tool traverses the source tree, parsing all .gitignore files, and
prints file paths that are ignored by git.

The output is similar to 'git ls-files --ignored --directory --others
--exclude-per-directory=.gitignore', except

[1] Not sorted
[2] No trailing slash for directories

[2] is intentional because tar's --exclude-from option cannot handle
trailing slashes.

[How to test this tool]

$ git clean -dfx
$ make -s -j$(nproc) defconfig all # or allmodconifg or whatever
$ git archive -o ../linux1.tar --prefix=./ HEAD
$ tar tf ../linux1.tar | LANG=C sort > ../file-list1 # files emitted by 'git archive'
$ make scripts_package
HOSTCC scripts/list-gitignored
$ scripts/list-gitignored --prefix=./ -o ../exclude-list
$ tar cf ../linux2.tar --exclude-from=../exclude-list .
$ tar tf ../linux2.tar | LANG=C sort > ../file-list2 # files emitted by 'tar'
$ diff ../file-list1 ../file-list2 | grep -E '^(<|>)'
< ./Documentation/devicetree/bindings/.yamllint
< ./drivers/clk/.kunitconfig
< ./drivers/gpu/drm/tests/.kunitconfig
< ./drivers/hid/.kunitconfig
< ./fs/ext4/.kunitconfig
< ./fs/fat/.kunitconfig
< ./kernel/kcsan/.kunitconfig
< ./lib/kunit/.kunitconfig
< ./mm/kfence/.kunitconfig
< ./tools/testing/selftests/arm64/tags/
< ./tools/testing/selftests/arm64/tags/.gitignore
< ./tools/testing/selftests/arm64/tags/Makefile
< ./tools/testing/selftests/arm64/tags/run_tags_test.sh
< ./tools/testing/selftests/arm64/tags/tags_test.c
< ./tools/testing/selftests/kvm/.gitignore
< ./tools/testing/selftests/kvm/Makefile
< ./tools/testing/selftests/kvm/config
< ./tools/testing/selftests/kvm/settings

The source tarball contains most of files that are tracked by git. You
see some diffs, but it is just because some .gitignore files are wrong.

$ git ls-files -i -c --exclude-per-directory=.gitignore
Documentation/devicetree/bindings/.yamllint
drivers/clk/.kunitconfig
drivers/gpu/drm/tests/.kunitconfig
drivers/hid/.kunitconfig
fs/ext4/.kunitconfig
fs/fat/.kunitconfig
kernel/kcsan/.kunitconfig
lib/kunit/.kunitconfig
mm/kfence/.kunitconfig
tools/testing/selftests/arm64/tags/.gitignore
tools/testing/selftests/arm64/tags/Makefile
tools/testing/selftests/arm64/tags/run_tags_test.sh
tools/testing/selftests/arm64/tags/tags_test.c
tools/testing/selftests/kvm/.gitignore
tools/testing/selftests/kvm/Makefile
tools/testing/selftests/kvm/config
tools/testing/selftests/kvm/settings

[1]: https://lore.kernel.org/all/20230128173843.765212-1-masahiroy@kernel.org/

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>

show more ...