17 Jun, 2020
1 commit
-
commit 90ceddcb495008ac8ba7a3dce297841efcd7d584 upstream.
Simplify gen_btf logic to make it work with llvm-objcopy. The existing
'file format' and 'architecture' parsing logic is brittle and does not
work with llvm-objcopy/llvm-objdump.'file format' output of llvm-objdump>=11 will match GNU objdump, but
'architecture' (bfdarch) may not..BTF in .tmp_vmlinux.btf is non-SHF_ALLOC. Add the SHF_ALLOC flag
because it is part of vmlinux image used for introspection. C code
can reference the section via linker script defined __start_BTF and
__stop_BTF. This fixes a small problem that previous .BTF had the
SHF_WRITE flag (objcopy -I binary -O elf* synthesized .data).Additionally, `objcopy -I binary` synthesized symbols
_binary__btf_vmlinux_bin_start and _binary__btf_vmlinux_bin_stop (not
used elsewhere) are replaced with more commonplace __start_BTF and
__stop_BTF.Add 2>/dev/null because GNU objcopy (but not llvm-objcopy) warns
"empty loadable segment detected at vaddr=0xffffffff81000000, is this intentional?"We use a dd command to change the e_type field in the ELF header from
ET_EXEC to ET_REL so that lld will accept .btf.vmlinux.bin.o. Accepting
ET_EXEC as an input file is an extremely rare GNU ld feature that lld
does not intend to support, because this is error-prone.The output section description .BTF in include/asm-generic/vmlinux.lds.h
avoids potential subtle orphan section placement issues and suppresses
--orphan-handling=warn warnings.Fixes: df786c9b9476 ("bpf: Force .BTF section start to zero when dumping from vmlinux")
Fixes: cb0cc635c7a9 ("powerpc: Include .BTF section")
Reported-by: Nathan Chancellor
Signed-off-by: Fangrui Song
Signed-off-by: Daniel Borkmann
Tested-by: Stanislav Fomichev
Tested-by: Andrii Nakryiko
Reviewed-by: Stanislav Fomichev
Reviewed-by: Kees Cook
Acked-by: Andrii Nakryiko
Acked-by: Michael Ellerman (powerpc)
Link: https://github.com/ClangBuiltLinux/linux/issues/871
Link: https://lore.kernel.org/bpf/20200318222746.173648-1-maskray@google.com
Signed-off-by: Maria Teguiani
Tested-by: Matthias Maennich
Signed-off-by: Greg Kroah-Hartman
27 May, 2020
1 commit
-
[ Upstream commit af73d78bd384aa9b8789aa6e7ddbb165f971276f ]
When CONFIG_DEBUG_INFO is enabled, the two kallsyms linking steps spend
time collecting and writing the dwarf sections to the temporary output
files. kallsyms does not need this information, and leaving it off
halves their linking time. This is especially noticeable without
CONFIG_DEBUG_INFO_REDUCED. The BTF linking stage, however, does still
need those details.Refactor the BTF and kallsyms generation stages slightly for more
regularized temporary names. Skip debug during kallsyms links.
Additionally move "info BTF" to the correct place since commit
8959e39272d6 ("kbuild: Parameterize kallsyms generation and correct
reporting"), which added "info LD ..." to vmlinux_link calls.For a full debug info build with BTF, my link time goes from 1m06s to
0m54s, saving about 12 seconds, or 18%.Signed-off-by: Kees Cook
Signed-off-by: Daniel Borkmann
Acked-by: Andrii Nakryiko
Link: https://lore.kernel.org/bpf/202003031814.4AEA3351@keescook
Signed-off-by: Sasha Levin
24 Feb, 2020
1 commit
-
[ Upstream commit 2a67a6ccb01f21b854715d86ff6432a18b97adb3 ]
When trying to compile with CONFIG_DEBUG_INFO_BTF enabled, I got this
error:% make -s
Failed to generate BTF for vmlinux
Try to disable CONFIG_DEBUG_INFO_BTF
make[3]: *** [vmlinux] Error 1Compiling again without -s shows the true error (that pahole is
missing), but since this is fatal, we should show the error
unconditionally on stderr as well, not silence it using the `info`
function. With this patch:% make -s
BTF: .tmp_vmlinux.btf: pahole (pahole) is not available
Failed to generate BTF for vmlinux
Try to disable CONFIG_DEBUG_INFO_BTF
make[3]: *** [vmlinux] Error 1Signed-off-by: Chris Down
Signed-off-by: Daniel Borkmann
Acked-by: Andrii Nakryiko
Link: https://lore.kernel.org/bpf/20200122000110.GA310073@chrisdown.name
Signed-off-by: Sasha Levin
26 Jan, 2020
1 commit
-
commit df786c9b947639aedbc7bb44b5dae2a7824af360 upstream.
While trying to figure out why fentry_fexit selftest doesn't pass for me
(old pahole, broken BTF), I found out that my latest patch can break vmlinux
.BTF generation. objcopy preserves section start when doing --only-section,
so there is a chance (depending on where pahole inserts .BTF section) to
have leading empty zeroes. Let's explicitly force section offset to zero.Before:
$ objcopy --set-section-flags .BTF=alloc -O binary \
--only-section=.BTF vmlinux .btf.vmlinux.bin
$ xxd .btf.vmlinux.bin | head -n1
00000000: 0000 0000 0000 0000 0000 0000 0000 0000 ................After:
$ objcopy --change-section-address .BTF=0 \
--set-section-flags .BTF=alloc -O binary \
--only-section=.BTF vmlinux .btf.vmlinux.bin
$ xxd .btf.vmlinux.bin | head -n1
00000000: 9feb 0100 1800 0000 0000 0000 80e1 1c00 ................
^BTF magicAs part of this change, I'm also dropping '2>/dev/null' from objcopy
invocation to be able to catch possible other issues (objcopy doesn't
produce any warnings for me anymore, it did before with --dump-section).Fixes: da5fb18225b4 ("bpf: Support pre-2.25-binutils objcopy for vmlinux BTF")
Signed-off-by: Stanislav Fomichev
Signed-off-by: Daniel Borkmann
Acked-by: John Fastabend
Cc: Andrii Nakryiko
Link: https://lore.kernel.org/bpf/20191127225759.39923-1-sdf@google.com
Signed-off-by: Greg Kroah-Hartman
18 Jan, 2020
1 commit
-
commit da5fb18225b49b97bb37c51bcbbb2990a507c364 upstream.
If vmlinux BTF generation fails, but CONFIG_DEBUG_INFO_BTF is set,
.BTF section of vmlinux is empty and kernel will prohibit
BPF loading and return "in-kernel BTF is malformed".--dump-section argument to binutils' objcopy was added in version 2.25.
When using pre-2.25 binutils, BTF generation silently fails. Convert
to --only-section which is present on pre-2.25 binutils.Documentation/process/changes.rst states that binutils 2.21+
is supported, not sure those standards apply to BPF subsystem.v2:
* exit and print an error if gen_btf fails (John Fastabend)v3:
* resend with Andrii's Acked-by/Tested-by tagsFixes: 341dfcf8d78ea ("btf: expose BTF info through sysfs")
Signed-off-by: Stanislav Fomichev
Signed-off-by: Alexei Starovoitov
Tested-by: Andrii Nakryiko
Acked-by: Andrii Nakryiko
Cc: John Fastabend
Link: https://lore.kernel.org/bpf/20191127161410.57327-1-sdf@google.com
Signed-off-by: Greg Kroah-Hartman
20 Sep, 2019
1 commit
-
Pull Kbuild updates from Masahiro Yamada:
- add modpost warn exported symbols marked as 'static' because 'static'
and EXPORT_SYMBOL is an odd combination- break the build early if gold linker is used
- optimize the Bison rule to produce .c and .h files by a single
pattern rule- handle PREEMPT_RT in the module vermagic and UTS_VERSION
- warn CONFIG options leaked to the user-space except existing ones
- make single targets work properly
- rebuild modules when module linker scripts are updated
- split the module final link stage into scripts/Makefile.modfinal
- fix the missed error code in merge_config.sh
- improve the error message displayed on the attempt of the O= build in
unclean source tree- remove 'clean-dirs' syntax
- disable -Wimplicit-fallthrough warning for Clang
- add CONFIG_CC_OPTIMIZE_FOR_SIZE_O3 for ARC
- remove ARCH_{CPP,A,C}FLAGS variables
- add $(BASH) to run bash scripts
- change *CFLAGS_.o to take the relative path to $(obj)
instead of the basename- stop suppressing Clang's -Wunused-function warnings when W=1
- fix linux/export.h to avoid genksyms calculating CRC of trimmed
exported symbols- misc cleanups
* tag 'kbuild-v5.4' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (63 commits)
genksyms: convert to SPDX License Identifier for lex.l and parse.y
modpost: use __section in the output to *.mod.c
modpost: use MODULE_INFO() for __module_depends
export.h, genksyms: do not make genksyms calculate CRC of trimmed symbols
export.h: remove defined(__KERNEL__), which is no longer needed
kbuild: allow Clang to find unused static inline functions for W=1 build
kbuild: rename KBUILD_ENABLE_EXTRA_GCC_CHECKS to KBUILD_EXTRA_WARN
kbuild: refactor scripts/Makefile.extrawarn
merge_config.sh: ignore unwanted grep errors
kbuild: change *FLAGS_.o to take the path relative to $(obj)
modpost: add NOFAIL to strndup
modpost: add guid_t type definition
kbuild: add $(BASH) to run scripts with bash-extension
kbuild: remove ARCH_{CPP,A,C}FLAGS
kbuild,arc: add CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE_O3 for ARC
kbuild: Do not enable -Wimplicit-fallthrough for clang for now
kbuild: clean up subdir-ymn calculation in Makefile.clean
kbuild: remove unneeded '+' marker from cmd_clean
kbuild: remove clean-dirs syntax
kbuild: check clean srctree even earlier
...
07 Sep, 2019
1 commit
-
${@:2} is BASH-specific extension, which makes link-vmlinux.sh rely on
BASH. Use shift and ${@} instead to fix this issue.Reported-by: Stephen Rothwell
Fixes: 341dfcf8d78e ("btf: expose BTF info through sysfs")
Cc: Stephen Rothwell
Cc: Masahiro Yamada
Signed-off-by: Andrii Nakryiko
Acked-by: Yonghong Song
Reviewed-by: Masahiro Yamada
Signed-off-by: Alexei Starovoitov
21 Aug, 2019
2 commits
-
Building s390 kernel with CONFIG_DEBUG_INFO_BTF fails, because
CONFIG_OUTPUT_FORMAT is not defined. As a matter of fact, this variable
appears to be x86-only, so other arches might be affected as well.Fix by obtaining this value from objdump output, just like it's already
done for bin_arch. The exact objdump invocation is "inspired" by
arch/powerpc/boot/wrapper.Also, use LANG=C for the existing bin_arch objdump invocation to avoid
potential build issues on systems with non-English locale.Fixes: 341dfcf8d78e ("btf: expose BTF info through sysfs")
Signed-off-by: Ilya Leoshkevich
Signed-off-by: Daniel Borkmann -
When kallsyms generation happens, temporary vmlinux outputs are linked
but the quiet make output didn't report it, giving the impression that
the prior command is taking longer than expected.Instead, report the linking step explicitly. While at it, this
consolidates the repeated "kallsyms generation step" into a single
function and removes the existing copy/pasting.Signed-off-by: Kees Cook
Signed-off-by: Masahiro Yamada
14 Aug, 2019
2 commits
-
Daniel Borkmann says:
====================
The following pull-request contains BPF updates for your *net-next* tree.There is a small merge conflict in libbpf (Cc Andrii so he's in the loop
as well):for (i = 1; i info = BTF_INFO_ENC(BTF_KIND_INT, 0, 0);
<<<<<<< HEAD
/*
* using size = 1 is the safest choice, 4 will be too
* big and cause kernel BTF validation failure if
* original variable took less than 4 bytes
*/
t->size = 1;
*(int *)(t+1) = BTF_INT_ENC(0, 0, 8);
} else if (!has_datasec && kind == BTF_KIND_DATASEC) {
=======
t->size = sizeof(int);
*(int *)(t + 1) = BTF_INT_ENC(0, 0, 32);
} else if (!has_datasec && btf_is_datasec(t)) {
>>>>>>> 72ef80b5ee131e96172f19e74b4f98fa3404efe8
/* replace DATASEC with STRUCT */Conflict is between the two commits 1d4126c4e119 ("libbpf: sanitize VAR to
conservative 1-byte INT") and b03bc6853c0e ("libbpf: convert libbpf code to
use new btf helpers"), so we need to pick the sanitation fixup as well as
use the new btf_is_datasec() helper and the whitespace cleanup. Looks like
the following:[...]
if (!has_datasec && btf_is_var(t)) {
/* replace VAR with INT */
t->info = BTF_INFO_ENC(BTF_KIND_INT, 0, 0);
/*
* using size = 1 is the safest choice, 4 will be too
* big and cause kernel BTF validation failure if
* original variable took less than 4 bytes
*/
t->size = 1;
*(int *)(t + 1) = BTF_INT_ENC(0, 0, 8);
} else if (!has_datasec && btf_is_datasec(t)) {
/* replace DATASEC with STRUCT */
[...]The main changes are:
1) Addition of core parts of compile once - run everywhere (co-re) effort,
that is, relocation of fields offsets in libbpf as well as exposure of
kernel's own BTF via sysfs and loading through libbpf, from Andrii.More info on co-re: http://vger.kernel.org/bpfconf2019.html#session-2
and http://vger.kernel.org/lpc-bpf2018.html#session-22) Enable passing input flags to the BPF flow dissector to customize parsing
and allowing it to stop early similar to the C based one, from Stanislav.3) Add a BPF helper function that allows generating SYN cookies from XDP and
tc BPF, from Petar.4) Add devmap hash-based map type for more flexibility in device lookup for
redirects, from Toke.5) Improvements to XDP forwarding sample code now utilizing recently enabled
devmap lookups, from Jesper.6) Add support for reporting the effective cgroup progs in bpftool, from Jakub
and Takshak.7) Fix reading kernel config from bpftool via /proc/config.gz, from Peter.
8) Fix AF_XDP umem pages mapping for 32 bit architectures, from Ivan.
9) Follow-up to add two more BPF loop tests for the selftest suite, from Alexei.
10) Add perf event output helper also for other skb-based program types, from Allan.
11) Fix a co-re related compilation error in selftests, from Yonghong.
====================Signed-off-by: Jakub Kicinski
-
Expose kernel's BTF under the name vmlinux to be more uniform with using
kernel module names as file names in the future.Fixes: 341dfcf8d78e ("btf: expose BTF info through sysfs")
Suggested-by: Daniel Borkmann
Signed-off-by: Andrii Nakryiko
Signed-off-by: Daniel Borkmann
13 Aug, 2019
1 commit
-
Make .BTF section allocated and expose its contents through sysfs.
/sys/kernel/btf directory is created to contain all the BTFs present
inside kernel. Currently there is only kernel's main BTF, represented as
/sys/kernel/btf/kernel file. Once kernel modules' BTFs are supported,
each module will expose its BTF as /sys/kernel/btf/ file.Current approach relies on a few pieces coming together:
1. pahole is used to take almost final vmlinux image (modulo .BTF and
kallsyms) and generate .BTF section by converting DWARF info into
BTF. This section is not allocated and not mapped to any segment,
though, so is not yet accessible from inside kernel at runtime.
2. objcopy dumps .BTF contents into binary file and subsequently
convert binary file into linkable object file with automatically
generated symbols _binary__btf_kernel_bin_start and
_binary__btf_kernel_bin_end, pointing to start and end, respectively,
of BTF raw data.
3. final vmlinux image is generated by linking this object file (and
kallsyms, if necessary). sysfs_btf.c then creates
/sys/kernel/btf/kernel file and exposes embedded BTF contents through
it. This allows, e.g., libbpf and bpftool access BTF info at
well-known location, without resorting to searching for vmlinux image
on disk (location of which is not standardized and vmlinux image
might not be even available in some scenarios, e.g., inside qemu
during testing).Alternative approach using .incbin assembler directive to embed BTF
contents directly was attempted but didn't work, because sysfs_proc.o is
not re-compiled during link-vmlinux.sh stage. This is required, though,
to update embedded BTF data (initially empty data is embedded, then
pahole generates BTF info and we need to regenerate sysfs_btf.o with
updated contents, but it's too late at that point).If BTF couldn't be generated due to missing or too old pahole,
sysfs_btf.c handles that gracefully by detecting that
_binary__btf_kernel_bin_start (weak symbol) is 0 and not creating
/sys/kernel/btf at all.v2->v3:
- added Documentation/ABI/testing/sysfs-kernel-btf (Greg K-H);
- created proper kobject (btf_kobj) for btf directory (Greg K-H);
- undo v2 change of reusing vmlinux, as it causes extra kallsyms pass
due to initially missing __binary__btf_kernel_bin_{start/end} symbols;v1->v2:
- allow kallsyms stage to re-use vmlinux generated by gen_btf();Reviewed-by: Greg Kroah-Hartman
Signed-off-by: Andrii Nakryiko
Signed-off-by: Daniel Borkmann
31 Jul, 2019
1 commit
-
Since commit ff9b45c55b26 ("kbuild: modpost: read modules.order instead
of $(MODVERDIR)/*.mod"), 'make vmlinux' emits a warning, like this:$ make defconfig vmlinux
[ snip ]
LD vmlinux.o
cat: modules.order: No such file or directory
MODPOST vmlinux.o
MODINFO modules.builtin.modinfo
KSYM .tmp_kallsyms1.o
KSYM .tmp_kallsyms2.o
LD vmlinux
SORTEX vmlinux
SYSMAP System.mapWhen building only vmlinux, KBUILD_MODULES is not set. Hence, the
modules.order is not generated. For the vmlinux modpost, it is not
necessary at all.Separate scripts/Makefile.modpost for the vmlinux/modules stages.
This works more efficiently because the vmlinux modpost does not
need to include .*.cmd files.Fixes: ff9b45c55b26 ("kbuild: modpost: read modules.order instead of $(MODVERDIR)/*.mod")
Signed-off-by: Masahiro Yamada
09 May, 2019
1 commit
-
Pull Kbuild updates from Masahiro Yamada:
- allow users to invoke 'make' out of the source tree
- refactor scripts/mkmakefile
- deprecate KBUILD_SRC, which was used to track the source tree
location for O= build.- fix recordmcount.pl in case objdump output is localized
- turn unresolved symbols in external modules to errors from warnings
by default; pass KBUILD_MODPOST_WARN=1 to get them back to warnings- generate modules.builtin.modinfo to collect .modinfo data from
built-in modules- misc Makefile cleanups
* tag 'kbuild-v5.2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (21 commits)
.gitignore: add more all*.config patterns
moduleparam: Save information about built-in modules in separate file
Remove MODULE_ALIAS() calls that take undefined macro
.gitignore: add leading and trailing slashes to generated directories
scripts/tags.sh: fix direct execution of scripts/tags.sh
scripts: override locale from environment when running recordmcount.pl
samples: kobject: allow CONFIG_SAMPLE_KOBJECT to become y
samples: seccomp: turn CONFIG_SAMPLE_SECCOMP into a bool option
kbuild: move Documentation to vmlinux-alldirs
kbuild: move samples/ to KBUILD_VMLINUX_OBJS
modpost: make KBUILD_MODPOST_WARN also configurable for external modules
kbuild: check arch/$(SRCARCH)/include/generated before out-of-tree build
kbuild: remove unneeded dependency for include/config/kernel.release
memory: squash drivers/memory/Makefile.asm-offsets
kbuild: use $(srctree) instead of KBUILD_SRC to check out-of-tree build
kbuild: mkmakefile: generate a simple wrapper of top Makefile
kbuild: mkmakefile: do not check the generated Makefile marker
kbuild: allow Kbuild to start from any directory
kbuild: pass $(MAKECMDGOALS) to sub-make as is
kbuild: fix warning "overriding recipe for target 'Makefile'"
...
07 May, 2019
1 commit
-
Problem:
When a kernel module is compiled as a separate module, some important
information about the kernel module is available via .modinfo section of
the module. In contrast, when the kernel module is compiled into the
kernel, that information is not available.Information about built-in modules is necessary in the following cases:
1. When it is necessary to find out what additional parameters can be
passed to the kernel at boot time.2. When you need to know which module names and their aliases are in
the kernel. This is very useful for creating an initrd image.Proposal:
The proposed patch does not remove .modinfo section with module
information from the vmlinux at the build time and saves it into a
separate file after kernel linking. So, the kernel does not increase in
size and no additional information remains in it. Information is stored
in the same format as in the separate modules (null-terminated string
array). Because the .modinfo section is already exported with a separate
modules, we are not creating a new API.It can be easily read in the userspace:
$ tr '\0' '\n' < modules.builtin.modinfo
ext4.softdep=pre: crc32c
ext4.license=GPL
ext4.description=Fourth Extended Filesystem
ext4.author=Remy Card, Stephen Tweedie, Andrew Morton, Andreas Dilger, Theodore Ts'o and others
ext4.alias=fs-ext4
ext4.alias=ext3
ext4.alias=fs-ext3
ext4.alias=ext2
ext4.alias=fs-ext2
md_mod.alias=block-major-9-*
md_mod.alias=md
md_mod.description=MD RAID framework
md_mod.license=GPL
md_mod.parmtype=create_on_open:bool
md_mod.parmtype=start_dirty_degraded:int
...Co-Developed-by: Gleb Fotengauer-Malinovskiy
Signed-off-by: Gleb Fotengauer-Malinovskiy
Signed-off-by: Alexey Gladkov
Acked-by: Jessica Yu
Signed-off-by: Masahiro Yamada
06 May, 2019
1 commit
-
When BTF generation is enabled through CONFIG_DEBUG_INFO_BTF,
scripts/link-vmlinux.sh detects if pahole version is too old and
gracefully continues build process, skipping BTF generation build step.
But if pahole is not available, build will still fail. This patch adds
check for whether pahole exists at all and bails out gracefully, if not.Cc: Alexei Starovoitov
Reported-by: Yonghong Song
Fixes: e83b9f55448a ("kbuild: add ability to generate BTF type info for vmlinux")
Signed-off-by: Andrii Nakryiko
Signed-off-by: Daniel Borkmann
16 Apr, 2019
1 commit
-
When CONFIG_DEBUG_INFO_BTF is enabled but available version of pahole is too
old to support BTF generation, build script is supposed to emit warning and
proceed with the build. Due to using exit instead of return from BASH function,
existing handling code prematurely exits exit code 0, not completing some of
the build steps. This patch fixes issue by correctly returning just from
gen_btf() function only.Fixes: e83b9f55448a ("kbuild: add ability to generate BTF type info for vmlinux")
Cc: Masahiro Yamada
Cc: Arnaldo Carvalho de Melo
Cc: Daniel Borkmann
Cc: Alexei Starovoitov
Cc: Yonghong Song
Cc: Martin KaFai Lau
Signed-off-by: Andrii Nakryiko
Acked-by: Song Liu
Signed-off-by: Daniel Borkmann
03 Apr, 2019
1 commit
-
This patch adds new config option to trigger generation of BTF type
information from DWARF debuginfo for vmlinux and kernel modules through
pahole, which in turn relies on libbpf for btf_dedup() algorithm.The intent is to record compact type information of all types used
inside kernel, including all the structs/unions/typedefs/etc. This
enables BPF's compile-once-run-everywhere ([0]) approach, in which
tracing programs that are inspecting kernel's internal data (e.g.,
struct task_struct) can be compiled on a system running some kernel
version, but would be possible to run on other kernel versions (and
configurations) without recompilation, even if the layout of structs
changed and/or some of the fields were added, removed, or renamed.This is only possible if BPF loader can get kernel type info to adjust
all the offsets correctly. This patch is a first time in this direction,
making sure that BTF type info is part of Linux kernel image in
non-loadable ELF section.BTF deduplication ([1]) algorithm typically provides 100x savings
compared to DWARF data, so resulting .BTF section is not big as is
typically about 2MB in size.[0] http://vger.kernel.org/lpc-bpf2018.html#session-2
[1] https://facebookmicrosites.github.io/bpf/blog/2018/11/14/btf-enhancement.htmlCc: Masahiro Yamada
Cc: Arnaldo Carvalho de Melo
Cc: Daniel Borkmann
Cc: Alexei Starovoitov
Cc: Yonghong Song
Cc: Martin KaFai Lau
Signed-off-by: Andrii Nakryiko
Acked-by: David S. Miller
Acked-by: Alexei Starovoitov
Acked-by: Daniel Borkmann
Signed-off-by: Daniel Borkmann
14 Mar, 2019
1 commit
-
As commit 423a8155facf ("kbuild: Fix reading of .config in
link-vmlinux.sh") addressed, some shells fail to perform '.' if
${KCONFIG_CONFIG} does not contain a slash at all.Instead, we can source include/config/auto.conf, which obviously
contain slashes, and we do not expect its file path overridden by
a user. Perhaps, the performance might be slightly better since
unset CONFIG options are stripped from include/config/auto.conf.scripts/setlocalversion already works this way.
Signed-off-by: Masahiro Yamada
28 Jan, 2019
2 commits
-
The top Makefile does not need to export KBUILD_VMLINUX_INIT and
KBUILD_VMLINUX_MAIN separately.Put every built-in.a into KBUILD_VMLINUX_OBJS. The order of
$(head-y), $(init-y), $(core-y), ... is still retained.Signed-off-by: Masahiro Yamada
-
The symbol table in the final archive is unneeded; the linker does not
require the symbol table after the --whole-archive option. Every object
file in the archive is included in the link anyway.Pass thin archives from subdirectories directly to the linker, and
remove the final archiving step.Fix up the document and comments as well.
Signed-off-by: Masahiro Yamada
Acked-by: Nicholas Piggin
24 Aug, 2018
1 commit
-
Commit a0f97e06a43c ("kbuild: enable 'make CFLAGS=...' to add
additional options to CC") renamed CFLAGS to KBUILD_CFLAGS.Commit 222d394d30e7 ("kbuild: enable 'make AFLAGS=...' to add
additional options to AS") renamed AFLAGS to KBUILD_AFLAGS.Commit 06c5040cdb13 ("kbuild: enable 'make CPPFLAGS=...' to add
additional options to CPP") renamed CPPFLAGS to KBUILD_CPPFLAGS.For some reason, LDFLAGS was not renamed.
Using a well-known variable like LDFLAGS may result in accidental
override of the variable.Kbuild generally uses KBUILD_ prefixed variables for the internally
appended options, so here is one more conversion to sanitize the
naming convention.I did not touch Makefiles under tools/ since the tools build system
is a different world.Signed-off-by: Masahiro Yamada
Acked-by: Kirill A. Shutemov
Reviewed-by: Palmer Dabbelt
17 May, 2018
1 commit
-
CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX was selected by BLACKFIN, METAG.
They were removed by commit 4ba66a976072 ("arch: remove blackfin port"),
commit bb6fb6dfcc17 ("metag: Remove arch/metag/"), respectively.No more architecture enables CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX,
hence the --symbol-prefix option is unnecessary.Signed-off-by: Masahiro Yamada
Reviewed-by: Sam Ravnborg
26 Mar, 2018
3 commits
-
stat(1) is not standardized and different implementations have their own
(conflicting) flags for querying the size of a file.ls(1) provides the same information (value of st.st_size) in the 5th
column, except when the file is a character or block device. This output
is standardized[0]. The -n option turns on -l, which writes lines
formatted like"%s %u %s %s %u %s %s\n", , ,
, , , ,
but instead of writing the and , it writes the
numeric owner and group IDs (this avoids /etc/passwd and /etc/group
lookups as well as potential field splitting issues).The field is specified as "the value that would be returned for
the file in the st_size field of struct stat".To avoid duplicating logic in several locations in the tree, create
scripts/file-size.sh and update callers to use that instead of stat(1).[0] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/ls.html#tag_20_73_10
Signed-off-by: Michael Forney
Signed-off-by: Masahiro Yamada -
Incremental linking is gone, so rename built-in.o to built-in.a, which
is the usual extension for archive files.This patch does two things, first is a simple search/replace:
git grep -l 'built-in\.o' | xargs sed -i 's/built-in\.o/built-in\.a/g'
The second is to invert nesting of nested text manipulations to avoid
filtering built-in.a out from libs-y2:-libs-y2 := $(filter-out %.a, $(patsubst %/, %/built-in.a, $(libs-y)))
+libs-y2 := $(patsubst %/, %/built-in.a, $(filter-out %.a, $(libs-y)))Signed-off-by: Nicholas Piggin
Signed-off-by: Masahiro Yamada -
This removes the old `ld -r` incremental link option, which has not
been selected by any architecture since June 2017.Signed-off-by: Nicholas Piggin
Signed-off-by: Masahiro Yamada
02 Mar, 2018
1 commit
-
GCC_PLUGINS_CFLAGS is already in the environment, so it is superfluous
to add it in commandline of final build of init/.Signed-off-by: Cao jin
Signed-off-by: Masahiro Yamada
18 Nov, 2017
1 commit
-
Pull Kbuild updates from Masahiro Yamada:
"One of the most remarkable improvements in this cycle is, Kbuild is
now able to cache the result of shell commands. Some variables are
expensive to compute, for example, $(call cc-option,...) invokes the
compiler. It is not efficient to redo this computation every time,
even when we are not actually building anything. Kbuild creates a
hidden file ".cache.mk" that contains invoked shell commands and their
results. The speed-up should be noticeable.Summary:
- Fix arch build issues (hexagon, sh)
- Clean up various Makefiles and scripts
- Fix wrong usage of {CFLAGS,LDFLAGS}_MODULE in arch Makefiles
- Cache variables that are expensive to compute
- Improve cc-ldopton and ld-option for Clang
- Optimize output directory creation"
* tag 'kbuild-v4.15' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (30 commits)
kbuild: move coccicheck help from scripts/Makefile.help to top Makefile
sh: decompressor: add shipped files to .gitignore
frv: .gitignore: ignore vmlinux.lds
selinux: remove unnecessary assignment to subdir-
kbuild: specify FORCE in Makefile.headersinst as .PHONY target
kbuild: remove redundant mkdir from ./Kbuild
kbuild: optimize object directory creation for incremental build
kbuild: create object directories simpler and faster
kbuild: filter-out PHONY targets from "targets"
kbuild: remove redundant $(wildcard ...) for cmd_files calculation
kbuild: create directory for make cache only when necessary
sh: select KBUILD_DEFCONFIG depending on ARCH
kbuild: fix linker feature test macros when cross compiling with Clang
kbuild: shrink .cache.mk when it exceeds 1000 lines
kbuild: do not call cc-option before KBUILD_CFLAGS initialization
kbuild: Cache a few more calls to the compiler
kbuild: Add a cache for generated variables
kbuild: add forward declaration of default target to Makefile.asm-generic
kbuild: remove KBUILD_SUBDIR_ASFLAGS and KBUILD_SUBDIR_CCFLAGS
hexagon/kbuild: replace CFLAGS_MODULE with KBUILD_CFLAGS_MODULE
...
02 Nov, 2017
1 commit
-
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.By default all files without license information are under the default
license of the kernel, which is GPL version 2.Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if
Reviewed-by: Philippe Ombredanne
Reviewed-by: Thomas Gleixner
Signed-off-by: Greg Kroah-Hartman
09 Oct, 2017
1 commit
-
Since commit 1f2bfbd00e46 ("kbuild: link of vmlinux moved to a
script"), it is easy to increment .version without using a temporary
file .old_version.I do not see anybody who creates the .tmp_version. Probably it is a
left-over of commit 4e25d8bb9550fb ("[PATCH] kbuild: adjust .version
updating"). Just remove it.Signed-off-by: Masahiro Yamada
30 Jun, 2017
3 commits
-
The thin archives build currently puts all lib.a and built-in.o
files together and links them with --whole-archive.This works because thin archives can recursively refer to thin
archives. However some architectures include libgcc.a, which may
not be a thin archive, or it may not be constructed with the "P"
option, in which case its contents do not get linked correctly.So don't pull .a libs into the root built-in.o archive. These
libs should already have symbol tables and indexes built, so they
can be direct linker inputs. Move them out of the --whole-archive
option, which restore the conditional linking behaviour of lib.a
to thin archives builds.Signed-off-by: Nicholas Piggin
Signed-off-by: Masahiro Yamada -
The P option makes ar do full path name matching and can prevent ar
from discarding files with duplicate names in some cases of creating
thin archives from thin archives. The sh architecture in particular
loses some object files from its kernel/cpu/sh*/ directories without
this option.This could be a bug in binutils ar, but the P option should not cause
any negative effects so it is safe to use to work around this with.Signed-off-by: Nicholas Piggin
Signed-off-by: Masahiro Yamada -
Close the --whole-archives option with --no-whole-archive. Some
architectures end up including additional .o and files multiple
times after this, and they get duplicate symbols when they are
brought under the --whole-archives option.This matches more closely with the incremental final link.
Signed-off-by: Nicholas Piggin
Signed-off-by: Masahiro Yamada
29 Nov, 2016
2 commits
-
The root built-in.o archive is currently generated before all object
files are built for the final link, due to final build of init/ after
version update. In practice it seems like it doesn't matter because
the archive symbol table does not change, but it is more logical to
create the final archive as the last step.Signed-off-by: Nicholas Piggin
Signed-off-by: Michal Marek -
kallsyms generation is not foolproof, due to some linkers adding
symbols (e.g., branch trampolines) when a binary size changes.
Have it attempt a 3rd pass automatically if the kallsyms size changes
in the 2nd pass.This allows powerpc64 allyesconfig to build without adding another
pass when it's not required.This can be solved other ways by directing the linker not to add labels
on branch stubs, or to move kallsyms near the end of the image. The
former is undesirable for debugging/tracing, and the latter is a more
significant change that requires more testing and review.Signed-off-by: Nicholas Piggin
Signed-off-by: Michal Marek
09 Sep, 2016
1 commit
-
ld -r is an incremental link used to create built-in.o files in build
subdirectories. It produces relocatable object files containing all
its input files, and these are are then pulled together and relocated
in the final link. Aside from the bloat, this constrains the final
link relocations, which has bitten large powerpc builds with
unresolvable relocations in the final link.Alan Modra has recommended the kernel use thin archives for linking.
This is an alternative and means that the linker has more information
available to it when it links the kernel.This patch enables a config option architectures can select, which
causes all built-in.o files to be built as thin archives. built-in.o
files in subdirectories do not get symbol table or index attached,
which improves speed and size. The final link pass creates a
built-in.o archive in the root output directory which includes the
symbol table and index. The linker then uses takes this file to link.The --whole-archive linker option is required, because the linker now
has visibility to every individual object file, and it will otherwise
just completely avoid including those without external references
(consider a file with EXPORT_SYMBOL or initcall or hardware exceptions
as its only entry points). The traditional built works "by luck" as
built-in.o files are large enough that they're going to get external
references. However this optimisation is unpredictable for the kernel
(due to above external references), ineffective at culling unused, and
costly because the .o files have to be searched for references.
Superior alternatives for link-time culling should be used instead.Build characteristics for inclink vs thinarc, on a small powerpc64le
pseries VM with a modest .config:inclink thinarc
sizes
vmlinux 15 618 680 15 625 028
sum of all built-in.o 56 091 808 1 054 334
sum excluding root built-in.o 151 430find -name built-in.o | xargs rm ; time make vmlinux
real 22.772s 21.143s
user 13.280s 13.430s
sys 4.310s 2.750s- Final kernel pulled in only about 6K more, which shows how
ineffective the object file culling is.
- Build performance looks improved due to less pagecache activity.
On IO constrained systems it could be a bigger win.
- Build size saving is significant.Side note, the toochain understands archives, so there's some tricks,
$ ar t built-in.o # list all files you linked with
$ size built-in.o # and their sizes
$ objdump -d built-in.o # disassembly (unrelocated) with filenamesImplementation by sfr, minor tweaks by npiggin.
Signed-off-by: Stephen Rothwell
Signed-off-by: Nicholas Piggin
Signed-off-by: Michal Marek
08 Jun, 2016
1 commit
-
This patch allows to build the whole kernel with GCC plugins. It was ported from
grsecurity/PaX. The infrastructure supports building out-of-tree modules and
building in a separate directory. Cross-compilation is supported too.
Currently the x86, arm, arm64 and uml architectures enable plugins.The directory of the gcc plugins is scripts/gcc-plugins. You can use a file or a directory
there. The plugins compile with these options:
* -fno-rtti: gcc is compiled with this option so the plugins must use it too
* -fno-exceptions: this is inherited from gcc too
* -fasynchronous-unwind-tables: this is inherited from gcc too
* -ggdb: it is useful for debugging a plugin (better backtrace on internal
errors)
* -Wno-narrowing: to suppress warnings from gcc headers (ipa-utils.h)
* -Wno-unused-variable: to suppress warnings from gcc headers (gcc_version
variable, plugin-version.h)The infrastructure introduces a new Makefile target called gcc-plugins. It
supports all gcc versions from 4.5 to 6.0. The scripts/gcc-plugin.sh script
chooses the proper host compiler (gcc-4.7 can be built by either gcc or g++).
This script also checks the availability of the included headers in
scripts/gcc-plugins/gcc-common.h.The gcc-common.h header contains frequently included headers for GCC plugins
and it has a compatibility layer for the supported gcc versions.The gcc-generate-*-pass.h headers automatically generate the registration
structures for GIMPLE, SIMPLE_IPA, IPA and RTL passes.Note that 'make clean' keeps the *.so files (only the distclean or mrproper
targets clean all) because they are needed for out-of-tree modules.Based on work created by the PaX Team.
Signed-off-by: Emese Revfy
Acked-by: Kees Cook
Signed-off-by: Michal Marek
08 Apr, 2016
1 commit
-
Now that we no longer emit .stubs symbols into a section VMA loaded
at absolute address 0x1000, we can drop the ARM-specific override that
sets a lower limit based on CONFIG_PAGE_OFFSET, below which symbols are
filtered from the kallsyms output.Acked-by: Nicolas Pitre
Acked-by: Chris Brandt
Signed-off-by: Ard Biesheuvel
Signed-off-by: Russell King
25 Mar, 2016
1 commit
-
Pull kbuild updates from Michal Marek:
- make dtbs_install fix
- Error handling fix fixdep and link-vmlinux.sh
- __UNIQUE_ID fix for clang
- Fix for if_changed_* to suppress the "is up to date." message
- The kernel is built with -Werror=incompatible-pointer-types
* 'kbuild' of git://git.kernel.org/pub/scm/linux/kernel/git/mmarek/kbuild:
kbuild: Add option to turn incompatible pointer check into error
kbuild: suppress annoying "... is up to date." message
kbuild: fixdep: Check fstat(2) return value
scripts/link-vmlinux.sh: force error on kallsyms failure
Kbuild: provide a __UNIQUE_ID for clang
dtbsinstall: don't move target directory out of the way
16 Mar, 2016
1 commit
-
Similar to how relative extables are implemented, it is possible to emit
the kallsyms table in such a way that it contains offsets relative to
some anchor point in the kernel image rather than absolute addresses.On 64-bit architectures, it cuts the size of the kallsyms address table
in half, since offsets between kernel symbols can typically be expressed
in 32 bits. This saves several hundreds of kilobytes of permanent
.rodata on average. In addition, the kallsyms address table is no
longer subject to dynamic relocation when CONFIG_RELOCATABLE is in
effect, so the relocation work done after decompression now doesn't have
to do relocation updates for all these values. This saves up to 24
bytes (i.e., the size of a ELF64 RELA relocation table entry) per value,
which easily adds up to a couple of megabytes of uncompressed __init
data on ppc64 or arm64. Even if these relocation entries typically
compress well, the combined size reduction of 2.8 MB uncompressed for a
ppc64_defconfig build (of which 2.4 MB is __init data) results in a ~500
KB space saving in the compressed image.Since it is useful for some architectures (like x86) to retain the
ability to emit absolute values as well, this patch also adds support
for capturing both absolute and relative values when
KALLSYMS_ABSOLUTE_PERCPU is in effect, by emitting absolute per-cpu
addresses as positive 32-bit values, and addresses relative to the
lowest encountered relative symbol as negative values, which are
subtracted from the runtime address of this base symbol to produce the
actual address.Support for the above is enabled by default for all architectures except
IA-64 and Tile-GX, whose symbols are too far apart to capture in this
manner.Signed-off-by: Ard Biesheuvel
Tested-by: Guenter Roeck
Reviewed-by: Kees Cook
Tested-by: Kees Cook
Cc: Heiko Carstens
Cc: Michael Ellerman
Cc: Ingo Molnar
Cc: H. Peter Anvin
Cc: Benjamin Herrenschmidt
Cc: Michal Marek
Cc: Rusty Russell
Cc: Arnd Bergmann
Signed-off-by: Andrew Morton
Signed-off-by: Linus Torvalds