20 Aug, 2020
1 commit
-
Change the format of processed-schema* from yaml to json to speed up
validation. With json output, using xargs and appending the output won't
work since json has explicit list begin and end characters. Instead,
we pass the schema files as a list in a temp file.The parsing time for the processed schema goes down from ~2sec to 70ms.
Also, 'make dtbs_check' becomes 33% faster.Some error messages are affected by this change. For example, "True was
expected" becomes "... is not of type 'boolean'". The order of messages
is also changed.Signed-off-by: Andrei Ziureaev
Signed-off-by: Rob Herring
10 Aug, 2020
3 commits
-
Pull Kbuild updates from Masahiro Yamada:
- run the checker (e.g. sparse) after the compiler
- remove unneeded cc-option tests for old compiler flags
- fix tar-pkg to install dtbs
- introduce ccflags-remove-y and asflags-remove-y syntax
- allow to trace functions in sub-directories of lib/
- introduce hostprogs-always-y and userprogs-always-y syntax
- various Makefile cleanups
* tag 'kbuild-v5.9' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild:
kbuild: stop filtering out $(GCC_PLUGINS_CFLAGS) from cc-option base
kbuild: include scripts/Makefile.* only when relevant CONFIG is enabled
kbuild: introduce hostprogs-always-y and userprogs-always-y
kbuild: sort hostprogs before passing it to ifneq
kbuild: move host .so build rules to scripts/gcc-plugins/Makefile
kbuild: Replace HTTP links with HTTPS ones
kbuild: trace functions in subdirectories of lib/
kbuild: introduce ccflags-remove-y and asflags-remove-y
kbuild: do not export LDFLAGS_vmlinux
kbuild: always create directories of targets
powerpc/boot: add DTB to 'targets'
kbuild: buildtar: add dtbs support
kbuild: remove cc-option test of -ffreestanding
kbuild: remove cc-option test of -fno-stack-protector
Revert "kbuild: Create directory for target DTB"
kbuild: run the checker after the compiler -
To build host programs, you need to add the program names to 'hostprogs'
to use the necessary build rule, but it is not enough to build them
because there is no dependency.There are two types of host programs: built as the prerequisite of
another (e.g. gen_crc32table in lib/Makefile), or always built when
Kbuild visits the Makefile (e.g. genksyms in scripts/genksyms/Makefile).The latter is typical in Makefiles under scripts/, which contains host
programs globally used during the kernel build. To build them, you need
to add them to both 'hostprogs' and 'always-y'.This commit adds hostprogs-always-y as a shorthand.
The same applies to user programs. net/bpfilter/Makefile builds
bpfilter_umh on demand, hence always-y is unneeded. In contrast,
programs under samples/ are added to both 'userprogs' and 'always-y'
so they are always built when Kbuild visits the Makefiles.userprogs-always-y works as a shorthand.
Signed-off-by: Masahiro Yamada
Acked-by: Miguel Ojeda -
CFLAGS_REMOVE_.o filters out flags when compiling a particular
object, but there is no convenient way to do that for every object in
a directory.Add ccflags-remove-y and asflags-remove-y to make it easily.
Use ccflags-remove-y to clean up some Makefiles.
The add/remove order works as follows:
[1] KBUILD_CFLAGS specifies compiler flags used globally
[2] ccflags-y adds compiler flags for all objects in the
current Makefile[3] ccflags-remove-y removes compiler flags for all objects in the
current Makefile (New feature)[4] CFLAGS_ adds compiler flags per file.
[5] CFLAGS_REMOVE_ removes compiler flags per file.
Having [3] before [4] allows us to remove flags from most (but not all)
objects in the current Makefile.For example, kernel/trace/Makefile removes $(CC_FLAGS_FTRACE)
from all objects in the directory, then adds it back to
trace_selftest_dynamic.o and CFLAGS_trace_kprobe_selftest.oThe same applies to lib/livepatch/Makefile.
Please note ccflags-remove-y has no effect to the sub-directories.
In contrast, the previous notation got rid of compiler flags also from
all the sub-directories.The following are not affected because they have no sub-directories:
arch/arm/boot/compressed/
arch/powerpc/xmon/
arch/sh/
kernel/trace/However, lib/ has several sub-directories.
To keep the behavior, I added ccflags-remove-y to all Makefiles
in subdirectories of lib/, except the following:lib/vdso/Makefile - Kbuild does not descend into this Makefile
lib/raid/test/Makefile - This is not used for the kernel buildI think commit 2464a609ded0 ("ftrace: do not trace library functions")
excluded too much. In the next commit, I will remove ccflags-remove-y
from the sub-directories of lib/.Suggested-by: Sami Tolvanen
Signed-off-by: Masahiro Yamada
Acked-by: Steven Rostedt (VMware)
Acked-by: Michael Ellerman (powerpc)
Acked-by: Brendan Higgins (KUnit)
Tested-by: Anders Roxell
31 Jul, 2020
1 commit
-
- Add the zstd and zstd22 cmds to scripts/Makefile.lib
- Add the HAVE_KERNEL_ZSTD and KERNEL_ZSTD options
Architecture specific support is still needed for decompression.
Signed-off-by: Nick Terrell
Signed-off-by: Ingo Molnar
Tested-by: Sedat Dilek
Reviewed-by: Kees Cook
Link: https://lore.kernel.org/r/20200730190841.2071656-4-nickrterrell@gmail.com
07 Jul, 2020
1 commit
-
This reverts commit 77479b38e2f58890eb221a0418357502a5b41cd6.
Since commit 8a78756eb545 ("kbuild: create object directories simpler
and faster"), all directories for 'targets' are created.'mkdir -p $(dir ${dtc-tmp})' is no longer needed.
Signed-off-by: Masahiro Yamada
30 Jun, 2020
2 commits
-
There are two processed schema files:
- processed-schema-examples.yaml
Used for 'make dt_binding_check'. This is always a full schema.
- processed-schema.yaml
Used for 'make dtbs_check'. This may be a full schema, or a smaller
subset if DT_SCHEMA_FILES is given by a user.If DT_SCHEMA_FILES is not specified, they are the same. You can copy
the former to the latter instead of running dt-mk-schema twice. This
saves the cpu time a lot when you do 'make dt_binding_check dtbs_check'
because building the full schema takes a couple of seconds.If DT_SCHEMA_FILES is specified, processed-schema.yaml is generated
based on the specified yaml files.Signed-off-by: Masahiro Yamada
Link: https://lore.kernel.org/r/20200625170434.635114-4-masahiroy@kernel.org
Signed-off-by: Rob Herring -
Sync with upstream dtc primarily to pickup the I2C bus check fixes. The
interrupt_provider check is noisy, so turn it off for now.This adds the following commits from upstream:
9d7888cbf19c dtc: Consider one-character strings as strings
8259d59f59de checks: Improve i2c reg property checking
fdabcf2980a4 checks: Remove warning for I2C_OWN_SLAVE_ADDRESS
2478b1652c8d libfdt: add extern "C" for C++
f68bfc2668b2 libfdt: trivial typo fix
7be250b4d059 libfdt: Correct condition for reordering blocks
81e0919a3e21 checks: Add interrupt provider test
85e5d839847a Makefile: when building libfdt only, do not add unneeded deps
b28464a550c5 Fix some potential unaligned accesses in dtcSigned-off-by: Rob Herring
14 Jun, 2020
1 commit
-
Pull more Kbuild updates from Masahiro Yamada:
- fix build rules in binderfs sample
- fix build errors when Kbuild recurses to the top Makefile
- covert '---help---' in Kconfig to 'help'
* tag 'kbuild-v5.8-2' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild:
treewide: replace '---help---' in Kconfig files with 'help'
kbuild: fix broken builds because of GZIP,BZIP2,LZOP variables
samples: binderfs: really compile this sample and fix build issues
12 Jun, 2020
1 commit
-
Merge the state of the locking kcsan branch before the read/write_once()
and the atomics modifications got merged.Squash the fallout of the rebase on top of the read/write once and atomic
fallback work into the merge. The history of the original branch is
preserved in tag locking-kcsan-2020-06-02.Signed-off-by: Thomas Gleixner
11 Jun, 2020
1 commit
-
Redefine GZIP, BZIP2, LZOP variables as KGZIP, KBZIP2, KLZOP resp.
GZIP, BZIP2, LZOP env variables are reserved by the tools. The original
attempt to redefine them internally doesn't work in makefiles/scripts
intercall scenarios, e.g., "make GZIP=gzip bindeb-pkg" and results in
broken builds. There can be other broken build commands because of this,
so the universal solution is to use non-reserved env variables for the
compression tools.Fixes: 8dfb61dcbace ("kbuild: add variables for compression tools")
Signed-off-by: Denis Efremov
Tested-by: Guenter Roeck
Signed-off-by: Masahiro Yamada
06 Jun, 2020
1 commit
-
Allow user to use alternative implementations of compression tools,
such as pigz, pbzip2, pxz. For example, multi-threaded tools to
speed up the build:
$ make GZIP=pigz BZIP2=pbzip2Variables _GZIP, _BZIP2, _LZOP are used internally because original env
vars are reserved by the tools. The use of GZIP in gzip tool is obsolete
since 2015. However, alternative implementations (e.g., pigz) still rely
on it. BZIP2, BZIP, LZOP vars are not obsolescent.The credit goes to @grsecurity.
As a sidenote, for multi-threaded lzma, xz compression one can use:
$ export XZ_OPT="--threads=0"Signed-off-by: Denis Efremov
Signed-off-by: Masahiro Yamada
03 Jun, 2020
1 commit
-
Make modules.order depend on $(obj-m), and use if_changed to build it.
This will avoid unneeded update of modules.order, which will be useful
to optimize the modpost stage.Currently, the second pass of modpost is always invoked. By checking the
timestamp of modules.order, we can avoid the unneeded modpost.Signed-off-by: Masahiro Yamada
01 Jun, 2020
1 commit
-
Remove the unneeded variables, __subdir-y and __subdir-m.
Signed-off-by: Masahiro Yamada
25 May, 2020
2 commits
-
built-in.a contains the built-in object paths from the current and sub
directories.module.order collects the module paths from the current and sub
directories.Make their build rules look more symmetrical.
Signed-off-by: Masahiro Yamada
-
Save $(addprefix ...) for subdir-obj-y.
Signed-off-by: Masahiro Yamada
17 May, 2020
1 commit
-
always, hostprogs-y, and hostprogs-m are deprecated.
There is no user in upstream code, but I will keep them for external
modules. I want to remove them entirely someday. Prompt downstream
users for the migration.Signed-off-by: Masahiro Yamada
12 May, 2020
2 commits
-
cmd_dtc takes the additional parameter $(2) to select the target
format, dtb or yaml. This makes things complicated when it is used
with cmd_and_fixdep and if_changed_rule. I actually stumbled on this.
See commit 3d4b2238684a ("kbuild: fix DT binding schema rule again to
avoid needless rebuilds").Extract the suffix part of the target instead of passing the parameter.
Fortunately, this works for both $(obj)/%.dtb and $(obj)/%.dt.yaml .Signed-off-by: Masahiro Yamada
-
This omits system headers from the generated header dependency.
System headers are not updated unless you upgrade the compiler. Nor do
they contain CONFIG options, so fixdep does not need to parse them.Having said that, the effect of this optimization will be quite small
because the kernel code generally does not include system headers
except . Host programs include a lot of system headers,
but there are not so many in the kernel tree.At first, keeping system headers in .*.cmd files might be useful to
detect the compiler update, but there is no guarantee that
is included from every file. So, I implemented a more reliable way in
the previous commit.Signed-off-by: Masahiro Yamada
23 Apr, 2020
1 commit
-
Since commit 7a0496056064 ("kbuild: fix DT binding schema rule to detect
command line changes"), this rule is every time re-run even if you change
nothing.cmd_dtc takes one additional parameter to pass to the -O option of dtc.
We need to pass 'yaml' to if_changed_rule. Otherwise, cmd-check invoked
from if_changed_rule is false positive.Fixes: 7a0496056064 ("kbuild: fix DT binding schema rule to detect command line changes")
Signed-off-by: Masahiro Yamada
13 Apr, 2020
1 commit
-
Resolve these conflicts:
arch/x86/Kconfig
arch/x86/kernel/MakefileDo a minor "evil merge" to move the KCSAN entry up a bit by a few lines
in the Kconfig to reduce the probability of future conflicts.Signed-off-by: Ingo Molnar
21 Mar, 2020
2 commits
-
Signed-off-by: Ingo Molnar
-
Conflicts:
arch/x86/purgatory/MakefileSigned-off-by: Ingo Molnar
03 Mar, 2020
1 commit
-
Most folks only run dt_binding_check on the single schema they care about
by setting DT_SCHEMA_FILES. That means example is only checked against
that one schema which is not always sufficient.Let's address this by splitting processed-schema.yaml into 2 files: one
that's always all schemas for the examples and one that's just the schema
in DT_SCHEMA_FILES for dtbs.Co-developed-by: Masahiro Yamada
Signed-off-by: Rob Herring
Signed-off-by: Masahiro Yamada
27 Feb, 2020
2 commits
-
This trailing semicolon is unneeded.
Signed-off-by: Masahiro Yamada
Acked-by: Rob Herring -
This if_change_rule is not working properly; it cannot detect any
command line change.The reason is because cmd-check in scripts/Kbuild.include compares
$(cmd_$@) and $(cmd_$1), but cmd_dtc_dt_yaml does not exist here.For if_change_rule to work properly, the stem part of cmd_* and rule_*
must match. Because this cmd_and_fixdep invokes cmd_dtc, this rule must
be named rule_dtc.Fixes: 4f0e3a57d6eb ("kbuild: Add support for DT binding schema checks")
Signed-off-by: Masahiro Yamada
Acked-by: Rob Herring
04 Feb, 2020
1 commit
-
In old days, the "host-progs" syntax was used for specifying host
programs. It was renamed to the current "hostprogs-y" in 2004.It is typically useful in scripts/Makefile because it allows Kbuild to
selectively compile host programs based on the kernel configuration.This commit renames like follows:
always -> always-y
hostprogs-y -> hostprogsSo, scripts/Makefile will look like this:
always-$(CONFIG_BUILD_BIN2C) += ...
always-$(CONFIG_KALLSYMS) += ...
...
hostprogs := $(always-y) $(always-m)I think this makes more sense because a host program is always a host
program, irrespective of the kernel configuration. We want to specify
which ones to compile by CONFIG options, so always-y will be handier.The "always", "hostprogs-y", "hostprogs-m" will be kept for backward
compatibility for a while.Signed-off-by: Masahiro Yamada
07 Jan, 2020
3 commits
-
When compiling, Kbuild passes KBUILD_BASENAME (basename of the object)
and KBUILD_MODNAME (basename of the module).This commit adds another one, KBUILD_MODFILE, which is the path of
the module. (or, the path of the module it would end up in if it were
compiled as a module.)The next commit will use this to generate modules.builtin without
tristate.conf.Signed-off-by: Masahiro Yamada
-
Make $(squote)$(quote)...$(quote)$(squote) a helper macro.
I will reuse it in the next commit.Signed-off-by: Masahiro Yamada
-
Both 'obj-y += foo/' and 'obj-m += foo/' request Kbuild to visit the
sub-directory foo/, but the difference is that only the former combines
foo/built-in.a into the built-in.a of the current directory because
everything in sub-directories visited by obj-m is supposed to be modular.So, it makes sense to create built-in.a only if that sub-directory is
reachable by the chain of obj-y. Otherwise, built-in.a will not be
linked into vmlinux anyway. For the same reason, it is pointless to
compile obj-y objects in the directory visited by obj-m.Signed-off-by: Masahiro Yamada
30 Dec, 2019
1 commit
-
Conflicts:
init/main.c
lib/Kconfig.debugSigned-off-by: Ingo Molnar
16 Nov, 2019
1 commit
-
Kernel Concurrency Sanitizer (KCSAN) is a dynamic data-race detector for
kernel space. KCSAN is a sampling watchpoint-based data-race detector.
See the included Documentation/dev-tools/kcsan.rst for more details.This patch adds basic infrastructure, but does not yet enable KCSAN for
any architecture.Signed-off-by: Marco Elver
Acked-by: Paul E. McKenney
Signed-off-by: Paul E. McKenney
14 Nov, 2019
1 commit
-
There are both positive and negative options about this feature.
At first, I thought it was a good idea, but actually Linus stated a
negative opinion (https://lkml.org/lkml/2019/9/29/227). I admit it
is ugly and annoying.The baseline I'd like to keep is the compile-test of uapi headers.
(Otherwise, kernel developers have no way to ensure the correctness
of the exported headers.)I will maintain a small build rule in usr/include/Makefile.
Remove the other header test functionality.Signed-off-by: Masahiro Yamada
01 Oct, 2019
1 commit
-
Commit 40df759e2b9e ("kbuild: Fix build with binutils
Reviewed-by: Nick Desaulniers
Tested-by: Nick Desaulniers
06 Sep, 2019
1 commit
-
KBUILD_ENABLE_EXTRA_GCC_CHECKS started as a switch to add extra warning
options for GCC, but now it is a historical misnomer since we use it
also for Clang, DTC, and even kernel-doc.Rename it to more sensible, shorter KBUILD_EXTRA_WARN.
For the backward compatibility, KBUILD_ENABLE_EXTRA_GCC_CHECKS is still
supported (but not advertised in the documentation).I also fixed up 'make help', and updated the documentation.
Signed-off-by: Masahiro Yamada
Reviewed-by: Nathan Chancellor
Reviewed-by: Nick Desaulniers
Reviewed-by: Sedat Dilek
04 Sep, 2019
2 commits
-
Kbuild provides per-file compiler flag addition/removal:
CFLAGS_.o
CFLAGS_REMOVE_.o
AFLAGS_.o
AFLAGS_REMOVE_.o
CPPFLAGS_.lds
HOSTCFLAGS_.o
HOSTCXXFLAGS_.oThe is the filename of the target with its directory and
suffix stripped.This syntax comes into a trouble when two files with the same basename
appear in one Makefile, for example:obj-y += foo.o
obj-y += dir/foo.o
CFLAGS_foo.o :=Here, the applies to both foo.o and dir/foo.o
The real world problem is:
scripts/kconfig/util.c
scripts/kconfig/lxdialog/util.cBoth files are compiled into scripts/kconfig/mconf, but only the
latter should be given with the ncurses flags.It is more sensible to use the relative path to the Makefile, like this:
obj-y += foo.o
CFLAGS_foo.o :=
obj-y += dir/foo.o
CFLAGS_dir/foo.o :=At first, I attempted to replace $(basetarget) with $*. The $* variable
is replaced with the stem ('%') part in a pattern rule. This works with
most of cases, but does not for explicit rules.For example, arch/ia64/lib/Makefile reuses rule_as_o_S in its own
explicit rules, so $* will be empty, resulting in ignoring the per-file
AFLAGS.I introduced a new variable, target-stem, which can be used also from
explicit rules.Signed-off-by: Masahiro Yamada
Acked-by: Marc Zyngier -
CONFIG_SHELL falls back to sh when bash is not installed on the system,
but nobody is testing such a case since bash is usually installed.
So, shell scripts invoked by CONFIG_SHELL are only tested with bash.It makes it difficult to test whether the hashbang #!/bin/sh is real.
For example, #!/bin/sh in arch/powerpc/kernel/prom_init_check.sh is
false. (I fixed it up)Besides, some shell scripts invoked by CONFIG_SHELL use bash-extension
and #!/bin/bash is specified as the hashbang, while CONFIG_SHELL may
not always be set to bash.Probably, the right thing to do is to introduce BASH, which is bash by
default, and always set CONFIG_SHELL to sh. Replace $(CONFIG_SHELL)
with $(BASH) for bash scripts.If somebody tries to add bash-extension to a #!/bin/sh script, it will
be caught in testing because /bin/sh is a symlink to dash on some major
distributions.Signed-off-by: Masahiro Yamada
22 Aug, 2019
1 commit
-
Makefile.lib is included by Makefile.modfinal as well as Makefile.build.
Move modkern_cflags to Makefile.lib in order to simplify cmd_cc_o_c
in Makefile.modfinal. Move modkern_cflags as well for consistency.Signed-off-by: Masahiro Yamada
15 Aug, 2019
1 commit
-
Currently, Kbuild treats an object as multi-used when any of
$(foo-objs), $(foo-y), $(foo-m) is set. It makes more sense to
check $(foo-) as well.In the context of foo-$(CONFIG_FOO_FEATURE1), CONFIG_FOO_FEATURE1
could be unset.Signed-off-by: Masahiro Yamada
14 Aug, 2019
1 commit
-
Flex and bison are used for kconfig, dtc, genksyms, all of which are
host programs. I never imagine the kernel embeds a parser or a lexer.Move the flex and bison rules to scripts/Makefile.host. This file is
included only when hostprogs-y etc. is present in the Makefile in the
directory. So, parsing these rules are skipped in most of directories.Signed-off-by: Masahiro Yamada