03 Sep, 2020

2 commits

  • commit e4a42c82e943b97ce124539fcd7a47445b43fa0d upstream.

    Redefine GZIP, BZIP2, LZOP variables as KGZIP, KBZIP2, KLZOP resp.
    GZIP, BZIP2, LZOP env variables are reserved by the tools. The original
    attempt to redefine them internally doesn't work in makefiles/scripts
    intercall scenarios, e.g., "make GZIP=gzip bindeb-pkg" and results in
    broken builds. There can be other broken build commands because of this,
    so the universal solution is to use non-reserved env variables for the
    compression tools.

    Fixes: 8dfb61dcbace ("kbuild: add variables for compression tools")
    Signed-off-by: Denis Efremov
    Tested-by: Guenter Roeck
    Signed-off-by: Masahiro Yamada
    Cc: Matthias Maennich
    Signed-off-by: Greg Kroah-Hartman

    Denis Efremov
     
  • commit 8dfb61dcbaceb19a5ded5e9c9dcf8d05acc32294 upstream.

    Allow user to use alternative implementations of compression tools,
    such as pigz, pbzip2, pxz. For example, multi-threaded tools to
    speed up the build:
    $ make GZIP=pigz BZIP2=pbzip2

    Variables _GZIP, _BZIP2, _LZOP are used internally because original env
    vars are reserved by the tools. The use of GZIP in gzip tool is obsolete
    since 2015. However, alternative implementations (e.g., pigz) still rely
    on it. BZIP2, BZIP, LZOP vars are not obsolescent.

    The credit goes to @grsecurity.

    As a sidenote, for multi-threaded lzma, xz compression one can use:
    $ export XZ_OPT="--threads=0"

    Signed-off-by: Denis Efremov
    Signed-off-by: Masahiro Yamada
    Signed-off-by: Matthias Maennich
    Signed-off-by: Greg Kroah-Hartman

    Denis Efremov
     

02 May, 2020

1 commit

  • commit 3d4b2238684ac919394eba7fb51bb7eeeec6ab57 upstream.

    Since commit 7a0496056064 ("kbuild: fix DT binding schema rule to detect
    command line changes"), this rule is every time re-run even if you change
    nothing.

    cmd_dtc takes one additional parameter to pass to the -O option of dtc.

    We need to pass 'yaml' to if_changed_rule. Otherwise, cmd-check invoked
    from if_changed_rule is false positive.

    Fixes: 7a0496056064 ("kbuild: fix DT binding schema rule to detect command line changes")
    Signed-off-by: Masahiro Yamada
    Signed-off-by: Greg Kroah-Hartman

    Masahiro Yamada
     

05 Mar, 2020

2 commits

  • commit fcbb8461fd2376ba3782b5b8bd440c929b8e4980 upstream.

    There are both positive and negative options about this feature.
    At first, I thought it was a good idea, but actually Linus stated a
    negative opinion (https://lkml.org/lkml/2019/9/29/227). I admit it
    is ugly and annoying.

    The baseline I'd like to keep is the compile-test of uapi headers.
    (Otherwise, kernel developers have no way to ensure the correctness
    of the exported headers.)

    I will maintain a small build rule in usr/include/Makefile.
    Remove the other header test functionality.

    Signed-off-by: Masahiro Yamada
    [ added to 5.4.y due to start of build warnings from backported patches
    because of this feature - gregkh]
    Signed-off-by: Greg Kroah-Hartman

    Masahiro Yamada
     
  • commit 7a04960560640ac5b0b89461f7757322b57d0c7a upstream.

    This if_change_rule is not working properly; it cannot detect any
    command line change.

    The reason is because cmd-check in scripts/Kbuild.include compares
    $(cmd_$@) and $(cmd_$1), but cmd_dtc_dt_yaml does not exist here.

    For if_change_rule to work properly, the stem part of cmd_* and rule_*
    must match. Because this cmd_and_fixdep invokes cmd_dtc, this rule must
    be named rule_dtc.

    Fixes: 4f0e3a57d6eb ("kbuild: Add support for DT binding schema checks")
    Signed-off-by: Masahiro Yamada
    Acked-by: Rob Herring
    Signed-off-by: Greg Kroah-Hartman

    Masahiro Yamada
     

01 Oct, 2019

1 commit


06 Sep, 2019

1 commit

  • KBUILD_ENABLE_EXTRA_GCC_CHECKS started as a switch to add extra warning
    options for GCC, but now it is a historical misnomer since we use it
    also for Clang, DTC, and even kernel-doc.

    Rename it to more sensible, shorter KBUILD_EXTRA_WARN.

    For the backward compatibility, KBUILD_ENABLE_EXTRA_GCC_CHECKS is still
    supported (but not advertised in the documentation).

    I also fixed up 'make help', and updated the documentation.

    Signed-off-by: Masahiro Yamada
    Reviewed-by: Nathan Chancellor
    Reviewed-by: Nick Desaulniers
    Reviewed-by: Sedat Dilek

    Masahiro Yamada
     

04 Sep, 2019

2 commits

  • Kbuild provides per-file compiler flag addition/removal:

    CFLAGS_.o
    CFLAGS_REMOVE_.o
    AFLAGS_.o
    AFLAGS_REMOVE_.o
    CPPFLAGS_.lds
    HOSTCFLAGS_.o
    HOSTCXXFLAGS_.o

    The is the filename of the target with its directory and
    suffix stripped.

    This syntax comes into a trouble when two files with the same basename
    appear in one Makefile, for example:

    obj-y += foo.o
    obj-y += dir/foo.o
    CFLAGS_foo.o :=

    Here, the applies to both foo.o and dir/foo.o

    The real world problem is:

    scripts/kconfig/util.c
    scripts/kconfig/lxdialog/util.c

    Both files are compiled into scripts/kconfig/mconf, but only the
    latter should be given with the ncurses flags.

    It is more sensible to use the relative path to the Makefile, like this:

    obj-y += foo.o
    CFLAGS_foo.o :=
    obj-y += dir/foo.o
    CFLAGS_dir/foo.o :=

    At first, I attempted to replace $(basetarget) with $*. The $* variable
    is replaced with the stem ('%') part in a pattern rule. This works with
    most of cases, but does not for explicit rules.

    For example, arch/ia64/lib/Makefile reuses rule_as_o_S in its own
    explicit rules, so $* will be empty, resulting in ignoring the per-file
    AFLAGS.

    I introduced a new variable, target-stem, which can be used also from
    explicit rules.

    Signed-off-by: Masahiro Yamada
    Acked-by: Marc Zyngier

    Masahiro Yamada
     
  • CONFIG_SHELL falls back to sh when bash is not installed on the system,
    but nobody is testing such a case since bash is usually installed.
    So, shell scripts invoked by CONFIG_SHELL are only tested with bash.

    It makes it difficult to test whether the hashbang #!/bin/sh is real.
    For example, #!/bin/sh in arch/powerpc/kernel/prom_init_check.sh is
    false. (I fixed it up)

    Besides, some shell scripts invoked by CONFIG_SHELL use bash-extension
    and #!/bin/bash is specified as the hashbang, while CONFIG_SHELL may
    not always be set to bash.

    Probably, the right thing to do is to introduce BASH, which is bash by
    default, and always set CONFIG_SHELL to sh. Replace $(CONFIG_SHELL)
    with $(BASH) for bash scripts.

    If somebody tries to add bash-extension to a #!/bin/sh script, it will
    be caught in testing because /bin/sh is a symlink to dash on some major
    distributions.

    Signed-off-by: Masahiro Yamada

    Masahiro Yamada
     

22 Aug, 2019

1 commit


15 Aug, 2019

1 commit


14 Aug, 2019

2 commits

  • Flex and bison are used for kconfig, dtc, genksyms, all of which are
    host programs. I never imagine the kernel embeds a parser or a lexer.

    Move the flex and bison rules to scripts/Makefile.host. This file is
    included only when hostprogs-y etc. is present in the Makefile in the
    directory. So, parsing these rules are skipped in most of directories.

    Signed-off-by: Masahiro Yamada

    Masahiro Yamada
     
  • We generally expect bison to create not only a C file, but also a
    header, which will be included from the lexer.

    Currently, Kbuild generates them in separate rules. So, for instance,
    when building Kconfig, you will notice bison is invoked twice:

    HOSTCC scripts/kconfig/conf.o
    HOSTCC scripts/kconfig/confdata.o
    HOSTCC scripts/kconfig/expr.o
    LEX scripts/kconfig/lexer.lex.c
    YACC scripts/kconfig/parser.tab.h
    HOSTCC scripts/kconfig/lexer.lex.o
    YACC scripts/kconfig/parser.tab.c
    HOSTCC scripts/kconfig/parser.tab.o
    HOSTCC scripts/kconfig/preprocess.o
    HOSTCC scripts/kconfig/symbol.o
    HOSTLD scripts/kconfig/conf

    Make handles such cases nicely in pattern rules [1]. Merge the two
    rules so that one invokcation of bison can generate both of them.

    HOSTCC scripts/kconfig/conf.o
    HOSTCC scripts/kconfig/confdata.o
    HOSTCC scripts/kconfig/expr.o
    LEX scripts/kconfig/lexer.lex.c
    YACC scripts/kconfig/parser.tab.[ch]
    HOSTCC scripts/kconfig/lexer.lex.o
    HOSTCC scripts/kconfig/parser.tab.o
    HOSTCC scripts/kconfig/preprocess.o
    HOSTCC scripts/kconfig/symbol.o
    HOSTLD scripts/kconfig/conf

    [1] Pattern rule

    GNU Make manual says:
    "Pattern rules may have more than one target. Unlike normal rules,
    this does not act as many different rules with the same prerequisites
    and recipe. If a pattern rule has multiple targets, make knows that
    the rule's recipe is responsible for making all of the targets. The
    recipe is executed only once to make all the targets. When searching
    for a pattern rule to match a target, the target patterns of a rule
    other than the one that matches the target in need of a rule are
    incidental: make worries only about giving a recipe and prerequisites
    to the file presently in question. However, when this file's recipe is
    run, the other targets are marked as having been updated themselves."

    https://www.gnu.org/software/make/manual/html_node/Pattern-Intro.html

    Signed-off-by: Masahiro Yamada

    Masahiro Yamada
     

27 Jul, 2019

1 commit


17 Jul, 2019

1 commit


10 Jul, 2019

2 commits

  • A missing compression utility or other errors were not picked up by make
    and an empty kernel image was produced. By removing the &&, errors will
    no longer be ignored.

    Signed-off-by: Harald Seiler
    Signed-off-by: Masahiro Yamada

    Harald Seiler
     
  • Commit 25b146c5b8ce ("kbuild: allow Kbuild to start from any directory")
    deprecated KBUILD_SRCTREE.

    It is only used in tools/testing/selftest/ to distinguish out-of-tree
    build. Replace it with a new boolean flag, building_out_of_srctree.

    I also replaced the conditional ($(srctree),.) because the next commit
    will allow an absolute path to be used for $(srctree) even when building
    in the source tree.

    Signed-off-by: Masahiro Yamada

    Masahiro Yamada
     

09 Jul, 2019

2 commits

  • In my view, most of headers can be self-contained. So, it would be
    tedious to add every header to header-test-y explicitly. We usually
    end up with "all headers with some exceptions".

    There are two types in exceptions:

    [1] headers that are never compiled as standalone units

    For examples, include/linux/compiler-gcc.h is not intended for
    direct inclusion. We should always exclude such ones.

    [2] headers that are conditionally compiled as standalone units

    Some headers can be compiled only for particular architectures.
    For example, include/linux/arm-cci.h can be compiled only for
    arm/arm64 because it requires to exist.
    Clang can compile include/soc/nps/mtm.h only for arc because
    it contains an arch-specific register in inline assembler.

    So, you can write Makefile like this:

    header-test- += linux/compiler-gcc.h
    header-test-$(CONFIG_ARM) += linux/arm-cci.h
    header-test-$(CONFIG_ARM64) += linux/arm-cci.h
    header-test-$(CONFIG_ARC) += soc/nps/mtm.h

    The new syntax header-test-pattern-y will be useful to specify
    "the rest".

    The typical usage is like this:

    header-test-pattern-y += */*.h

    This will add all the headers in sub-directories to the test coverage,
    excluding $(header-test-). In this regards, header-test-pattern-y
    behaves like a weaker variant of header-test-y.

    Caveat:
    The patterns in header-test-pattern-y are prefixed with $(srctree)/$(src)/
    but not $(objtree)/$(obj)/. Stale generated headers are often left over
    when you traverse the git history without cleaning. Wildcard patterns for
    $(objtree) may match to stale headers, which could fail to compile.
    One pitfall is $(srctree)/$(src)/ and $(objtree)/$(obj)/ point to the
    same directory for in-tree building. So, header-test-pattern-y should
    be used with care since it can potentially match to stale headers.

    Caveat2:
    You could use wildcard for header-test-. For example,

    header-test- += asm-generic/%

    ... will exclude headers in asm-generic directory. Unfortunately, the
    wildcard character is '%' instead of '*' here because this is evaluated
    by $(filter-out ...) whereas header-test-pattern-y is evaluated by
    $(wildcard ...). This is a kludge, but seems useful in some places...

    Signed-off-by: Masahiro Yamada
    Tested-by: Jani Nikula

    Masahiro Yamada
     
  • header-test-y does not work with headers in sub-directories.

    For example, you may want to write a Makefile, like this:

    include/linux/Kbuild:

    header-test-y += mtd/nand.h

    This entry will create a wrapper include/linux/mtd/nand.hdrtest.c
    with the following content:

    #include "mtd/nand.h"

    To make this work, we need to add $(srctree)/include/linux to the
    header search path. It would be tedious to add ccflags-y.

    Instead, we could change the *.hdrtest.c rule to wrap:

    #include "nand.h"

    This works for in-tree build since #include "..." searches in the
    relative path from the header with this directive. For O=... build,
    we need to add $(srctree)/include/linux/mtd to the header search path,
    which will be even more tedious.

    After all, I thought it would be handier to compile headers directly
    without creating wrappers.

    I added a new build rule to compile %.h into %.h.s

    The target is %.h.s instead of %.h.o because it is slightly faster.
    Also, as for GCC, an empty assembly is smaller than an empty object.

    I wrote the build rule:

    $(CC) $(c_flags) -S -o $@ -x c /dev/null -include $<

    instead of:

    $(CC) $(c_flags) -S -o $@ -x c $<

    Both work fine with GCC, but the latter is bad for Clang.

    This comes down to the difference in the -Wunused-function policy.
    GCC does not warn about unused 'static inline' functions at all.
    Clang does not warn about the ones in included headers, but does
    about the ones in the source. So, we should handle headers as
    headers, not as source files.

    In fact, this has been hidden since commit abb2ea7dfd82 ("compiler,
    clang: suppress warning for unused static inline functions"), but we
    should not rely on that.

    Signed-off-by: Masahiro Yamada
    Acked-by: Jani Nikula
    Tested-by: Jani Nikula

    Masahiro Yamada
     

15 Jun, 2019

1 commit

  • Sometimes it's useful to be able to explicitly ensure certain headers
    remain self-contained, i.e. that they are compilable as standalone
    units, by including and/or forward declaring everything they depend on.

    Add special target header-test-y where individual Makefiles can add
    headers to be tested if CONFIG_HEADER_TEST is enabled. This will
    generate a dummy C file per header that gets built as part of extra-y.

    Signed-off-by: Jani Nikula
    Reviewed-by: Sam Ravnborg
    Signed-off-by: Masahiro Yamada

    Jani Nikula
     

22 May, 2019

1 commit


18 May, 2019

1 commit

  • The 'addtree' and 'flags' in scripts/Kbuild.include are so compilecated
    and ugly.

    As I mentioned in [1], Kbuild should stop automatic prefixing of header
    search path options.

    I fixed up (almost) all Makefiles in the kernel. Now 'addtree' and
    'flags' have been removed.

    Kbuild still caters to add $(srctree)/$(src) and $(objtree)/$(obj)
    to the header search path for O= building, but never touches extra
    compiler options from ccflags-y etc.

    [1]: https://patchwork.kernel.org/patch/9632347/

    Signed-off-by: Masahiro Yamada

    Masahiro Yamada
     

02 Apr, 2019

1 commit

  • KBUILD_SRC was conventionally used for some different purposes:
    [1] To remember the source tree path
    [2] As a flag to check if sub-make is already done
    [3] As a flag to check if Kbuild runs out of tree

    For [1], we do not need to remember it because the top Makefile
    can compute it by $(realpath $(dir $(lastword $(MAKEFILE_LIST))))

    [2] has been replaced with self-commenting 'sub_make_done'.

    For [3], we can distinguish in-tree/out-of-tree by comparing
    $(srctree) and '.'

    This commit converts [3] to prepare for the KBUILD_SRC removal.

    Signed-off-by: Masahiro Yamada

    Masahiro Yamada
     

14 Mar, 2019

1 commit


28 Jan, 2019

4 commits


06 Jan, 2019

4 commits


30 Dec, 2018

1 commit

  • Pull Kbuild updates from Masahiro Yamada:
    "Kbuild core:
    - remove unneeded $(call cc-option,...) switches
    - consolidate Clang compiler flags into CLANG_FLAGS
    - announce the deprecation of SUBDIRS
    - fix single target build for external module
    - simplify the dependencies of 'prepare' stage targets
    - allow fixdep to directly write to .*.cmd files
    - simplify dependency generation for CONFIG_TRIM_UNUSED_KSYMS
    - change if_changed_rule to accept multi-line recipe
    - move .SECONDARY special target to scripts/Kbuild.include
    - remove redundant 'set -e'
    - improve parallel execution for CONFIG_HEADERS_CHECK
    - misc cleanups

    Treewide fixes and cleanups
    - set Clang flags correctly for PowerPC boot images
    - fix UML build error with CONFIG_GCC_PLUGINS
    - remove unneeded patterns from .gitignore files
    - refactor firmware/Makefile
    - remove unneeded rules for *offsets.s
    - avoid unneeded regeneration of intermediate .s files
    - clean up ./Kbuild

    Modpost:
    - remove unused -M, -K options
    - fix false positive warnings about section mismatch
    - use simple devtable lookup instead of linker magic
    - misc cleanups

    Coccinelle:
    - relax boolinit.cocci checks for overall consistency
    - fix warning messages of boolinit.cocci

    Other tools:
    - improve -dirty check of scripts/setlocalversion
    - add a tool to generate compile_commands.json from .*.cmd files"

    * tag 'kbuild-v4.21' of git://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild: (51 commits)
    kbuild: remove unused cmd_gentimeconst
    kbuild: remove $(obj)/ prefixes in ./Kbuild
    treewide: add intermediate .s files to targets
    treewide: remove explicit rules for *offsets.s
    firmware: refactor firmware/Makefile
    firmware: remove unnecessary patterns from .gitignore
    scripts: remove unnecessary ihex2fw and check-lc_ctypes from .gitignore
    um: remove unused filechk_gen_header in Makefile
    scripts: add a tool to produce a compile_commands.json file
    kbuild: add -Werror=implicit-int flag unconditionally
    kbuild: add -Werror=strict-prototypes flag unconditionally
    kbuild: add -fno-PIE flag unconditionally
    scripts: coccinelle: Correct warning message
    scripts: coccinelle: only suggest true/false in files that already use them
    kbuild: handle part-of-module correctly for *.ll and *.symtypes
    kbuild: refactor part-of-module
    kbuild: refactor quiet_modtag
    kbuild: remove redundant quiet_modtag for $(obj-m)
    kbuild: refactor Makefile.asm-generic
    user/Makefile: Fix typo and capitalization in comment section
    ...

    Linus Torvalds
     

14 Dec, 2018

1 commit


13 Dec, 2018

1 commit

  • This adds the build infrastructure for checking DT binding schema
    documents and validating dts files using the binding schema.

    Check DT binding schema documents:
    make dt_binding_check

    Build dts files and check using DT binding schema:
    make dtbs_check

    Optionally, DT_SCHEMA_FILES can be passed in with a schema file(s) to
    use for validation. This makes it easier to find and fix errors
    generated by a specific schema.

    Currently, the validation targets are separate from a normal build to
    avoid a hard dependency on the external DT schema project and because
    there are lots of warnings generated.

    Cc: Jonathan Corbet
    Cc: Mark Rutland
    Acked-by: Masahiro Yamada
    Cc: Michal Marek
    Cc: linux-doc@vger.kernel.org
    Cc: devicetree@vger.kernel.org
    Cc: linux-kbuild@vger.kernel.org
    Signed-off-by: Rob Herring

    Rob Herring
     

01 Dec, 2018

1 commit


30 Nov, 2018

2 commits


02 Oct, 2018

1 commit

  • There is nothing arch specific about building dtb files other than their
    location under /arch/*/boot/dts/. Keeping each arch aligned is a pain.
    The dependencies and supported targets are all slightly different.
    Also, a cross-compiler for each arch is needed, but really the host
    compiler preprocessor is perfectly fine for building dtbs. Move the
    build rules to a common location and remove the arch specific ones. This
    is done in a single step to avoid warnings about overriding rules.

    The build dependencies had been a mixture of 'scripts' and/or 'prepare'.
    These pull in several dependencies some of which need a target compiler
    (specifically devicetable-offsets.h) and aren't needed to build dtbs.
    All that is really needed is dtc, so adjust the dependencies to only be
    dtc.

    This change enables support 'dtbs_install' on some arches which were
    missing the target.

    Acked-by: Will Deacon
    Acked-by: Paul Burton
    Acked-by: Ley Foon Tan
    Acked-by: Masahiro Yamada
    Cc: Michal Marek
    Cc: Vineet Gupta
    Cc: Russell King
    Cc: Catalin Marinas
    Cc: Yoshinori Sato
    Cc: Michal Simek
    Cc: Ralf Baechle
    Cc: James Hogan
    Cc: Benjamin Herrenschmidt
    Cc: Paul Mackerras
    Cc: Michael Ellerman
    Cc: Chris Zankel
    Cc: Max Filippov
    Cc: linux-kbuild@vger.kernel.org
    Cc: linux-snps-arc@lists.infradead.org
    Cc: linux-arm-kernel@lists.infradead.org
    Cc: uclinux-h8-devel@lists.sourceforge.jp
    Cc: linux-mips@linux-mips.org
    Cc: nios2-dev@lists.rocketboards.org
    Cc: linuxppc-dev@lists.ozlabs.org
    Cc: linux-xtensa@linux-xtensa.org
    Signed-off-by: Rob Herring

    Rob Herring
     

24 Aug, 2018

1 commit

  • Commit a0f97e06a43c ("kbuild: enable 'make CFLAGS=...' to add
    additional options to CC") renamed CFLAGS to KBUILD_CFLAGS.

    Commit 222d394d30e7 ("kbuild: enable 'make AFLAGS=...' to add
    additional options to AS") renamed AFLAGS to KBUILD_AFLAGS.

    Commit 06c5040cdb13 ("kbuild: enable 'make CPPFLAGS=...' to add
    additional options to CPP") renamed CPPFLAGS to KBUILD_CPPFLAGS.

    For some reason, LDFLAGS was not renamed.

    Using a well-known variable like LDFLAGS may result in accidental
    override of the variable.

    Kbuild generally uses KBUILD_ prefixed variables for the internally
    appended options, so here is one more conversion to sanitize the
    naming convention.

    I did not touch Makefiles under tools/ since the tools build system
    is a different world.

    Signed-off-by: Masahiro Yamada
    Acked-by: Kirill A. Shutemov
    Reviewed-by: Palmer Dabbelt

    Masahiro Yamada