18 Aug, 2020

1 commit


10 Aug, 2020

1 commit

  • To build host programs, you need to add the program names to 'hostprogs'
    to use the necessary build rule, but it is not enough to build them
    because there is no dependency.

    There are two types of host programs: built as the prerequisite of
    another (e.g. gen_crc32table in lib/Makefile), or always built when
    Kbuild visits the Makefile (e.g. genksyms in scripts/genksyms/Makefile).

    The latter is typical in Makefiles under scripts/, which contains host
    programs globally used during the kernel build. To build them, you need
    to add them to both 'hostprogs' and 'always-y'.

    This commit adds hostprogs-always-y as a shorthand.

    The same applies to user programs. net/bpfilter/Makefile builds
    bpfilter_umh on demand, hence always-y is unneeded. In contrast,
    programs under samples/ are added to both 'userprogs' and 'always-y'
    so they are always built when Kbuild visits the Makefiles.

    userprogs-always-y works as a shorthand.

    Signed-off-by: Masahiro Yamada
    Acked-by: Miguel Ojeda

    Masahiro Yamada
     

25 Mar, 2020

1 commit


04 Feb, 2020

1 commit

  • In old days, the "host-progs" syntax was used for specifying host
    programs. It was renamed to the current "hostprogs-y" in 2004.

    It is typically useful in scripts/Makefile because it allows Kbuild to
    selectively compile host programs based on the kernel configuration.

    This commit renames like follows:

    always -> always-y
    hostprogs-y -> hostprogs

    So, scripts/Makefile will look like this:

    always-$(CONFIG_BUILD_BIN2C) += ...
    always-$(CONFIG_KALLSYMS) += ...
    ...
    hostprogs := $(always-y) $(always-m)

    I think this makes more sense because a host program is always a host
    program, irrespective of the kernel configuration. We want to specify
    which ones to compile by CONFIG options, so always-y will be handier.

    The "always", "hostprogs-y", "hostprogs-m" will be kept for backward
    compatibility for a while.

    Signed-off-by: Masahiro Yamada

    Masahiro Yamada
     

14 Sep, 2019

2 commits

  • I used the C comment style (/* ... */) for the flex and bison files
    as in Kconfig (scripts/kconfig/{lexer.l,parser.y})

    Signed-off-by: Masahiro Yamada

    Masahiro Yamada
     
  • Arnd Bergmann reported false-positive modpost warnings detected by his
    randconfig testing of linux-next.

    Actually, this happens under the combination of CONFIG_MODVERSIONS
    and CONFIG_TRIM_UNUSED_KSYMS since commit 15bfc2348d54 ("modpost:
    check for static EXPORT_SYMBOL* functions").

    For example, arch/arm/config/multi_v7_defconfig + CONFIG_MODVERSIONS
    + CONFIG_TRIM_UNUSED_KSYMS produces the following false-positives:

    WARNING: "__lshrdi3" [vmlinux] is a static (unknown)
    WARNING: "__ashrdi3" [vmlinux] is a static (unknown)
    WARNING: "__aeabi_lasr" [vmlinux] is a static (unknown)
    WARNING: "__aeabi_llsr" [vmlinux] is a static (unknown)
    WARNING: "ftrace_set_clr_event" [vmlinux] is a static (unknown)
    WARNING: "__muldi3" [vmlinux] is a static (unknown)
    WARNING: "__aeabi_ulcmp" [vmlinux] is a static (unknown)
    WARNING: "__ucmpdi2" [vmlinux] is a static (unknown)
    WARNING: "__aeabi_lmul" [vmlinux] is a static (unknown)
    WARNING: "__bswapsi2" [vmlinux] is a static (unknown)
    WARNING: "__bswapdi2" [vmlinux] is a static (unknown)
    WARNING: "__ashldi3" [vmlinux] is a static (unknown)
    WARNING: "__aeabi_llsl" [vmlinux] is a static (unknown)

    The root cause of the problem is not in the modpost, but in the
    implementation of CONFIG_TRIM_UNUSED_KSYMS.

    If there is at least one untrimmed symbol in the file, genksyms is
    invoked to calculate CRC of *all* the exported symbols in that file
    even if some of them have been trimmed due to no caller existing.

    As a result, .tmp_*.ver files contain CRC of trimmed symbols, thus
    unneeded, orphan __crc* symbols are added to objects. It had been
    harmless until recently.

    With commit 15bfc2348d54 ("modpost: check for static EXPORT_SYMBOL*
    functions"), it is now harmful because the bogus __crc* symbols make
    modpost call sym_update_crc() to add the symbols to the hash table,
    but there is no one that clears the ->is_static member.

    I gave Fixes to the first commit that uncovered the issue, but the
    potential problem has long existed since commit f235541699bc
    ("export.h: allow for per-symbol configurable EXPORT_SYMBOL()").

    Fixes: 15bfc2348d54 ("modpost: check for static EXPORT_SYMBOL* functions")
    Reported-by: Arnd Bergmann
    Signed-off-by: Masahiro Yamada
    Tested-by: Arnd Bergmann

    Masahiro Yamada
     

06 Sep, 2019

1 commit

  • KBUILD_ENABLE_EXTRA_GCC_CHECKS started as a switch to add extra warning
    options for GCC, but now it is a historical misnomer since we use it
    also for Clang, DTC, and even kernel-doc.

    Rename it to more sensible, shorter KBUILD_EXTRA_WARN.

    For the backward compatibility, KBUILD_ENABLE_EXTRA_GCC_CHECKS is still
    supported (but not advertised in the documentation).

    I also fixed up 'make help', and updated the documentation.

    Signed-off-by: Masahiro Yamada
    Reviewed-by: Nathan Chancellor
    Reviewed-by: Nick Desaulniers
    Reviewed-by: Sedat Dilek

    Masahiro Yamada
     

14 Aug, 2019

1 commit

  • We generally expect bison to create not only a C file, but also a
    header, which will be included from the lexer.

    Currently, Kbuild generates them in separate rules. So, for instance,
    when building Kconfig, you will notice bison is invoked twice:

    HOSTCC scripts/kconfig/conf.o
    HOSTCC scripts/kconfig/confdata.o
    HOSTCC scripts/kconfig/expr.o
    LEX scripts/kconfig/lexer.lex.c
    YACC scripts/kconfig/parser.tab.h
    HOSTCC scripts/kconfig/lexer.lex.o
    YACC scripts/kconfig/parser.tab.c
    HOSTCC scripts/kconfig/parser.tab.o
    HOSTCC scripts/kconfig/preprocess.o
    HOSTCC scripts/kconfig/symbol.o
    HOSTLD scripts/kconfig/conf

    Make handles such cases nicely in pattern rules [1]. Merge the two
    rules so that one invokcation of bison can generate both of them.

    HOSTCC scripts/kconfig/conf.o
    HOSTCC scripts/kconfig/confdata.o
    HOSTCC scripts/kconfig/expr.o
    LEX scripts/kconfig/lexer.lex.c
    YACC scripts/kconfig/parser.tab.[ch]
    HOSTCC scripts/kconfig/lexer.lex.o
    HOSTCC scripts/kconfig/parser.tab.o
    HOSTCC scripts/kconfig/preprocess.o
    HOSTCC scripts/kconfig/symbol.o
    HOSTLD scripts/kconfig/conf

    [1] Pattern rule

    GNU Make manual says:
    "Pattern rules may have more than one target. Unlike normal rules,
    this does not act as many different rules with the same prerequisites
    and recipe. If a pattern rule has multiple targets, make knows that
    the rule's recipe is responsible for making all of the targets. The
    recipe is executed only once to make all the targets. When searching
    for a pattern rule to match a target, the target patterns of a rule
    other than the one that matches the target in need of a rule are
    incidental: make worries only about giving a recipe and prerequisites
    to the file presently in question. However, when this file's recipe is
    run, the other targets are marked as having been updated themselves."

    https://www.gnu.org/software/make/manual/html_node/Pattern-Intro.html

    Signed-off-by: Masahiro Yamada

    Masahiro Yamada
     

24 Jun, 2019

1 commit

  • __uint128_t crops up in a few files that export symbols to modules, so
    teach genksyms about it and the other GCC built-in 128-bit integer types
    so that we don't end up skipping the CRC generation for some symbols due
    to the parser failing to spot them:

    | WARNING: EXPORT symbol "kernel_neon_begin" [vmlinux] version
    | generation failed, symbol will not be versioned.
    | ld: arch/arm64/kernel/fpsimd.o: relocation R_AARCH64_ABS32 against
    | `__crc_kernel_neon_begin' can not be used when making a shared
    | object
    | ld: arch/arm64/kernel/fpsimd.o:(.data+0x0): dangerous relocation:
    | unsupported relocation

    Reported-by: Arnd Bergmann
    Signed-off-by: Will Deacon
    Signed-off-by: Masahiro Yamada

    Will Deacon
     

31 May, 2019

1 commit

  • Based on 1 normalized pattern(s):

    this program is free software you can redistribute it and or modify
    it under the terms of the gnu general public license as published by
    the free software foundation either version 2 of the license or at
    your option any later version this program is distributed in the
    hope that it will be useful but without any warranty without even
    the implied warranty of merchantability or fitness for a particular
    purpose see the gnu general public license for more details you
    should have received a copy of the gnu general public license along
    with this program if not write to the free software foundation inc
    59 temple place suite 330 boston ma 02111 1307 usa

    extracted by the scancode license scanner the SPDX license identifier

    GPL-2.0-or-later

    has been chosen to replace the boilerplate/reference in 1334 file(s).

    Signed-off-by: Thomas Gleixner
    Reviewed-by: Allison Randal
    Reviewed-by: Richard Fontana
    Cc: linux-spdx@vger.kernel.org
    Link: https://lkml.kernel.org/r/20190527070033.113240726@linutronix.de
    Signed-off-by: Greg Kroah-Hartman

    Thomas Gleixner
     

21 May, 2019

1 commit

  • Add SPDX license identifiers to all files which:

    - Have no license information of any form

    - Have EXPORT_.*_SYMBOL_GPL inside which was used in the
    initial scan/conversion to ignore the file

    These files fall under the project license, GPL v2 only. The resulting SPDX
    license identifier is:

    GPL-2.0-only

    Signed-off-by: Thomas Gleixner
    Signed-off-by: Greg Kroah-Hartman

    Thomas Gleixner
     

18 May, 2019

1 commit

  • Currently, the Kbuild core manipulates header search paths in a crazy
    way [1].

    To fix this mess, I want all Makefiles to add explicit $(srctree)/ to
    the search paths in the srctree. Some Makefiles are already written in
    that way, but not all. The goal of this work is to make the notation
    consistent, and finally get rid of the gross hacks.

    Having whitespaces after -I does not matter since commit 48f6e3cf5bc6
    ("kbuild: do not drop -I without parameter").

    [1]: https://patchwork.kernel.org/patch/9632347/

    Signed-off-by: Masahiro Yamada

    Masahiro Yamada
     

17 May, 2018

1 commit

  • CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX was selected by BLACKFIN, METAG.
    They were removed by commit 4ba66a976072 ("arch: remove blackfin port"),
    commit bb6fb6dfcc17 ("metag: Remove arch/metag/"), respectively.

    No more architecture enables CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX,
    hence the -s (--symbol-prefix) option is unnecessary.

    Signed-off-by: Masahiro Yamada
    Reviewed-by: Sam Ravnborg

    Masahiro Yamada
     

05 May, 2018

1 commit


07 Apr, 2018

4 commits

  • Files generated by if_changed* must be added to 'targets' to include
    *.cmd files. Otherwise, they would be regenerated every time.

    The build system automatically adds objects to 'targets' where
    appropriate, such as obj-y, extra-y, etc. but does nothing for
    intermediate files. So, each Makefile needs to add them by itself.

    There are some common cases where objects are generated by chained
    rules. Lexers and parsers are compiled like follows:

    %.lex.o
    Acked-by: Frank Rowand

    Masahiro Yamada
     
  • Now that the kernel build supports flex and bison, remove the _shipped
    files and generate them during the build instead.

    There are no more shipped lexer and parser, so I ripped off the rules
    in scripts/Malefile.lib that were used for REGENERATE_PARSERS.

    The genksyms parser has ambiguous grammar, which would emit warnings:

    scripts/genksyms/parse.y: warning: 9 shift/reduce conflicts [-Wconflicts-sr]
    scripts/genksyms/parse.y: warning: 5 reduce/reduce conflicts [-Wconflicts-rr]

    They are normally suppressed, but displayed when W=1 is given.

    Signed-off-by: Masahiro Yamada

    Masahiro Yamada
     
  • Files suffixed by .lex.c, .tab.[ch] are generated lexers, parsers,
    respectively. Clean them up globally from the top Makefile.

    Some of the final host programs those lexer/parser are linked into
    are necessary for building external modules, but the intermediates
    are unneeded. They can be cleaned away by 'make clean' instead of
    'make mrproper'.

    Signed-off-by: Masahiro Yamada
    Acked-by: Frank Rowand

    Masahiro Yamada
     
  • These patterns are common to host programs that require lexer and parser.
    Move them to the top .gitignore.

    Signed-off-by: Masahiro Yamada
    Acked-by: Frank Rowand

    Masahiro Yamada
     

13 Jan, 2018

1 commit


02 Nov, 2017

1 commit

  • Many source files in the tree are missing licensing information, which
    makes it harder for compliance tools to determine the correct license.

    By default all files without license information are under the default
    license of the kernel, which is GPL version 2.

    Update the files which contain no license information with the 'GPL-2.0'
    SPDX license identifier. The SPDX identifier is a legally binding
    shorthand, which can be used instead of the full boiler plate text.

    This patch is based on work done by Thomas Gleixner and Kate Stewart and
    Philippe Ombredanne.

    How this work was done:

    Patches were generated and checked against linux-4.14-rc6 for a subset of
    the use cases:
    - file had no licensing information it it.
    - file was a */uapi/* one with no licensing information in it,
    - file was a */uapi/* one with existing licensing information,

    Further patches will be generated in subsequent months to fix up cases
    where non-standard license headers were used, and references to license
    had to be inferred by heuristics based on keywords.

    The analysis to determine which SPDX License Identifier to be applied to
    a file was done in a spreadsheet of side by side results from of the
    output of two independent scanners (ScanCode & Windriver) producing SPDX
    tag:value files created by Philippe Ombredanne. Philippe prepared the
    base worksheet, and did an initial spot review of a few 1000 files.

    The 4.13 kernel was the starting point of the analysis with 60,537 files
    assessed. Kate Stewart did a file by file comparison of the scanner
    results in the spreadsheet to determine which SPDX license identifier(s)
    to be applied to the file. She confirmed any determination that was not
    immediately clear with lawyers working with the Linux Foundation.

    Criteria used to select files for SPDX license identifier tagging was:
    - Files considered eligible had to be source code files.
    - Make and config files were included as candidates if they contained >5
    lines of source
    - File already had some variant of a license header in it (even if
    Reviewed-by: Philippe Ombredanne
    Reviewed-by: Thomas Gleixner
    Signed-off-by: Greg Kroah-Hartman

    Greg Kroah-Hartman
     

09 Sep, 2017

1 commit

  • I had stupidly missed one special use of 'is_reserved_word()' when I
    converted the code to avoid gperf.

    I had changed that function to return the token ID directly rather than
    a pointer to the token descriptor structure, but that meant that the
    test for "is this a reserved word" changed from checking the return
    value against NULL, to checking that it wasn't negative.

    And while I had converted the main token parser over, I missed the
    special case of the typeof phrase handling. And since our dependency
    chain for genksyms does not include the genksyms program itself
    changing, my kernel rebuild didn't show the problem.

    Fixes: bb3290d91695 ("Remove gperf usage from toolchain")
    Reported-by: Masahiro Yamada
    Signed-off-by: Linus Torvalds

    Linus Torvalds
     

20 Aug, 2017

1 commit

  • It turns out that gperf-3.1 changed types in the generated code in ways
    that aren't even trivially detectable without having to generate a test-file.

    It's just not worth using tools and libraries from clowns that don't
    understand or care about compatibility. So get rid of gperf.

    Signed-off-by: Linus Torvalds

    Linus Torvalds
     

06 Jun, 2017

1 commit

  • When compiling with -Wsuggest-attribute=format in HOSTCFLAGS, gcc
    complains that error_with_pos() may be declared with a printf format
    attribute:

    scripts/genksyms/genksyms.c:726:3: warning: function might be
    possible candidate for ‘gnu_printf’ format attribute
    [-Wsuggest-attribute=format]
    vfprintf(stderr, fmt, args);
    ^~~~~~~~

    This would allow catching printf-format errors at compile time in
    callers to error_with_pos(). Add this attribute.

    Signed-off-by: Nicolas Iooss
    Signed-off-by: Masahiro Yamada

    Nicolas Iooss
     

11 Mar, 2017

1 commit


04 Feb, 2017

1 commit

  • This add the kbuild infrastructure that will allow architectures to emit
    vmlinux symbol CRCs as 32-bit offsets to another location in the kernel
    where the actual value is stored. This works around problems with CRCs
    being mistaken for relocatable symbols on kernels that self relocate at
    runtime (i.e., powerpc with CONFIG_RELOCATABLE=y)

    For the kbuild side of things, this comes down to the following:

    - introducing a Kconfig symbol MODULE_REL_CRCS

    - adding a -R switch to genksyms to instruct it to emit the CRC symbols
    as references into the .rodata section

    - making modpost distinguish such references from absolute CRC symbols
    by the section index (SHN_ABS)

    - making kallsyms disregard non-absolute symbols with a __crc_ prefix

    Signed-off-by: Ard Biesheuvel
    Signed-off-by: Linus Torvalds

    Ard Biesheuvel
     

05 Jan, 2017

2 commits

  • Regenerate the parser after d920f7c6628c ("genksyms: Fix segfault with
    invalid declarations").

    Reported-and-tested-by: Borislav Petkov
    Signed-off-by: Michal Marek

    Michal Marek
     
  • Do not try to recover too early and segfault when parsing invalid
    declarations such as

    echo 'int (int);' | scripts/genksyms/genksyms
    echo 'int a, (int);' | scripts/genksyms/genksyms
    echo 'extern void *__inline_memcpy((void *), (const void *), (__kernel_size_t));' | scripts/genksyms/genksyms

    The last one was a real-life bug with
    include/asm-generic/asm-prototypes.h on x86_64.

    Reported-and-tested-by: Borislav Petkov
    Signed-off-by: Michal Marek

    Michal Marek
     

29 Nov, 2016

2 commits

  • Regenerate the keyword table and parser after commit 0efdb2282343
    ("kbuild/genksyms: handle va_list type").

    Signed-off-by: Michal Marek

    Michal Marek
     
  • genksyms currently does not handle va_list. Add the __builtin_va_list
    keyword as a type. This reduces the amount of syntax errors thrown,
    but so far no export symbol has a type with a va_list argument, so
    there is currently no bug in the end result.

    Note: this patch does not regenerate shipped parser files.

    Signed-off-by: Nicholas Piggin
    Signed-off-by: Michal Marek

    Nicholas Piggin
     

26 Aug, 2016

2 commits

  • Update the lexer after 4fab91605a6b ("kbuild: genksyms fix for typeof
    handling").

    Signed-off-by: Michal Marek

    Michal Marek
     
  • The tokenizer misses counting an open-parenthesis when parsing a
    non-trivial typeof beginning with an open-parenthesis. This function
    in include/linux/ceph/libceph.h

    static type *lookup_##name(struct rb_root *root,
    typeof(((type *)0)->keyfld) key)

    When instantiated in net/ceph/mon_client.c, causes subsequent symbols
    including an EXPORT_SYMBOL in that file to be lost.

    Signed-off-by: Nicholas Piggin
    Signed-off-by: Michal Marek

    Nicholas Piggin
     

20 Apr, 2016

1 commit


09 Dec, 2015

1 commit

  • The reference files use spaces to separate tokens, however, we must
    preserve spaces inside string literals. Currently the only case in the
    tree is struct edac_raw_error_desc in :

    $ KBUILD_SYMTYPES=1 make -s drivers/edac/amd64_edac.symtypes
    $ mv drivers/edac/amd64_edac.{symtypes,symref}
    $ KBUILD_SYMTYPES=1 make -s drivers/edac/amd64_edac.symtypes
    drivers/edac/amd64_edac.c:527: warning: amd64_get_dram_hole_info: modversion changed because of changes in struct edac_raw_error_desc

    Signed-off-by: Michal Marek

    Michal Marek
     

20 Aug, 2015

2 commits

  • Rebuild the parser after commit 1c722503fa81 (genksyms: Duplicate
    function pointer type definitions segfault), using bison 2.7.

    Signed-off-by: Michal Marek

    Michal Marek
     
  • I noticed that genksyms will segfault when it sees duplicate function
    pointer type declaration when I placed the same function pointer
    definition in two separate headers in a local branch as an intermediate
    step of some refactoring. This can be reproduced by piping the following
    minimal test case into `genksyms -r /dev/null` or alternatively, putting
    it into a C file attempting a build:

    typedef int (*f)();
    typedef int (*f)();

    Attaching gdb to genksyms to understand this failure is useless without
    changing CFLAGS to emit debuginfo. Once you have debuginfo, you will
    find that the failure is that `char *s` was NULL and the program
    executed `while(*s)`. At which point, further debugging requires
    familiarity with compiler front end / parser development.

    What happens is that flex identifies the first instance of the token "f"
    as IDENT and the yacc parser adds it to the symbol table. On the second
    instance, flex will identify "f" as TYPE, which triggers an error case
    in the yacc parser. Given that TYPE would have been IDENT had it not
    been in the symbol table, the the segmentaion fault could be avoided by
    treating TYPE as IDENT in the affected rule.

    Some might consider placing identical function pointer type declarations
    in different headers to be poor style might consider a failure to be
    beneficial. However, failing through a segmentation fault makes the
    cause non-obvious and can waste the time of anyone who encounters it.

    Signed-off-by: Richard Yao
    Acked-by: Madhuri Yechuri
    Signed-off-by: Michal Marek

    Richard Yao
     

04 Apr, 2014

1 commit

  • Recent increased use of typeof() throughout the tree resulted in a
    number of symbols (25 in a typical distro config of ours) not getting a
    proper CRC calculated for them anymore, due to the parser in genksyms
    not coping with several of these uses (interestingly in the majority of
    [if not all] cases the problem is due to the use of typeof() in code
    preceding a certain export, not in the declaration/definition of the
    exported function/object itself; I wasn't able to find a way to address
    this more general parser shortcoming).

    The use of parameter_declaration is a little more relaxed than would be
    ideal (permitting not just a bare type specification, but also one with
    identifier), but since the same code is being passed through an actual
    compiler, there's no apparent risk of allowing through any broken code.

    Otoh using parameter_declaration instead of the ad hoc
    "decl_specifier_seq '*'" / "decl_specifier_seq" pair allows all types to
    be handled rather than just plain ones and pointers to plain ones.

    Signed-off-by: Jan Beulich
    Cc: Michal Marek
    Signed-off-by: Andrew Morton
    Signed-off-by: Linus Torvalds

    Jan Beulich
     

20 Mar, 2013

1 commit

  • Pass symbol-prefix to genksyms instead of arch, so that the decision
    what symbol prefix to use is kept in one place.

    Basically genksyms used to take a -a $ARCH argument and it used that to
    determine whether to add an underscore symbol prefix. It's now changed
    to take a -s $SYMBOL_PREFIX argument so that the caller decides whether
    a symbol prefix is required. The build system then uses
    CONFIG_HAVE_UNDERSCORE_SYMBOL_PREFIX to determine whether to pass the
    argument.

    Signed-off-by: James Hogan
    Signed-off-by: Rusty Russell

    James Hogan
     

03 Mar, 2013

1 commit


08 Jan, 2012

1 commit


11 Oct, 2011

1 commit