Eric Lee / smarc-fsl-linux-kernel

07 Dec, 2011

1 commit

  • ac99b862f   jump_label: Provide jump_label_key initializers ... Browse Code »

    Provide two initializers for jump_label_key that initialize it enabled
    or disabled. Also modify all jump_label code to allow for jump_labels to be
    initialized enabled.

    Signed-off-by: Peter Zijlstra
    Cc: Jason Baron
    Link: http://lkml.kernel.org/n/tip-p40e3yj21b68y03z1yv825e7@git.kernel.org
    Signed-off-by: Ingo Molnar

    Peter Zijlstra
     

06 Dec, 2011

1 commit

  • b20295207   perf, core: Rate limit perf_sched_events jump_label patching ... Browse Code »

    jump_lable patching is very expensive operation that involves pausing all
    cpus. The patching of perf_sched_events jump_label is easily controllable
    from userspace by unprivileged user.

    When te user runs a loop like this:

    "while true; do perf stat -e cycles true; done"

    ... the performance of my test application that just increments a counter
    for one second drops by 4%.

    This is on a 16 cpu box with my test application using only one of
    them. An impact on a real server doing real work will be worse.

    Performance of KVM PMU drops nearly 50% due to jump_lable for "perf
    record" since KVM PMU implementation creates and destroys perf event
    frequently.

    This patch introduces a way to rate limit jump_label patching and uses
    it to fix the above problem.

    I believe that as jump_label use will spread the problem will become more
    common and thus solving it in a generic code is appropriate. Also fixing
    it in the perf code would result in moving jump_label accounting logic to
    perf code with all the ifdefs in case of JUMP_LABEL=n kernel. With this
    patch all details are nicely hidden inside jump_label code.

    Signed-off-by: Gleb Natapov
    Acked-by: Jason Baron
    Signed-off-by: Peter Zijlstra
    Link: http://lkml.kernel.org/r/20111127155909.GO2557@redhat.com
    Signed-off-by: Ingo Molnar

    Gleb Natapov
     

26 Oct, 2011

4 commits

27 Jul, 2011

1 commit

  • 60063497a   atomic: use <linux/atomic.h> ... Browse Code »

    This allows us to move duplicated code in
    (atomic_inc_not_zero() for now) to

    Signed-off-by: Arun Sharma
    Reviewed-by: Eric Dumazet
    Cc: Ingo Molnar
    Cc: David Miller
    Cc: Eric Dumazet
    Acked-by: Mike Frysinger
    Signed-off-by: Andrew Morton
    Signed-off-by: Linus Torvalds

    Arun Sharma
     

05 Apr, 2011

1 commit

  • d430d3d7e   jump label: Introduce static_branch() interface ... Browse Code »

    Introduce:

    static __always_inline bool static_branch(struct jump_label_key *key);

    instead of the old JUMP_LABEL(key, label) macro.

    In this way, jump labels become really easy to use:

    Define:

    struct jump_label_key jump_key;

    Can be used as:

    if (static_branch(&jump_key))
    do unlikely code

    enable/disale via:

    jump_label_inc(&jump_key);
    jump_label_dec(&jump_key);

    that's it!

    For the jump labels disabled case, the static_branch() becomes an
    atomic_read(), and jump_label_inc()/dec() are simply atomic_inc(),
    atomic_dec() operations. We show testing results for this change below.

    Thanks to H. Peter Anvin for suggesting the 'static_branch()' construct.

    Since we now require a 'struct jump_label_key *key', we can store a pointer into
    the jump table addresses. In this way, we can enable/disable jump labels, in
    basically constant time. This change allows us to completely remove the previous
    hashtable scheme. Thanks to Peter Zijlstra for this re-write.

    Testing:

    I ran a series of 'tbench 20' runs 5 times (with reboots) for 3
    configurations, where tracepoints were disabled.

    jump label configured in
    avg: 815.6

    jump label *not* configured in (using atomic reads)
    avg: 800.1

    jump label *not* configured in (regular reads)
    avg: 803.4

    Signed-off-by: Peter Zijlstra
    LKML-Reference:
    Signed-off-by: Jason Baron
    Suggested-by: H. Peter Anvin
    Tested-by: David Daney
    Acked-by: Ralf Baechle
    Acked-by: David S. Miller
    Acked-by: Mathieu Desnoyers
    Signed-off-by: Steven Rostedt

    Jason Baron
     

30 Oct, 2010

1 commit

  • 45f81b1c9   jump label: Add work around to i386 gcc asm goto bug ... Browse Code »

    On i386 (not x86_64) early implementations of gcc would have a bug
    with asm goto causing it to produce code like the following:

    (This was noticed by Peter Zijlstra)

    56 pushl 0
    67 nopl jmp 0x6f
    popl
    jmp 0x8c

    6f mov
    test
    je 0x8c

    8c mov
    call *(%esp)

    The jump added in the asm goto skipped over the popl that matched
    the pushl 0, which lead up to a quick crash of the system when
    the jump was enabled. The nopl is defined in the asm goto () statement
    and when tracepoints are enabled, the nop changes to a jump to the label
    that was specified by the asm goto. asm goto is suppose to tell gcc that
    the code in the asm might jump to an external label. Here gcc obviously
    fails to make that work.

    The bug report for gcc is here:

    http://gcc.gnu.org/bugzilla/show_bug.cgi?id=46226

    The bug only appears on x86 when not compiled with
    -maccumulate-outgoing-args. This option is always set on x86_64 and it
    is also the work around for a function graph tracer i386 bug.
    (See commit: 746357d6a526d6da9d89a2ec645b28406e959c2e)
    This explains why the bug only showed up on i386 when function graph
    tracer was not enabled.

    This patch now adds a CONFIG_JUMP_LABEL option that is default
    off instead of using jump labels by default. When jump labels are
    enabled, the -maccumulate-outgoing-args will be used (causing a
    slightly larger kernel image on i386). This option will exist
    until we have a way to detect if the gcc compiler in use is safe
    to use on all configurations without the work around.

    Note, there exists such a test, but for now we will keep the enabling
    of jump label as a manual option.

    Archs that know the compiler is safe with asm goto, may choose to
    select JUMP_LABEL and enable it by default.

    Reported-by: Ingo Molnar
    Cause-discovered-by: Peter Zijlstra
    Cc: Peter Zijlstra
    Cc: Thomas Gleixner
    Cc: Jason Baron
    Cc: H. Peter Anvin
    Cc: David Daney
    Cc: Mathieu Desnoyers
    Cc: Masami Hiramatsu
    Cc: David Miller
    Cc: Richard Henderson
    LKML-Reference:
    Signed-off-by: Steven Rostedt

    Steven Rostedt
     

28 Oct, 2010

1 commit

  • 91bad2f8d   jump label: Fix deadlock b/w jump_label_mutex vs. text_mutex ... Browse Code »

    register_kprobe() downs the 'text_mutex' and then calls
    jump_label_text_reserved(), which downs the 'jump_label_mutex'.
    However, the jump label code takes those mutexes in the reverse
    order.

    Fix by requiring the caller of jump_label_text_reserved() to do
    the jump label locking via the newly added: jump_label_lock(),
    jump_label_unlock(). Currently, kprobes is the only user
    of jump_label_text_reserved().

    Reported-by: Ingo Molnar
    Acked-by: Masami Hiramatsu
    Signed-off-by: Jason Baron
    LKML-Reference:
    Signed-off-by: Steven Rostedt

    Jason Baron
     

19 Oct, 2010

2 commits

23 Sep, 2010

2 commits

  • 4c3ef6d79   jump label: Add jump_label_text_reserved() to reserve jump points ... Browse Code »

    Add a jump_label_text_reserved(void *start, void *end), so that other
    pieces of code that want to modify kernel text, can first verify that
    jump label has not reserved the instruction.

    Acked-by: Masami Hiramatsu
    Signed-off-by: Jason Baron
    LKML-Reference:
    Signed-off-by: Steven Rostedt

    Jason Baron
     
  • bf5438fca   jump label: Base patch for jump label ... Browse Code »

    base patch to implement 'jump labeling'. Based on a new 'asm goto' inline
    assembly gcc mechanism, we can now branch to labels from an 'asm goto'
    statment. This allows us to create a 'no-op' fastpath, which can subsequently
    be patched with a jump to the slowpath code. This is useful for code which
    might be rarely used, but which we'd like to be able to call, if needed.
    Tracepoints are the current usecase that these are being implemented for.

    Acked-by: David S. Miller
    Signed-off-by: Jason Baron
    LKML-Reference:

    [ cleaned up some formating ]

    Signed-off-by: Steven Rostedt

    Jason Baron