06 Oct, 2012

1 commit

  • The logic in do_raw_spin_lock() attempts to acquire a spinlock by invoking
    arch_spin_trylock() in a loop with a delay between each attempt. Now
    consider the following situation in a 2 CPU system:

    1. CPU-0 continually acquires and releases a spinlock in a
    tight loop; it stays in this loop until some condition X
    is satisfied. X can only be satisfied by another CPU.

    2. CPU-1 tries to acquire the same spinlock, in an attempt
    to satisfy the aforementioned condition X. However, it
    never sees the unlocked value of the lock because the
    debug spinlock code uses trylock instead of just lock;
    it checks at all the wrong moments - whenever CPU-0 has
    locked the lock.

    Now in the absence of debug spinlocks, the architecture specific spinlock
    code can correctly allow CPU-1 to wait in a "queue" (e.g., ticket
    spinlocks), ensuring that it acquires the lock at some point. However,
    with the debug spinlock code, livelock can easily occur due to the use of
    try_lock, which obviously cannot put the CPU in that "queue". This
    queueing mechanism is implemented in both x86 and ARM spinlock code.

    Note that the situation mentioned above is not hypothetical. A real
    problem was encountered where CPU-0 was running hrtimer_cancel with
    interrupts disabled, and CPU-1 was attempting to run the hrtimer that
    CPU-0 was trying to cancel.

    Address this by actually attempting arch_spin_lock once it is suspected
    that there is a spinlock lockup. If we're in a situation that is
    described above, the arch_spin_lock should succeed; otherwise other
    timeout mechanisms (e.g., watchdog) should alert the system of a lockup.
    Therefore, if there is a genuine system problem and the spinlock can't be
    acquired, the end result (irrespective of this change being present) is
    the same. If there is a livelock caused by the debug code, this change
    will allow the lock to be acquired, depending on the implementation of the
    lower level arch specific spinlock code.

    [akpm@linux-foundation.org: tweak comment]
    Signed-off-by: Vikram Mulukutla
    Cc: Thomas Gleixner
    Cc: Peter Zijlstra
    Signed-off-by: Andrew Morton
    Signed-off-by: Linus Torvalds

    Vikram Mulukutla
     

31 Jul, 2012

1 commit

  • If there are two spinlocks embedded in a structure that kallsyms knows
    about and one of the spinlocks locks up we will print the name of the
    containing structure instead of the address of the lock. This is quite
    bad, so let's use %pS instead of %ps so we get an offset in addition to
    the symbol so we can determine which particular lock is having problems.

    Signed-off-by: Stephen Boyd
    Signed-off-by: Andrew Morton
    Signed-off-by: Linus Torvalds

    Stephen Boyd
     

06 Jun, 2012

1 commit

  • On an over-committed KVM system we got a:

    "BUG: spinlock lockup on CPU#2, swapper/2/0"

    message on the heavily contended virtio blk spinlock.

    While we might want to reconsider the locking of virtio-blk
    (lock is held while switching to the host) this patch tries to
    make the message clearer: the lockup is only suspected.

    Signed-off-by: Christian Borntraeger
    Cc: Rusty Russell
    Cc: Michael S. Tsirkin
    Cc: Linus Torvalds
    Cc: Andrew Morton
    Cc: Peter Zijlstra
    Link: http://lkml.kernel.org/r/1338283124-7063-1-git-send-email-borntraeger@de.ibm.com
    Signed-off-by: Ingo Molnar

    Christian Borntraeger
     

30 May, 2012

1 commit

  • When a spinlock warning is printed we usually get

    BUG: spinlock bad magic on CPU#0, modprobe/111
    lock: 0xdff09f38, .magic: 00000000, .owner: /0, .owner_cpu: 0

    but it's nicer to print the symbol for the lock if we have it so that we
    can avoid 'grep dff09f38 /proc/kallsyms' to find out which lock it was.
    Use kallsyms to print the symbol name so we get something a bit easier to
    read

    BUG: spinlock bad magic on CPU#0, modprobe/112
    lock: test_lock, .magic: 00000000, .owner: /-1, .owner_cpu: 0

    If the lock is not in kallsyms %ps will fall back to printing the address
    directly.

    Signed-off-by: Stephen Boyd
    Cc: Ingo Molnar
    Cc: Peter Zijlstra
    Signed-off-by: Andrew Morton
    Signed-off-by: Linus Torvalds

    Stephen Boyd
     

08 Mar, 2012

1 commit


01 Nov, 2011

1 commit

  • When SPIN_BUG_ON is triggered, the lock owner information is reported.
    But it is omitted when spinlock lockup is detected.

    This information is useful especially on the architectures which don't
    implement trigger_all_cpu_backtrace() that is called just after detecting
    lockup. So report it and also avoid message format duplication.

    Signed-off-by: Akinobu Mita
    Cc: Ingo Molnar
    Signed-off-by: Andrew Morton
    Signed-off-by: Linus Torvalds

    Akinobu Mita
     

15 Dec, 2009

7 commits

  • The name space hierarchy for the internal lock functions is now a bit
    backwards. raw_spin* functions map to _spin* which use __spin*, while
    we would like to have _raw_spin* and __raw_spin*.

    _raw_spin* is already used by lock debugging, so rename those funtions
    to do_raw_spin* to free up the _raw_spin* name space.

    No functional change.

    Signed-off-by: Thomas Gleixner
    Acked-by: Peter Zijlstra
    Acked-by: Ingo Molnar

    Thomas Gleixner
     
  • Now that the raw_spin name space is freed up, we can implement
    raw_spinlock and the related functions which are used to annotate the
    locks which are not converted to sleeping spinlocks in preempt-rt.

    A side effect is that only such locks can be used with the low level
    lock fsunctions which circumvent lockdep.

    For !rt spin_* functions are mapped to the raw_spin* implementations.

    Signed-off-by: Thomas Gleixner
    Acked-by: Peter Zijlstra
    Acked-by: Ingo Molnar

    Thomas Gleixner
     
  • Name space cleanup for rwlock functions. No functional change.

    Signed-off-by: Thomas Gleixner
    Acked-by: Peter Zijlstra
    Acked-by: David S. Miller
    Acked-by: Ingo Molnar
    Cc: linux-arch@vger.kernel.org

    Thomas Gleixner
     
  • Not strictly necessary for -rt as -rt does not have non sleeping
    rwlocks, but it's odd to not have a consistent naming convention.

    No functional change.

    Signed-off-by: Thomas Gleixner
    Acked-by: Peter Zijlstra
    Acked-by: David S. Miller
    Acked-by: Ingo Molnar
    Cc: linux-arch@vger.kernel.org

    Thomas Gleixner
     
  • Name space cleanup. No functional change.

    Signed-off-by: Thomas Gleixner
    Acked-by: Peter Zijlstra
    Acked-by: David S. Miller
    Acked-by: Ingo Molnar
    Cc: linux-arch@vger.kernel.org

    Thomas Gleixner
     
  • Further name space cleanup. No functional change

    Signed-off-by: Thomas Gleixner
    Acked-by: Peter Zijlstra
    Acked-by: David S. Miller
    Acked-by: Ingo Molnar
    Cc: linux-arch@vger.kernel.org

    Thomas Gleixner
     
  • The raw_spin* namespace was taken by lockdep for the architecture
    specific implementations. raw_spin_* would be the ideal name space for
    the spinlocks which are not converted to sleeping locks in preempt-rt.

    Linus suggested to convert the raw_ to arch_ locks and cleanup the
    name space instead of using an artifical name like core_spin,
    atomic_spin or whatever

    No functional change.

    Signed-off-by: Thomas Gleixner
    Acked-by: Peter Zijlstra
    Acked-by: David S. Miller
    Acked-by: Ingo Molnar
    Cc: linux-arch@vger.kernel.org

    Thomas Gleixner
     

20 Oct, 2007

1 commit

  • The task_struct->pid member is going to be deprecated, so start
    using the helpers (task_pid_nr/task_pid_vnr/task_pid_nr_ns) in
    the kernel.

    The first thing to start with is the pid, printed to dmesg - in
    this case we may safely use task_pid_nr(). Besides, printks produce
    more (much more) than a half of all the explicit pid usage.

    [akpm@linux-foundation.org: git-drm went and changed lots of stuff]
    Signed-off-by: Pavel Emelyanov
    Cc: Dave Airlie
    Signed-off-by: Andrew Morton
    Signed-off-by: Linus Torvalds

    Pavel Emelyanov
     

07 Dec, 2006

1 commit

  • When a spinlock lockup occurs, arrange for the NMI code to emit an all-cpu
    backtrace, so we get to see which CPU is holding the lock, and where.

    Cc: Andi Kleen
    Cc: Ingo Molnar
    Cc: Badari Pulavarty
    Signed-off-by: Andrew Morton
    Signed-off-by: Andi Kleen

    Andrew Morton
     

11 Oct, 2006

1 commit


30 Sep, 2006

1 commit


06 Aug, 2006

1 commit

  • We've confirmed that the debug version of write_lock() can get stuck for long
    enough to cause NMI watchdog timeouts and hence a crash.

    We don't know why, yet. Disable it for now.

    Also disable the similar read_lock() code. Just in case.

    Thanks to Dave Olson for reporting and testing.

    Acked-by: Ingo Molnar
    Cc:
    Signed-off-by: Andrew Morton
    Signed-off-by: Linus Torvalds

    Andrew Morton
     

04 Jul, 2006

2 commits

  • Use the lock validator framework to prove spinlock and rwlock locking
    correctness.

    Signed-off-by: Ingo Molnar
    Signed-off-by: Arjan van de Ven
    Signed-off-by: Andrew Morton
    Signed-off-by: Linus Torvalds

    Ingo Molnar
     
  • Generic lock debugging:

    - generalized lock debugging framework. For example, a bug in one lock
    subsystem turns off debugging in all lock subsystems.

    - got rid of the caller address passing (__IP__/__IP_DECL__/etc.) from
    the mutex/rtmutex debugging code: it caused way too much prototype
    hackery, and lockdep will give the same information anyway.

    - ability to do silent tests

    - check lock freeing in vfree too.

    - more finegrained debugging options, to allow distributions to
    turn off more expensive debugging features.

    There's no separate 'held mutexes' list anymore - but there's a 'held locks'
    stack within lockdep, which unifies deadlock detection across all lock
    classes. (this is independent of the lockdep validation stuff - lockdep first
    checks whether we are holding a lock already)

    Here are the current debugging options:

    CONFIG_DEBUG_MUTEXES=y
    CONFIG_DEBUG_LOCK_ALLOC=y

    which do:

    config DEBUG_MUTEXES
    bool "Mutex debugging, basic checks"

    config DEBUG_LOCK_ALLOC
    bool "Detect incorrect freeing of live mutexes"

    Signed-off-by: Ingo Molnar
    Signed-off-by: Arjan van de Ven
    Signed-off-by: Andrew Morton
    Signed-off-by: Linus Torvalds

    Ingo Molnar
     

01 Jul, 2006

1 commit


08 Feb, 2006

1 commit

  • The spinlock-debug wait-loop was using loops_per_jiffy to detect too long
    spinlock waits - but on fast CPUs this led to a way too fast timeout and false
    messages.

    The fix is to include a __delay(1) call in the loop, to correctly approximate
    the intended delay timeout of 1 second. The code assumes that every
    architecture implements __delay(1) to last around 1/(loops_per_jiffy*HZ)
    seconds.

    Signed-off-by: Ingo Molnar
    Cc: Andi Kleen
    Signed-off-by: Andrew Morton
    Signed-off-by: Linus Torvalds

    Ingo Molnar
     

11 Jan, 2006

1 commit


21 Dec, 2005

1 commit

  • When a spinlock debugging check hits, we print the CPU number as an
    informational thing - but there is no guarantee that preemption is off
    at that point - hence we should use raw_smp_processor_id(). Otherwise
    DEBUG_PREEMPT will print a warning.

    With this fix the warning goes away and only the spinlock-debugging info
    is printed.

    Signed-off-by: Ingo Molnar
    Signed-off-by: Linus Torvalds

    Ingo Molnar
     

11 Sep, 2005

1 commit

  • This patch (written by me and also containing many suggestions of Arjan van
    de Ven) does a major cleanup of the spinlock code. It does the following
    things:

    - consolidates and enhances the spinlock/rwlock debugging code

    - simplifies the asm/spinlock.h files

    - encapsulates the raw spinlock type and moves generic spinlock
    features (such as ->break_lock) into the generic code.

    - cleans up the spinlock code hierarchy to get rid of the spaghetti.

    Most notably there's now only a single variant of the debugging code,
    located in lib/spinlock_debug.c. (previously we had one SMP debugging
    variant per architecture, plus a separate generic one for UP builds)

    Also, i've enhanced the rwlock debugging facility, it will now track
    write-owners. There is new spinlock-owner/CPU-tracking on SMP builds too.
    All locks have lockup detection now, which will work for both soft and hard
    spin/rwlock lockups.

    The arch-level include files now only contain the minimally necessary
    subset of the spinlock code - all the rest that can be generalized now
    lives in the generic headers:

    include/asm-i386/spinlock_types.h | 16
    include/asm-x86_64/spinlock_types.h | 16

    I have also split up the various spinlock variants into separate files,
    making it easier to see which does what. The new layout is:

    SMP | UP
    ----------------------------|-----------------------------------
    asm/spinlock_types_smp.h | linux/spinlock_types_up.h
    linux/spinlock_types.h | linux/spinlock_types.h
    asm/spinlock_smp.h | linux/spinlock_up.h
    linux/spinlock_api_smp.h | linux/spinlock_api_up.h
    linux/spinlock.h | linux/spinlock.h

    /*
    * here's the role of the various spinlock/rwlock related include files:
    *
    * on SMP builds:
    *
    * asm/spinlock_types.h: contains the raw_spinlock_t/raw_rwlock_t and the
    * initializers
    *
    * linux/spinlock_types.h:
    * defines the generic type and initializers
    *
    * asm/spinlock.h: contains the __raw_spin_*()/etc. lowlevel
    * implementations, mostly inline assembly code
    *
    * (also included on UP-debug builds:)
    *
    * linux/spinlock_api_smp.h:
    * contains the prototypes for the _spin_*() APIs.
    *
    * linux/spinlock.h: builds the final spin_*() APIs.
    *
    * on UP builds:
    *
    * linux/spinlock_type_up.h:
    * contains the generic, simplified UP spinlock type.
    * (which is an empty structure on non-debug builds)
    *
    * linux/spinlock_types.h:
    * defines the generic type and initializers
    *
    * linux/spinlock_up.h:
    * contains the __raw_spin_*()/etc. version of UP
    * builds. (which are NOPs on non-debug, non-preempt
    * builds)
    *
    * (included on UP-non-debug builds:)
    *
    * linux/spinlock_api_up.h:
    * builds the _spin_*() APIs.
    *
    * linux/spinlock.h: builds the final spin_*() APIs.
    */

    All SMP and UP architectures are converted by this patch.

    arm, i386, ia64, ppc, ppc64, s390/s390x, x64 was build-tested via
    crosscompilers. m32r, mips, sh, sparc, have not been tested yet, but should
    be mostly fine.

    From: Grant Grundler

    Booted and lightly tested on a500-44 (64-bit, SMP kernel, dual CPU).
    Builds 32-bit SMP kernel (not booted or tested). I did not try to build
    non-SMP kernels. That should be trivial to fix up later if necessary.

    I converted bit ops atomic_hash lock to raw_spinlock_t. Doing so avoids
    some ugly nesting of linux/*.h and asm/*.h files. Those particular locks
    are well tested and contained entirely inside arch specific code. I do NOT
    expect any new issues to arise with them.

    If someone does ever need to use debug/metrics with them, then they will
    need to unravel this hairball between spinlocks, atomic ops, and bit ops
    that exist only because parisc has exactly one atomic instruction: LDCW
    (load and clear word).

    From: "Luck, Tony"

    ia64 fix

    Signed-off-by: Ingo Molnar
    Signed-off-by: Arjan van de Ven
    Signed-off-by: Grant Grundler
    Cc: Matthew Wilcox
    Signed-off-by: Hirokazu Takata
    Signed-off-by: Mikael Pettersson
    Signed-off-by: Benoit Boissinot
    Signed-off-by: Andrew Morton
    Signed-off-by: Linus Torvalds

    Ingo Molnar