06 Feb, 2016
1 commit
-
Some servers experienced fatal deadlocks because of a combination of
bugs, leading to multiple cpus calling dump_stack().The checksumming bug was fixed in commit 34ae6a1aa054 ("ipv6: update
skb->csum when CE mark is propagated").The second problem is a faulty locking in dump_stack()
CPU1 runs in process context and calls dump_stack(), grabs dump_lock.
CPU2 receives a TCP packet under softirq, grabs socket spinlock, and
call dump_stack() from netdev_rx_csum_fault().dump_stack() spins on atomic_cmpxchg(&dump_lock, -1, 2), since
dump_lock is owned by CPU1While dumping its stack, CPU1 is interrupted by a softirq, and happens
to process a packet for the TCP socket locked by CPU2.CPU1 spins forever in spin_lock() : deadlock
Stack trace on CPU1 looked like :
NMI backtrace for cpu 1
RIP: _raw_spin_lock+0x25/0x30
...
Call Trace:
tcp_v6_rcv+0x243/0x620
ip6_input_finish+0x11f/0x330
ip6_input+0x38/0x40
ip6_rcv_finish+0x3c/0x90
ipv6_rcv+0x2a9/0x500
process_backlog+0x461/0xaa0
net_rx_action+0x147/0x430
__do_softirq+0x167/0x2d0
call_softirq+0x1c/0x30
do_softirq+0x3f/0x80
irq_exit+0x6e/0xc0
smp_call_function_single_interrupt+0x35/0x40
call_function_single_interrupt+0x6a/0x70
printk+0x4d/0x4f
printk_address+0x31/0x33
print_trace_address+0x33/0x3c
print_context_stack+0x7f/0x119
dump_trace+0x26b/0x28e
show_trace_log_lvl+0x4f/0x5c
show_stack_log_lvl+0x104/0x113
show_stack+0x42/0x44
dump_stack+0x46/0x58
netdev_rx_csum_fault+0x38/0x3c
__skb_checksum_complete_head+0x6e/0x80
__skb_checksum_complete+0x11/0x20
tcp_rcv_established+0x2bd5/0x2fd0
tcp_v6_do_rcv+0x13c/0x620
sk_backlog_rcv+0x15/0x30
release_sock+0xd2/0x150
tcp_recvmsg+0x1c1/0xfc0
inet_recvmsg+0x7d/0x90
sock_recvmsg+0xaf/0xe0
___sys_recvmsg+0x111/0x3b0
SyS_recvmsg+0x5c/0xb0
system_call_fastpath+0x16/0x1bFixes: b58d977432c8 ("dump_stack: serialize the output from dump_stack()")
Signed-off-by: Eric Dumazet
Cc: Alex Thorlton
Cc:
Signed-off-by: Andrew Morton
Signed-off-by: Linus Torvalds
06 May, 2014
1 commit
-
As requested by Linus add explicit __visible to the asmlinkage users.
This marks functions visible to assembler.Tree sweep for rest of tree.
Signed-off-by: Andi Kleen
Link: http://lkml.kernel.org/r/1398984278-29319-4-git-send-email-andi@firstfloor.org
Signed-off-by: H. Peter Anvin
07 Aug, 2013
1 commit
-
dump_stack is used from assembler code, so make it visible.
Signed-off-by: Andi Kleen
Link: http://lkml.kernel.org/r/1375740170-7446-15-git-send-email-andi@firstfloor.org
Signed-off-by: H. Peter Anvin
04 Jul, 2013
1 commit
-
Add functionality to serialize the output from dump_stack() to avoid
mangling of the output when dump_stack is called simultaneously from
multiple cpus.[akpm@linux-foundation.org: fix comment indenting, avoid inclusion of files - use where possiblem fix uniprocessor build (__dump_stack undefined), remove unneeded ifdef around smp.h inclusion]
Signed-off-by: Alex Thorlton
Reported-by: Russ Anderson
Reviewed-by: Robin Holt
Cc: Vineet Gupta
Cc: David S. Miller
Cc: Richard Kuo
Cc: Jesper Nilsson
Signed-off-by: Andrew Morton
Signed-off-by: Linus Torvalds
01 May, 2013
1 commit
-
Both dump_stack() and show_stack() are currently implemented by each
architecture. show_stack(NULL, NULL) dumps the backtrace for the
current task as does dump_stack(). On some archs, dump_stack() prints
extra information - pid, utsname and so on - in addition to the
backtrace while the two are identical on other archs.The usages in arch-independent code of the two functions indicate
show_stack(NULL, NULL) should print out bare backtrace while
dump_stack() is used for debugging purposes when something went wrong,
so it does make sense to print additional information on the task which
triggered dump_stack().There's no reason to require archs to implement two separate but mostly
identical functions. It leads to unnecessary subtle information.This patch expands the dummy fallback dump_stack() implementation in
lib/dump_stack.c such that it prints out debug information (taken from
x86) and invokes show_stack(NULL, NULL) and drops arch-specific
dump_stack() implementations in all archs except blackfin. Blackfin's
dump_stack() does something wonky that I don't understand.Debug information can be printed separately by calling
dump_stack_print_info() so that arch-specific dump_stack()
implementation can still emit the same debug information. This is used
in blackfin.This patch brings the following behavior changes.
* On some archs, an extra level in backtrace for show_stack() could be
printed. This is because the top frame was determined in
dump_stack() on those archs while generic dump_stack() can't do that
reliably. It can be compensated by inlining dump_stack() but not
sure whether that'd be necessary.* Most archs didn't use to print debug info on dump_stack(). They do
now.An example WARN dump follows.
WARNING: at kernel/workqueue.c:4841 init_workqueues+0x35/0x505()
Hardware name: empty
Modules linked in:
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.9.0-rc1-work+ #9
0000000000000009 ffff88007c861e08 ffffffff81c614dc ffff88007c861e48
ffffffff8108f50f ffffffff82228240 0000000000000040 ffffffff8234a03c
0000000000000000 0000000000000000 0000000000000000 ffff88007c861e58
Call Trace:
[] dump_stack+0x19/0x1b
[] warn_slowpath_common+0x7f/0xc0
[] warn_slowpath_null+0x1a/0x20
[] init_workqueues+0x35/0x505
...v2: CPU number added to the generic debug info as requested by s390
folks and dropped the s390 specific dump_stack(). This loses %ksp
from the debug message which the maintainers think isn't important
enough to keep the s390-specific dump_stack() implementation.dump_stack_print_info() is moved to kernel/printk.c from
lib/dump_stack.c. Because linkage is per objecct file,
dump_stack_print_info() living in the same lib file as generic
dump_stack() means that archs which implement custom dump_stack()
- at this point, only blackfin - can't use dump_stack_print_info()
as that will bring in the generic version of dump_stack() too. v1
The v1 patch broke build on blackfin due to this issue. The build
breakage was reported by Fengguang Wu.Signed-off-by: Tejun Heo
Acked-by: David S. Miller
Acked-by: Vineet Gupta
Acked-by: Jesper Nilsson
Acked-by: Vineet Gupta
Acked-by: Martin Schwidefsky [s390 bits]
Cc: Heiko Carstens
Cc: Mike Frysinger
Cc: Fengguang Wu
Cc: Bjorn Helgaas
Cc: Sam Ravnborg
Acked-by: Richard Kuo [hexagon bits]
Signed-off-by: Andrew Morton
Signed-off-by: Linus Torvalds
08 Mar, 2012
1 commit
-
For files only using THIS_MODULE and/or EXPORT_SYMBOL, map
them onto including export.h -- or if the file isn't even
using those, then just delete the include. Fix up any implicit
include dependencies that were being masked by module.h along
the way.Signed-off-by: Paul Gortmaker
17 Apr, 2005
1 commit
-
Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.Let it rip!
