25 Dec, 2017
1 commit
-
commit 3382290ed2d5e275429cef510ab21889d3ccd164 upstream.
[ Note, this is a Git cherry-pick of the following commit:
506458efaf15 ("locking/barriers: Convert users of lockless_dereference() to READ_ONCE()")
... for easier x86 PTI code testing and back-porting. ]
READ_ONCE() now has an implicit smp_read_barrier_depends() call, so it
can be used instead of lockless_dereference() without any change in
semantics.Signed-off-by: Will Deacon
Cc: Linus Torvalds
Cc: Paul E. McKenney
Cc: Peter Zijlstra
Cc: Thomas Gleixner
Link: http://lkml.kernel.org/r/1508840570-22169-4-git-send-email-will.deacon@arm.com
Signed-off-by: Ingo Molnar
Signed-off-by: Greg Kroah-Hartman
02 Nov, 2017
1 commit
-
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.By default all files without license information are under the default
license of the kernel, which is GPL version 2.Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if
Reviewed-by: Philippe Ombredanne
Reviewed-by: Thomas Gleixner
Signed-off-by: Greg Kroah-Hartman
20 Oct, 2017
1 commit
-
Because many of RCU's files have not been included into docbook, a
number of errors have accumulated. This commit fixes them.Signed-off-by: Paul E. McKenney
Signed-off-by: Linus Torvalds
20 Apr, 2017
1 commit
-
sparse is unhappy about this code in hlist_add_tail_rcu:
struct hlist_node *i, *last = NULL;
for (i = hlist_first_rcu(h); i; i = hlist_next_rcu(i))
last = i;This is because hlist_next_rcu and hlist_next_rcu return
__rcu pointers.It's a false positive - it's a write side primitive and so
does not need to be called in a read side critical section.The following trivial patch disables the warning
without changing the behaviour in any way.Note: __hlist_for_each_rcu would also remove the warning but it would be
confusing since it calls rcu_derefence and is designed to run in the rcu
read side critical section.Signed-off-by: Michael S. Tsirkin
Reviewed-by: Steven Rostedt (VMware)
Signed-off-by: Paul E. McKenney
01 Nov, 2016
1 commit
-
This commit consolidates the debug checking for list_add_rcu() into the
new single __list_add_valid() debug function. Notably, this commit fixes
the sanity check that was added in commit 17a801f4bfeb ("list_debug:
WARN for adding something already in the list"), which wasn't checking
RCU-protected lists.Signed-off-by: Kees Cook
Acked-by: Steven Rostedt
Signed-off-by: Paul E. McKenney
Acked-by: Rik van Riel
24 Apr, 2016
1 commit
-
Conflicts were two cases of simple overlapping changes,
nothing serious.In the UDP case, we need to add a hlist_add_tail_rcu()
to linux/rculist.h, because we've moved UDP socket handling
away from using nulls lists.Signed-off-by: David S. Miller
10 Mar, 2016
1 commit
-
This is a convenience function that returns the next entry in an RCU
list or NULL if at the end of the list.Signed-off-by: Tom Herbert
Signed-off-by: David S. Miller
08 Dec, 2015
2 commits
-
…2015.12.04b' and 'torture.2015.12.05a' into HEAD
doc.2015.12.05a: Documentation updates
exp.2015.12.07a: Expedited grace-period updates
fixes.2015.12.07a: Miscellaneous fixes
list.2015.12.04b: Linked-list updates
torture.2015.12.05a: Torture-test updates -
Although list_for_each_entry_rcu() can in theory be used anywhere
preemption is disabled, it can result in calls to lockdep, which cannot
be used in certain constrained execution environments, such as exception
handlers that do not map the entire kernel into their address spaces.
This commit therefore adds list_entry_lockless() and
list_for_each_entry_lockless(), which never invoke lockdep and can
therefore safely be used from these constrained environments, but only
as long as those environments are non-preemptible (or items are never
deleted from the list).Use synchronize_sched(), call_rcu_sched(), or synchronize_sched_expedited()
in updates for the needed grace periods. Of course, if items are never
deleted from the list, there is no need to wait for grace periods.Signed-off-by: Alexey Kardashevskiy
Signed-off-by: Paul E. McKenney
05 Dec, 2015
1 commit
-
The list_splice_init_rcu() can be used as a stack onto which full lists
are pushed, but queue-like behavior is now needed by some security
policies. This requires a list_splice_tail_init_rcu().This commit therefore supplies a list_splice_tail_init_rcu() by
pulling code common it and to list_splice_init_rcu() into a new
__list_splice_init_rcu() function. This new function is based on the
existing list_splice_init_rcu() implementation.Signed-off-by: Petko Manolov
Cc: Mimi Zohar
Signed-off-by: Paul E. McKenney
07 Oct, 2015
1 commit
-
The current list_entry_rcu() implementation copies the pointer to a stack
variable, then invokes rcu_dereference_raw() on it. This results in an
additional store-load pair. Now, most compilers will emit normal store
and load instructions, which might seem to be of negligible overhead,
but this results in a load-hit-store situation that can cause surprisingly
long pipeline stalls, even on modern microprocessors. The problem is
that it takes time for the store to get the store buffer updated, which
can delay the subsequent load, which immediately follows.This commit therefore switches to the lockless_dereference() primitive,
which does not expect the __rcu annotations (that are anyway not present
in the list_head structure) and which, like rcu_dereference_raw(),
does not check for an enclosing RCU read-side critical section.
Most importantly, it does not copy the pointer, thus avoiding the
load-hit-store overhead.Signed-off-by: Patrick Marlier
[ paulmck: Switched to lockless_dereference() to suppress sparse warnings. ]
Signed-off-by: Paul E. McKenney
Reviewed-by: Josh Triplett
28 May, 2015
2 commits
-
This fixes the following sparse warnings:
make C=1 CF=-D__CHECK_ENDIAN__ net/tipc/name_table.o
net/tipc/name_table.c:977:17: error: incompatible types in comparison expression (different address spaces)
net/tipc/name_table.c:977:17: error: incompatible types in comparison expression (different address spaces)To silence these spare complaints, an RCU annotation should be added to
"next" pointer of hlist_node structure through hlist_next_rcu() macro
when iterating over a hlist with hlist_for_each_entry_from_rcu().Signed-off-by: Ying Xue
Signed-off-by: Paul E. McKenney -
This commit moves from the old ACCESS_ONCE() API to the new READ_ONCE()
and WRITE_ONCE() APIs.Signed-off-by: Paul E. McKenney
[ paulmck: Updated to include kernel/torture.c as suggested by Jason Low. ]
07 Jan, 2015
1 commit
-
This fixes the following sparse warnings:
make C=1 CF=-D__CHECK_ENDIAN__ net/ipv6/addrconf.o
net/ipv6/addrconf.c:3495:9: error: incompatible types in comparison expression (different address spaces)
net/ipv6/addrconf.c:3495:9: error: incompatible types in comparison expression (different address spaces)
net/ipv6/addrconf.c:3495:9: error: incompatible types in comparison expression (different address spaces)
net/ipv6/addrconf.c:3495:9: error: incompatible types in comparison expression (different address spaces)To silence these spare complaints, an RCU annotation should be added to
"next" pointer of hlist_node structure through hlist_next_rcu() macro
when iterating over a hlist with hlist_for_each_entry_continue_rcu_bh().By the way, this commit also resolves the same error appearing in
hlist_for_each_entry_continue_rcu().Signed-off-by: Ying Xue
Signed-off-by: Paul E. McKenney
13 Dec, 2014
1 commit
-
Pull trivial tree update from Jiri Kosina:
"Usual stuff: documentation updates, printk() fixes, etc"* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial: (24 commits)
intel_ips: fix a type in error message
cpufreq: cpufreq-dt: Move newline to end of error message
ps3rom: fix error return code
treewide: fix typo in printk and Kconfig
ARM: dts: bcm63138: change "interupts" to "interrupts"
Replace mentions of "list_struct" to "list_head"
kernel: trace: fix printk message
scsi: mpt2sas: fix ioctl in comment
zbud, zswap: change module author email
clocksource: Fix 'clcoksource' typo in comment
arm: fix wording of "Crotex" in CONFIG_ARCH_EXYNOS3 help
gpio: msm-v1: make boolean argument more obvious
usb: Fix typo in usb-serial-simple.c
PCI: Fix comment typo 'COMFIG_PM_OPS'
powerpc: Fix comment typo 'CONIFG_8xx'
powerpc: Fix comment typos 'CONFiG_ALTIVEC'
clk: st: Spelling s/stucture/structure/
isci: Spelling s/stucture/structure/
usb: gadget: zero: Spelling s/infrastucture/infrastructure/
treewide: Fix company name in module descriptions
...
09 Dec, 2014
1 commit
-
Convert tipc name table read-write lock to RCU. After this change,
a new spin lock is used to protect name table on write side while
RCU is applied on read side.Signed-off-by: Ying Xue
Reviewed-by: Erik Hugne
Reviewed-by: Jon Maloy
Tested-by: Erik Hugne
Signed-off-by: David S. Miller
20 Nov, 2014
1 commit
-
There's no such thing as "list_struct".
Signed-off-by: Andrey Utkin
Acked-by: Steven Rostedt
Acked-by: Mauro Carvalho Chehab
Acked-by: Paul E. McKenney
Acked-by: Alex Deucher
Signed-off-by: Jiri Kosina
07 Aug, 2014
1 commit
-
All other add functions for lists have the new item as first argument
and the position where it is added as second argument. This was changed
for no good reason in this function and makes using it unnecessary
confusing.The name was changed to hlist_add_behind() to cause unconverted code to
generate a compile error instead of using the wrong parameter order.[akpm@linux-foundation.org: coding-style fixes]
Signed-off-by: Ken Helias
Cc: "Paul E. McKenney"
Acked-by: Jeff Kirsher [intel driver bits]
Cc: Hugh Dickins
Cc: Christoph Hellwig
Signed-off-by: Andrew Morton
Signed-off-by: Linus Torvalds
18 Feb, 2014
1 commit
-
This commit outdents expression-statement macros, thus repairing a few
line-length complaints. Also fix some spacing errors called out by
checkpatch.pl.Signed-off-by: Joe Perches
Signed-off-by: Paul E. McKenney
Reviewed-by: Josh Triplett
13 Dec, 2013
1 commit
-
Function prototypes don't need to have the "extern" keyword since this
is the default behavior. Its explicit use is redundant. This commit
therefore removes them.Signed-off-by: Teodora Baluta
Signed-off-by: Paul E. McKenney
24 Sep, 2013
1 commit
-
The list_splice_init_rcu() function allows a list visible to RCU readers
to be spliced into another list visible to RCU readers. This is OK,
except for the use of INIT_LIST_HEAD(), which does pointer updates
without doing anything to make those updates safe for concurrent readers.Of course, most of the time INIT_LIST_HEAD() is being used in reader-free
contexts, such as initialization or cleanup, so it is OK for it to update
pointers in an unsafe-for-RCU-readers manner. This commit therefore
creates an INIT_LIST_HEAD_RCU() that uses ACCESS_ONCE() to make the updates
reader-safe. The reason that we can use ACCESS_ONCE() instead of the more
typical rcu_assign_pointer() is that list_splice_init_rcu() is updating the
pointers to reference something that is already visible to readers, so
that there is no problem with pre-initialized values.Signed-off-by: Paul E. McKenney
19 Aug, 2013
1 commit
-
list_first_or_null() should test whether the list is empty and return
pointer to the first entry if not in a RCU safe manner. It's broken
in several ways.* It compares __kernel @__ptr with __rcu @__next triggering the
following sparse warning.net/core/dev.c:4331:17: error: incompatible types in comparison expression (different address spaces)
* It doesn't perform rcu_dereference*() and computes the entry address
using container_of() directly from the __rcu pointer which is
inconsitent with other rculist interface. As a result, all three
in-kernel users - net/core/dev.c, macvlan, cgroup - are buggy. They
dereference the pointer w/o going through read barrier.* While ->next dereference passes through list_next_rcu(), the
compiler is still free to fetch ->next more than once and thus
nullify the "__ptr != __next" condition check.Fix it by making list_first_or_null_rcu() dereference ->next directly
using ACCESS_ONCE() and then use list_entry_rcu() on it like other
rculist accessors.v2: Paul pointed out that the compiler may fetch the pointer more than
once nullifying the condition check. ACCESS_ONCE() added on
->next dereference.v3: Restored () around macro param which was accidentally removed.
Spotted by Paul.Signed-off-by: Tejun Heo
Reported-by: Fengguang Wu
Cc: Dipankar Sarma
Cc: "Paul E. McKenney"
Cc: "David S. Miller"
Cc: Li Zefan
Cc: Patrick McHardy
Cc: stable@vger.kernel.org
Signed-off-by: Paul E. McKenney
Reviewed-by: Josh Triplett
29 May, 2013
1 commit
-
As rcu_dereference_raw() under RCU debug config options can add quite a
bit of checks, and that tracing uses rcu_dereference_raw(), these checks
happen with the function tracer. The function tracer also happens to trace
these debug checks too. This added overhead can livelock the system.Add a new interface to RCU for both rcu_dereference_raw_notrace() as well
as hlist_for_each_entry_rcu_notrace() as the hlist iterator uses the
rcu_dereference_raw() as well, and is used a bit with the function tracer.Link: http://lkml.kernel.org/r/20130528184209.304356745@goodmis.org
Acked-by: Paul E. McKenney
Signed-off-by: Steven Rostedt
28 Feb, 2013
1 commit
-
I'm not sure why, but the hlist for each entry iterators were conceived
list_for_each_entry(pos, head, member)
The hlist ones were greedy and wanted an extra parameter:
hlist_for_each_entry(tpos, pos, head, member)
Why did they need an extra pos parameter? I'm not quite sure. Not only
they don't really need it, it also prevents the iterator from looking
exactly like the list iterator, which is unfortunate.Besides the semantic patch, there was some manual work required:
- Fix up the actual hlist iterators in linux/list.h
- Fix up the declaration of other iterators based on the hlist ones.
- A very small amount of places were using the 'node' parameter, this
was modified to use 'obj->member' instead.
- Coccinelle didn't handle the hlist_for_each_entry_safe iterator
properly, so those had to be fixed up manually.The semantic patch which is mostly the work of Peter Senna Tschudin is here:
@@
iterator name hlist_for_each_entry, hlist_for_each_entry_continue, hlist_for_each_entry_from, hlist_for_each_entry_rcu, hlist_for_each_entry_rcu_bh, hlist_for_each_entry_continue_rcu_bh, for_each_busy_worker, ax25_uid_for_each, ax25_for_each, inet_bind_bucket_for_each, sctp_for_each_hentry, sk_for_each, sk_for_each_rcu, sk_for_each_from, sk_for_each_safe, sk_for_each_bound, hlist_for_each_entry_safe, hlist_for_each_entry_continue_rcu, nr_neigh_for_each, nr_neigh_for_each_safe, nr_node_for_each, nr_node_for_each_safe, for_each_gfn_indirect_valid_sp, for_each_gfn_sp, for_each_host;type T;
expression a,c,d,e;
identifier b;
statement S;
@@-T b;
[akpm@linux-foundation.org: drop bogus change from net/ipv4/raw.c]
[akpm@linux-foundation.org: drop bogus hunk from net/ipv6/raw.c]
[akpm@linux-foundation.org: checkpatch fixes]
[akpm@linux-foundation.org: fix warnings]
[akpm@linux-foudnation.org: redo intrusive kvm changes]
Tested-by: Peter Senna Tschudin
Acked-by: Paul E. McKenney
Signed-off-by: Sasha Levin
Cc: Wu Fengguang
Cc: Marcelo Tosatti
Cc: Gleb Natapov
Signed-off-by: Andrew Morton
Signed-off-by: Linus Torvalds
14 Nov, 2012
1 commit
-
The list_for_each_continue_rcu() macro is no longer used, so this commit
removes it. The list_for_each_entry_continue_rcu() macro should be
used instead.Signed-off-by: Paul E. McKenney
25 Apr, 2012
2 commits
-
The list_first_entry_rcu() macro is inherently unsafe because it cannot
be applied to an empty list. But because RCU readers do not exclude
updaters, a list might become empty between the time that list_empty()
claimed it was non-empty and the time that list_first_entry_rcu() is
invoked. Therefore, the list_empty() test cannot be separated from the
list_first_entry_rcu() call. This commit therefore combines these to
macros to create a new list_first_or_null_rcu() macro that replaces
the old (and unsafe) list_first_entry_rcu() macro.This patch incorporates Paul's review comments on the previous version of
this patch available here:https://lkml.org/lkml/2012/4/2/536
This patch cannot break any upstream code because list_first_entry_rcu()
is not being used anywhere in the kernel (tested with grep(1)), and any
external code using it is probably broken as a result of using it.Signed-off-by: Michel Machado
CC: "Paul E. McKenney"
CC: Dipankar Sarma
Signed-off-by: Paul E. McKenney -
* Make __list_add_rcu check the next->prev and prev->next pointers
just like __list_add does.
* Make list_del_rcu use __list_del_entry, which does the same checking
at deletion time.Has been running for a week here without anything being tripped up,
but it seems worth adding for completeness just in case something
ever does corrupt those lists.Signed-off-by: Dave Jones
Signed-off-by: Paul E. McKenney
21 Jul, 2011
1 commit
-
If the list to be spliced is empty, then list_splice_init_rcu() has
nothing to do. Unfortunately, list_splice_init_rcu() does not check
the list to be spliced; it instead checks the list to be spliced into.
This results in memory leaks given current usage. This commit
therefore fixes the empty-list check.Signed-off-by: Jan H. Schönherr
Signed-off-by: Paul E. McKenney
20 May, 2011
2 commits
-
This is removes the use of software prefetching from the regular list
iterators. We don't want it. If you do want to prefetch in some
iterator of yours, go right ahead. Just don't expect the iterator to do
it, since normally the downsides are bigger than the upsides.It also replaces with , because the
use of LIST_POISON ends up needing it. is sadly not
self-contained, and including prefetch.h just happened to hide that.Suggested by David Miller (networking has a lot of regular lists that
are often empty or a single entry, and prefetching is not going to do
anything but add useless instructions).Acked-by: Ingo Molnar
Acked-by: David S. Miller
Cc: linux-arch@vger.kernel.org
Signed-off-by: Linus Torvalds -
They not only increase the code footprint, they actually make things
slower rather than faster. On internationally acclaimed benchmarks
("make -j16" on an already fully built kernel source tree) the hlist
prefetching slows down the build by up to 1%.(Almost all of it comes from hlist_for_each_entry_rcu() as used by
avc_has_perm_noaudit(), which is very hot due to all the pathname
lookups to see if there is anything to do).The cause seems to be two-fold:
- on at least some Intel cores, prefetch(NULL) ends up with some
microarchitectural stall due to the TLB miss that it incurs. The
hlist case triggers this very commonly, since the NULL pointer is the
last entry in the list.- the prefetch appears to cause more D$ activity, probably because it
prefetches hash list entries that are never actually used (because we
ended the search early due to a hit).Regardless, the numbers clearly say that the implicit prefetching is
simply a bad idea. If some _particular_ user of the hlist iterators
wants to prefetch the next list entry, they can do so themselves
explicitly, rather than depend on all list iterators doing so
implicitly.Acked-by: Ingo Molnar
Acked-by: David S. Miller
Cc: linux-arch@vger.kernel.org
Signed-off-by: Linus Torvalds
18 Dec, 2010
2 commits
-
Signed-off-by: Paul E. McKenney
-
This restores parentheses blance.
Signed-off-by: Mariusz Kozlowski
Signed-off-by: Paul E. McKenney
21 Aug, 2010
1 commit
-
Because list_empty() does not dereference any RCU-protected pointers, and
further does not pass such pointers to the caller (so that the caller
does not dereference them either), it is safe to use list_empty() on
RCU-protected lists. There is no need for a list_empty_rcu(). This
commit adds a comment stating this explicitly.Requested-by: Andrew Morton
Signed-off-by: Paul E. McKenney
20 Aug, 2010
1 commit
-
This avoids warnings from missing __rcu annotations
in the rculist implementation, making it possible to
use the same lists in both RCU and non-RCU cases.We can add rculist annotations later, together with
lockdep support for rculist, which is missing as well,
but that may involve changing all the users.Signed-off-by: Arnd Bergmann
Signed-off-by: Paul E. McKenney
Cc: Pavel Emelyanov
Cc: Sukadev Bhattiprolu
Reviewed-by: Josh Triplett
04 May, 2010
1 commit
-
Add hlist_for_each_entry_rcu_bh() and
hlist_for_each_entry_continue_rcu_bh() macros, and use them in
ipv6_get_ifaddr(), if6_get_first() and if6_get_next() to fix lockdeps
warnings.Signed-off-by: Eric Dumazet
Reviewed-by: "Paul E. McKenney"
Signed-off-by: David S. Miller
21 Mar, 2010
1 commit
-
Convert from reader/writer lock to RCU and spinlock for addrconf
hash list.Adds an additional helper macro for hlist_for_each_entry_continue_rcu
to handle the continue case.Signed-off-by: Stephen Hemminger
Signed-off-by: David S. Miller
01 Mar, 2010
1 commit
-
Conflicts:
drivers/firmware/iscsi_ibft.c
25 Feb, 2010
1 commit
-
The theory is that use of bare rcu_dereference() is more prone
to error than use of the RCU list-traversal primitives.
Therefore, disable lockdep RCU read-side critical-section
checking in these primitives for the time being. Once all of
the rcu_dereference() uses have been dealt with, it may be time
to re-enable lockdep checking for the RCU list-traversal
primitives.Signed-off-by: Paul E. McKenney
Cc: laijs@cn.fujitsu.com
Cc: dipankar@in.ibm.com
Cc: mathieu.desnoyers@polymtl.ca
Cc: josh@joshtriplett.org
Cc: dvhltc@us.ibm.com
Cc: niv@us.ibm.com
Cc: peterz@infradead.org
Cc: rostedt@goodmis.org
Cc: Valdis.Kletnieks@vt.edu
Cc: dhowells@redhat.com
LKML-Reference:
Signed-off-by: Ingo Molnar
23 Feb, 2010
1 commit
-
Many usages of seq_file use RCU protected lists, so non RCU
iterators will not work safely.Signed-off-by: Stephen Hemminger
Signed-off-by: David S. Miller
11 Nov, 2009
1 commit
-
This adds an RCU macro for continuing search, useful for some
network devices like vlan.Signed-off-by: Stephen Hemminger
Acked-by: Eric Dumazet
Signed-off-by: David S. Miller