Eric Lee / linux-smarc-t335x-v3.2

Commit 3f9d35b9514da6757ca98831372518f9eeb71b33

Authored by Eric Dumazet 2010-11-12 06:05:08 +0800

Committed by Linus Torvalds 2010-11-12 23:55:30 +0800

Exists in master and in 4 other branches

atomic: add atomic_inc_not_zero_hint()

Followup of perf tools session in Netfilter WorkShop 2010

In the network stack we make high usage of atomic_inc_not_zero() in
contexts we know the probable value of atomic before increment (2 for udp
sockets for example)

Using a special version of atomic_inc_not_zero() giving this hint can help
processor to use less bus transactions.

On x86 (MESI protocol) for example, this avoids entering Shared state,
because "lock cmpxchg" issues an RFO (Read For Ownership)

akpm: Adds a new include/linux/atomic.h.  This means that new code should
henceforth include linux/atomic.h and not asm/atomic.h.  The presence of
include/linux/atomic.h will in fact cause checkpatch.pl to warn about use
of asm/atomic.h.  The new include/linux/atomic.h becomes the place where
arch-neutral atomic_t code should be placed.

[akpm@linux-foundation.org: coding-style fixes]
Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Arnaldo Carvalho de Melo <acme@infradead.org>
Cc: David Miller <davem@davemloft.net>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Cc: Nick Piggin <npiggin@kernel.dk>
Reviewed-by: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Showing 1 changed file with 37 additions and 0 deletions Side-by-side Diff

include/linux/atomic.h

include/linux/atomic.h

Diff comments View file @ 3f9d35b

	1	+#ifndef _LINUX_ATOMIC_H
	2	+#define _LINUX_ATOMIC_H
	3	+#include <asm/atomic.h>
	4	+
	5	+/**
	6	+ * atomic_inc_not_zero_hint - increment if not null
	7	+ * @v: pointer of type atomic_t
	8	+ * @hint: probable value of the atomic before the increment
	9	+ *
	10	+ * This version of atomic_inc_not_zero() gives a hint of probable
	11	+ * value of the atomic. This helps processor to not read the memory
	12	+ * before doing the atomic read/modify/write cycle, lowering
	13	+ * number of bus transactions on some arches.
	14	+ *
	15	+ * Returns: 0 if increment was not done, 1 otherwise.
	16	+ */
	17	+#ifndef atomic_inc_not_zero_hint
	18	+static inline int atomic_inc_not_zero_hint(atomic_t *v, int hint)
	19	+{
	20	+ int val, c = hint;
	21	+
	22	+ /* sanity test, should be removed by compiler if hint is a constant */
	23	+ if (!hint)
	24	+ return atomic_inc_not_zero(v);
	25	+
	26	+ do {
	27	+ val = atomic_cmpxchg(v, c, c + 1);
	28	+ if (val == c)
	29	+ return 1;
	30	+ c = val;
	31	+ } while (c);
	32	+
	33	+ return 0;
	34	+}
	35	+#endif
	36	+
	37	+#endif /* _LINUX_ATOMIC_H */