20 Sep, 2016
1 commit
-
Install the callbacks via the state machine. CPU-hotplug multinstance support
is used with the nocalls() version. Maybe parts of padata_alloc() could be
moved into the online callback so that we could invoke ->startup callback for
instance and drop get_online_cpus().Signed-off-by: Sebastian Andrzej Siewior
Cc: Steffen Klassert
Cc: Peter Zijlstra
Cc: linux-crypto@vger.kernel.org
Cc: rt@linutronix.de
Link: http://lkml.kernel.org/r/20160906170457.32393-14-bigeasy@linutronix.de
Signed-off-by: Thomas Gleixner
20 May, 2016
2 commits
-
A recent cleanup removed some exported functions that were not used
anywhere, which in turn exposed the fact that some other functions in
the same file are only used in some configurations.We now get a warning about them when CONFIG_HOTPLUG_CPU is disabled:
kernel/padata.c:670:12: error: '__padata_remove_cpu' defined but not used [-Werror=unused-function]
static int __padata_remove_cpu(struct padata_instance *pinst, int cpu)
^~~~~~~~~~~~~~~~~~~
kernel/padata.c:650:12: error: '__padata_add_cpu' defined but not used [-Werror=unused-function]
static int __padata_add_cpu(struct padata_instance *pinst, int cpu)This rearranges the code so the __padata_remove_cpu/__padata_add_cpu
functions are within the #ifdef that protects the code that calls them.[akpm@linux-foundation.org: coding-style fixes]
Fixes: 4ba6d78c671e ("kernel/padata.c: removed unused code")
Signed-off-by: Arnd Bergmann
Cc: Richard Cochran
Cc: Steffen Klassert
Signed-off-by: Andrew Morton
Signed-off-by: Linus Torvalds -
By accident I stumbled across code that has never been used. This
driver has EXPORT_SYMBOL functions, and the only user of the code is
pcrypt.c, but this only uses a subset of the exported symbols.According to 'git log -G', the functions, padata_set_cpumasks,
padata_add_cpu, and padata_remove_cpu have never been used since they
were first introduced. This patch removes the unused code.On one 64 bit build, with CRYPTO_PCRYPT built in, the text is more than
4k smaller.kbuild_hp> size $KBUILD_OUTPUT/vmlinux
text data bss dec hex filename
10566658 4678360 1122304 16367322 f9beda vmlinux
10561984 4678360 1122304 16362648 f9ac98 vmlinuxOn another config, 32 bit, the saving is about 0.5k bytes.
kbuild_hp-x86> size $KBUILD_OUTPUT/vmlinux
6012005 2409513 2785280 11206798 ab008e vmlinux
6011491 2409513 2785280 11206284 aafe8c vmlinuxSigned-off-by: Richard Cochran
Cc: Steffen Klassert
Cc: Herbert Xu
Cc: "David S. Miller"
Signed-off-by: Andrew Morton
Signed-off-by: Linus Torvalds
14 Feb, 2015
1 commit
-
printk and friends can now format bitmaps using '%*pb[l]'. cpumask
and nodemask also provide cpumask_pr_args() and nodemask_pr_args()
respectively which can be used to generate the two printf arguments
necessary to format the specified cpu/nodemask.Signed-off-by: Tejun Heo
Cc: Steffen Klassert
Signed-off-by: Andrew Morton
Signed-off-by: Linus Torvalds
05 Dec, 2013
1 commit
-
A kernel with enabled lockdep complains about the wrong usage of
rcu_dereference() under a rcu_read_lock_bh() protected region.===============================
[ INFO: suspicious RCU usage. ]
3.13.0-rc1+ #126 Not tainted
-------------------------------
linux/kernel/padata.c:115 suspicious rcu_dereference_check() usage!other info that might help us debug this:
rcu_scheduler_active = 1, debug_locks = 1
1 lock held by cryptomgr_test/153:
#0: (rcu_read_lock_bh){.+....}, at: [] padata_do_parallel+0x5/0x270Fix that by using rcu_dereference_bh() instead.
Signed-off-by: Mathias Krause
Acked-by: Steffen Klassert
Signed-off-by: Herbert Xu
30 Oct, 2013
1 commit
-
Using a spinlock to atomically increase a counter sounds wrong -- we've
atomic_t for this!Also move 'seq_nr' to a different cache line than 'lock' to reduce cache
line trashing. This has the nice side effect of decreasing the size of
struct parallel_data from 192 to 128 bytes for a x86-64 build, e.g.
occupying only two instead of three cache lines.Those changes results in a 5% performance increase on an IPsec test run
using pcrypt.Btw. the seq_lock spinlock was never explicitly initialized -- one more
reason to get rid of it.Signed-off-by: Mathias Krause
Acked-by: Steffen Klassert
Signed-off-by: Herbert Xu
29 Aug, 2013
2 commits
-
padata_cpu_callback() takes pinst->lock, to avoid taking
an uninitialized lock, register the notifier after it's
initialization.Signed-off-by: Richard Weinberger
Acked-by: Steffen Klassert
Signed-off-by: Herbert Xu -
…PARE and CPU_UP_CANCELED
Share code between CPU_ONLINE and CPU_DOWN_FAILED, same to
CPU_DOWN_PREPARE and CPU_UP_CANCELED.It will fix 2 bugs:
"not check the return value of __padata_remove_cpu() and __padata_add_cpu()".
"need add 'break' between CPU_UP_CANCELED and CPU_DOWN_FAILED".Signed-off-by: Chen Gang <gang.chen@asianux.com>
Acked-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
06 Dec, 2012
1 commit
-
For bottom halves off, __this_cpu_read is better.
Signed-off-by: Shan Wei
Reviewed-by: Christoph Lameter
Acked-by: Steffen Klassert
Signed-off-by: Herbert Xu
29 Mar, 2012
3 commits
-
We don't remove the cpu that went offline from our cpumasks
on cpu hotplug. This got lost somewhere along the line, so
restore it. This fixes a hang of the padata instance on cpu
hotplug.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu -
We use the active cpumask to determine the superset of cpus
to use for parallelization. However, the active cpumask is
for internal usage of the scheduler and therefore not the
appropriate cpumask for these purposes. So use the online
cpumask instead.Reported-by: Peter Zijlstra
Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu -
Add a reference to the padata api documentation at Documentation/padata.txt
Suggested-by: Peter Zijlstra
Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu
14 Mar, 2012
2 commits
-
When padata_do_parallel() is called from multiple cpus for the same
padata instance, we can get object reordering on sequence number wrap
because testing for sequence number wrap and reseting the sequence
number must happen atomically but is implemented with two atomic
operations. This patch fixes this by converting the sequence number
from atomic_t to an unsigned int and protect the access with a
spin_lock. As a side effect, we get rid of the sequence number wrap
handling because the seqence number wraps back to null now without
the need to do anything.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu -
When a padata object is queued to the serialization queue, another
cpu might process and free the padata object. So don't dereference
it after queueing to the serialization queue.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu
31 Oct, 2011
1 commit
-
The changed files were only including linux/module.h for the
EXPORT_SYMBOL infrastructure, and nothing else. Revector them
onto the isolated export header for faster compile times.Nothing to see here but a whole lot of instances of:
-#include
+#includeThis commit is only changing the kernel dir; next targets
will probably be mm, fs, the arch dirs, etc.Signed-off-by: Paul Gortmaker
31 Mar, 2011
1 commit
-
Fixes generated by 'codespell' and manually reviewed.
Signed-off-by: Lucas De Marchi
05 Aug, 2010
1 commit
-
* git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (39 commits)
random: Reorder struct entropy_store to remove padding on 64bits
padata: update API documentation
padata: Remove padata_get_cpumask
crypto: pcrypt - Update pcrypt cpumask according to the padata cpumask notifier
crypto: pcrypt - Rename pcrypt_instance
padata: Pass the padata cpumasks to the cpumask_change_notifier chain
padata: Rearrange set_cpumask functions
padata: Rename padata_alloc functions
crypto: pcrypt - Dont calulate a callback cpu on empty callback cpumask
padata: Check for valid cpumasks
padata: Allocate cpumask dependend recources in any case
padata: Fix cpu index counting
crypto: geode_aes - Convert pci_table entries to PCI_VDEVICE (if PCI_ANY_ID is used)
pcrypt: Added sysfs interface to pcrypt
padata: Added sysfs primitives to padata subsystem
padata: Make two separate cpumasks
padata: update documentation
padata: simplify serialization mechanism
padata: make padata_do_parallel to return zero on success
padata: Handle empty padata cpumasks
...
31 Jul, 2010
4 commits
-
A function that copies the padata cpumasks to a user buffer
is a bit error prone. The cpumask can change any time so we
can't be sure to have the right cpumask when using this function.
A user who is interested in the padata cpumasks should register
to the padata cpumask notifier chain instead. Users of
padata_get_cpumask are already updated, so we can remove it.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu -
We pass a pointer to the new padata cpumasks to the cpumask_change_notifier
chain. So users can access the cpumasks without the need of an extra
padata_get_cpumask function.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu -
padata_set_cpumask needs to be protected by a lock. We make
__padata_set_cpumasks unlocked and static. So this function
can be used by the exported and locked padata_set_cpumask and
padata_set_cpumasks functions.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu -
We rename padata_alloc to padata_alloc_possible because this
function allocates a padata_instance and uses the cpu_possible
mask for parallel and serial workers. Also we rename __padata_alloc
to padata_alloc to avoid to export underlined functions. Underlined
functions are considered to be private to padata. Users are updated
accordingly.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu
26 Jul, 2010
3 commits
-
Now that we allow to change the cpumasks from userspace, we have
to check for valid cpumasks in padata_do_parallel. This patch adds
the necessary check. This fixes a division by zero crash if the
parallel cpumask contains no active cpu.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu -
The cpumask separation work assumes the cpumask dependend recources
present regardless of valid or invalid cpumasks. With this patch
we allocate the cpumask dependend recources in any case. This fixes
two NULL pointer dereference crashes in padata_replace and in
padata_get_cpumask.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu -
The counting of the cpu index got lost with a recent commit.
This patch restores it. This fixes a hang of the parallel worker
threads on cpu hotplug.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu
19 Jul, 2010
2 commits
-
Added sysfs primitives to padata subsystem. Now API user may
embedded kobject each padata instance contains into any sysfs
hierarchy. For now padata sysfs interface provides only
two objects:
serial_cpumask [RW] - cpumask for serial workers
parallel_cpumask [RW] - cpumask for parallel workersSigned-off-by: Dan Kruchinin
Signed-off-by: Herbert Xu -
The aim of this patch is to make two separate cpumasks
for padata parallel and serial workers respectively.
It allows user to make more thin and sophisticated configurations
of padata framework. For example user may bind parallel and serial workers to non-intersecting
CPU groups to gain better performance. Also each padata instance has notifiers chain for its
cpumasks now. If either parallel or serial or both masks were changed all
interested subsystems will get notification about that. It's especially useful
if padata user uses algorithm for callback CPU selection according to serial cpumask.Signed-off-by: Dan Kruchinin
Signed-off-by: Herbert Xu
14 Jul, 2010
5 commits
-
We count the number of processed objects on a percpu basis,
so we need to go through all the percpu reorder queues to calculate
the sequence number of the next object that needs serialization.
This patch changes this to count the number of processed objects
global. So we can calculate the sequence number and the percpu
reorder queue of the next object that needs serialization without
searching through the percpu reorder queues. This avoids some
accesses to memory of foreign cpus.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu -
To return -EINPROGRESS on success in padata_do_parallel was
considered to be odd. This patch changes this to return zero
on success. Also the only user of padata, pcrypt is adapted to
convert a return of zero to -EINPROGRESS within the crypto layer.
This also removes the pcrypt fallback if padata_do_parallel
was called on a not running padata instance as we can't handle it
anymore. This fallback was unused, so it's save to remove it.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu -
This patch fixes a bug when the padata cpumask does not
intersect with the active cpumask. In this case we get a
division by zero in padata_alloc_pd and we end up with a
useless padata instance. Padata can end up with an empty
cpumask for two reasons:1. A user removed the last cpu that belongs to the padata
cpumask and the active cpumask.2. The last cpu that belongs to the padata cpumask and the
active cpumask goes offline.We introduce a function padata_validate_cpumask to check if the padata
cpumask does intersect with the active cpumask. If the cpumasks do not
intersect we mark the instance as invalid, so it can't be used. We do not
allocate the cpumask dependend recources in this case. This fixes the
division by zero and keeps the padate instance in a consistent state.It's not possible to trigger this bug by now because the only padata user,
pcrypt uses always the possible cpumask.Reported-by: Dan Kruchinin
Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu -
This patch makes padata_stop to block until the padata
instance is unused. Also we split padata_stop to a locked
and a unlocked version. This is in preparation to be able
to change the cpumask after a call to patata stop.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu -
This patch introduces the PADATA_INVALID flag which is
checked on padata start. This will be used to mark a padata
instance as invalid, if the padata cpumask does not intersect
with the active cpumask. we change padata_start to return an
error if the PADATA_INVALID is set. Also we adapt the only
padata user, pcrypt to this change.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu
03 Jun, 2010
1 commit
-
MAX_SEQ_NR is used in padata_alloc_pd() like this:
pd->max_seq_nr = (MAX_SEQ_NR / num_cpus) * num_cpus - 1;
It needs parenthesis or the divide by num_cpus takes precedence over the
subtraction.Signed-off-by: Dan Carpenter
Acked-by: Steffen Klassert
Signed-off-by: Herbert Xu
28 May, 2010
1 commit
-
By the previous modification, the cpu notifier can return encapsulate
errno value. This converts the cpu notifiers for kernel/*.cSigned-off-by: Akinobu Mita
Cc: Ingo Molnar
Cc: Peter Zijlstra
Signed-off-by: Andrew Morton
Signed-off-by: Linus Torvalds
19 May, 2010
4 commits
-
Add get_online_cpus/put_online_cpus to ensure that no cpu goes
offline during the flushing of the padata percpu queues.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu -
Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu -
yield was used to wait until all references of the internal control
structure in use are dropped before it is freed. This patch implements
padata_flush_queues which actively flushes the padata percpu queues
in this case.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu -
padata_get_next needs to check whether the next object that
need serialization must be parallel processed by the local cpu.
This check was wrong implemented and returned always true,
so the try_again loop in padata_reorder was never taken. This
can lead to object leaks in some rare cases due to a race that
appears with the trylock in padata_reorder. The try_again loop
was not a good idea after all, because a cpu could take that
loop frequently, so we handle this with a timer instead.This patch adds a timer to handle the race that appears with
the trylock. If cpu1 queues an object to the reorder queue while
cpu2 holds the pd->lock but left the while loop in padata_reorder
already, cpu2 can't care for this object and cpu1 exits because
it can't get the lock. Usually the next cpu that takes the lock
cares for this object too. We need the timer just if this object
was the last one that arrives to the reorder queues. The timer
function sends it out in this case.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu
03 May, 2010
3 commits
-
This patch puts get_online_cpus/put_online_cpus around the places
we modify the padata cpumask to ensure that no cpu goes offline
during this operation.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu -
padata_alloc_pd set up queues for all possible cpus.
This patch changes this to set up the queues just for
the used cpus.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu -
might_sleep() was placed before mutex_lock() in some places.
We remove them because mutex_lock() does might_sleep() too.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu