23 Jul, 2020
3 commits
-
There's no reason to have two interfaces when there's only one caller.
Removing _possible saves text and simplifies future changes.Signed-off-by: Daniel Jordan
Cc: Herbert Xu
Cc: Steffen Klassert
Cc: linux-crypto@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Herbert Xu -
padata_stop() has two callers and is unnecessary in both cases. When
pcrypt calls it before padata_free(), it's being unloaded so there are
no outstanding padata jobs[0]. When __padata_free() calls it, it's
either along the same path or else pcrypt initialization failed, which
of course means there are also no outstanding jobs.Removing it simplifies padata and saves text.
[0] https://lore.kernel.org/linux-crypto/20191119225017.mjrak2fwa5vccazl@gondor.apana.org.au/
Signed-off-by: Daniel Jordan
Cc: Herbert Xu
Cc: Steffen Klassert
Cc: linux-crypto@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Herbert Xu -
padata_start() is only used right after pcrypt allocates an instance
with all possible CPUs, when PADATA_INVALID can't happen, so there's no
need for a separate "start" step. It can be done during allocation to
save text, make using padata easier, and avoid unneeded calls in the
future.Signed-off-by: Daniel Jordan
Cc: Herbert Xu
Cc: Steffen Klassert
Cc: linux-crypto@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Herbert Xu
16 Jul, 2020
1 commit
-
The flag CRYPTO_ALG_ASYNC is "inherited" in the sense that when a
template is instantiated, the template will have CRYPTO_ALG_ASYNC set if
any of the algorithms it uses has CRYPTO_ALG_ASYNC set.We'd like to add a second flag (CRYPTO_ALG_ALLOCATES_MEMORY) that gets
"inherited" in the same way. This is difficult because the handling of
CRYPTO_ALG_ASYNC is hardcoded everywhere. Address this by:- Add CRYPTO_ALG_INHERITED_FLAGS, which contains the set of flags that
have these inheritance semantics.- Add crypto_algt_inherited_mask(), for use by template ->create()
methods. It returns any of these flags that the user asked to be
unset and thus must be passed in the 'mask' to crypto_grab_*().- Also modify crypto_check_attr_type() to handle computing the 'mask'
so that most templates can just use this.- Make crypto_grab_*() propagate these flags to the template instance
being created so that templates don't have to do this themselves.Make crypto/simd.c propagate these flags too, since it "wraps" another
algorithm, similar to a template.Based on a patch by Mikulas Patocka
(https://lore.kernel.org/r/alpine.LRH.2.02.2006301414580.30526@file01.intranet.prod.int.rdu2.redhat.com).Signed-off-by: Eric Biggers
Signed-off-by: Herbert Xu
06 Mar, 2020
1 commit
-
Simplify the error handling in pcrypt_create_aead() by taking advantage
of crypto_grab_aead() now handling an ERR_PTR() name and by taking
advantage of crypto_drop_aead() now accepting (as a no-op) a spawn that
hasn't been grabbed yet.This required also making padata_free_shell() accept a NULL argument.
Signed-off-by: Eric Biggers
Signed-off-by: Herbert Xu
09 Jan, 2020
1 commit
-
Initializing a crypto_aead_spawn currently requires:
1. Set spawn->base.inst to point to the instance.
2. Call crypto_grab_aead().But there's no reason for these steps to be separate, and in fact this
unneeded complication has caused at least one bug, the one fixed by
commit 6db43410179b ("crypto: adiantum - initialize crypto_spawn::inst")So just make crypto_grab_aead() take the instance as an argument.
To keep the function calls from getting too unwieldy due to this extra
argument, also introduce a 'mask' variable into the affected places
which weren't already using one.Signed-off-by: Eric Biggers
Signed-off-by: Herbert Xu
11 Dec, 2019
4 commits
-
Since commit 63d3578892dc ("crypto: pcrypt - remove padata cpumask
notifier") this feature is unused, so get rid of it.Signed-off-by: Daniel Jordan
Cc: Eric Biggers
Cc: Herbert Xu
Cc: Jonathan Corbet
Cc: Steffen Klassert
Cc: linux-crypto@vger.kernel.org
Cc: linux-doc@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Herbert Xu -
We should not be modifying the original request's MAY_SLEEP flag
upon completion. It makes no sense to do so anyway.Reported-by: Eric Biggers
Fixes: 5068c7a883d1 ("crypto: pcrypt - Add pcrypt crypto...")
Signed-off-by: Herbert Xu
Tested-by: Eric Biggers
Signed-off-by: Herbert Xu -
If the pcrypt template is used multiple times in an algorithm, then a
deadlock occurs because all pcrypt instances share the same
padata_instance, which completes requests in the order submitted. That
is, the inner pcrypt request waits for the outer pcrypt request while
the outer request is already waiting for the inner.This patch fixes this by allocating a set of queues for each pcrypt
instance instead of using two global queues. In order to maintain
the existing user-space interface, the pinst structure remains global
so any sysfs modifications will apply to every pcrypt instance.Note that when an update occurs we have to allocate memory for
every pcrypt instance. Should one of the allocations fail we
will abort the update without rolling back changes already made.The new per-instance data structure is called padata_shell and is
essentially a wrapper around parallel_data.Reproducer:
#include
#include
#includeint main()
{
struct sockaddr_alg addr = {
.salg_type = "aead",
.salg_name = "pcrypt(pcrypt(rfc4106-gcm-aesni))"
};
int algfd, reqfd;
char buf[32] = { 0 };algfd = socket(AF_ALG, SOCK_SEQPACKET, 0);
bind(algfd, (void *)&addr, sizeof(addr));
setsockopt(algfd, SOL_ALG, ALG_SET_KEY, buf, 20);
reqfd = accept(algfd, 0, 0);
write(reqfd, buf, 32);
read(reqfd, buf, 16);
}Reported-by: syzbot+56c7151cad94eec37c521f0e47d2eee53f9361c4@syzkaller.appspotmail.com
Fixes: 5068c7a883d1 ("crypto: pcrypt - Add pcrypt crypto parallelization wrapper")
Signed-off-by: Herbert Xu
Tested-by: Eric Biggers
Signed-off-by: Herbert Xu -
On module unload of pcrypt we must unregister the crypto algorithms
first and then tear down the padata structure. As otherwise the
crypto algorithms are still alive and can be used while the padata
structure is being freed.Fixes: 5068c7a883d1 ("crypto: pcrypt - Add pcrypt crypto...")
Cc:
Signed-off-by: Herbert Xu
13 Sep, 2019
4 commits
-
With pcrypt's cpumask no longer used, take the CPU hotplug lock inside
padata_alloc_possible.Useful later in the series for avoiding nested acquisition of the CPU
hotplug lock in padata when padata_alloc_possible is allocating an
unbound workqueue.Without this patch, this nested acquisition would happen later in the
series:pcrypt_init_padata
get_online_cpus
alloc_padata_possible
alloc_padata
alloc_workqueue(WQ_UNBOUND) // later in the series
alloc_and_link_pwqs
apply_wqattrs_lock
get_online_cpus // recursive rwsem acquisitionSigned-off-by: Daniel Jordan
Acked-by: Steffen Klassert
Cc: Herbert Xu
Cc: Lai Jiangshan
Cc: Peter Zijlstra
Cc: Tejun Heo
Cc: linux-crypto@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Herbert Xu -
Now that padata_do_parallel takes care of finding an alternate callback
CPU, there's no need for pcrypt's callback cpumask, so remove it and the
notifier callback that keeps it in sync.Signed-off-by: Daniel Jordan
Acked-by: Steffen Klassert
Cc: Herbert Xu
Cc: Lai Jiangshan
Cc: Peter Zijlstra
Cc: Tejun Heo
Cc: linux-crypto@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Herbert Xu -
padata_do_parallel currently returns -EINVAL if the callback CPU isn't
in the callback cpumask.pcrypt tries to prevent this situation by keeping its own callback
cpumask in sync with padata's and checks that the callback CPU it passes
to padata is valid. Make padata handle this instead.padata_do_parallel now takes a pointer to the callback CPU and updates
it for the caller if an alternate CPU is used. Overall behavior in
terms of which callback CPUs are chosen stays the same.Prepares for removal of the padata cpumask notifier in pcrypt, which
will fix a lockdep complaint about nested acquisition of the CPU hotplug
lock later in the series.Signed-off-by: Daniel Jordan
Acked-by: Steffen Klassert
Cc: Herbert Xu
Cc: Lai Jiangshan
Cc: Peter Zijlstra
Cc: Tejun Heo
Cc: linux-crypto@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Herbert Xu -
Move workqueue allocation inside of padata to prepare for further
changes to how padata uses workqueues.Guarantees the workqueue is created with max_active=1, which padata
relies on to work correctly. No functional change.Signed-off-by: Daniel Jordan
Acked-by: Steffen Klassert
Cc: Herbert Xu
Cc: Jonathan Corbet
Cc: Lai Jiangshan
Cc: Peter Zijlstra
Cc: Tejun Heo
Cc: linux-crypto@vger.kernel.org
Cc: linux-doc@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Herbert Xu
05 Jun, 2019
1 commit
-
Based on 1 normalized pattern(s):
this program is free software you can redistribute it and or modify
it under the terms and conditions of the gnu general public license
version 2 as published by the free software foundation this program
is distributed in the hope it will be useful but without any
warranty without even the implied warranty of merchantability or
fitness for a particular purpose see the gnu general public license
for more details you should have received a copy of the gnu general
public license along with this program if not write to the free
software foundation inc 51 franklin st fifth floor boston ma 02110
1301 usaextracted by the scancode license scanner the SPDX license identifier
GPL-2.0-only
has been chosen to replace the boilerplate/reference in 111 file(s).
Signed-off-by: Thomas Gleixner
Reviewed-by: Alexios Zavras
Reviewed-by: Allison Randal
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190530000436.567572064@linutronix.de
Signed-off-by: Greg Kroah-Hartman
18 Apr, 2019
1 commit
-
Use subsys_initcall for registration of all templates and generic
algorithm implementations, rather than module_init. Then change
cryptomgr to use arch_initcall, to place it before the subsys_initcalls.This is needed so that when both a generic and optimized implementation
of an algorithm are built into the kernel (not loadable modules), the
generic implementation is registered before the optimized one.
Otherwise, the self-tests for the optimized implementation are unable to
allocate the generic implementation for the new comparison fuzz tests.Note that on arm, a side effect of this change is that self-tests for
generic implementations may run before the unaligned access handler has
been installed. So, unaligned accesses will crash the kernel. This is
arguably a good thing as it makes it easier to detect that type of bug.Signed-off-by: Eric Biggers
Signed-off-by: Herbert Xu
28 Dec, 2018
1 commit
-
Pull crypto updates from Herbert Xu:
"API:
- Add 1472-byte test to tcrypt for IPsec
- Reintroduced crypto stats interface with numerous changes
- Support incremental algorithm dumpsAlgorithms:
- Add xchacha12/20
- Add nhpoly1305
- Add adiantum
- Add streebog hash
- Mark cts(cbc(aes)) as FIPS allowedDrivers:
- Improve performance of arm64/chacha20
- Improve performance of x86/chacha20
- Add NEON-accelerated nhpoly1305
- Add SSE2 accelerated nhpoly1305
- Add AVX2 accelerated nhpoly1305
- Add support for 192/256-bit keys in gcmaes AVX
- Add SG support in gcmaes AVX
- ESN for inline IPsec tx in chcr
- Add support for CryptoCell 703 in ccree
- Add support for CryptoCell 713 in ccree
- Add SM4 support in ccree
- Add SM3 support in ccree
- Add support for chacha20 in caam/qi2
- Add support for chacha20 + poly1305 in caam/jr
- Add support for chacha20 + poly1305 in caam/qi2
- Add AEAD cipher support in cavium/nitrox"* 'linus' of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (130 commits)
crypto: skcipher - remove remnants of internal IV generators
crypto: cavium/nitrox - Fix build with !CONFIG_DEBUG_FS
crypto: salsa20-generic - don't unnecessarily use atomic walk
crypto: skcipher - add might_sleep() to skcipher_walk_virt()
crypto: x86/chacha - avoid sleeping under kernel_fpu_begin()
crypto: cavium/nitrox - Added AEAD cipher support
crypto: mxc-scc - fix build warnings on ARM64
crypto: api - document missing stats member
crypto: user - remove unused dump functions
crypto: chelsio - Fix wrong error counter increments
crypto: chelsio - Reset counters on cxgb4 Detach
crypto: chelsio - Handle PCI shutdown event
crypto: chelsio - cleanup:send addr as value in function argument
crypto: chelsio - Use same value for both channel in single WR
crypto: chelsio - Swap location of AAD and IV sent in WR
crypto: chelsio - remove set but not used variable 'kctx_len'
crypto: ux500 - Use proper enum in hash_set_dma_transfer
crypto: ux500 - Use proper enum in cryp_set_dma_transfer
crypto: aesni - Add scatter/gather avx stubs, and use them in C
crypto: aesni - Introduce partial block macro
..
28 Nov, 2018
1 commit
-
Now that synchronize_rcu() waits for bh-disable regions of code as
well as RCU read-side critical sections, the synchronize_rcu_bh() in
pcrypt_cpumask_change_notify() can be replaced by synchronize_rcu().
This commit therefore makes this change.Signed-off-by: Paul E. McKenney
Cc: Steffen Klassert
Cc:
Acked-by: Herbert Xu
09 Nov, 2018
1 commit
-
Passing string 'name' as the format specifier is potentially hazardous
because name could (although very unlikely to) have a format specifier
embedded in it causing issues when parsing the non-existent arguments
to these. Follow best practice by using the "%s" format string for
the string 'name'.Cleans up clang warning:
crypto/pcrypt.c:397:40: warning: format string is not a string literal
(potentially insecure) [-Wformat-security]Fixes: a3fb1e330dd2 ("pcrypt: Added sysfs interface to pcrypt")
Signed-off-by: Colin Ian King
Signed-off-by: Herbert Xu
22 Dec, 2017
1 commit
-
pcrypt is using the old way of freeing instances, where the ->free()
method specified in the 'struct crypto_template' is passed a pointer to
the 'struct crypto_instance'. But the crypto_instance is being
kfree()'d directly, which is incorrect because the memory was actually
allocated as an aead_instance, which contains the crypto_instance at a
nonzero offset. Thus, the wrong pointer was being kfree()'d.Fix it by switching to the new way to free aead_instance's where the
->free() method is specified in the aead_instance itself.Reported-by: syzbot
Fixes: 0496f56065e0 ("crypto: pcrypt - Add support for new AEAD interface")
Cc: # v4.2+
Signed-off-by: Eric Biggers
Signed-off-by: Herbert Xu
17 Aug, 2015
1 commit
-
This patch removes the CRYPTO_ALG_AEAD_NEW flag now that everyone
has been converted.Signed-off-by: Herbert Xu
14 Jul, 2015
1 commit
-
This patch allows the CRYPTO_ALG_AEAD_NEW flag to be propagated.
It also restores the ASYNC bit that went missing during the AEAD
conversion.Signed-off-by: Herbert Xu
03 Jun, 2015
1 commit
-
This patch converts pcrypt over to the new AEAD interface.
Signed-off-by: Herbert Xu
25 May, 2015
1 commit
-
The variable tfm_count is accessed by multiple threads without
locking. This patch converts it to an atomic_t.Signed-off-by: Herbert Xu
22 May, 2015
1 commit
-
As AEAD has switched over to using frontend types, the function
crypto_init_spawn must not be used since it does not specify a
frontend type. Otherwise it leads to a crash when the spawn is
used.This patch fixes it by switching over to crypto_grab_aead instead.
Fixes: 5d1d65f8bea6 ("crypto: aead - Convert top level interface to new style")
Signed-off-by: Herbert Xu
13 May, 2015
1 commit
-
This patch uses the crypto_aead_set_reqsize helper to avoid directly
touching the internals of aead.Signed-off-by: Herbert Xu
26 Nov, 2014
1 commit
-
This adds the module loading prefix "crypto-" to the template lookup
as well.For example, attempting to load 'vfat(blowfish)' via AF_ALG now correctly
includes the "crypto-" prefix at every level, correctly rejecting "vfat":net-pf-38
algif-hash
crypto-vfat(blowfish)
crypto-vfat(blowfish)-all
crypto-vfatReported-by: Mathias Krause
Signed-off-by: Kees Cook
Acked-by: Mathias Krause
Signed-off-by: Herbert Xu
05 Dec, 2013
1 commit
-
A kernel with enabled lockdep complains about the wrong usage of
rcu_dereference() under a rcu_read_lock_bh() protected region.===============================
[ INFO: suspicious RCU usage. ]
3.13.0-rc1+ #126 Not tainted
-------------------------------
linux/crypto/pcrypt.c:81 suspicious rcu_dereference_check() usage!other info that might help us debug this:
rcu_scheduler_active = 1, debug_locks = 1
1 lock held by cryptomgr_test/153:
#0: (rcu_read_lock_bh){.+....}, at: [] pcrypt_do_parallel.isra.2+0x5/0x200Fix that by using rcu_dereference_bh() instead.
Signed-off-by: Mathias Krause
Cc: "David S. Miller"
Acked-by: Steffen Klassert
Signed-off-by: Herbert Xu
04 Jul, 2013
1 commit
-
For the workqueue creation interfaces that do not expect format strings,
make sure they cannot accidently be parsed that way. Additionally, clean
up calls made with a single parameter that would be handled as a format
string. Many callers are passing potentially dynamic string content, so
use "%s" in those cases to avoid any potential accidents.Signed-off-by: Kees Cook
Signed-off-by: Andrew Morton
Signed-off-by: Linus Torvalds
29 Mar, 2012
1 commit
-
We use the active cpumask to determine the superset of cpus
to use for parallelization. However, the active cpumask is
for internal usage of the scheduler and therefore not the
appropriate cpumask for these purposes. So use the online
cpumask instead.Reported-by: Peter Zijlstra
Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu
14 Jan, 2011
1 commit
-
* git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6: (46 commits)
hwrng: via_rng - Fix memory scribbling on some CPUs
crypto: padlock - Move padlock.h into include/crypto
hwrng: via_rng - Fix asm constraints
crypto: n2 - use __devexit not __exit in n2_unregister_algs
crypto: mark crypto workqueues CPU_INTENSIVE
crypto: mv_cesa - dont return PTR_ERR() of wrong pointer
crypto: ripemd - Set module author and update email address
crypto: omap-sham - backlog handling fix
crypto: gf128mul - Remove experimental tag
crypto: af_alg - fix af_alg memory_allocated data type
crypto: aesni-intel - Fixed build with binutils 2.16
crypto: af_alg - Make sure sk_security is initialized on accept()ed sockets
net: Add missing lockdep class names for af_alg
include: Install linux/if_alg.h for user-space crypto API
crypto: omap-aes - checkpatch --file warning fixes
crypto: omap-aes - initialize aes module once per request
crypto: omap-aes - unnecessary code removed
crypto: omap-aes - error handling implementation improved
crypto: omap-aes - redundant locking is removed
crypto: omap-aes - DMA initialization fixes for OMAP off mode
...
04 Jan, 2011
1 commit
-
kcrypto_wq and pcrypt->wq's are used to run ciphers and may consume
considerable amount of CPU cycles. Mark both as CPU_INTENSIVE so that
they don't block other work items.As the workqueues are primarily used to burn CPU cycles, concurrency
levels shouldn't matter much and are left at 1. A higher value may be
beneficial and needs investigation.Signed-off-by: Tejun Heo
Signed-off-by: Herbert Xu
12 Nov, 2010
1 commit
-
kobject_put is called from padata_free for the padata kobject.
The kobject's release function frees the padata instance,
so don't call kobject_put for the padata kobject from pcrypt.Reported-and-tested-by: Randy Dunlap
Signed-off-by: Steffen Klassert
Signed-off-by: Linus Torvalds
31 Jul, 2010
3 commits
-
The padata cpumask change notifier passes a padata_cpumask to the
notifier chain. So we use this cpumask instead of asking padata for
the cpumask.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu -
In the crypto-layer an instance refers usually to a crypto instance.
The struct pcrypt_instance is not related to a crypto instance.
It rather contains the padata informations, so we rename it to
padata_pcrypt. The functions that handle this struct are renamed
accordingly.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu -
We rename padata_alloc to padata_alloc_possible because this
function allocates a padata_instance and uses the cpu_possible
mask for parallel and serial workers. Also we rename __padata_alloc
to padata_alloc to avoid to export underlined functions. Underlined
functions are considered to be private to padata. Users are updated
accordingly.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu
26 Jul, 2010
1 commit
-
If the callback cpumask is empty, we crash with a division by zero
when we try to calculate a callback cpu. So we don't update the callback
cpu in pcrypt_do_parallel if the callback cpumask is empty.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu
19 Jul, 2010
2 commits
-
Added sysfs interface to pcrypt. Now pcrypt subsystem creates two
sysfs directories with corresponding padata sysfs objects:
/sys/kernel/pcrypt/[pencrypt|pdecrypt]Signed-off-by: Dan Kruchinin
Signed-off-by: Herbert Xu -
The aim of this patch is to make two separate cpumasks
for padata parallel and serial workers respectively.
It allows user to make more thin and sophisticated configurations
of padata framework. For example user may bind parallel and serial workers to non-intersecting
CPU groups to gain better performance. Also each padata instance has notifiers chain for its
cpumasks now. If either parallel or serial or both masks were changed all
interested subsystems will get notification about that. It's especially useful
if padata user uses algorithm for callback CPU selection according to serial cpumask.Signed-off-by: Dan Kruchinin
Signed-off-by: Herbert Xu
14 Jul, 2010
1 commit
-
To return -EINPROGRESS on success in padata_do_parallel was
considered to be odd. This patch changes this to return zero
on success. Also the only user of padata, pcrypt is adapted to
convert a return of zero to -EINPROGRESS within the crypto layer.
This also removes the pcrypt fallback if padata_do_parallel
was called on a not running padata instance as we can't handle it
anymore. This fallback was unused, so it's save to remove it.Signed-off-by: Steffen Klassert
Signed-off-by: Herbert Xu