Eric Lee / smarc-ti-linux-kernel

Download as
Browse Code ยป

Commit 0b6b098efcddac2bf4e2a895c9b655560bbfcee4

Authored by Mathias Krause
Committed by Herbert Xu
1 parent cfc6f11b76
Exists in master and in 16 other branches dlt-processor-sdk-linux-03.00.00.04, processor-sdk-linux-01.00.00, processor-sdk-linux-02.00.01, smarc-ti-linux-3.14.y, smarc-ti-linux-3.15.y, smarc-ti-lsk-linux-4.1.y, smarct3x-processor-sdk-04.01.00.06, smarct3x-processor-sdk-linux-02.00.01, smarct3x-processor-sdk-linux-03.00.00.04, smarct4x-800-processor-sdk-linux-02.00.01, smarct4x-processor-sdk-04.01.00.06, smarct4x-processor-sdk-linux-02.00.01, smarct4x-processor-sdk-linux-03.00.00.04, ti-linux-3.14.y, ti-linux-3.15.y, ti-lsk-linux-4.1.y

padata: make the sequence counter an atomic_t 

Using a spinlock to atomically increase a counter sounds wrong -- we've
atomic_t for this!

Also move 'seq_nr' to a different cache line than 'lock' to reduce cache
line trashing. This has the nice side effect of decreasing the size of
struct parallel_data from 192 to 128 bytes for a x86-64 build, e.g.
occupying only two instead of three cache lines.

Those changes results in a 5% performance increase on an IPsec test run
using pcrypt.

Btw. the seq_lock spinlock was never explicitly initialized -- one more
reason to get rid of it.

Signed-off-by: Mathias Krause <mathias.krause@secunet.com>
Acked-by: Steffen Klassert <steffen.klassert@secunet.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Showing 2 changed files with 5 additions and 7 deletions Side-by-side Diff

include/linux/padata.h
Diff comments   View file @ 0b6b098
... ... @@ -129,10 +129,9 @@
129 129 struct padata_serial_queue __percpu *squeue;
130 130 atomic_t reorder_objects;
131 131 atomic_t refcnt;
  132 + atomic_t seq_nr;
132 133 struct padata_cpumask cpumask;
133 134 spinlock_t lock ____cacheline_aligned;
134   - spinlock_t seq_lock;
135   - unsigned int seq_nr;
136 135 unsigned int processed;
137 136 struct timer_list timer;
138 137 };
... ... @@ -46,6 +46,7 @@
46 46  
47 47 static int padata_cpu_hash(struct parallel_data *pd)
48 48 {
  49 + unsigned int seq_nr;
49 50 int cpu_index;
50 51  
51 52 /*
... ... @@ -53,10 +54,8 @@
53 54 * seq_nr mod. number of cpus in use.
54 55 */
55 56  
56   - spin_lock(&pd->seq_lock);
57   - cpu_index = pd->seq_nr % cpumask_weight(pd->cpumask.pcpu);
58   - pd->seq_nr++;
59   - spin_unlock(&pd->seq_lock);
  57 + seq_nr = atomic_inc_return(&pd->seq_nr);
  58 + cpu_index = seq_nr % cpumask_weight(pd->cpumask.pcpu);
60 59  
61 60 return padata_index_to_cpu(pd, cpu_index);
62 61 }
... ... @@ -429,7 +428,7 @@
429 428 padata_init_pqueues(pd);
430 429 padata_init_squeues(pd);
431 430 setup_timer(&pd->timer, padata_reorder_timer, (unsigned long)pd);
432   - pd->seq_nr = 0;
  431 + atomic_set(&pd->seq_nr, -1);
433 432 atomic_set(&pd->reorder_objects, 0);
434 433 atomic_set(&pd->refcnt, 0);
435 434 pd->pinst = pinst;