21 Sep, 2020

1 commit


19 Jun, 2019

1 commit

  • Based on 2 normalized pattern(s):

    this program is free software you can redistribute it and or modify
    it under the terms of the gnu general public license version 2 as
    published by the free software foundation

    this program is free software you can redistribute it and or modify
    it under the terms of the gnu general public license version 2 as
    published by the free software foundation #

    extracted by the scancode license scanner the SPDX license identifier

    GPL-2.0-only

    has been chosen to replace the boilerplate/reference in 4122 file(s).

    Signed-off-by: Thomas Gleixner
    Reviewed-by: Enrico Weigelt
    Reviewed-by: Kate Stewart
    Reviewed-by: Allison Randal
    Cc: linux-spdx@vger.kernel.org
    Link: https://lkml.kernel.org/r/20190604081206.933168790@linutronix.de
    Signed-off-by: Greg Kroah-Hartman

    Thomas Gleixner
     

13 Apr, 2019

1 commit

  • Rather than dereferencing a pointer to a bucket and then passing the
    result to rht_ptr(), we now pass in the pointer and do the dereference
    in rht_ptr().

    This requires that we pass in the tbl and hash as well to support RCU
    checks, and means that the various rht_for_each functions can expect a
    pointer that can be dereferenced without further care.

    There are two places where we dereference a bucket pointer
    where there is no testable protection - in each case we know
    that we much have exclusive access without having taken a lock.
    The previous code used rht_dereference() to pretend that holding
    the mutex provided protects, but holding the mutex never provides
    protection for accessing buckets.

    So instead introduce rht_ptr_exclusive() that can be used when
    there is known to be exclusive access without holding any locks.

    Signed-off-by: NeilBrown
    Signed-off-by: David S. Miller

    NeilBrown
     

08 Apr, 2019

1 commit

  • This patch changes rhashtables to use a bit_spin_lock on BIT(1) of the
    bucket pointer to lock the hash chain for that bucket.

    The benefits of a bit spin_lock are:
    - no need to allocate a separate array of locks.
    - no need to have a configuration option to guide the
    choice of the size of this array
    - locking cost is often a single test-and-set in a cache line
    that will have to be loaded anyway. When inserting at, or removing
    from, the head of the chain, the unlock is free - writing the new
    address in the bucket head implicitly clears the lock bit.
    For __rhashtable_insert_fast() we ensure this always happens
    when adding a new key.
    - even when lockings costs 2 updates (lock and unlock), they are
    in a cacheline that needs to be read anyway.

    The cost of using a bit spin_lock is a little bit of code complexity,
    which I think is quite manageable.

    Bit spin_locks are sometimes inappropriate because they are not fair -
    if multiple CPUs repeatedly contend of the same lock, one CPU can
    easily be starved. This is not a credible situation with rhashtable.
    Multiple CPUs may want to repeatedly add or remove objects, but they
    will typically do so at different buckets, so they will attempt to
    acquire different locks.

    As we have more bit-locks than we previously had spinlocks (by at
    least a factor of two) we can expect slightly less contention to
    go with the slightly better cache behavior and reduced memory
    consumption.

    To enhance type checking, a new struct is introduced to represent the
    pointer plus lock-bit
    that is stored in the bucket-table. This is "struct rhash_lock_head"
    and is empty. A pointer to this needs to be cast to either an
    unsigned lock, or a "struct rhash_head *" to be useful.
    Variables of this type are most often called "bkt".

    Previously "pprev" would sometimes point to a bucket, and sometimes a
    ->next pointer in an rhash_head. As these are now different types,
    pprev is NULL when it would have pointed to the bucket. In that case,
    'blk' is used, together with correct locking protocol.

    Signed-off-by: NeilBrown
    Signed-off-by: David S. Miller

    NeilBrown
     

22 Feb, 2019

2 commits


01 Feb, 2019

1 commit

  • The test_insert_dup() function from lib/test_rhashtable.c passes a
    pointer to a stack object to rhltable_init(). Allocate the hash table
    dynamically to avoid that the following is reported with object
    debugging enabled:

    ODEBUG: object (ptrval) is on stack (ptrval), but NOT annotated.
    WARNING: CPU: 0 PID: 1 at lib/debugobjects.c:368 __debug_object_init+0x312/0x480
    Modules linked in:
    EIP: __debug_object_init+0x312/0x480
    Call Trace:
    ? debug_object_init+0x1a/0x20
    ? __init_work+0x16/0x30
    ? rhashtable_init+0x1e1/0x460
    ? sched_clock_cpu+0x57/0xe0
    ? rhltable_init+0xb/0x20
    ? test_insert_dup+0x32/0x20f
    ? trace_hardirqs_on+0x38/0xf0
    ? ida_dump+0x10/0x10
    ? jhash+0x130/0x130
    ? my_hashfn+0x30/0x30
    ? test_rht_init+0x6aa/0xab4
    ? ida_dump+0x10/0x10
    ? test_rhltable+0xc5c/0xc5c
    ? do_one_initcall+0x67/0x28e
    ? trace_hardirqs_off+0x22/0xe0
    ? restore_all_kernel+0xf/0x70
    ? trace_hardirqs_on_thunk+0xc/0x10
    ? restore_all_kernel+0xf/0x70
    ? kernel_init_freeable+0x142/0x213
    ? rest_init+0x230/0x230
    ? kernel_init+0x10/0x110
    ? schedule_tail_wrapper+0x9/0xc
    ? ret_from_fork+0x19/0x24

    Cc: Thomas Graf
    Cc: Herbert Xu
    Cc: netdev@vger.kernel.org
    Cc: linux-kernel@vger.kernel.org
    Signed-off-by: Bart Van Assche
    Acked-by: Herbert Xu
    Signed-off-by: David S. Miller

    Bart Van Assche
     

19 Dec, 2018

1 commit

  • This is one of only two files that initialize a semaphore to a negative
    value. We don't really need the two semaphores here at all, but can do
    the same thing in more conventional and more effient way, by using a
    single waitqueue and an atomic thread counter.

    This gets us a little bit closer to eliminating classic semaphores from
    the kernel. It also fixes a corner case where we fail to continue after
    one of the threads fails to start up.

    An alternative would be to use a split kthread_create()+wake_up_process()
    and completely eliminate the separate synchronization.

    Acked-by: Phil Sutter
    Signed-off-by: Arnd Bergmann
    Acked-by: Herbert Xu
    Signed-off-by: David S. Miller

    Arnd Bergmann
     

22 Jun, 2018

2 commits

  • This "feature" is unused, undocumented, and untested and so doesn't
    really belong. A patch is under development to properly implement
    support for detecting when a search gets diverted down a different
    chain, which the common purpose of nulls markers.

    This patch actually fixes a bug too. The table resizing allows a
    table to grow to 2^31 buckets, but the hash is truncated to 27 bits -
    any growth beyond 2^27 is wasteful an ineffective.

    This patch results in NULLS_MARKER(0) being used for all chains,
    and leaves the use of rht_is_a_null() to test for it.

    Acked-by: Herbert Xu
    Signed-off-by: NeilBrown
    Signed-off-by: David S. Miller

    NeilBrown
     
  • print_ht in rhashtable_test calls rht_dereference() with neither
    RCU protection or the mutex. This triggers an RCU warning.
    So take the mutex to silence the warning.

    Acked-by: Herbert Xu
    Signed-off-by: NeilBrown
    Signed-off-by: David S. Miller

    NeilBrown
     

13 Jun, 2018

1 commit

  • The vzalloc() function has no 2-factor argument form, so multiplication
    factors need to be wrapped in array_size(). This patch replaces cases of:

    vzalloc(a * b)

    with:
    vzalloc(array_size(a, b))

    as well as handling cases of:

    vzalloc(a * b * c)

    with:

    vzalloc(array3_size(a, b, c))

    This does, however, attempt to ignore constant size factors like:

    vzalloc(4 * 1024)

    though any constants defined via macros get caught up in the conversion.

    Any factors with a sizeof() of "unsigned char", "char", and "u8" were
    dropped, since they're redundant.

    The Coccinelle script used for this was:

    // Fix redundant parens around sizeof().
    @@
    type TYPE;
    expression THING, E;
    @@

    (
    vzalloc(
    - (sizeof(TYPE)) * E
    + sizeof(TYPE) * E
    , ...)
    |
    vzalloc(
    - (sizeof(THING)) * E
    + sizeof(THING) * E
    , ...)
    )

    // Drop single-byte sizes and redundant parens.
    @@
    expression COUNT;
    typedef u8;
    typedef __u8;
    @@

    (
    vzalloc(
    - sizeof(u8) * (COUNT)
    + COUNT
    , ...)
    |
    vzalloc(
    - sizeof(__u8) * (COUNT)
    + COUNT
    , ...)
    |
    vzalloc(
    - sizeof(char) * (COUNT)
    + COUNT
    , ...)
    |
    vzalloc(
    - sizeof(unsigned char) * (COUNT)
    + COUNT
    , ...)
    |
    vzalloc(
    - sizeof(u8) * COUNT
    + COUNT
    , ...)
    |
    vzalloc(
    - sizeof(__u8) * COUNT
    + COUNT
    , ...)
    |
    vzalloc(
    - sizeof(char) * COUNT
    + COUNT
    , ...)
    |
    vzalloc(
    - sizeof(unsigned char) * COUNT
    + COUNT
    , ...)
    )

    // 2-factor product with sizeof(type/expression) and identifier or constant.
    @@
    type TYPE;
    expression THING;
    identifier COUNT_ID;
    constant COUNT_CONST;
    @@

    (
    vzalloc(
    - sizeof(TYPE) * (COUNT_ID)
    + array_size(COUNT_ID, sizeof(TYPE))
    , ...)
    |
    vzalloc(
    - sizeof(TYPE) * COUNT_ID
    + array_size(COUNT_ID, sizeof(TYPE))
    , ...)
    |
    vzalloc(
    - sizeof(TYPE) * (COUNT_CONST)
    + array_size(COUNT_CONST, sizeof(TYPE))
    , ...)
    |
    vzalloc(
    - sizeof(TYPE) * COUNT_CONST
    + array_size(COUNT_CONST, sizeof(TYPE))
    , ...)
    |
    vzalloc(
    - sizeof(THING) * (COUNT_ID)
    + array_size(COUNT_ID, sizeof(THING))
    , ...)
    |
    vzalloc(
    - sizeof(THING) * COUNT_ID
    + array_size(COUNT_ID, sizeof(THING))
    , ...)
    |
    vzalloc(
    - sizeof(THING) * (COUNT_CONST)
    + array_size(COUNT_CONST, sizeof(THING))
    , ...)
    |
    vzalloc(
    - sizeof(THING) * COUNT_CONST
    + array_size(COUNT_CONST, sizeof(THING))
    , ...)
    )

    // 2-factor product, only identifiers.
    @@
    identifier SIZE, COUNT;
    @@

    vzalloc(
    - SIZE * COUNT
    + array_size(COUNT, SIZE)
    , ...)

    // 3-factor product with 1 sizeof(type) or sizeof(expression), with
    // redundant parens removed.
    @@
    expression THING;
    identifier STRIDE, COUNT;
    type TYPE;
    @@

    (
    vzalloc(
    - sizeof(TYPE) * (COUNT) * (STRIDE)
    + array3_size(COUNT, STRIDE, sizeof(TYPE))
    , ...)
    |
    vzalloc(
    - sizeof(TYPE) * (COUNT) * STRIDE
    + array3_size(COUNT, STRIDE, sizeof(TYPE))
    , ...)
    |
    vzalloc(
    - sizeof(TYPE) * COUNT * (STRIDE)
    + array3_size(COUNT, STRIDE, sizeof(TYPE))
    , ...)
    |
    vzalloc(
    - sizeof(TYPE) * COUNT * STRIDE
    + array3_size(COUNT, STRIDE, sizeof(TYPE))
    , ...)
    |
    vzalloc(
    - sizeof(THING) * (COUNT) * (STRIDE)
    + array3_size(COUNT, STRIDE, sizeof(THING))
    , ...)
    |
    vzalloc(
    - sizeof(THING) * (COUNT) * STRIDE
    + array3_size(COUNT, STRIDE, sizeof(THING))
    , ...)
    |
    vzalloc(
    - sizeof(THING) * COUNT * (STRIDE)
    + array3_size(COUNT, STRIDE, sizeof(THING))
    , ...)
    |
    vzalloc(
    - sizeof(THING) * COUNT * STRIDE
    + array3_size(COUNT, STRIDE, sizeof(THING))
    , ...)
    )

    // 3-factor product with 2 sizeof(variable), with redundant parens removed.
    @@
    expression THING1, THING2;
    identifier COUNT;
    type TYPE1, TYPE2;
    @@

    (
    vzalloc(
    - sizeof(TYPE1) * sizeof(TYPE2) * COUNT
    + array3_size(COUNT, sizeof(TYPE1), sizeof(TYPE2))
    , ...)
    |
    vzalloc(
    - sizeof(TYPE1) * sizeof(THING2) * (COUNT)
    + array3_size(COUNT, sizeof(TYPE1), sizeof(TYPE2))
    , ...)
    |
    vzalloc(
    - sizeof(THING1) * sizeof(THING2) * COUNT
    + array3_size(COUNT, sizeof(THING1), sizeof(THING2))
    , ...)
    |
    vzalloc(
    - sizeof(THING1) * sizeof(THING2) * (COUNT)
    + array3_size(COUNT, sizeof(THING1), sizeof(THING2))
    , ...)
    |
    vzalloc(
    - sizeof(TYPE1) * sizeof(THING2) * COUNT
    + array3_size(COUNT, sizeof(TYPE1), sizeof(THING2))
    , ...)
    |
    vzalloc(
    - sizeof(TYPE1) * sizeof(THING2) * (COUNT)
    + array3_size(COUNT, sizeof(TYPE1), sizeof(THING2))
    , ...)
    )

    // 3-factor product, only identifiers, with redundant parens removed.
    @@
    identifier STRIDE, SIZE, COUNT;
    @@

    (
    vzalloc(
    - (COUNT) * STRIDE * SIZE
    + array3_size(COUNT, STRIDE, SIZE)
    , ...)
    |
    vzalloc(
    - COUNT * (STRIDE) * SIZE
    + array3_size(COUNT, STRIDE, SIZE)
    , ...)
    |
    vzalloc(
    - COUNT * STRIDE * (SIZE)
    + array3_size(COUNT, STRIDE, SIZE)
    , ...)
    |
    vzalloc(
    - (COUNT) * (STRIDE) * SIZE
    + array3_size(COUNT, STRIDE, SIZE)
    , ...)
    |
    vzalloc(
    - COUNT * (STRIDE) * (SIZE)
    + array3_size(COUNT, STRIDE, SIZE)
    , ...)
    |
    vzalloc(
    - (COUNT) * STRIDE * (SIZE)
    + array3_size(COUNT, STRIDE, SIZE)
    , ...)
    |
    vzalloc(
    - (COUNT) * (STRIDE) * (SIZE)
    + array3_size(COUNT, STRIDE, SIZE)
    , ...)
    |
    vzalloc(
    - COUNT * STRIDE * SIZE
    + array3_size(COUNT, STRIDE, SIZE)
    , ...)
    )

    // Any remaining multi-factor products, first at least 3-factor products
    // when they're not all constants...
    @@
    expression E1, E2, E3;
    constant C1, C2, C3;
    @@

    (
    vzalloc(C1 * C2 * C3, ...)
    |
    vzalloc(
    - E1 * E2 * E3
    + array3_size(E1, E2, E3)
    , ...)
    )

    // And then all remaining 2 factors products when they're not all constants.
    @@
    expression E1, E2;
    constant C1, C2;
    @@

    (
    vzalloc(C1 * C2, ...)
    |
    vzalloc(
    - E1 * E2
    + array_size(E1, E2)
    , ...)
    )

    Signed-off-by: Kees Cook

    Kees Cook
     

07 Mar, 2018

1 commit


11 Dec, 2017

1 commit

  • Most callers of rhashtable_walk_start don't care about a resize event
    which is indicated by a return value of -EAGAIN. So calls to
    rhashtable_walk_start are wrapped wih code to ignore -EAGAIN. Something
    like this is common:

    ret = rhashtable_walk_start(rhiter);
    if (ret && ret != -EAGAIN)
    goto out;

    Since zero and -EAGAIN are the only possible return values from the
    function this check is pointless. The condition never evaluates to true.

    This patch changes rhashtable_walk_start to return void. This simplifies
    code for the callers that ignore -EAGAIN. For the few cases where the
    caller cares about the resize event, particularly where the table can be
    walked in mulitple parts for netlink or seq file dump, the function
    rhashtable_walk_start_check has been added that returns -EAGAIN on a
    resize event.

    Signed-off-by: Tom Herbert
    Acked-by: Herbert Xu
    Signed-off-by: David S. Miller

    Tom Herbert
     

22 Sep, 2017

1 commit

  • kbuild test robot reported a section mismatch warning w. gcc 4.x:
    WARNING: lib/test_rhashtable.o(.text+0x139e):
    Section mismatch in reference from the function rhltable_insert.clone.3() to the variable .init.data:rhlt

    so remove this annotation.

    Fixes: cdd4de372ea06 ("test_rhashtable: add test case for rhl_table interface")
    Reported-by: kbuild test robot
    Signed-off-by: Florian Westphal
    Signed-off-by: David S. Miller

    Florian Westphal
     

20 Sep, 2017

4 commits


26 Jul, 2017

1 commit


25 Jul, 2017

1 commit

  • During concurrent access testing, threadfunc() concatenated thread ID
    and object index to create a unique key like so:

    | tdata->objs[i].value = (tdata->id << 16) | i;

    This breaks if a user passes an entries parameter of 64k or higher,
    since 'i' might use more than 16 bits then. Effectively, this will lead
    to duplicate keys in the table.

    Fix the problem by introducing a struct holding object and thread ID and
    using that as key instead of a single integer type field.

    Fixes: f4a3e90ba5739 ("rhashtable-test: extend to test concurrency")
    Reported by: Manuel Messner
    Signed-off-by: Phil Sutter
    Acked-by: Herbert Xu
    Signed-off-by: David S. Miller

    Phil Sutter
     

09 Aug, 2016

1 commit


05 Apr, 2016

1 commit

  • In certain cases, the 802.11 mesh pathtable code wants to
    iterate over all of the entries in the forwarding table from
    the receive path, which is inside an RCU read-side critical
    section. Enable walks inside atomic sections by allowing
    GFP_ATOMIC allocations for the walker state.

    Change all existing callsites to pass in GFP_KERNEL.

    Acked-by: Thomas Graf
    Signed-off-by: Bob Copeland
    [also adjust gfs2/glock.c and rhashtable tests]
    Signed-off-by: Johannes Berg

    Bob Copeland
     

24 Nov, 2015

4 commits

  • This is rather a hack to expose the current issue with rhashtable to
    under high pressure sometimes return -ENOMEM even though system memory
    is not exhausted and a consecutive insert may succeed.

    Signed-off-by: Phil Sutter
    Signed-off-by: David S. Miller

    Phil Sutter
     
  • A maximum table size of 64k entries is insufficient for the multiple
    threads test even in default configuration (10 threads * 50000 objects =
    500000 objects in total). Since we know how many objects will be
    inserted, calculate the max size unless overridden by parameter.

    Note that specifying the exact number of objects upon table init won't
    suffice as that value is being rounded down to the next power of two -
    anticipate this by rounding up to the next power of two in beforehand.

    Signed-off-by: Phil Sutter
    Signed-off-by: David S. Miller

    Phil Sutter
     
  • After adding cond_resched() calls to threadfunc(), a surprisingly high
    rate of insert failures occurred probably due to table resizes getting a
    better chance to run in background. To not soften up the remaining
    tests, retry inserts until they either succeed or fail permanently.

    Also change the non-threaded test to retry insert operations, too.

    Suggested-by: Thomas Graf
    Signed-off-by: Phil Sutter
    Signed-off-by: David S. Miller

    Phil Sutter
     
  • This should fix for soft lockup bugs triggered on slow systems.

    Signed-off-by: Phil Sutter
    Signed-off-by: David S. Miller

    Phil Sutter
     

18 Aug, 2015

1 commit

  • After having tested insertion, lookup, table walk and removal, spawn a
    number of threads running operations on the same rhashtable. Each of
    them will:

    1) insert it's own set of objects,
    2) lookup every successfully inserted object and finally
    3) remove objects in several rounds until all of them have been removed,
    making sure the remaining ones are still found after each round.

    This should put a good amount of load onto the system and due to
    synchronising thread startup via two semaphores also extensive
    concurrent table access.

    The default number of ten threads returned within half a second on my
    local VM with two cores. Running 200 threads took about four seconds. If
    slow systems suffer too much from this though, the default could be
    lowered or even set to zero so this extended test does not run at all by
    default.

    Signed-off-by: Phil Sutter
    Acked-by: Thomas Graf
    Signed-off-by: David S. Miller

    Phil Sutter
     

21 Jul, 2015

1 commit


06 May, 2015

1 commit

  • A 64bit division went in unnoticed. Use do_div() to accomodate
    non 64bit architectures.

    Reported-by: kbuild test robot
    Fixes: 1aa661f5c3df ("rhashtable-test: Measure time to insert, remove & traverse entries")
    Signed-off-by: Thomas Graf
    Signed-off-by: David S. Miller

    Thomas Graf
     

04 May, 2015

6 commits


04 Apr, 2015

1 commit


24 Mar, 2015

1 commit

  • This patch adds the missing bits to allow multiple rehashes. The
    read-side as well as remove already handle this correctly. So it's
    only the rehasher and insertion that need modification to handle
    this.

    Note that this patch doesn't actually enable it so for now rehashing
    is still only performed by the worker thread.

    This patch also disables the explicit expand/shrink interface because
    the table is meant to expand and shrink automatically, and continuing
    to export these interfaces unnecessarily complicates the life of the
    rehasher since the rehash process is now composed of two parts.

    Signed-off-by: Herbert Xu
    Acked-by: Thomas Graf
    Signed-off-by: David S. Miller

    Herbert Xu
     

21 Mar, 2015

1 commit


19 Mar, 2015

1 commit


15 Mar, 2015

1 commit

  • This patch adds a rehash counter to bucket_table to indicate
    the last bucket that has been rehashed. This serves two purposes:

    1. Any bucket that has been rehashed can never gain a new object.
    2. If the rehash counter reaches the size of the table, the table
    will forever remain empty.

    This patch also downsizes bucket_table->size to an unsigned int
    since we do not support sizes greater than 32 bits yet.

    Signed-off-by: Herbert Xu
    Signed-off-by: David S. Miller

    Herbert Xu