13 Dec, 2011

1 commit

  • When disabling the MMU, it is necessary to take out a 1:1 identity map
    of the reset code so that it can safely be executed with and without
    the MMU active. To avoid the situation where the physical address of the
    reset code aliases with the virtual address of the active stack (which
    cannot be included in the 1:1 mapping), it is desirable to change to a
    new stack at a location which is less likely to alias.

    This code adds a new lib function, call_with_stack:

    void call_with_stack(void (*fn)(void *), void *arg, void *sp);

    which changes the stack to point at the sp parameter, before invoking
    fn(arg) with the new stack selected.

    Reviewed-by: Nicolas Pitre
    Reviewed-by: Catalin Marinas
    Signed-off-by: Dave Martin
    Signed-off-by: Will Deacon

    Will Deacon
     

27 Nov, 2011

1 commit

  • The bitops functions (e.g. _test_and_set_bit) on ARM do not have unwind
    annotations and therefore the kernel cannot backtrace out of them on a
    fatal error (for example, NULL pointer dereference).

    This patch annotates the bitops assembly macros with UNWIND annotations
    so that we can produce a meaningful backtrace on error. Callers of the
    macros are modified to pass their function name as a macro parameter,
    enforcing that the macros are used as standalone function implementations.

    Acked-by: Dave Martin
    Signed-off-by: Will Deacon
    Signed-off-by: Russell King

    Will Deacon
     

25 Oct, 2011

1 commit


17 Oct, 2011

2 commits


02 Oct, 2011

1 commit


08 Aug, 2011

1 commit

  • Since commit 1eb19a12bd22 ("lib/sha1: use the git implementation of
    SHA-1"), the ARM SHA1 routines no longer work. The reason? They
    depended on the larger 320-byte workspace, and now the sha1 workspace is
    just 16 words (64 bytes). So the assembly version would overwrite the
    stack randomly.

    The optimized asm version is also probably slower than the new improved
    C version, so there's no reason to keep it around. At least that was
    the case in git, where what appears to be the same assembly language
    version was removed two years ago because the optimized C BLK_SHA1 code
    was faster.

    Reported-and-tested-by: Joachim Eastwood
    Cc: Andreas Schwab
    Cc: Nicolas Pitre
    Signed-off-by: Linus Torvalds

    Linus Torvalds
     

13 Jul, 2011

1 commit


28 May, 2011

1 commit

  • The software division functions never had unwinding annotations
    added. Currently, when a division by zero occurs the backtrace shown
    will stop at Ldiv0 or some completely unrelated function. Add
    unwinding annotations in hopes of getting a more useful backtrace
    when a division by zero occurs.

    Signed-off-by: Laura Abbott
    Acked-by: Dave Martin
    Signed-off-by: Russell King

    Laura Abbott
     

20 Mar, 2011

1 commit


22 Feb, 2011

1 commit

  • Add pud_offset() et.al. between the pgd and pmd code in preparation of
    using pgtable-nopud.h rather than 4level-fixup.h.

    This incorporates a fix from Jamie Iles for
    uaccess_with_memcpy.c.

    Signed-off-by: Russell King

    Russell King
     

20 Feb, 2011

1 commit

  • The kernel doesn't officially need to interwork, but using BX
    wherever appropriate will help educate people into good assembler
    coding habits.

    BX is appropriate here because this code is predicated on
    __LINUX_ARM_ARCH__ >= 6

    Signed-off-by: Dave Martin
    Acked-by: Nicolas Pitre
    Signed-off-by: Russell King

    Dave Martin
     

03 Feb, 2011

2 commits


11 Jan, 2011

1 commit


07 Jan, 2011

1 commit


25 Nov, 2010

1 commit

  • The find_next_bit, find_first_bit, find_next_zero_bit
    and find_first_zero_bit functions were not properly
    clamping to the maxbit argument at the bit level. They
    were instead only checking maxbit at the byte level.
    To fix this, add a compare and a conditional move
    instruction to the end of the common bit-within-the-
    byte code used by all the functions and be sure not to
    clobber the maxbit argument before it is used.

    Cc:
    Reviewed-by: Nicolas Pitre
    Tested-by: Stephen Warren
    Signed-off-by: James Jones
    Signed-off-by: Russell King

    James Jones
     

04 Nov, 2010

1 commit

  • This patch removes the domain switching functionality via the set_fs and
    __switch_to functions on cores that have a TLS register.

    Currently, the ioremap and vmalloc areas share the same level 1 page
    tables and therefore have the same domain (DOMAIN_KERNEL). When the
    kernel domain is modified from Client to Manager (via the __set_fs or in
    the __switch_to function), the XN (eXecute Never) bit is overridden and
    newer CPUs can speculatively prefetch the ioremap'ed memory.

    Linux performs the kernel domain switching to allow user-specific
    functions (copy_to/from_user, get/put_user etc.) to access kernel
    memory. In order for these functions to work with the kernel domain set
    to Client, the patch modifies the LDRT/STRT and related instructions to
    the LDR/STR ones.

    The user pages access rights are also modified for kernel read-only
    access rather than read/write so that the copy-on-write mechanism still
    works. CPU_USE_DOMAINS gets disabled only if the hardware has a TLS register
    (CPU_32v6K is defined) since writing the TLS value to the high vectors page
    isn't possible.

    The user addresses passed to the kernel are checked by the access_ok()
    function so that they do not point to the kernel space.

    Tested-by: Anton Vorontsov
    Cc: Tony Lindgren
    Signed-off-by: Catalin Marinas
    Signed-off-by: Russell King

    Catalin Marinas
     

04 Aug, 2010

1 commit

  • * 'devel' of master.kernel.org:/home/rmk/linux-2.6-arm: (291 commits)
    ARM: AMBA: Add pclk support to AMBA bus infrastructure
    ARM: 6278/2: fix regression in RealView after the introduction of pclk
    ARM: 6277/1: mach-shmobile: Allow users to select HZ, default to 128
    ARM: 6276/1: mach-shmobile: remove duplicate NR_IRQS_LEGACY
    ARM: 6246/1: mmci: support larger MMCIDATALENGTH register
    ARM: 6245/1: mmci: enable hardware flow control on Ux500 variants
    ARM: 6244/1: mmci: add variant data and default MCICLOCK support
    ARM: 6243/1: mmci: pass power_mode to the translate_vdd callback
    ARM: 6274/1: add global control registers definition header file for nuc900
    mx2_camera: fix type of dma buffer virtual address pointer
    mx2_camera: Add soc_camera support for i.MX25/i.MX27
    arm/imx/gpio: add spinlock protection
    ARM: Add support for the LPC32XX arch
    ARM: LPC32XX: Arch config menu supoport and makefiles
    ARM: LPC32XX: Phytec 3250 platform support
    ARM: LPC32XX: Misc support functions
    ARM: LPC32XX: Serial support code
    ARM: LPC32XX: System suspend support
    ARM: LPC32XX: GPIO, timer, and IRQ drivers
    ARM: LPC32XX: Clock driver
    ...

    Linus Torvalds
     

26 Jul, 2010

1 commit

  • Using the parent functions frame pointer to access our arguments is
    completely wrong, whether or not we're building with frame pointers
    or not. What we should be using is the stack pointer to get at the
    word above the registers we stacked ourselves.

    Reported-by: Bosko Radivojevic
    Tested-by: Bosko Radivojevic
    Signed-off-by: Russell King

    Russell King
     

24 Jun, 2010

1 commit


08 May, 2010

1 commit


21 Apr, 2010

1 commit

  • /tmp/ccJ3ssZW.s: Assembler messages:
    /tmp/ccJ3ssZW.s:1952: Error: can't resolve `.text' {.text section} - `.LFB1077'

    This is caused because:

    .section .data
    .section .text
    .section .text
    .previous

    does not return us to the .text section, but the .data section; this
    makes use of .previous dangerous if the ordering of previous sections
    is not known.

    Fix up the other users of .previous; .pushsection and .popsection are
    a safer pairing to use than .section and .previous.

    Signed-off-by: Russell King

    Russell King
     

05 Apr, 2010

1 commit


30 Mar, 2010

2 commits

  • …it slab.h inclusion from percpu.h

    percpu.h is included by sched.h and module.h and thus ends up being
    included when building most .c files. percpu.h includes slab.h which
    in turn includes gfp.h making everything defined by the two files
    universally available and complicating inclusion dependencies.

    percpu.h -> slab.h dependency is about to be removed. Prepare for
    this change by updating users of gfp and slab facilities include those
    headers directly instead of assuming availability. As this conversion
    needs to touch large number of source files, the following script is
    used as the basis of conversion.

    http://userweb.kernel.org/~tj/misc/slabh-sweep.py

    The script does the followings.

    * Scan files for gfp and slab usages and update includes such that
    only the necessary includes are there. ie. if only gfp is used,
    gfp.h, if slab is used, slab.h.

    * When the script inserts a new include, it looks at the include
    blocks and try to put the new include such that its order conforms
    to its surrounding. It's put in the include block which contains
    core kernel includes, in the same order that the rest are ordered -
    alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
    doesn't seem to be any matching order.

    * If the script can't find a place to put a new include (mostly
    because the file doesn't have fitting include block), it prints out
    an error message indicating which .h file needs to be added to the
    file.

    The conversion was done in the following steps.

    1. The initial automatic conversion of all .c files updated slightly
    over 4000 files, deleting around 700 includes and adding ~480 gfp.h
    and ~3000 slab.h inclusions. The script emitted errors for ~400
    files.

    2. Each error was manually checked. Some didn't need the inclusion,
    some needed manual addition while adding it to implementation .h or
    embedding .c file was more appropriate for others. This step added
    inclusions to around 150 files.

    3. The script was run again and the output was compared to the edits
    from #2 to make sure no file was left behind.

    4. Several build tests were done and a couple of problems were fixed.
    e.g. lib/decompress_*.c used malloc/free() wrappers around slab
    APIs requiring slab.h to be added manually.

    5. The script was run on all .h files but without automatically
    editing them as sprinkling gfp.h and slab.h inclusions around .h
    files could easily lead to inclusion dependency hell. Most gfp.h
    inclusion directives were ignored as stuff from gfp.h was usually
    wildly available and often used in preprocessor macros. Each
    slab.h inclusion directive was examined and added manually as
    necessary.

    6. percpu.h was updated not to include slab.h.

    7. Build test were done on the following configurations and failures
    were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
    distributed build env didn't work with gcov compiles) and a few
    more options had to be turned off depending on archs to make things
    build (like ipr on powerpc/64 which failed due to missing writeq).

    * x86 and x86_64 UP and SMP allmodconfig and a custom test config.
    * powerpc and powerpc64 SMP allmodconfig
    * sparc and sparc64 SMP allmodconfig
    * ia64 SMP allmodconfig
    * s390 SMP allmodconfig
    * alpha SMP allmodconfig
    * um on x86_64 SMP allmodconfig

    8. percpu.h modifications were reverted so that it could be applied as
    a separate patch and serve as bisection point.

    Given the fact that I had only a couple of failures from tests on step
    6, I'm fairly confident about the coverage of this conversion patch.
    If there is a breakage, it's likely to be something in one of the arch
    headers which should be easily discoverable easily on most builds of
    the specific arch.

    Signed-off-by: Tejun Heo <tj@kernel.org>
    Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
    Cc: Ingo Molnar <mingo@redhat.com>
    Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>

    Tejun Heo
     
  • When compiling the kernel to Thumb-2, using a 16-bit NOP in the
    memmove() implementation causes the preceding ADD PC instruction to
    branch incorrectly in the middle of a 32-bit LDR or STR instruction. The
    memmove() code is now similar to the memcpy() template.

    Signed-off-by: Catalin Marinas
    Signed-off-by: Russell King

    Catalin Marinas
     

19 Sep, 2009

1 commit


16 Sep, 2009

2 commits

  • Optimized version of copy_page() was written with assumption that cache
    line size is 32 bytes. On Cortex-A8 cache line size is 64 bytes.

    This patch tries to generalize copy_page() to work with any cache line
    size if cache line size is multiple of 16 and page size is multiple of
    two cache line size.

    After this optimization we've got ~25% speedup on OMAP3(tested in
    userspace).

    There is test for kernelspace which trigger copy-on-write after fork():

    #include
    #include
    #include

    #define BUF_SIZE (10000*4096)
    #define NFORK 200

    int main(int argc, char **argv)
    {
    char *buf = malloc(BUF_SIZE);
    int i;

    memset(buf, 0, BUF_SIZE);

    for(i = 0; i < NFORK; i++) {
    if (fork()) {
    wait(NULL);
    } else {
    int j;

    for(j = 0; j < BUF_SIZE; j+= 4096)
    buf[j] = (j & 0xFF) + 1;
    break;
    }
    }

    free(buf);
    return 0;
    }

    Before optimization this test takes ~66 seconds, after optimization
    takes ~56 seconds.

    Signed-off-by: Siarhei Siamashka
    Signed-off-by: Kirill A. Shutemov
    Signed-off-by: Russell King

    Kirill A. Shutemov
     
  • Due to problems at cam.org, my nico@cam.org email address is no longer
    valid. FRom now on, nico@fluxnic.net should be used instead.

    Signed-off-by: Nicolas Pitre
    Signed-off-by: Linus Torvalds

    Nicolas Pitre
     

15 Aug, 2009

1 commit


14 Aug, 2009

1 commit

  • Before this patch enabling and disabling irqs in assembler code and by
    the hardware wasn't tracked completly.

    I had to transpose two instructions in arch/arm/lib/bitops.h because
    restore_irqs doesn't preserve the flags with CONFIG_TRACE_IRQFLAGS=y

    Signed-off-by: Uwe Kleine-König
    Cc: Russell King
    Cc: Peter Zijlstra
    Cc: Ingo Molnar

    Signed-off-by: Uwe Kleine-König

    Uwe Kleine-König
     

24 Jul, 2009

2 commits


14 Jun, 2009

1 commit


30 May, 2009

4 commits

  • Previous size thresholds were guessed from various user space benchmarks
    using a kernel with and without the alternative uaccess option. This
    is however not as precise as a kernel based test to measure the real
    speed of each method.

    This adds a simple test bench to show the time needed for each method.
    With this, the optimal size treshold for the alternative implementation
    can be determined with more confidence. It appears that the optimal
    threshold for both copy_to_user and clear_user is around 64 bytes. This
    is not a surprise knowing that the memcpy and memset implementations
    need at least 64 bytes to achieve maximum throughput.

    One might suggest that such test be used to determine the optimal
    threshold at run time instead, but results are near enough to 64 on
    tested targets concerned by this alternative copy_to_user implementation,
    so adding some overhead associated with a variable threshold is probably
    not worth it for now.

    Signed-off-by: Nicolas Pitre

    Nicolas Pitre
     
  • Because the alternate copy_to_user implementation has a higher setup cost
    than the standard implementation, the size of the memory area to copy
    is tested and the standard implementation invoked instead when that size
    is too small. Still, that test is made after the processor has preserved
    a bunch of registers on the stack which have to be reloaded right away
    needlessly in that case, causing a measurable performance regression
    compared to plain usage of the standard implementation only.

    To make the size test overhead negligible, let's factorize it out of
    the alternate copy_to_user function where it is clear to the compiler
    that no stack frame is needed. Thanks to CONFIG_ARM_UNWIND allowing
    for frame pointers to be disabled and tail call optimization to kick in,
    the overhead in the small copy case becomes only 3 assembly instructions.

    A similar trick is applied to clear_user as well.

    Signed-off-by: Nicolas Pitre

    Nicolas Pitre
     
  • This implements {copy_to,clear}_user() by faulting in the userland
    pages and then using the regular kernel mem{cpy,set}() to copy the
    data (while holding the page table lock). This is a win if the regular
    mem{cpy,set}() implementations are faster than the user copy functions,
    which is the case e.g. on Feroceon, where 8-word STMs (which memcpy()
    uses under the right conditions) give significantly higher memory write
    throughput than a sequence of individual 32bit stores.

    Here are numbers for page sized buffers on some Feroceon cores:

    - copy_to_user on Orion5x goes from 51 MB/s to 83 MB/s
    - clear_user on Orion5x goes from 89MB/s to 314MB/s
    - copy_to_user on Kirkwood goes from 240 MB/s to 356 MB/s
    - clear_user on Kirkwood goes from 367 MB/s to 1108 MB/s
    - copy_to_user on Disco-Duo goes from 248 MB/s to 398 MB/s
    - clear_user on Disco-Duo goes from 328 MB/s to 1741 MB/s

    Because the setup cost is non negligible, this is worthwhile only if
    the amount of data to copy is large enough. The operation falls back
    to the standard implementation when the amount of data is below a certain
    threshold. This threshold was determined empirically, however some targets
    could benefit from a lower runtime determined value for optimal results
    eventually.

    In the copy_from_user() case, this technique does not provide any
    worthwhile performance gain due to the fact that any kind of read access
    allocates the cache and subsequent 32bit loads are just as fast as the
    equivalent 8-word LDM.

    Signed-off-by: Lennert Buytenhek
    Signed-off-by: Nicolas Pitre
    Tested-by: Martin Michlmayr

    Lennert Buytenhek
     
  • This allows for optional alternative implementations of __copy_to_user
    and __clear_user, with a possible runtime fallback to the standard
    version when the alternative provides no gain over that standard
    version. This is done by making the standard __copy_to_user into a weak
    alias for the symbol __copy_to_user_std. Same thing for __clear_user.

    Those two functions are particularly good candidates to have alternative
    implementations for, since they rely on the STRT instruction which has
    lower performances than STM instructions on some CPU cores such as
    the ARM1176 and Marvell Feroceon.

    Signed-off-by: Nicolas Pitre

    Nicolas Pitre
     

29 May, 2009

1 commit

  • Mathieu Desnoyers pointed out that the ARM barriers were lacking:

    - cmpxchg, xchg and atomic add return need memory barriers on
    architectures which can reorder the relative order in which memory
    read/writes can be seen between CPUs, which seems to include recent
    ARM architectures. Those barriers are currently missing on ARM.

    - test_and_xxx_bit were missing SMP barriers.

    So put these barriers in. Provide separate atomic_add/atomic_sub
    operations which do not require barriers.

    Reported-Reviewed-and-Acked-by: Mathieu Desnoyers
    Signed-off-by: Russell King

    Russell King
     

27 Nov, 2008

1 commit