29 Jul, 2010

1 commit


18 May, 2010

1 commit

  • …/git/tip/linux-2.6-tip

    * 'x86-atomic-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip:
    x86: Fix LOCK_PREFIX_HERE for uniprocessor build
    x86, atomic64: In selftest, distinguish x86-64 from 586+
    x86-32: Fix atomic64_inc_not_zero return value convention
    lib: Fix atomic64_inc_not_zero test
    lib: Fix atomic64_add_unless return value convention
    x86-32: Fix atomic64_add_unless return value convention
    lib: Fix atomic64_add_unless test
    x86: Implement atomic[64]_dec_if_positive()
    lib: Only test atomic64_dec_if_positive on archs having it
    x86-32: Rewrite 32-bit atomic64 functions in assembly
    lib: Add self-test for atomic64_t
    x86-32: Allow UP/SMP lock replacement in cmpxchg64
    x86: Add support for lock prefix in alternatives

    Linus Torvalds
     

30 Apr, 2010

1 commit


10 Mar, 2010

1 commit

  • The PEBS+LBR decoding magic needs the insn_get_length() infrastructure
    to be able to decode x86 instruction length.

    So split it out of KPROBES dependency and make it enabled when either
    KPROBES or PERF_EVENTS is enabled.

    Cc: Peter Zijlstra
    Cc: Masami Hiramatsu
    Cc: Frederic Weisbecker
    Cc: Paul Mackerras
    Cc: Arnaldo Carvalho de Melo
    LKML-Reference:
    Signed-off-by: Ingo Molnar

    Ingo Molnar
     

01 Mar, 2010

2 commits


26 Feb, 2010

1 commit

  • This patch replaces atomic64_32.c with two assembly implementations,
    one for 386/486 machines using pushf/cli/popf and one for 586+ machines
    using cmpxchg8b.

    The cmpxchg8b implementation provides the following advantages over the
    current one:

    1. Implements atomic64_add_unless, atomic64_dec_if_positive and
    atomic64_inc_not_zero

    2. Uses the ZF flag changed by cmpxchg8b instead of doing a comparison

    3. Uses custom register calling conventions that reduce or eliminate
    register moves to suit cmpxchg8b

    4. Reads the initial value instead of using cmpxchg8b to do that.
    Currently we use lock xaddl and movl, which seems the fastest.

    5. Does not use the lock prefix for atomic64_set
    64-bit writes are already atomic, so we don't need that.
    We still need it for atomic64_read to avoid restoring a value
    changed in the meantime.

    6. Allocates registers as well or better than gcc

    The 386 implementation provides support for 386 and 486 machines.
    386/486 SMP is not supported (we dropped it), but such support can be
    added easily if desired.

    A pure assembly implementation is required due to the custom calling
    conventions, and desire to use %ebp in atomic64_add_return (we need
    7 registers...), as well as the ability to use pushf/popf in the 386
    code without an intermediate pop/push.

    The parameter names are changed to match the convention in atomic_64.h

    Changes in v3 (due to rebasing to tip/x86/asm):
    - Patches atomic64_32.h instead of atomic_32.h
    - Uses the CALL alternative mechanism from commit
    1b1d9258181bae199dc940f4bd0298126b9a73d9

    Changes in v2:
    - Merged 386 and cx8 support in the same patch
    - 386 support now done in assembly, C code no longer used at all
    - cmpxchg64 is used for atomic64_cmpxchg
    - stop using macros, use one-line inline functions instead
    - miscellanous changes and improvements

    Signed-off-by: Luca Barbieri
    LKML-Reference:
    Signed-off-by: H. Peter Anvin

    Luca Barbieri
     

06 Feb, 2010

1 commit

  • Iomem has no special significance on x86. Use the standard mem*
    functions instead of trying to call other versions. Some fixups
    are needed to match the function prototypes.

    Signed-off-by: Brian Gerst
    LKML-Reference:
    Signed-off-by: H. Peter Anvin

    Brian Gerst
     

23 Jan, 2010

1 commit

  • Add wbinvd_on_cpu and wbinvd_on_all_cpus stubs for executing wbinvd on a
    particular CPU.

    [ hpa: renamed lib/smp.c to lib/cache-smp.c ]
    [ hpa: wbinvd_on_all_cpus() returns int, but wbinvd() returns
    void. Thus, the former cannot be a macro for the latter,
    replace with an inline function. ]

    Signed-off-by: Borislav Petkov
    LKML-Reference:
    Signed-off-by: H. Peter Anvin

    Borislav Petkov
     

14 Jan, 2010

1 commit

  • This one is much faster than the spinlock based fallback rwsem code,
    with certain artifical benchmarks having shown 300%+ improvement on
    threaded page faults etc.

    Again, note the 32767-thread limit here. So this really does need that
    whole "make rwsem_count_t be 64-bit and fix the BIAS values to match"
    extension on top of it, but that is conceptually a totally independent
    issue.

    NOT TESTED! The original patch that this all was based on were tested by
    KAMEZAWA Hiroyuki, but maybe I screwed up something when I created the
    cleaned-up series, so caveat emptor..

    Also note that it _may_ be a good idea to mark some more registers
    clobbered on x86-64 in the inline asms instead of saving/restoring them.
    They are inline functions, but they are only used in places where there
    are not a lot of live registers _anyway_, so doing for example the
    clobbers of %r8-%r11 in the asm wouldn't make the fast-path code any
    worse, and would make the slow-path code smaller.

    (Not that the slow-path really matters to that degree. Saving a few
    unnecessary registers is the _least_ of our problems when we hit the slow
    path. The instruction/cycle counting really only matters in the fast
    path).

    Signed-off-by: Linus Torvalds
    LKML-Reference:
    Signed-off-by: H. Peter Anvin

    Linus Torvalds
     

20 Dec, 2009

1 commit

  • …git/tip/linux-2.6-tip

    * 'x86-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip:
    x86, irq: Allow 0xff for /proc/irq/[n]/smp_affinity on an 8-cpu system
    Makefile: Unexport LC_ALL instead of clearing it
    x86: Fix objdump version check in arch/x86/tools/chkobjdump.awk
    x86: Reenable TSC sync check at boot, even with NONSTOP_TSC
    x86: Don't use POSIX character classes in gen-insn-attr-x86.awk
    Makefile: set LC_CTYPE, LC_COLLATE, LC_NUMERIC to C
    x86: Increase MAX_EARLY_RES; insufficient on 32-bit NUMA
    x86: Fix checking of SRAT when node 0 ram is not from 0
    x86, cpuid: Add "volatile" to asm in native_cpuid()
    x86, msr: msrs_alloc/free for CONFIG_SMP=n
    x86, amd: Get multi-node CPU info from NodeId MSR instead of PCI config space
    x86: Add IA32_TSC_AUX MSR and use it
    x86, msr/cpuid: Register enough minors for the MSR and CPUID drivers
    initramfs: add missing decompressor error check
    bzip2: Add missing checks for malloc returning NULL
    bzip2/lzma/gzip: pre-boot malloc doesn't return NULL on failure

    Linus Torvalds
     

17 Dec, 2009

1 commit

  • Randy Dunlap reported the following build error:

    "When CONFIG_SMP=n, CONFIG_X86_MSR=m:

    ERROR: "msrs_free" [drivers/edac/amd64_edac_mod.ko] undefined!
    ERROR: "msrs_alloc" [drivers/edac/amd64_edac_mod.ko] undefined!"

    This is due to the fact that is conditioned on
    CONFIG_SMP and in the UP case we have only the stubs in the header.
    Fork off SMP functionality into a new file (msr-smp.c) and build
    msrs_{alloc,free} unconditionally.

    Reported-by: Randy Dunlap
    Cc: H. Peter Anvin
    Signed-off-by: Borislav Petkov
    LKML-Reference:
    Signed-off-by: H. Peter Anvin

    Borislav Petkov
     

08 Dec, 2009

1 commit

  • Delete empty or incomplete inat-tables.c if gen-insn-attr-x86.awk
    failed, because it causes a build error if user tries to build
    kernel next time.

    Reported-by: Arkadiusz Miskiewicz
    Signed-off-by: Masami Hiramatsu
    Cc: systemtap
    Cc: DLE
    Cc: Jens Axboe
    Cc: Frederic Weisbecker
    LKML-Reference:
    Signed-off-by: Ingo Molnar

    Masami Hiramatsu
     

07 Dec, 2009

1 commit


17 Oct, 2009

1 commit


01 Oct, 2009

2 commits

  • Conditionaly compile cmpxchg8b_emu.o and EXPORT_SYMBOL(cmpxchg8b_emu).

    This reduces the kernel size a bit.

    Signed-off-by: Eric Dumazet
    Cc: Arjan van de Ven
    Cc: Martin Schwidefsky
    Cc: John Stultz
    Cc: Peter Zijlstra
    Cc: Linus Torvalds
    LKML-Reference:
    Signed-off-by: Ingo Molnar

    Eric Dumazet
     
  • cmpxchg64() today generates, to quote Linus, "barf bag" code.

    cmpxchg64() is about to get used in the scheduler to fix a bug there,
    but it's a prerequisite that cmpxchg64() first be made non-sucking.

    This patch turns cmpxchg64() into an efficient implementation that
    uses the alternative() mechanism to just use the raw instruction on
    all modern systems.

    Note: the fallback is NOT smp safe, just like the current fallback
    is not SMP safe. (Interested parties with i486 based SMP systems
    are welcome to submit fix patches for that.)

    Signed-off-by: Arjan van de Ven
    Acked-by: Linus Torvalds
    [ fixed asm constraint bug ]
    Fixed-by: Eric Dumazet
    Cc: Martin Schwidefsky
    Cc: John Stultz
    Cc: Peter Zijlstra
    LKML-Reference:
    Signed-off-by: Ingo Molnar

    Arjan van de Ven
     

24 Sep, 2009

1 commit


05 Sep, 2009

1 commit

  • Change msr-reg.o to obj-y (it will be included in virtually every
    kernel since it is used by the initialization code for AMD processors)
    and add a separate C file to export its symbols to modules, so that
    msr.ko can use them; on uniprocessors we bypass the helper functions
    in msr.o and use the accessor functions directly via inlines.

    Signed-off-by: H. Peter Anvin
    LKML-Reference:
    Cc: Borislav Petkov

    H. Peter Anvin
     

01 Sep, 2009

1 commit

  • native_{rdmsr,wrmsr}_safe_regs are two new interfaces which allow
    presetting of a subset of eight x86 GPRs before executing the rd/wrmsr
    instructions. This is needed at least on AMD K8 for accessing an erratum
    workaround MSR.

    Originally based on an idea by H. Peter Anvin.

    Signed-off-by: Borislav Petkov
    LKML-Reference:
    Signed-off-by: H. Peter Anvin

    Borislav Petkov
     

27 Aug, 2009

1 commit

  • Add x86 instruction decoder to arch-specific libraries. This decoder
    can decode x86 instructions used in kernel into prefix, opcode, modrm,
    sib, displacement and immediates. This can also show the length of
    instructions.

    This version introduces instruction attributes for decoding
    instructions.
    The instruction attribute tables are generated from the opcode map file
    (x86-opcode-map.txt) by the generator script(gen-insn-attr-x86.awk).

    Currently, the opcode maps are based on opcode maps in Intel(R) 64 and
    IA-32 Architectures Software Developers Manual Vol.2: Appendix.A,
    and consist of below two types of opcode tables.

    1-byte/2-bytes/3-bytes opcodes, which has 256 elements, are
    written as below;

    Table: table-name
    Referrer: escaped-name
    opcode: mnemonic|GrpXXX [operand1[,operand2...]] [(extra1)[,(extra2)...] [| 2nd-mnemonic ...]
    (or)
    opcode: escape # escaped-name
    EndTable

    Group opcodes, which has 8 elements, are written as below;

    GrpTable: GrpXXX
    reg: mnemonic [operand1[,operand2...]] [(extra1)[,(extra2)...] [| 2nd-mnemonic ...]
    EndTable

    These opcode maps include a few SSE and FP opcodes (for setup), because
    those opcodes are used in the kernel.

    Signed-off-by: Masami Hiramatsu
    Signed-off-by: Jim Keniston
    Acked-by: H. Peter Anvin
    Cc: Ananth N Mavinakayanahalli
    Cc: Avi Kivity
    Cc: Andi Kleen
    Cc: Christoph Hellwig
    Cc: Frank Ch. Eigler
    Cc: Ingo Molnar
    Cc: Jason Baron
    Cc: K.Prasad
    Cc: Lai Jiangshan
    Cc: Li Zefan
    Cc: Przemysław Pawełczyk
    Cc: Roland McGrath
    Cc: Sam Ravnborg
    Cc: Srikar Dronamraju
    Cc: Steven Rostedt
    Cc: Tom Zanussi
    Cc: Vegard Nossum
    LKML-Reference:
    Signed-off-by: Frederic Weisbecker

    Masami Hiramatsu
     

04 Jul, 2009

1 commit

  • atomic64_t primitives are used by a handful of drivers,
    so export the APIs consistently. These were inlined
    before.

    Also mark atomic64_32.o a core object, so that the symbols
    are available even if not linked to core kernel pieces.

    Cc: Eric Dumazet
    Cc: Linus Torvalds
    Cc: Peter Zijlstra
    Cc: Mike Galbraith
    Cc: Paul Mackerras
    Cc: Arnaldo Carvalho de Melo
    Cc: Frederic Weisbecker
    Cc: David Howells
    Cc: Andrew Morton
    Cc: Arnd Bergmann
    LKML-Reference:
    Signed-off-by: Ingo Molnar

    Ingo Molnar
     

03 Jul, 2009

1 commit

  • Linus noted that the atomic64_t primitives are all inlines
    currently which is crazy because these functions have a large
    register footprint anyway.

    Move them to a separate file: arch/x86/lib/atomic64_32.c

    Also, while at it, rename all uses of 'unsigned long long' to
    the much shorter u64.

    This makes the appearance of the prototypes a lot nicer - and
    it also uncovered a few bugs where (yet unused) API variants
    had 'long' as their return type instead of u64.

    [ More intrusive changes are not yet done in this patch. ]

    Reported-by: Linus Torvalds
    Cc: Eric Dumazet
    Cc: Peter Zijlstra
    Cc: Mike Galbraith
    Cc: Paul Mackerras
    Cc: Arnaldo Carvalho de Melo
    Cc: Frederic Weisbecker
    Cc: David Howells
    Cc: Andrew Morton
    Cc: Arnd Bergmann
    LKML-Reference:
    Signed-off-by: Ingo Molnar

    Ingo Molnar
     

10 Jun, 2009

1 commit

  • Add a struct representing a 64bit MSR pair consisting of a low and high
    register part and convert msr_info to use it. Also, rename msr-on-cpu.c
    to msr.c.

    Side note: Put the cpumask.h include in __KERNEL__ space thus fixing an
    allmodconfig build failure in the headers_check target.

    CC: H. Peter Anvin
    Signed-off-by: Borislav Petkov

    Borislav Petkov
     

04 Sep, 2008

1 commit

  • Impact: performance optimization

    I did some rebenchmarking with modern compilers and dropping
    -funroll-loops makes the function consistently go faster by a few
    percent. So drop that flag.

    Thanks to Richard Guenther for a hint.

    Signed-off-by: Andi Kleen
    Signed-off-by: H. Peter Anvin

    Andi Kleen
     

14 Jul, 2008

1 commit


09 Jul, 2008

3 commits


24 May, 2008

1 commit

  • This patch adds latency tracing for critical timings
    (how long interrupts are disabled for).

    "irqsoff" is added to /debugfs/tracing/available_tracers

    Note:
    tracing_max_latency
    also holds the max latency for irqsoff (in usecs).
    (default to large number so one must start latency tracing)

    tracing_thresh
    threshold (in usecs) to always print out if irqs off
    is detected to be longer than stated here.
    If irq_thresh is non-zero, then max_irq_latency
    is ignored.

    Here's an example of a trace with ftrace_enabled = 0

    =======
    preemption latency trace v1.1.5 on 2.6.24-rc7
    Signed-off-by: Ingo Molnar
    --------------------------------------------------------------------
    latency: 100 us, #3/3, CPU#1 | (M:rt VP:0, KP:0, SP:0 HP:0 #P:2)
    -----------------
    | task: swapper-0 (uid:0 nice:0 policy:0 rt_prio:0)
    -----------------
    => started at: _spin_lock_irqsave+0x2a/0xb7
    => ended at: _spin_unlock_irqrestore+0x32/0x5f

    _------=> CPU#
    / _-----=> irqs-off
    | / _----=> need-resched
    || / _---=> hardirq/softirq
    ||| / _--=> preempt-depth
    |||| /
    ||||| delay
    cmd pid ||||| time | caller
    \ / ||||| \ | /
    swapper-0 1d.s3 0us+: _spin_lock_irqsave+0x2a/0xb7 (e1000_update_stats+0x47/0x64c [e1000])
    swapper-0 1d.s3 100us : _spin_unlock_irqrestore+0x32/0x5f (e1000_update_stats+0x641/0x64c [e1000])
    swapper-0 1d.s3 100us : trace_hardirqs_on_caller+0x75/0x89 (_spin_unlock_irqrestore+0x32/0x5f)

    vim:ft=help
    =======

    And this is a trace with ftrace_enabled == 1

    =======
    preemption latency trace v1.1.5 on 2.6.24-rc7
    --------------------------------------------------------------------
    latency: 102 us, #12/12, CPU#1 | (M:rt VP:0, KP:0, SP:0 HP:0 #P:2)
    -----------------
    | task: swapper-0 (uid:0 nice:0 policy:0 rt_prio:0)
    -----------------
    => started at: _spin_lock_irqsave+0x2a/0xb7
    => ended at: _spin_unlock_irqrestore+0x32/0x5f

    _------=> CPU#
    / _-----=> irqs-off
    | / _----=> need-resched
    || / _---=> hardirq/softirq
    ||| / _--=> preempt-depth
    |||| /
    ||||| delay
    cmd pid ||||| time | caller
    \ / ||||| \ | /
    swapper-0 1dNs3 0us+: _spin_lock_irqsave+0x2a/0xb7 (e1000_update_stats+0x47/0x64c [e1000])
    swapper-0 1dNs3 46us : e1000_read_phy_reg+0x16/0x225 [e1000] (e1000_update_stats+0x5e2/0x64c [e1000])
    swapper-0 1dNs3 46us : e1000_swfw_sync_acquire+0x10/0x99 [e1000] (e1000_read_phy_reg+0x49/0x225 [e1000])
    swapper-0 1dNs3 46us : e1000_get_hw_eeprom_semaphore+0x12/0xa6 [e1000] (e1000_swfw_sync_acquire+0x36/0x99 [e1000])
    swapper-0 1dNs3 47us : __const_udelay+0x9/0x47 (e1000_read_phy_reg+0x116/0x225 [e1000])
    swapper-0 1dNs3 47us+: __delay+0x9/0x50 (__const_udelay+0x45/0x47)
    swapper-0 1dNs3 97us : preempt_schedule+0xc/0x84 (__delay+0x4e/0x50)
    swapper-0 1dNs3 98us : e1000_swfw_sync_release+0xc/0x55 [e1000] (e1000_read_phy_reg+0x211/0x225 [e1000])
    swapper-0 1dNs3 99us+: e1000_put_hw_eeprom_semaphore+0x9/0x35 [e1000] (e1000_swfw_sync_release+0x50/0x55 [e1000])
    swapper-0 1dNs3 101us : _spin_unlock_irqrestore+0xe/0x5f (e1000_update_stats+0x641/0x64c [e1000])
    swapper-0 1dNs3 102us : _spin_unlock_irqrestore+0x32/0x5f (e1000_update_stats+0x641/0x64c [e1000])
    swapper-0 1dNs3 102us : trace_hardirqs_on_caller+0x75/0x89 (_spin_unlock_irqrestore+0x32/0x5f)

    vim:ft=help
    =======

    Signed-off-by: Steven Rostedt
    Signed-off-by: Ingo Molnar
    Signed-off-by: Thomas Gleixner

    Steven Rostedt
     

27 Apr, 2008

2 commits

  • x86 has been switched to the generic versions of find_first_bit
    and find_first_zero_bit, but the original versions were retained.
    This patch just removes the now unused x86-specific versions.

    also update UML.

    Signed-off-by: Alexander van Heukelum
    Signed-off-by: Ingo Molnar

    Alexander van Heukelum
     
  • The versions with inline assembly are in fact slower on the machines I
    tested them on (in userspace) (Athlon XP 2800+, p4-like Xeon 2.8GHz, AMD
    Opteron 270). The i386-version needed a fix similar to 06024f21 to avoid
    crashing the benchmark.

    Benchmark using: gcc -fomit-frame-pointer -Os. For each bitmap size
    1...512, for each possible bitmap with one bit set, for each possible
    offset: find the position of the first bit starting at offset. If you
    follow ;). Times include setup of the bitmap and checking of the
    results.

    Athlon Xeon Opteron 32/64bit
    x86-specific: 0m3.692s 0m2.820s 0m3.196s / 0m2.480s
    generic: 0m2.622s 0m1.662s 0m2.100s / 0m1.572s

    If the bitmap size is not a multiple of BITS_PER_LONG, and no set
    (cleared) bit is found, find_next_bit (find_next_zero_bit) returns a
    value outside of the range [0, size]. The generic version always returns
    exactly size. The generic version also uses unsigned long everywhere,
    while the x86 versions use a mishmash of int, unsigned (int), long and
    unsigned long.

    Using the generic version does give a slightly bigger kernel, though.

    defconfig: text data bss dec hex filename
    x86-specific: 4738555 481232 626688 5846475 5935cb vmlinux (32 bit)
    generic: 4738621 481232 626688 5846541 59360d vmlinux (32 bit)
    x86-specific: 5392395 846568 724424 6963387 6a40bb vmlinux (64 bit)
    generic: 5392458 846568 724424 6963450 6a40fa vmlinux (64 bit)

    Signed-off-by: Alexander van Heukelum
    Signed-off-by: Ingo Molnar

    Alexander van Heukelum
     

06 Feb, 2008

1 commit


30 Jan, 2008

1 commit

  • Trivial unification of Makefiles for the
    x86 specific library part.
    Linking order is slightly modified but should be harmless.

    Tested doing a defconfig build before and after and saw
    no build changes.

    It adds almost as many lines as it deletes - bacause
    I broke a few lines up fo readability in the Makefile.

    Signed-off-by: Sam Ravnborg
    Cc: "H. Peter Anvin"
    Signed-off-by: Ingo Molnar
    Signed-off-by: Thomas Gleixner

    Sam Ravnborg
     

11 Oct, 2007

2 commits