Commit c9b02a24130e3ff14a553d966a79f46cf806b037

Authored by Zachary Amsden
Committed by Linus Torvalds
1 parent e7a2ff593c

[PATCH] i386: use set_pte macros in a couple places where they were missing

Also, setting PDPEs in PAE mode does not require atomic operations, since the
PDPEs are cached by the processor, and only reloaded on an explicit or
implicit reload of CR3.

Since the four PDPEs must always be present in an active root, and the kernel
PDPE is never updated, we are safe even from SMIs and interrupts / NMIs using
task gates (which reload CR3).  Actually, much of this is moot, since the user
PDPEs are never updated either, and the only usage of task gates is by the
doublefault handler.  It appears the only place PGDs get updated in PAE mode
is in init_low_mappings() / zap_low_mapping() for initial page table creation
and recovery from ACPI sleep state, and these sites are safe by inspection.
Getting rid of the cmpxchg8b saves code space and 720 cycles in pgd_alloc on
P4.

Signed-off-by: Zachary Amsden <zach@vmware.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>

Showing 3 changed files with 5 additions and 4 deletions Side-by-side Diff

... ... @@ -349,7 +349,7 @@
349 349 * All user-space mappings are explicitly cleared after
350 350 * SMP startup.
351 351 */
352   - pgd_base[0] = pgd_base[USER_PTRS_PER_PGD];
  352 + set_pgd(&pgd_base[0], pgd_base[USER_PTRS_PER_PGD]);
353 353 #endif
354 354 }
355 355  
arch/i386/mm/pageattr.c
... ... @@ -12,6 +12,7 @@
12 12 #include <asm/uaccess.h>
13 13 #include <asm/processor.h>
14 14 #include <asm/tlbflush.h>
  15 +#include <asm/pgalloc.h>
15 16  
16 17 static DEFINE_SPINLOCK(cpa_lock);
17 18 static struct list_head df_list = LIST_HEAD_INIT(df_list);
... ... @@ -52,8 +53,8 @@
52 53 addr = address & LARGE_PAGE_MASK;
53 54 pbase = (pte_t *)page_address(base);
54 55 for (i = 0; i < PTRS_PER_PTE; i++, addr += PAGE_SIZE) {
55   - pbase[i] = pfn_pte(addr >> PAGE_SHIFT,
56   - addr == address ? prot : PAGE_KERNEL);
  56 + set_pte(&pbase[i], pfn_pte(addr >> PAGE_SHIFT,
  57 + addr == address ? prot : PAGE_KERNEL));
57 58 }
58 59 return base;
59 60 }
include/asm-i386/pgtable-3level.h
... ... @@ -64,7 +64,7 @@
64 64 #define set_pmd(pmdptr,pmdval) \
65 65 set_64bit((unsigned long long *)(pmdptr),pmd_val(pmdval))
66 66 #define set_pud(pudptr,pudval) \
67   - set_64bit((unsigned long long *)(pudptr),pud_val(pudval))
  67 + (*(pudptr) = (pudval))
68 68  
69 69 /*
70 70 * Pentium-II erratum A13: in PAE mode we explicitly have to flush