Commit 25e4df5bae333a06cd2c9b88baf14432652dc9f7

Authored by Zachary Amsden
Committed by Linus Torvalds
1 parent 6606c3e0da

[PATCH] paravirt: combine flush accessed dirty.patch

Remove ptep_test_and_clear_{dirty|young} from i386, and instead use the
dominating functions, ptep_clear_flush_{dirty|young}.  This allows the TLB
page flush to be contained in the same macro, and allows for an eager
optimization - if reading the PTE initially returned dirty/accessed, we can
assume the fact that no subsequent update to the PTE which cleared accessed /
dirty has occurred, as the only way A/D bits can change without holding the
page table lock is if a remote processor clears them.  This eliminates an
extra branch which came from the generic version of the code, as we know that
no other CPU could have cleared the A/D bit, so the flush will always be
needed.

We still export these two defines, even though we do not actually define
the macros in the i386 code:

 #define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG
 #define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_DIRTY

The reason for this is that the only use of these functions is within the
generic clear_flush functions, and we want a strong guarantee that there
are no other users of these functions, so we want to prevent the generic
code from defining them for us.

Signed-off-by: Zachary Amsden <zach@vmware.com>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Jeremy Fitzhardinge <jeremy@xensource.com>
Cc: Andi Kleen <ak@suse.de>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>

Showing 1 changed file with 28 additions and 13 deletions Side-by-side Diff

include/asm-i386/pgtable.h
... ... @@ -262,21 +262,36 @@
262 262 } \
263 263 } while (0)
264 264  
  265 +/*
  266 + * We don't actually have these, but we want to advertise them so that
  267 + * we can encompass the flush here.
  268 + */
265 269 #define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_DIRTY
266   -static inline int ptep_test_and_clear_dirty(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep)
267   -{
268   - if (!pte_dirty(*ptep))
269   - return 0;
270   - return test_and_clear_bit(_PAGE_BIT_DIRTY, &ptep->pte_low);
271   -}
272   -
273 270 #define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG
274   -static inline int ptep_test_and_clear_young(struct vm_area_struct *vma, unsigned long addr, pte_t *ptep)
275   -{
276   - if (!pte_young(*ptep))
277   - return 0;
278   - return test_and_clear_bit(_PAGE_BIT_ACCESSED, &ptep->pte_low);
279   -}
  271 +
  272 +#define __HAVE_ARCH_PTEP_CLEAR_DIRTY_FLUSH
  273 +#define ptep_clear_flush_dirty(vma, address, ptep) \
  274 +({ \
  275 + int __dirty; \
  276 + __dirty = pte_dirty(*(ptep)); \
  277 + if (__dirty) { \
  278 + clear_bit(_PAGE_BIT_DIRTY, &(ptep)->pte_low); \
  279 + flush_tlb_page(vma, address); \
  280 + } \
  281 + __dirty; \
  282 +})
  283 +
  284 +#define __HAVE_ARCH_PTEP_CLEAR_YOUNG_FLUSH
  285 +#define ptep_clear_flush_young(vma, address, ptep) \
  286 +({ \
  287 + int __young; \
  288 + __young = pte_young(*(ptep)); \
  289 + if (__young) { \
  290 + clear_bit(_PAGE_BIT_ACCESSED, &(ptep)->pte_low); \
  291 + flush_tlb_page(vma, address); \
  292 + } \
  293 + __young; \
  294 +})
280 295  
281 296 #define __HAVE_ARCH_PTEP_GET_AND_CLEAR_FULL
282 297 static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm, unsigned long addr, pte_t *ptep, int full)