Eric Lee / smarc-ti-linux-kernel

Download as
Browse Code »

Commit 5b40998ae35cf64561868370e6c9f3d3e94b6bf7

Authored by Vlastimil Babka
Committed by Linus Torvalds
1 parent 56afe477df
Exists in master and in 20 other branches dlt-processor-sdk-linux-03.00.00.04, newt-ti-linux-3.12.y, processor-sdk-linux-01.00.00, processor-sdk-linux-02.00.01, smarc-ti-linux-3.12.10, smarc-ti-linux-3.12.y, smarc-ti-linux-3.14.y, smarc-ti-linux-3.15.y, smarc-ti-lsk-linux-4.1.y, smarct3x-processor-sdk-04.01.00.06, smarct3x-processor-sdk-linux-02.00.01, smarct3x-processor-sdk-linux-03.00.00.04, smarct4x-800-processor-sdk-linux-02.00.01, smarct4x-processor-sdk-04.01.00.06, smarct4x-processor-sdk-linux-02.00.01, smarct4x-processor-sdk-linux-03.00.00.04, ti-linux-3.12.y, ti-linux-3.14.y, ti-linux-3.15.y, ti-lsk-linux-4.1.y

mm: munlock: remove redundant get_page/put_page pair on the fast path 

The performance of the fast path in munlock_vma_range() can be further
improved by avoiding atomic ops of a redundant get_page()/put_page() pair.

When calling get_page() during page isolation, we already have the pin
from follow_page_mask().  This pin will be then returned by
__pagevec_lru_add(), after which we do not reference the pages anymore.

After this patch, an 8% speedup was measured for munlocking a 56GB large
memory area with THP disabled.

Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
Reviewed-by: Jörn Engel <joern@logfs.org>
Acked-by: Mel Gorman <mgorman@suse.de>
Cc: Michel Lespinasse <walken@google.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@suse.cz>
Cc: Vlastimil Babka <vbabka@suse.cz>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Showing 1 changed file with 14 additions and 12 deletions Side-by-side Diff

... ... @@ -303,8 +303,10 @@
303 303 if (PageLRU(page)) {
304 304 lruvec = mem_cgroup_page_lruvec(page, zone);
305 305 lru = page_lru(page);
306   -
307   - get_page(page);
  306 + /*
  307 + * We already have pin from follow_page_mask()
  308 + * so we can spare the get_page() here.
  309 + */
308 310 ClearPageLRU(page);
309 311 del_page_from_lru_list(page, lruvec, lru);
310 312 } else {
311 313  
312 314  
313 315  
... ... @@ -336,24 +338,24 @@
336 338 lock_page(page);
337 339 if (!__putback_lru_fast_prepare(page, &pvec_putback,
338 340 &pgrescued)) {
339   - /* Slow path */
  341 + /*
  342 + * Slow path. We don't want to lose the last
  343 + * pin before unlock_page()
  344 + */
  345 + get_page(page); /* for putback_lru_page() */
340 346 __munlock_isolated_page(page);
341 347 unlock_page(page);
  348 + put_page(page); /* from follow_page_mask() */
342 349 }
343 350 }
344 351 }
345 352  
346   - /* Phase 3: page putback for pages that qualified for the fast path */
  353 + /*
  354 + * Phase 3: page putback for pages that qualified for the fast path
  355 + * This will also call put_page() to return pin from follow_page_mask()
  356 + */
347 357 if (pagevec_count(&pvec_putback))
348 358 __putback_lru_fast(&pvec_putback, pgrescued);
349   -
350   - /* Phase 4: put_page to return pin from follow_page_mask() */
351   - for (i = 0; i < nr; i++) {
352   - struct page *page = pvec->pages[i];
353   -
354   - if (page)
355   - put_page(page);
356   - }
357 359  
358 360 pagevec_reinit(pvec);
359 361 }