Commit 59d4b11371a0a23eef41138c122074aabaeaff4b

Authored by Mel Gorman
Committed by Jiri Slaby
1 parent 04bab05a95

mm: pagemap: avoid unnecessary overhead when tracepoints are deactivated

commit 24b7e5819ad5cbef2b7c7376510862aa8319d240 upstream.

This was formerly the series "Improve sequential read throughput" which
noted some major differences in performance of tiobench since 3.0.
While there are a number of factors, two that dominated were the
introduction of the fair zone allocation policy and changes to CFQ.

The behaviour of fair zone allocation policy makes more sense than
tiobench as a benchmark and CFQ defaults were not changed due to
insufficient benchmarking.

This series is what's left.  It's one functional fix to the fair zone
allocation policy when used on NUMA machines and a reduction of overhead
in general.  tiobench was used for the comparison despite its flaws as
an IO benchmark as in this case we are primarily interested in the
overhead of page allocator and page reclaim activity.

On UMA, it makes little difference to overhead

          3.16.0-rc3   3.16.0-rc3
             vanilla lowercost-v5
User          383.61      386.77
System        403.83      401.74
Elapsed      5411.50     5413.11

On a 4-socket NUMA machine it's a bit more noticable

          3.16.0-rc3   3.16.0-rc3
             vanilla lowercost-v5
User          746.94      802.00
System      65336.22    40852.33
Elapsed     27553.52    27368.46

This patch (of 6):

The LRU insertion and activate tracepoints take PFN as a parameter
forcing the overhead to the caller.  Move the overhead to the tracepoint
fast-assign method to ensure the cost is only incurred when the
tracepoint is active.

Signed-off-by: Mel Gorman <mgorman@suse.de>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Mel Gorman <mgorman@suse.de>
Signed-off-by: Jiri Slaby <jslaby@suse.cz>

Showing 2 changed files with 9 additions and 11 deletions Side-by-side Diff

include/trace/events/pagemap.h
... ... @@ -28,12 +28,10 @@
28 28  
29 29 TP_PROTO(
30 30 struct page *page,
31   - unsigned long pfn,
32   - int lru,
33   - unsigned long flags
  31 + int lru
34 32 ),
35 33  
36   - TP_ARGS(page, pfn, lru, flags),
  34 + TP_ARGS(page, lru),
37 35  
38 36 TP_STRUCT__entry(
39 37 __field(struct page *, page )
40 38  
... ... @@ -44,9 +42,9 @@
44 42  
45 43 TP_fast_assign(
46 44 __entry->page = page;
47   - __entry->pfn = pfn;
  45 + __entry->pfn = page_to_pfn(page);
48 46 __entry->lru = lru;
49   - __entry->flags = flags;
  47 + __entry->flags = trace_pagemap_flags(page);
50 48 ),
51 49  
52 50 /* Flag format is based on page-types.c formatting for pagemap */
53 51  
... ... @@ -64,9 +62,9 @@
64 62  
65 63 TRACE_EVENT(mm_lru_activate,
66 64  
67   - TP_PROTO(struct page *page, unsigned long pfn),
  65 + TP_PROTO(struct page *page),
68 66  
69   - TP_ARGS(page, pfn),
  67 + TP_ARGS(page),
70 68  
71 69 TP_STRUCT__entry(
72 70 __field(struct page *, page )
... ... @@ -75,7 +73,7 @@
75 73  
76 74 TP_fast_assign(
77 75 __entry->page = page;
78   - __entry->pfn = pfn;
  76 + __entry->pfn = page_to_pfn(page);
79 77 ),
80 78  
81 79 /* Flag format is based on page-types.c formatting for pagemap */
... ... @@ -437,7 +437,7 @@
437 437 SetPageActive(page);
438 438 lru += LRU_ACTIVE;
439 439 add_page_to_lru_list(page, lruvec, lru);
440   - trace_mm_lru_activate(page, page_to_pfn(page));
  440 + trace_mm_lru_activate(page);
441 441  
442 442 __count_vm_event(PGACTIVATE);
443 443 update_page_reclaim_stat(lruvec, file, 1);
... ... @@ -930,7 +930,7 @@
930 930 SetPageLRU(page);
931 931 add_page_to_lru_list(page, lruvec, lru);
932 932 update_page_reclaim_stat(lruvec, file, active);
933   - trace_mm_lru_insertion(page, page_to_pfn(page), lru, trace_pagemap_flags(page));
  933 + trace_mm_lru_insertion(page, lru);
934 934 }
935 935  
936 936 /*