Eric Lee / smarc-ti-linux-kernel

Download as
Browse Code ยป

Commit 35515de99f95c55f6c416fc7cc2b16832b9f58ee

Authored by Mel Gorman
Committed by Jiri Slaby
1 parent d6217476e1
Exists in ti-linux-3.12.y and in 2 other branches newt-ti-linux-3.12.y, smarc-ti-linux-3.12.y

mm: page_alloc: take the ALLOC_NO_WATERMARK check out of the fast path 

commit 5dab29113ca56335c78be3f98bf5ddf2ef8eb6a6 upstream.

ALLOC_NO_WATERMARK is set in a few cases.  Always by kswapd, always for
__GFP_MEMALLOC, sometimes for swap-over-nfs, tasks etc.  Each of these
cases are relatively rare events but the ALLOC_NO_WATERMARK check is an
unlikely branch in the fast path.  This patch moves the check out of the
fast path and after it has been determined that the watermarks have not
been met.  This helps the common fast path at the cost of making the slow
path slower and hitting kswapd with a performance cost.  It's a reasonable
tradeoff.

Signed-off-by: Mel Gorman <mgorman@suse.de>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Reviewed-by: Rik van Riel <riel@redhat.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Jan Kara <jack@suse.cz>
Cc: Michal Hocko <mhocko@suse.cz>
Cc: Hugh Dickins <hughd@google.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Theodore Ts'o <tytso@mit.edu>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Cc: Oleg Nesterov <oleg@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Mel Gorman <mgorman@suse.de>
Signed-off-by: Jiri Slaby <jslaby@suse.cz>

Showing 1 changed file with 5 additions and 3 deletions Side-by-side Diff

... ... @@ -1934,9 +1934,6 @@
1934 1934 (alloc_flags & ALLOC_CPUSET) &&
1935 1935 !cpuset_zone_allowed_softwall(zone, gfp_mask))
1936 1936 continue;
1937   - BUILD_BUG_ON(ALLOC_NO_WATERMARKS < NR_WMARK);
1938   - if (unlikely(alloc_flags & ALLOC_NO_WATERMARKS))
1939   - goto try_this_zone;
1940 1937 /*
1941 1938 * Distribute pages in proportion to the individual
1942 1939 * zone size to ensure fair page aging. The zone a
... ... @@ -1982,6 +1979,11 @@
1982 1979 if (!zone_watermark_ok(zone, order, mark,
1983 1980 classzone_idx, alloc_flags)) {
1984 1981 int ret;
  1982 +
  1983 + /* Checked here to keep the fast path fast */
  1984 + BUILD_BUG_ON(ALLOC_NO_WATERMARKS < NR_WMARK);
  1985 + if (alloc_flags & ALLOC_NO_WATERMARKS)
  1986 + goto try_this_zone;
1985 1987  
1986 1988 if (IS_ENABLED(CONFIG_NUMA) &&
1987 1989 !did_zlc_setup && nr_online_nodes > 1) {