mm: numa: Use a two-stage filter to restrict pages being migrated for unlikely t…

…ask<->node relationships Note: This two-stage filter was taken directly from the sched/numa patch "sched, numa, mm: Add the scanning page fault machinery" but is only a partial extraction. As the end result is not necessarily recognisable, the signed-offs-by had to be removed. Will be added back if requested. While it is desirable that all threads in a process run on its home node, this is not always possible or necessary. There may be more threads than exist within the node or the node might over-subscribed with unrelated processes. This can cause a situation whereby a page gets migrated off its home node because the threads clearing pte_numa were running off-node. This patch uses page->last_nid to build a two-stage filter before pages get migrated to avoid problems with short or unlikely task<->node relationships. Signed-off-by: Mel Gorman <mgorman@suse.de>

mm: numa: Use a two-stage filter to restrict pages being migrated for unlikely t…
…ask<->node relationships Note: This two-stage filter was taken directly from the sched/numa patch "sched, numa, mm: Add the scanning page fault machinery" but is only a partial extraction. As the end result is not necessarily recognisable, the signed-offs-by had to be removed. Will be added back if requested. While it is desirable that all threads in a process run on its home node, this is not always possible or necessary. There may be more threads than exist within the node or the node might over-subscribed with unrelated processes. This can cause a situation whereby a page gets migrated off its home node because the threads clearing pte_numa were running off-node. This patch uses page->last_nid to build a two-stage filter before pages get migrated to avoid problems with short or unlikely task<->node relationships. Signed-off-by: Mel Gorman <mgorman@suse.de>
Mel Gorman
1 parent bac0382c6a
Showing 1 changed file with 29 additions and 1 deletions Side-by-side Diff
mm/mempolicy.c
@@ -2317,8 +2317,36 @@
 	}
  
 	/* Migrate the page towards the node whose CPU is referencing it */
-	if (pol->flags & MPOL_F_MORON)
+	if (pol->flags & MPOL_F_MORON) {
+		int last_nid;
+
 		polnid = numa_node_id();
+
+		/*
+		 * Multi-stage node selection is used in conjunction
+		 * with a periodic migration fault to build a temporal
+		 * task<->page relation. By using a two-stage filter we
+		 * remove short/unlikely relations.
+		 *
+		 * Using P(p) ~ n_p / n_t as per frequentist
+		 * probability, we can equate a task's usage of a
+		 * particular page (n_p) per total usage of this
+		 * page (n_t) (in a given time-span) to a probability.
+		 *
+		 * Our periodic faults will sample this probability and
+		 * getting the same result twice in a row, given these
+		 * samples are fully independent, is then given by
+		 * P(n)^2, provided our sample period is sufficiently
+		 * short compared to the usage pattern.
+		 *
+		 * This quadric squishes small probabilities, making
+		 * it less likely we act on an unlikely task<->page
+		 * relation.
+		 */
+		last_nid = page_xchg_last_nid(page, polnid);
+		if (last_nid != polnid)
+			goto out;
+	}
  
 	if (curnid != polnid)
 		ret = polnid;
...	...	@@ -2317,8 +2317,36 @@
2317	2317	}
2318	2318
2319	2319	/* Migrate the page towards the node whose CPU is referencing it */
2320		- if (pol->flags & MPOL_F_MORON)
	2320	+ if (pol->flags & MPOL_F_MORON) {
	2321	+ int last_nid;
	2322	+
2321	2323	polnid = numa_node_id();
	2324	+
	2325	+ /*
	2326	+ * Multi-stage node selection is used in conjunction
	2327	+ * with a periodic migration fault to build a temporal
	2328	+ * task<->page relation. By using a two-stage filter we
	2329	+ * remove short/unlikely relations.
	2330	+ *
	2331	+ * Using P(p) ~ n_p / n_t as per frequentist
	2332	+ * probability, we can equate a task's usage of a
	2333	+ * particular page (n_p) per total usage of this
	2334	+ * page (n_t) (in a given time-span) to a probability.
	2335	+ *
	2336	+ * Our periodic faults will sample this probability and
	2337	+ * getting the same result twice in a row, given these
	2338	+ * samples are fully independent, is then given by
	2339	+ * P(n)^2, provided our sample period is sufficiently
	2340	+ * short compared to the usage pattern.
	2341	+ *
	2342	+ * This quadric squishes small probabilities, making
	2343	+ * it less likely we act on an unlikely task<->page
	2344	+ * relation.
	2345	+ */
	2346	+ last_nid = page_xchg_last_nid(page, polnid);
	2347	+ if (last_nid != polnid)
	2348	+ goto out;
	2349	+ }
2322	2350
2323	2351	if (curnid != polnid)
2324	2352	ret = polnid;