Eric Lee / smarc-ti-linux-kernel

Download as
Browse Code ยป

Commit ff9042b11a71c81238c70af168cd36b98a6d5a3c

Authored by Mel Gorman
Committed by Ingo Molnar
1 parent 0c3a775e1e
Exists in master and in 16 other branches dlt-processor-sdk-linux-03.00.00.04, processor-sdk-linux-01.00.00, processor-sdk-linux-02.00.01, smarc-ti-linux-3.14.y, smarc-ti-linux-3.15.y, smarc-ti-lsk-linux-4.1.y, smarct3x-processor-sdk-04.01.00.06, smarct3x-processor-sdk-linux-02.00.01, smarct3x-processor-sdk-linux-03.00.00.04, smarct4x-800-processor-sdk-linux-02.00.01, smarct4x-processor-sdk-04.01.00.06, smarct4x-processor-sdk-linux-02.00.01, smarct4x-processor-sdk-linux-03.00.00.04, ti-linux-3.14.y, ti-linux-3.15.y, ti-lsk-linux-4.1.y

mm: Wait for THP migrations to complete during NUMA hinting faults 

The locking for migrating THP is unusual. While normal page migration
prevents parallel accesses using a migration PTE, THP migration relies on
a combination of the page_table_lock, the page lock and the existance of
the NUMA hinting PTE to guarantee safety but there is a bug in the scheme.

If a THP page is currently being migrated and another thread traps a
fault on the same page it checks if the page is misplaced. If it is not,
then pmd_numa is cleared. The problem is that it checks if the page is
misplaced without holding the page lock meaning that the racing thread
can be migrating the THP when the second thread clears the NUMA bit
and faults a stale page.

This patch checks if the page is potentially being migrated and stalls
using the lock_page if it is potentially being migrated before checking
if the page is misplaced or not.

Signed-off-by: Mel Gorman <mgorman@suse.de>
Reviewed-by: Rik van Riel <riel@redhat.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1381141781-10992-6-git-send-email-mgorman@suse.de
Signed-off-by: Ingo Molnar <mingo@kernel.org>

Showing 1 changed file with 16 additions and 7 deletions Side-by-side Diff

... ... @@ -1295,13 +1295,14 @@
1295 1295 if (current_nid == numa_node_id())
1296 1296 count_vm_numa_event(NUMA_HINT_FAULTS_LOCAL);
1297 1297  
1298   - target_nid = mpol_misplaced(page, vma, haddr);
1299   - if (target_nid == -1) {
1300   - put_page(page);
1301   - goto clear_pmdnuma;
1302   - }
  1298 + /*
  1299 + * Acquire the page lock to serialise THP migrations but avoid dropping
  1300 + * page_table_lock if at all possible
  1301 + */
  1302 + if (trylock_page(page))
  1303 + goto got_lock;
1303 1304  
1304   - /* Acquire the page lock to serialise THP migrations */
  1305 + /* Serialise against migrationa and check placement check placement */
1305 1306 spin_unlock(&mm->page_table_lock);
1306 1307 lock_page(page);
1307 1308  
1308 1309  
1309 1310  
... ... @@ -1312,9 +1313,17 @@
1312 1313 put_page(page);
1313 1314 goto out_unlock;
1314 1315 }
1315   - spin_unlock(&mm->page_table_lock);
1316 1316  
  1317 +got_lock:
  1318 + target_nid = mpol_misplaced(page, vma, haddr);
  1319 + if (target_nid == -1) {
  1320 + unlock_page(page);
  1321 + put_page(page);
  1322 + goto clear_pmdnuma;
  1323 + }
  1324 +
1317 1325 /* Migrate the THP to the requested node */
  1326 + spin_unlock(&mm->page_table_lock);
1318 1327 migrated = migrate_misplaced_transhuge_page(mm, vma,
1319 1328 pmdp, pmd, addr, page, target_nid);
1320 1329 if (!migrated)