Commit a1fde08c74e90accd62d4cfdbf580d2ede938fe7

Authored by Linus Torvalds
1 parent 5895198c56

VM: skip the stack guard page lookup in get_user_pages only for mlock

The logic in __get_user_pages() used to skip the stack guard page lookup
whenever the caller wasn't interested in seeing what the actual page
was.  But Michel Lespinasse points out that there are cases where we
don't care about the physical page itself (so 'pages' may be NULL), but
do want to make sure a page is mapped into the virtual address space.

So using the existence of the "pages" array as an indication of whether
to look up the guard page or not isn't actually so great, and we really
should just use the FOLL_MLOCK bit.  But because that bit was only set
for the VM_LOCKED case (and not all vma's necessarily have it, even for
mlock()), we couldn't do that originally.

Fix that by moving the VM_LOCKED check deeper into the call-chain, which
actually simplifies many things.  Now mlock() gets simpler, and we can
also check for FOLL_MLOCK in __get_user_pages() and the code ends up
much more straightforward.

Reported-and-reviewed-by: Michel Lespinasse <walken@google.com>
Cc: stable@kernel.org
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Showing 2 changed files with 4 additions and 8 deletions Side-by-side Diff

... ... @@ -1359,7 +1359,7 @@
1359 1359 */
1360 1360 mark_page_accessed(page);
1361 1361 }
1362   - if (flags & FOLL_MLOCK) {
  1362 + if ((flags & FOLL_MLOCK) && (vma->vm_flags & VM_LOCKED)) {
1363 1363 /*
1364 1364 * The preliminary mapping check is mainly to avoid the
1365 1365 * pointless overhead of lock_page on the ZERO_PAGE
1366 1366  
... ... @@ -1552,10 +1552,9 @@
1552 1552 }
1553 1553  
1554 1554 /*
1555   - * If we don't actually want the page itself,
1556   - * and it's the stack guard page, just skip it.
  1555 + * For mlock, just skip the stack guard page.
1557 1556 */
1558   - if (!pages && stack_guard_page(vma, start))
  1557 + if ((gup_flags & FOLL_MLOCK) && stack_guard_page(vma, start))
1559 1558 goto next_page;
1560 1559  
1561 1560 do {
... ... @@ -162,7 +162,7 @@
162 162 VM_BUG_ON(end > vma->vm_end);
163 163 VM_BUG_ON(!rwsem_is_locked(&mm->mmap_sem));
164 164  
165   - gup_flags = FOLL_TOUCH;
  165 + gup_flags = FOLL_TOUCH | FOLL_MLOCK;
166 166 /*
167 167 * We want to touch writable mappings with a write fault in order
168 168 * to break COW, except for shared mappings because these don't COW
... ... @@ -177,9 +177,6 @@
177 177 */
178 178 if (vma->vm_flags & (VM_READ | VM_WRITE | VM_EXEC))
179 179 gup_flags |= FOLL_FORCE;
180   -
181   - if (vma->vm_flags & VM_LOCKED)
182   - gup_flags |= FOLL_MLOCK;
183 180  
184 181 return __get_user_pages(current, mm, addr, nr_pages, gup_flags,
185 182 NULL, NULL, nonblocking);