Eric Lee / linux-smarc-t335x-v3.2

Commit c40f6f8bbc4cbd2902671aacd587400ddca62627

Authored by Linus Torvalds 2009-01-10 06:00:58 +0800

2 parents 1a7d0f0bec cb6ff20807

Exists in master and in 4 other branches

Merge git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-2.6-nommu

* git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-2.6-nommu:
  NOMMU: Support XIP on initramfs
  NOMMU: Teach kobjsize() about VMA regions.
  FLAT: Don't attempt to expand the userspace stack to fill the space allocated
  FDPIC: Don't attempt to expand the userspace stack to fill the space allocated
  NOMMU: Improve procfs output using per-MM VMAs
  NOMMU: Make mmap allocation page trimming behaviour configurable.
  NOMMU: Make VMAs per MM as for MMU-mode linux
  NOMMU: Delete askedalloc and realalloc variables
  NOMMU: Rename ARM's struct vm_region
  NOMMU: Fix cleanup handling in ramfs_nommu_get_umapped_area()

Showing 29 changed files Inline Diff

Documentation/nommu-mmap.txt

Diff comments View file @ c40f6f8

 			 =============================
 			 NO-MMU MEMORY MAPPING SUPPORT
 			 =============================
 The kernel has limited support for memory mapping under no-MMU conditions, such
 as are used in uClinux environments. From the userspace point of view, memory
 mapping is made use of in conjunction with the mmap() system call, the shmat()
 call and the execve() system call. From the kernel's point of view, execve()
 mapping is actually performed by the binfmt drivers, which call back into the
 mmap() routines to do the actual work.
 Memory mapping behaviour also involves the way fork(), vfork(), clone() and
 ptrace() work. Under uClinux there is no fork(), and clone() must be supplied
 the CLONE_VM flag.
 The behaviour is similar between the MMU and no-MMU cases, but not identical;
 and it's also much more restricted in the latter case:
  (*) Anonymous mapping, MAP_PRIVATE
 	In the MMU case: VM regions backed by arbitrary pages; copy-on-write
 	across fork.
 	In the no-MMU case: VM regions backed by arbitrary contiguous runs of
 	pages.
  (*) Anonymous mapping, MAP_SHARED
 	These behave very much like private mappings, except that they're
 	shared across fork() or clone() without CLONE_VM in the MMU case. Since
 	the no-MMU case doesn't support these, behaviour is identical to
 	MAP_PRIVATE there.
  (*) File, MAP_PRIVATE, PROT_READ / PROT_EXEC, !PROT_WRITE
 	In the MMU case: VM regions backed by pages read from file; changes to
 	the underlying file are reflected in the mapping; copied across fork.
 	In the no-MMU case:
          - If one exists, the kernel will re-use an existing mapping to the
            same segment of the same file if that has compatible permissions,
            even if this was created by another process.
          - If possible, the file mapping will be directly on the backing device
            if the backing device has the BDI_CAP_MAP_DIRECT capability and
            appropriate mapping protection capabilities. Ramfs, romfs, cramfs
            and mtd might all permit this.
 	 - If the backing device device can't or won't permit direct sharing,
            but does have the BDI_CAP_MAP_COPY capability, then a copy of the
            appropriate bit of the file will be read into a contiguous bit of
            memory and any extraneous space beyond the EOF will be cleared
 	 - Writes to the file do not affect the mapping; writes to the mapping
 	   are visible in other processes (no MMU protection), but should not
 	   happen.
  (*) File, MAP_PRIVATE, PROT_READ / PROT_EXEC, PROT_WRITE
 	In the MMU case: like the non-PROT_WRITE case, except that the pages in
 	question get copied before the write actually happens. From that point
 	on writes to the file underneath that page no longer get reflected into
 	the mapping's backing pages. The page is then backed by swap instead.
 	In the no-MMU case: works much like the non-PROT_WRITE case, except
 	that a copy is always taken and never shared.
  (*) Regular file / blockdev, MAP_SHARED, PROT_READ / PROT_EXEC / PROT_WRITE
 	In the MMU case: VM regions backed by pages read from file; changes to
 	pages written back to file; writes to file reflected into pages backing
 	mapping; shared across fork.
 	In the no-MMU case: not supported.
  (*) Memory backed regular file, MAP_SHARED, PROT_READ / PROT_EXEC / PROT_WRITE
 	In the MMU case: As for ordinary regular files.
 	In the no-MMU case: The filesystem providing the memory-backed file
 	(such as ramfs or tmpfs) may choose to honour an open, truncate, mmap
 	sequence by providing a contiguous sequence of pages to map. In that
 	case, a shared-writable memory mapping will be possible. It will work
 	as for the MMU case. If the filesystem does not provide any such
 	support, then the mapping request will be denied.
  (*) Memory backed blockdev, MAP_SHARED, PROT_READ / PROT_EXEC / PROT_WRITE
 	In the MMU case: As for ordinary regular files.
 	In the no-MMU case: As for memory backed regular files, but the
 	blockdev must be able to provide a contiguous run of pages without
 	truncate being called. The ramdisk driver could do this if it allocated
 	all its memory as a contiguous array upfront.
  (*) Memory backed chardev, MAP_SHARED, PROT_READ / PROT_EXEC / PROT_WRITE
 	In the MMU case: As for ordinary regular files.
 	In the no-MMU case: The character device driver may choose to honour
 	the mmap() by providing direct access to the underlying device if it
 	provides memory or quasi-memory that can be accessed directly. Examples
 	of such are frame buffers and flash devices. If the driver does not
 	provide any such support, then the mapping request will be denied.
 ============================
 FURTHER NOTES ON NO-MMU MMAP
 ============================
- (*) A request for a private mapping of less than a page in size may not return
+ (*) A request for a private mapping of a file may return a buffer that is not
-     a page-aligned buffer. This is because the kernel calls kmalloc() to
+     page-aligned.  This is because XIP may take place, and the data may not be
-     allocate the buffer, not get_free_page().
+     paged aligned in the backing store.
- (*) A list of all the mappings on the system is visible through /proc/maps in
+ (*) A request for an anonymous mapping will always be page aligned.  If
-     no-MMU mode.
+     possible the size of the request should be a power of two otherwise some
+     of the space may be wasted as the kernel must allocate a power-of-2
+     granule but will only discard the excess if appropriately configured as
+     this has an effect on fragmentation.
+ (*) A list of all the private copy and anonymous mappings on the system is
+     visible through /proc/maps in no-MMU mode.
  (*) A list of all the mappings in use by a process is visible through
      /proc/<pid>/maps in no-MMU mode.
  (*) Supplying MAP_FIXED or a requesting a particular mapping address will
      result in an error.
  (*) Files mapped privately usually have to have a read method provided by the
      driver or filesystem so that the contents can be read into the memory
      allocated if mmap() chooses not to map the backing device directly. An
      error will result if they don't. This is most likely to be encountered
      with character device files, pipes, fifos and sockets.
 ==========================
 INTERPROCESS SHARED MEMORY
 ==========================
 Both SYSV IPC SHM shared memory and POSIX shared memory is supported in NOMMU
 mode.  The former through the usual mechanism, the latter through files created
 on ramfs or tmpfs mounts.
 =======
 FUTEXES
 =======
 Futexes are supported in NOMMU mode if the arch supports them.  An error will
 be given if an address passed to the futex system call lies outside the
 mappings made by a process or if the mapping in which the address lies does not
 support futexes (such as an I/O chardev mapping).
 =============
 NO-MMU MREMAP
 =============
 The mremap() function is partially supported.  It may change the size of a
 mapping, and may move it[*] if MREMAP_MAYMOVE is specified and if the new size
 of the mapping exceeds the size of the slab object currently occupied by the
 memory to which the mapping refers, or if a smaller slab object could be used.
 MREMAP_FIXED is not supported, though it is ignored if there's no change of
 address and the object does not need to be moved.
 Shared mappings may not be moved.  Shareable mappings may not be moved either,
 even if they are not currently shared.
 The mremap() function must be given an exact match for base address and size of
 a previously mapped object.  It may not be used to create holes in existing
 mappings, move parts of existing mappings or resize parts of mappings.  It must
 act on a complete mapping.
 [*] Not currently supported.
 ============================================
 PROVIDING SHAREABLE CHARACTER DEVICE SUPPORT
 ============================================
 To provide shareable character device support, a driver must provide a
 file->f_op->get_unmapped_area() operation. The mmap() routines will call this
 to get a proposed address for the mapping. This may return an error if it
 doesn't wish to honour the mapping because it's too long, at a weird offset,
 under some unsupported combination of flags or whatever.
 The driver should also provide backing device information with capabilities set
 to indicate the permitted types of mapping on such devices. The default is
 assumed to be readable and writable, not executable, and only shareable
 directly (can't be copied).
 The file->f_op->mmap() operation will be called to actually inaugurate the
 mapping. It can be rejected at that point. Returning the ENOSYS error will
 cause the mapping to be copied instead if BDI_CAP_MAP_COPY is specified.
 The vm_ops->close() routine will be invoked when the last mapping on a chardev
 is removed. An existing mapping will be shared, partially or not, if possible
 without notifying the driver.
 It is permitted also for the file->f_op->get_unmapped_area() operation to
 return -ENOSYS. This will be taken to mean that this operation just doesn't
 want to handle it, despite the fact it's got an operation. For instance, it
 might try directing the call to a secondary driver which turns out not to
 implement it. Such is the case for the framebuffer driver which attempts to
 direct the call to the device-specific driver. Under such circumstances, the
 mapping request will be rejected if BDI_CAP_MAP_COPY is not specified, and a
 copy mapped otherwise.
 IMPORTANT NOTE:
 	Some types of device may present a different appearance to anyone
 	looking at them in certain modes. Flash chips can be like this; for
 	instance if they're in programming or erase mode, you might see the
 	status reflected in the mapping, instead of the data.
 	In such a case, care must be taken lest userspace see a shared or a
 	private mapping showing such information when the driver is busy
 	controlling the device. Remember especially: private executable
 	mappings may still be mapped directly off the device under some
 	circumstances!
 ==============================================
 PROVIDING SHAREABLE MEMORY-BACKED FILE SUPPORT
 ==============================================
 Provision of shared mappings on memory backed files is similar to the provision
 of support for shared mapped character devices. The main difference is that the
 filesystem providing the service will probably allocate a contiguous collection
 of pages and permit mappings to be made on that.
 It is recommended that a truncate operation applied to such a file that
 increases the file size, if that file is empty, be taken as a request to gather
 enough pages to honour a mapping. This is required to support POSIX shared
 memory.
 Memory backed devices are indicated by the mapping's backing device info having
 the memory_backed flag set.
 ========================================
 PROVIDING SHAREABLE BLOCK DEVICE SUPPORT
 ========================================
 Provision of shared mappings on block device files is exactly the same as for
 character devices. If there isn't a real device underneath, then the driver
 should allocate sufficient contiguous memory to honour any supported mapping.
+=================================
+ADJUSTING PAGE TRIMMING BEHAVIOUR
+=================================
+NOMMU mmap automatically rounds up to the nearest power-of-2 number of pages
+when performing an allocation.  This can have adverse effects on memory
+fragmentation, and as such, is left configurable.  The default behaviour is to
+aggressively trim allocations and discard any excess pages back in to the page
+allocator.  In order to retain finer-grained control over fragmentation, this
+behaviour can either be disabled completely, or bumped up to a higher page
+watermark where trimming begins.
+Page trimming behaviour is configurable via the sysctl `vm.nr_trim_pages'.

Documentation/sysctl/vm.txt

Diff comments View file @ c40f6f8

 Documentation for /proc/sys/vm/*	kernel version 2.2.10
 	(c) 1998, 1999,  Rik van Riel <riel@nl.linux.org>
 For general info and legal blurb, please look in README.
 ==============================================================
 This file contains the documentation for the sysctl files in
 /proc/sys/vm and is valid for Linux kernel version 2.2.
 The files in this directory can be used to tune the operation
 of the virtual memory (VM) subsystem of the Linux kernel and
 the writeout of dirty data to disk.
 Default values and initialization routines for most of these
 files can be found in mm/swap.c.
 Currently, these files are in /proc/sys/vm:
 - overcommit_memory
 - page-cluster
 - dirty_ratio
 - dirty_background_ratio
 - dirty_expire_centisecs
 - dirty_writeback_centisecs
 - highmem_is_dirtyable   (only if CONFIG_HIGHMEM set)
 - max_map_count
 - min_free_kbytes
 - laptop_mode
 - block_dump
 - drop-caches
 - zone_reclaim_mode
 - min_unmapped_ratio
 - min_slab_ratio
 - panic_on_oom
 - oom_dump_tasks
 - oom_kill_allocating_task
 - mmap_min_address
 - numa_zonelist_order
 - nr_hugepages
 - nr_overcommit_hugepages
+- nr_trim_pages		(only if CONFIG_MMU=n)
 ==============================================================
 dirty_bytes, dirty_ratio, dirty_background_bytes,
 dirty_background_ratio, dirty_expire_centisecs,
 dirty_writeback_centisecs, highmem_is_dirtyable,
 vfs_cache_pressure, laptop_mode, block_dump, swap_token_timeout,
 drop-caches, hugepages_treat_as_movable:
 See Documentation/filesystems/proc.txt
 ==============================================================
 overcommit_memory:
 This value contains a flag that enables memory overcommitment.
 When this flag is 0, the kernel attempts to estimate the amount
 of free memory left when userspace requests more memory.
 When this flag is 1, the kernel pretends there is always enough
 memory until it actually runs out.
 When this flag is 2, the kernel uses a "never overcommit"
 policy that attempts to prevent any overcommit of memory.
 This feature can be very useful because there are a lot of
 programs that malloc() huge amounts of memory "just-in-case"
 and don't use much of it.
 The default value is 0.
 See Documentation/vm/overcommit-accounting and
 security/commoncap.c::cap_vm_enough_memory() for more information.
 ==============================================================
 overcommit_ratio:
 When overcommit_memory is set to 2, the committed address
 space is not permitted to exceed swap plus this percentage
 of physical RAM.  See above.
 ==============================================================
 page-cluster:
 The Linux VM subsystem avoids excessive disk seeks by reading
 multiple pages on a page fault. The number of pages it reads
 is dependent on the amount of memory in your machine.
 The number of pages the kernel reads in at once is equal to
 2 ^ page-cluster. Values above 2 ^ 5 don't make much sense
 for swap because we only cluster swap data in 32-page groups.
 ==============================================================
 max_map_count:
 This file contains the maximum number of memory map areas a process
 may have. Memory map areas are used as a side-effect of calling
 malloc, directly by mmap and mprotect, and also when loading shared
 libraries.
 While most applications need less than a thousand maps, certain
 programs, particularly malloc debuggers, may consume lots of them,
 e.g., up to one or two maps per allocation.
 The default value is 65536.
 ==============================================================
 min_free_kbytes:
 This is used to force the Linux VM to keep a minimum number
 of kilobytes free.  The VM uses this number to compute a pages_min
 value for each lowmem zone in the system.  Each lowmem zone gets
 a number of reserved free pages based proportionally on its size.
 Some minimal amount of memory is needed to satisfy PF_MEMALLOC
 allocations; if you set this to lower than 1024KB, your system will
 become subtly broken, and prone to deadlock under high loads.
 Setting this too high will OOM your machine instantly.
 ==============================================================
 percpu_pagelist_fraction
 This is the fraction of pages at most (high mark pcp->high) in each zone that
 are allocated for each per cpu page list.  The min value for this is 8.  It
 means that we don't allow more than 1/8th of pages in each zone to be
 allocated in any single per_cpu_pagelist.  This entry only changes the value
 of hot per cpu pagelists.  User can specify a number like 100 to allocate
 1/100th of each zone to each per cpu page list.
 The batch value of each per cpu pagelist is also updated as a result.  It is
 set to pcp->high/4.  The upper limit of batch is (PAGE_SHIFT * 8)
 The initial value is zero.  Kernel does not use this value at boot time to set
 the high water marks for each per cpu page list.
 ===============================================================
 zone_reclaim_mode:
 Zone_reclaim_mode allows someone to set more or less aggressive approaches to
 reclaim memory when a zone runs out of memory. If it is set to zero then no
 zone reclaim occurs. Allocations will be satisfied from other zones / nodes
 in the system.
 This is value ORed together of
 1	= Zone reclaim on
 2	= Zone reclaim writes dirty pages out
 4	= Zone reclaim swaps pages
 zone_reclaim_mode is set during bootup to 1 if it is determined that pages
 from remote zones will cause a measurable performance reduction. The
 page allocator will then reclaim easily reusable pages (those page
 cache pages that are currently not used) before allocating off node pages.
 It may be beneficial to switch off zone reclaim if the system is
 used for a file server and all of memory should be used for caching files
 from disk. In that case the caching effect is more important than
 data locality.
 Allowing zone reclaim to write out pages stops processes that are
 writing large amounts of data from dirtying pages on other nodes. Zone
 reclaim will write out dirty pages if a zone fills up and so effectively
 throttle the process. This may decrease the performance of a single process
 since it cannot use all of system memory to buffer the outgoing writes
 anymore but it preserve the memory on other nodes so that the performance
 of other processes running on other nodes will not be affected.
 Allowing regular swap effectively restricts allocations to the local
 node unless explicitly overridden by memory policies or cpuset
 configurations.
 =============================================================
 min_unmapped_ratio:
 This is available only on NUMA kernels.
 A percentage of the total pages in each zone.  Zone reclaim will only
 occur if more than this percentage of pages are file backed and unmapped.
 This is to insure that a minimal amount of local pages is still available for
 file I/O even if the node is overallocated.
 The default is 1 percent.
 =============================================================
 min_slab_ratio:
 This is available only on NUMA kernels.
 A percentage of the total pages in each zone.  On Zone reclaim
 (fallback from the local zone occurs) slabs will be reclaimed if more
 than this percentage of pages in a zone are reclaimable slab pages.
 This insures that the slab growth stays under control even in NUMA
 systems that rarely perform global reclaim.
 The default is 5 percent.
 Note that slab reclaim is triggered in a per zone / node fashion.
 The process of reclaiming slab memory is currently not node specific
 and may not be fast.
 =============================================================
 panic_on_oom
 This enables or disables panic on out-of-memory feature.
 If this is set to 0, the kernel will kill some rogue process,
 called oom_killer.  Usually, oom_killer can kill rogue processes and
 system will survive.
 If this is set to 1, the kernel panics when out-of-memory happens.
 However, if a process limits using nodes by mempolicy/cpusets,
 and those nodes become memory exhaustion status, one process
 may be killed by oom-killer. No panic occurs in this case.
 Because other nodes' memory may be free. This means system total status
 may be not fatal yet.
 If this is set to 2, the kernel panics compulsorily even on the
 above-mentioned.
 The default value is 0.
 1 and 2 are for failover of clustering. Please select either
 according to your policy of failover.
 =============================================================
 oom_dump_tasks
 Enables a system-wide task dump (excluding kernel threads) to be
 produced when the kernel performs an OOM-killing and includes such
 information as pid, uid, tgid, vm size, rss, cpu, oom_adj score, and
 name.  This is helpful to determine why the OOM killer was invoked
 and to identify the rogue task that caused it.
 If this is set to zero, this information is suppressed.  On very
 large systems with thousands of tasks it may not be feasible to dump
 the memory state information for each one.  Such systems should not
 be forced to incur a performance penalty in OOM conditions when the
 information may not be desired.
 If this is set to non-zero, this information is shown whenever the
 OOM killer actually kills a memory-hogging task.
 The default value is 0.
 =============================================================
 oom_kill_allocating_task
 This enables or disables killing the OOM-triggering task in
 out-of-memory situations.
 If this is set to zero, the OOM killer will scan through the entire
 tasklist and select a task based on heuristics to kill.  This normally
 selects a rogue memory-hogging task that frees up a large amount of
 memory when killed.
 If this is set to non-zero, the OOM killer simply kills the task that
 triggered the out-of-memory condition.  This avoids the expensive
 tasklist scan.
 If panic_on_oom is selected, it takes precedence over whatever value
 is used in oom_kill_allocating_task.
 The default value is 0.
 ==============================================================
 mmap_min_addr
 This file indicates the amount of address space  which a user process will
 be restricted from mmaping.  Since kernel null dereference bugs could
 accidentally operate based on the information in the first couple of pages
 of memory userspace processes should not be allowed to write to them.  By
 default this value is set to 0 and no protections will be enforced by the
 security module.  Setting this value to something like 64k will allow the
 vast majority of applications to work correctly and provide defense in depth
 against future potential kernel bugs.
 ==============================================================
 numa_zonelist_order
 This sysctl is only for NUMA.
 'where the memory is allocated from' is controlled by zonelists.
 (This documentation ignores ZONE_HIGHMEM/ZONE_DMA32 for simple explanation.
  you may be able to read ZONE_DMA as ZONE_DMA32...)
 In non-NUMA case, a zonelist for GFP_KERNEL is ordered as following.
 ZONE_NORMAL -> ZONE_DMA
 This means that a memory allocation request for GFP_KERNEL will
 get memory from ZONE_DMA only when ZONE_NORMAL is not available.
 In NUMA case, you can think of following 2 types of order.
 Assume 2 node NUMA and below is zonelist of Node(0)'s GFP_KERNEL
 (A) Node(0) ZONE_NORMAL -> Node(0) ZONE_DMA -> Node(1) ZONE_NORMAL
 (B) Node(0) ZONE_NORMAL -> Node(1) ZONE_NORMAL -> Node(0) ZONE_DMA.
 Type(A) offers the best locality for processes on Node(0), but ZONE_DMA
 will be used before ZONE_NORMAL exhaustion. This increases possibility of
 out-of-memory(OOM) of ZONE_DMA because ZONE_DMA is tend to be small.
 Type(B) cannot offer the best locality but is more robust against OOM of
 the DMA zone.
 Type(A) is called as "Node" order. Type (B) is "Zone" order.
 "Node order" orders the zonelists by node, then by zone within each node.
 Specify "[Nn]ode" for zone order
 "Zone Order" orders the zonelists by zone type, then by node within each
 zone.  Specify "[Zz]one"for zode order.
 Specify "[Dd]efault" to request automatic configuration.  Autoconfiguration
 will select "node" order in following case.
 (1) if the DMA zone does not exist or
 (2) if the DMA zone comprises greater than 50% of the available memory or
 (3) if any node's DMA zone comprises greater than 60% of its local memory and
     the amount of local memory is big enough.
 Otherwise, "zone" order will be selected. Default order is recommended unless
 this is causing problems for your system/application.
 ==============================================================
 nr_hugepages
 Change the minimum size of the hugepage pool.
 See Documentation/vm/hugetlbpage.txt
 ==============================================================
 nr_overcommit_hugepages
 Change the maximum size of the hugepage pool. The maximum is
 nr_hugepages + nr_overcommit_hugepages.
 See Documentation/vm/hugetlbpage.txt
+==============================================================
+nr_trim_pages
+This is available only on NOMMU kernels.
+This value adjusts the excess page trimming behaviour of power-of-2 aligned
+NOMMU mmap allocations.
+A value of 0 disables trimming of allocations entirely, while a value of 1
+trims excess pages aggressively. Any value >= 1 acts as the watermark where
+trimming of allocations is initiated.
+The default value is 1.
+See Documentation/nommu-mmap.txt for more information.

arch/arm/include/asm/mmu.h

Diff comments View file @ c40f6f8

 #ifndef __ARM_MMU_H
 #define __ARM_MMU_H
 #ifdef CONFIG_MMU
 typedef struct {
 #ifdef CONFIG_CPU_HAS_ASID
 	unsigned int id;
 #endif
 	unsigned int kvm_seq;
 } mm_context_t;
 #ifdef CONFIG_CPU_HAS_ASID
 #define ASID(mm)	((mm)->context.id & 255)
 #else
 #define ASID(mm)	(0)
 #endif
 #else
 /*
  * From nommu.h:
  *  Copyright (C) 2002, David McCullough <davidm@snapgear.com>
  *  modified for 2.6 by Hyok S. Choi <hyok.choi@samsung.com>
  */
 typedef struct {
-	struct vm_list_struct	*vmlist;
 	unsigned long		end_brk;
 } mm_context_t;
 #endif
 #endif

arch/arm/mm/dma-mapping.c

Diff comments View file @ c40f6f8

1	/*	1	/*
2	* linux/arch/arm/mm/dma-mapping.c	2	* linux/arch/arm/mm/dma-mapping.c
3	*	3	*
4	* Copyright (C) 2000-2004 Russell King	4	* Copyright (C) 2000-2004 Russell King
5	*	5	*
6	* This program is free software; you can redistribute it and/or modify	6	* This program is free software; you can redistribute it and/or modify
7	* it under the terms of the GNU General Public License version 2 as	7	* it under the terms of the GNU General Public License version 2 as
8	* published by the Free Software Foundation.	8	* published by the Free Software Foundation.
9	*	9	*
10	* DMA uncached mapping support.	10	* DMA uncached mapping support.
11	*/	11	*/
12	#include <linux/module.h>	12	#include <linux/module.h>
13	#include <linux/mm.h>	13	#include <linux/mm.h>
14	#include <linux/slab.h>	14	#include <linux/slab.h>
15	#include <linux/errno.h>	15	#include <linux/errno.h>
16	#include <linux/list.h>	16	#include <linux/list.h>
17	#include <linux/init.h>	17	#include <linux/init.h>
18	#include <linux/device.h>	18	#include <linux/device.h>
19	#include <linux/dma-mapping.h>	19	#include <linux/dma-mapping.h>
20		20
21	#include <asm/memory.h>	21	#include <asm/memory.h>
22	#include <asm/cacheflush.h>	22	#include <asm/cacheflush.h>
23	#include <asm/tlbflush.h>	23	#include <asm/tlbflush.h>
24	#include <asm/sizes.h>	24	#include <asm/sizes.h>
25		25
26	/* Sanity check size */	26	/* Sanity check size */
27	#if (CONSISTENT_DMA_SIZE % SZ_2M)	27	#if (CONSISTENT_DMA_SIZE % SZ_2M)
28	#error "CONSISTENT_DMA_SIZE must be multiple of 2MiB"	28	#error "CONSISTENT_DMA_SIZE must be multiple of 2MiB"
29	#endif	29	#endif
30		30
31	#define CONSISTENT_END (0xffe00000)	31	#define CONSISTENT_END (0xffe00000)
32	#define CONSISTENT_BASE (CONSISTENT_END - CONSISTENT_DMA_SIZE)	32	#define CONSISTENT_BASE (CONSISTENT_END - CONSISTENT_DMA_SIZE)
33		33
34	#define CONSISTENT_OFFSET(x) (((unsigned long)(x) - CONSISTENT_BASE) >> PAGE_SHIFT)	34	#define CONSISTENT_OFFSET(x) (((unsigned long)(x) - CONSISTENT_BASE) >> PAGE_SHIFT)
35	#define CONSISTENT_PTE_INDEX(x) (((unsigned long)(x) - CONSISTENT_BASE) >> PGDIR_SHIFT)	35	#define CONSISTENT_PTE_INDEX(x) (((unsigned long)(x) - CONSISTENT_BASE) >> PGDIR_SHIFT)
36	#define NUM_CONSISTENT_PTES (CONSISTENT_DMA_SIZE >> PGDIR_SHIFT)	36	#define NUM_CONSISTENT_PTES (CONSISTENT_DMA_SIZE >> PGDIR_SHIFT)
37		37
38		38
39	/*	39	/*
40	* These are the page tables (2MB each) covering uncached, DMA consistent allocations	40	* These are the page tables (2MB each) covering uncached, DMA consistent allocations
41	*/	41	*/
42	static pte_t *consistent_pte[NUM_CONSISTENT_PTES];	42	static pte_t *consistent_pte[NUM_CONSISTENT_PTES];
43	static DEFINE_SPINLOCK(consistent_lock);	43	static DEFINE_SPINLOCK(consistent_lock);
44		44
45	/*	45	/*
46	* VM region handling support.	46	* VM region handling support.
47	*	47	*
48	* This should become something generic, handling VM region allocations for	48	* This should become something generic, handling VM region allocations for
49	* vmalloc and similar (ioremap, module space, etc).	49	* vmalloc and similar (ioremap, module space, etc).
50	*	50	*
51	* I envisage vmalloc()'s supporting vm_struct becoming:	51	* I envisage vmalloc()'s supporting vm_struct becoming:
52	*	52	*
53	* struct vm_struct {	53	* struct vm_struct {
54	* struct vm_region region;	54	* struct vm_region region;
55	* unsigned long flags;	55	* unsigned long flags;
56	* struct page **pages;	56	* struct page **pages;
57	* unsigned int nr_pages;	57	* unsigned int nr_pages;
58	* unsigned long phys_addr;	58	* unsigned long phys_addr;
59	* };	59	* };
60	*	60	*
61	* get_vm_area() would then call vm_region_alloc with an appropriate	61	* get_vm_area() would then call vm_region_alloc with an appropriate
62	* struct vm_region head (eg):	62	* struct vm_region head (eg):
63	*	63	*
64	* struct vm_region vmalloc_head = {	64	* struct vm_region vmalloc_head = {
65	* .vm_list = LIST_HEAD_INIT(vmalloc_head.vm_list),	65	* .vm_list = LIST_HEAD_INIT(vmalloc_head.vm_list),
66	* .vm_start = VMALLOC_START,	66	* .vm_start = VMALLOC_START,
67	* .vm_end = VMALLOC_END,	67	* .vm_end = VMALLOC_END,
68	* };	68	* };
69	*	69	*
70	* However, vmalloc_head.vm_start is variable (typically, it is dependent on	70	* However, vmalloc_head.vm_start is variable (typically, it is dependent on
71	* the amount of RAM found at boot time.) I would imagine that get_vm_area()	71	* the amount of RAM found at boot time.) I would imagine that get_vm_area()
72	* would have to initialise this each time prior to calling vm_region_alloc().	72	* would have to initialise this each time prior to calling vm_region_alloc().
73	*/	73	*/
74	struct vm_region {	74	struct arm_vm_region {
75	struct list_head vm_list;	75	struct list_head vm_list;
76	unsigned long vm_start;	76	unsigned long vm_start;
77	unsigned long vm_end;	77	unsigned long vm_end;
78	struct page *vm_pages;	78	struct page *vm_pages;
79	int vm_active;	79	int vm_active;
80	};	80	};
81		81
82	static struct vm_region consistent_head = {	82	static struct arm_vm_region consistent_head = {
83	.vm_list = LIST_HEAD_INIT(consistent_head.vm_list),	83	.vm_list = LIST_HEAD_INIT(consistent_head.vm_list),
84	.vm_start = CONSISTENT_BASE,	84	.vm_start = CONSISTENT_BASE,
85	.vm_end = CONSISTENT_END,	85	.vm_end = CONSISTENT_END,
86	};	86	};
87		87
88	static struct vm_region *	88	static struct arm_vm_region *
89	vm_region_alloc(struct vm_region *head, size_t size, gfp_t gfp)	89	arm_vm_region_alloc(struct arm_vm_region *head, size_t size, gfp_t gfp)
90	{	90	{
91	unsigned long addr = head->vm_start, end = head->vm_end - size;	91	unsigned long addr = head->vm_start, end = head->vm_end - size;
92	unsigned long flags;	92	unsigned long flags;
93	struct vm_region c, new;	93	struct arm_vm_region c, new;
94		94
95	new = kmalloc(sizeof(struct vm_region), gfp);	95	new = kmalloc(sizeof(struct arm_vm_region), gfp);
96	if (!new)	96	if (!new)
97	goto out;	97	goto out;
98		98
99	spin_lock_irqsave(&consistent_lock, flags);	99	spin_lock_irqsave(&consistent_lock, flags);
100		100
101	list_for_each_entry(c, &head->vm_list, vm_list) {	101	list_for_each_entry(c, &head->vm_list, vm_list) {
102	if ((addr + size) < addr)	102	if ((addr + size) < addr)
103	goto nospc;	103	goto nospc;
104	if ((addr + size) <= c->vm_start)	104	if ((addr + size) <= c->vm_start)
105	goto found;	105	goto found;
106	addr = c->vm_end;	106	addr = c->vm_end;
107	if (addr > end)	107	if (addr > end)
108	goto nospc;	108	goto nospc;
109	}	109	}
110		110
111	found:	111	found:
112	/*	112	/*
113	* Insert this entry _before_ the one we found.	113	* Insert this entry _before_ the one we found.
114	*/	114	*/
115	list_add_tail(&new->vm_list, &c->vm_list);	115	list_add_tail(&new->vm_list, &c->vm_list);
116	new->vm_start = addr;	116	new->vm_start = addr;
117	new->vm_end = addr + size;	117	new->vm_end = addr + size;
118	new->vm_active = 1;	118	new->vm_active = 1;
119		119
120	spin_unlock_irqrestore(&consistent_lock, flags);	120	spin_unlock_irqrestore(&consistent_lock, flags);
121	return new;	121	return new;
122		122
123	nospc:	123	nospc:
124	spin_unlock_irqrestore(&consistent_lock, flags);	124	spin_unlock_irqrestore(&consistent_lock, flags);
125	kfree(new);	125	kfree(new);
126	out:	126	out:
127	return NULL;	127	return NULL;
128	}	128	}
129		129
130	static struct vm_region vm_region_find(struct vm_region head, unsigned long addr)	130	static struct arm_vm_region arm_vm_region_find(struct arm_vm_region head, unsigned long addr)
131	{	131	{
132	struct vm_region *c;	132	struct arm_vm_region *c;
133		133
134	list_for_each_entry(c, &head->vm_list, vm_list) {	134	list_for_each_entry(c, &head->vm_list, vm_list) {
135	if (c->vm_active && c->vm_start == addr)	135	if (c->vm_active && c->vm_start == addr)
136	goto out;	136	goto out;
137	}	137	}
138	c = NULL;	138	c = NULL;
139	out:	139	out:
140	return c;	140	return c;
141	}	141	}
142		142
143	#ifdef CONFIG_HUGETLB_PAGE	143	#ifdef CONFIG_HUGETLB_PAGE
144	#error ARM Coherent DMA allocator does not (yet) support huge TLB	144	#error ARM Coherent DMA allocator does not (yet) support huge TLB
145	#endif	145	#endif
146		146
147	static void *	147	static void *
148	__dma_alloc(struct device dev, size_t size, dma_addr_t handle, gfp_t gfp,	148	__dma_alloc(struct device dev, size_t size, dma_addr_t handle, gfp_t gfp,
149	pgprot_t prot)	149	pgprot_t prot)
150	{	150	{
151	struct page *page;	151	struct page *page;
152	struct vm_region *c;	152	struct arm_vm_region *c;
153	unsigned long order;	153	unsigned long order;
154	u64 mask = ISA_DMA_THRESHOLD, limit;	154	u64 mask = ISA_DMA_THRESHOLD, limit;
155		155
156	if (!consistent_pte[0]) {	156	if (!consistent_pte[0]) {
157	printk(KERN_ERR "%s: not initialised\n", __func__);	157	printk(KERN_ERR "%s: not initialised\n", __func__);
158	dump_stack();	158	dump_stack();
159	return NULL;	159	return NULL;
160	}	160	}
161		161
162	if (dev) {	162	if (dev) {
163	mask = dev->coherent_dma_mask;	163	mask = dev->coherent_dma_mask;
164		164
165	/*	165	/*
166	* Sanity check the DMA mask - it must be non-zero, and	166	* Sanity check the DMA mask - it must be non-zero, and
167	* must be able to be satisfied by a DMA allocation.	167	* must be able to be satisfied by a DMA allocation.
168	*/	168	*/
169	if (mask == 0) {	169	if (mask == 0) {
170	dev_warn(dev, "coherent DMA mask is unset\n");	170	dev_warn(dev, "coherent DMA mask is unset\n");
171	goto no_page;	171	goto no_page;
172	}	172	}
173		173
174	if ((~mask) & ISA_DMA_THRESHOLD) {	174	if ((~mask) & ISA_DMA_THRESHOLD) {
175	dev_warn(dev, "coherent DMA mask %#llx is smaller "	175	dev_warn(dev, "coherent DMA mask %#llx is smaller "
176	"than system GFP_DMA mask %#llx\n",	176	"than system GFP_DMA mask %#llx\n",
177	mask, (unsigned long long)ISA_DMA_THRESHOLD);	177	mask, (unsigned long long)ISA_DMA_THRESHOLD);
178	goto no_page;	178	goto no_page;
179	}	179	}
180	}	180	}
181		181
182	/*	182	/*
183	* Sanity check the allocation size.	183	* Sanity check the allocation size.
184	*/	184	*/
185	size = PAGE_ALIGN(size);	185	size = PAGE_ALIGN(size);
186	limit = (mask + 1) & ~mask;	186	limit = (mask + 1) & ~mask;
187	if ((limit && size >= limit) \|\|	187	if ((limit && size >= limit) \|\|
188	size >= (CONSISTENT_END - CONSISTENT_BASE)) {	188	size >= (CONSISTENT_END - CONSISTENT_BASE)) {
189	printk(KERN_WARNING "coherent allocation too big "	189	printk(KERN_WARNING "coherent allocation too big "
190	"(requested %#x mask %#llx)\n", size, mask);	190	"(requested %#x mask %#llx)\n", size, mask);
191	goto no_page;	191	goto no_page;
192	}	192	}
193		193
194	order = get_order(size);	194	order = get_order(size);
195		195
196	if (mask != 0xffffffff)	196	if (mask != 0xffffffff)
197	gfp \|= GFP_DMA;	197	gfp \|= GFP_DMA;
198		198
199	page = alloc_pages(gfp, order);	199	page = alloc_pages(gfp, order);
200	if (!page)	200	if (!page)
201	goto no_page;	201	goto no_page;
202		202
203	/*	203	/*
204	* Invalidate any data that might be lurking in the	204	* Invalidate any data that might be lurking in the
205	* kernel direct-mapped region for device DMA.	205	* kernel direct-mapped region for device DMA.
206	*/	206	*/
207	{	207	{
208	void *ptr = page_address(page);	208	void *ptr = page_address(page);
209	memset(ptr, 0, size);	209	memset(ptr, 0, size);
210	dmac_flush_range(ptr, ptr + size);	210	dmac_flush_range(ptr, ptr + size);
211	outer_flush_range(__pa(ptr), __pa(ptr) + size);	211	outer_flush_range(__pa(ptr), __pa(ptr) + size);
212	}	212	}
213		213
214	/*	214	/*
215	* Allocate a virtual address in the consistent mapping region.	215	* Allocate a virtual address in the consistent mapping region.
216	*/	216	*/
217	c = vm_region_alloc(&consistent_head, size,	217	c = arm_vm_region_alloc(&consistent_head, size,
218	gfp & ~(__GFP_DMA \| __GFP_HIGHMEM));	218	gfp & ~(__GFP_DMA \| __GFP_HIGHMEM));
219	if (c) {	219	if (c) {
220	pte_t *pte;	220	pte_t *pte;
221	struct page *end = page + (1 << order);	221	struct page *end = page + (1 << order);
222	int idx = CONSISTENT_PTE_INDEX(c->vm_start);	222	int idx = CONSISTENT_PTE_INDEX(c->vm_start);
223	u32 off = CONSISTENT_OFFSET(c->vm_start) & (PTRS_PER_PTE-1);	223	u32 off = CONSISTENT_OFFSET(c->vm_start) & (PTRS_PER_PTE-1);
224		224
225	pte = consistent_pte[idx] + off;	225	pte = consistent_pte[idx] + off;
226	c->vm_pages = page;	226	c->vm_pages = page;
227		227
228	split_page(page, order);	228	split_page(page, order);
229		229
230	/*	230	/*
231	* Set the "dma handle"	231	* Set the "dma handle"
232	*/	232	*/
233	*handle = page_to_dma(dev, page);	233	*handle = page_to_dma(dev, page);
234		234
235	do {	235	do {
236	BUG_ON(!pte_none(*pte));	236	BUG_ON(!pte_none(*pte));
237		237
238	/*	238	/*
239	* x86 does not mark the pages reserved...	239	* x86 does not mark the pages reserved...
240	*/	240	*/
241	SetPageReserved(page);	241	SetPageReserved(page);
242	set_pte_ext(pte, mk_pte(page, prot), 0);	242	set_pte_ext(pte, mk_pte(page, prot), 0);
243	page++;	243	page++;
244	pte++;	244	pte++;
245	off++;	245	off++;
246	if (off >= PTRS_PER_PTE) {	246	if (off >= PTRS_PER_PTE) {
247	off = 0;	247	off = 0;
248	pte = consistent_pte[++idx];	248	pte = consistent_pte[++idx];
249	}	249	}
250	} while (size -= PAGE_SIZE);	250	} while (size -= PAGE_SIZE);
251		251
252	/*	252	/*
253	* Free the otherwise unused pages.	253	* Free the otherwise unused pages.
254	*/	254	*/
255	while (page < end) {	255	while (page < end) {
256	__free_page(page);	256	__free_page(page);
257	page++;	257	page++;
258	}	258	}
259		259
260	return (void *)c->vm_start;	260	return (void *)c->vm_start;
261	}	261	}
262		262
263	if (page)	263	if (page)
264	__free_pages(page, order);	264	__free_pages(page, order);
265	no_page:	265	no_page:
266	*handle = ~0;	266	*handle = ~0;
267	return NULL;	267	return NULL;
268	}	268	}
269		269
270	/*	270	/*
271	* Allocate DMA-coherent memory space and return both the kernel remapped	271	* Allocate DMA-coherent memory space and return both the kernel remapped
272	* virtual and bus address for that space.	272	* virtual and bus address for that space.
273	*/	273	*/
274	void *	274	void *
275	dma_alloc_coherent(struct device dev, size_t size, dma_addr_t handle, gfp_t gfp)	275	dma_alloc_coherent(struct device dev, size_t size, dma_addr_t handle, gfp_t gfp)
276	{	276	{
277	void *memory;	277	void *memory;
278		278
279	if (dma_alloc_from_coherent(dev, size, handle, &memory))	279	if (dma_alloc_from_coherent(dev, size, handle, &memory))
280	return memory;	280	return memory;
281		281
282	if (arch_is_coherent()) {	282	if (arch_is_coherent()) {
283	void *virt;	283	void *virt;
284		284
285	virt = kmalloc(size, gfp);	285	virt = kmalloc(size, gfp);
286	if (!virt)	286	if (!virt)
287	return NULL;	287	return NULL;
288	*handle = virt_to_dma(dev, virt);	288	*handle = virt_to_dma(dev, virt);
289		289
290	return virt;	290	return virt;
291	}	291	}
292		292
293	return __dma_alloc(dev, size, handle, gfp,	293	return __dma_alloc(dev, size, handle, gfp,
294	pgprot_noncached(pgprot_kernel));	294	pgprot_noncached(pgprot_kernel));
295	}	295	}
296	EXPORT_SYMBOL(dma_alloc_coherent);	296	EXPORT_SYMBOL(dma_alloc_coherent);
297		297
298	/*	298	/*
299	* Allocate a writecombining region, in much the same way as	299	* Allocate a writecombining region, in much the same way as
300	* dma_alloc_coherent above.	300	* dma_alloc_coherent above.
301	*/	301	*/
302	void *	302	void *
303	dma_alloc_writecombine(struct device dev, size_t size, dma_addr_t handle, gfp_t gfp)	303	dma_alloc_writecombine(struct device dev, size_t size, dma_addr_t handle, gfp_t gfp)
304	{	304	{
305	return __dma_alloc(dev, size, handle, gfp,	305	return __dma_alloc(dev, size, handle, gfp,
306	pgprot_writecombine(pgprot_kernel));	306	pgprot_writecombine(pgprot_kernel));
307	}	307	}
308	EXPORT_SYMBOL(dma_alloc_writecombine);	308	EXPORT_SYMBOL(dma_alloc_writecombine);
309		309
310	static int dma_mmap(struct device dev, struct vm_area_struct vma,	310	static int dma_mmap(struct device dev, struct vm_area_struct vma,
311	void *cpu_addr, dma_addr_t dma_addr, size_t size)	311	void *cpu_addr, dma_addr_t dma_addr, size_t size)
312	{	312	{
313	unsigned long flags, user_size, kern_size;	313	unsigned long flags, user_size, kern_size;
314	struct vm_region *c;	314	struct arm_vm_region *c;
315	int ret = -ENXIO;	315	int ret = -ENXIO;
316		316
317	user_size = (vma->vm_end - vma->vm_start) >> PAGE_SHIFT;	317	user_size = (vma->vm_end - vma->vm_start) >> PAGE_SHIFT;
318		318
319	spin_lock_irqsave(&consistent_lock, flags);	319	spin_lock_irqsave(&consistent_lock, flags);
320	c = vm_region_find(&consistent_head, (unsigned long)cpu_addr);	320	c = arm_vm_region_find(&consistent_head, (unsigned long)cpu_addr);
321	spin_unlock_irqrestore(&consistent_lock, flags);	321	spin_unlock_irqrestore(&consistent_lock, flags);
322		322
323	if (c) {	323	if (c) {
324	unsigned long off = vma->vm_pgoff;	324	unsigned long off = vma->vm_pgoff;
325		325
326	kern_size = (c->vm_end - c->vm_start) >> PAGE_SHIFT;	326	kern_size = (c->vm_end - c->vm_start) >> PAGE_SHIFT;
327		327
328	if (off < kern_size &&	328	if (off < kern_size &&
329	user_size <= (kern_size - off)) {	329	user_size <= (kern_size - off)) {
330	ret = remap_pfn_range(vma, vma->vm_start,	330	ret = remap_pfn_range(vma, vma->vm_start,
331	page_to_pfn(c->vm_pages) + off,	331	page_to_pfn(c->vm_pages) + off,
332	user_size << PAGE_SHIFT,	332	user_size << PAGE_SHIFT,
333	vma->vm_page_prot);	333	vma->vm_page_prot);
334	}	334	}
335	}	335	}
336		336
337	return ret;	337	return ret;
338	}	338	}
339		339
340	int dma_mmap_coherent(struct device dev, struct vm_area_struct vma,	340	int dma_mmap_coherent(struct device dev, struct vm_area_struct vma,
341	void *cpu_addr, dma_addr_t dma_addr, size_t size)	341	void *cpu_addr, dma_addr_t dma_addr, size_t size)
342	{	342	{
343	vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);	343	vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
344	return dma_mmap(dev, vma, cpu_addr, dma_addr, size);	344	return dma_mmap(dev, vma, cpu_addr, dma_addr, size);
345	}	345	}
346	EXPORT_SYMBOL(dma_mmap_coherent);	346	EXPORT_SYMBOL(dma_mmap_coherent);
347		347
348	int dma_mmap_writecombine(struct device dev, struct vm_area_struct vma,	348	int dma_mmap_writecombine(struct device dev, struct vm_area_struct vma,
349	void *cpu_addr, dma_addr_t dma_addr, size_t size)	349	void *cpu_addr, dma_addr_t dma_addr, size_t size)
350	{	350	{
351	vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);	351	vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
352	return dma_mmap(dev, vma, cpu_addr, dma_addr, size);	352	return dma_mmap(dev, vma, cpu_addr, dma_addr, size);
353	}	353	}
354	EXPORT_SYMBOL(dma_mmap_writecombine);	354	EXPORT_SYMBOL(dma_mmap_writecombine);
355		355
356	/*	356	/*
357	* free a page as defined by the above mapping.	357	* free a page as defined by the above mapping.
358	* Must not be called with IRQs disabled.	358	* Must not be called with IRQs disabled.
359	*/	359	*/
360	void dma_free_coherent(struct device dev, size_t size, void cpu_addr, dma_addr_t handle)	360	void dma_free_coherent(struct device dev, size_t size, void cpu_addr, dma_addr_t handle)
361	{	361	{
362	struct vm_region *c;	362	struct arm_vm_region *c;
363	unsigned long flags, addr;	363	unsigned long flags, addr;
364	pte_t *ptep;	364	pte_t *ptep;
365	int idx;	365	int idx;
366	u32 off;	366	u32 off;
367		367
368	WARN_ON(irqs_disabled());	368	WARN_ON(irqs_disabled());
369		369
370	if (dma_release_from_coherent(dev, get_order(size), cpu_addr))	370	if (dma_release_from_coherent(dev, get_order(size), cpu_addr))
371	return;	371	return;
372		372
373	if (arch_is_coherent()) {	373	if (arch_is_coherent()) {
374	kfree(cpu_addr);	374	kfree(cpu_addr);
375	return;	375	return;
376	}	376	}
377		377
378	size = PAGE_ALIGN(size);	378	size = PAGE_ALIGN(size);
379		379
380	spin_lock_irqsave(&consistent_lock, flags);	380	spin_lock_irqsave(&consistent_lock, flags);
381	c = vm_region_find(&consistent_head, (unsigned long)cpu_addr);	381	c = arm_vm_region_find(&consistent_head, (unsigned long)cpu_addr);
382	if (!c)	382	if (!c)
383	goto no_area;	383	goto no_area;
384		384
385	c->vm_active = 0;	385	c->vm_active = 0;
386	spin_unlock_irqrestore(&consistent_lock, flags);	386	spin_unlock_irqrestore(&consistent_lock, flags);
387		387
388	if ((c->vm_end - c->vm_start) != size) {	388	if ((c->vm_end - c->vm_start) != size) {
389	printk(KERN_ERR "%s: freeing wrong coherent size (%ld != %d)\n",	389	printk(KERN_ERR "%s: freeing wrong coherent size (%ld != %d)\n",
390	__func__, c->vm_end - c->vm_start, size);	390	__func__, c->vm_end - c->vm_start, size);
391	dump_stack();	391	dump_stack();
392	size = c->vm_end - c->vm_start;	392	size = c->vm_end - c->vm_start;
393	}	393	}
394		394
395	idx = CONSISTENT_PTE_INDEX(c->vm_start);	395	idx = CONSISTENT_PTE_INDEX(c->vm_start);
396	off = CONSISTENT_OFFSET(c->vm_start) & (PTRS_PER_PTE-1);	396	off = CONSISTENT_OFFSET(c->vm_start) & (PTRS_PER_PTE-1);
397	ptep = consistent_pte[idx] + off;	397	ptep = consistent_pte[idx] + off;
398	addr = c->vm_start;	398	addr = c->vm_start;
399	do {	399	do {
400	pte_t pte = ptep_get_and_clear(&init_mm, addr, ptep);	400	pte_t pte = ptep_get_and_clear(&init_mm, addr, ptep);
401	unsigned long pfn;	401	unsigned long pfn;
402		402
403	ptep++;	403	ptep++;
404	addr += PAGE_SIZE;	404	addr += PAGE_SIZE;
405	off++;	405	off++;
406	if (off >= PTRS_PER_PTE) {	406	if (off >= PTRS_PER_PTE) {
407	off = 0;	407	off = 0;
408	ptep = consistent_pte[++idx];	408	ptep = consistent_pte[++idx];
409	}	409	}
410		410
411	if (!pte_none(pte) && pte_present(pte)) {	411	if (!pte_none(pte) && pte_present(pte)) {
412	pfn = pte_pfn(pte);	412	pfn = pte_pfn(pte);
413		413
414	if (pfn_valid(pfn)) {	414	if (pfn_valid(pfn)) {
415	struct page *page = pfn_to_page(pfn);	415	struct page *page = pfn_to_page(pfn);
416		416
417	/*	417	/*
418	* x86 does not mark the pages reserved...	418	* x86 does not mark the pages reserved...
419	*/	419	*/
420	ClearPageReserved(page);	420	ClearPageReserved(page);
421		421
422	__free_page(page);	422	__free_page(page);
423	continue;	423	continue;
424	}	424	}
425	}	425	}
426		426
427	printk(KERN_CRIT "%s: bad page in kernel page table\n",	427	printk(KERN_CRIT "%s: bad page in kernel page table\n",
428	__func__);	428	__func__);
429	} while (size -= PAGE_SIZE);	429	} while (size -= PAGE_SIZE);
430		430
431	flush_tlb_kernel_range(c->vm_start, c->vm_end);	431	flush_tlb_kernel_range(c->vm_start, c->vm_end);
432		432
433	spin_lock_irqsave(&consistent_lock, flags);	433	spin_lock_irqsave(&consistent_lock, flags);
434	list_del(&c->vm_list);	434	list_del(&c->vm_list);
435	spin_unlock_irqrestore(&consistent_lock, flags);	435	spin_unlock_irqrestore(&consistent_lock, flags);
436		436
437	kfree(c);	437	kfree(c);
438	return;	438	return;
439		439
440	no_area:	440	no_area:
441	spin_unlock_irqrestore(&consistent_lock, flags);	441	spin_unlock_irqrestore(&consistent_lock, flags);
442	printk(KERN_ERR "%s: trying to free invalid coherent area: %p\n",	442	printk(KERN_ERR "%s: trying to free invalid coherent area: %p\n",
443	__func__, cpu_addr);	443	__func__, cpu_addr);
444	dump_stack();	444	dump_stack();
445	}	445	}
446	EXPORT_SYMBOL(dma_free_coherent);	446	EXPORT_SYMBOL(dma_free_coherent);
447		447
448	/*	448	/*
449	* Initialise the consistent memory allocation.	449	* Initialise the consistent memory allocation.
450	*/	450	*/
451	static int __init consistent_init(void)	451	static int __init consistent_init(void)
452	{	452	{
453	pgd_t *pgd;	453	pgd_t *pgd;
454	pmd_t *pmd;	454	pmd_t *pmd;
455	pte_t *pte;	455	pte_t *pte;
456	int ret = 0, i = 0;	456	int ret = 0, i = 0;
457	u32 base = CONSISTENT_BASE;	457	u32 base = CONSISTENT_BASE;
458		458
459	do {	459	do {
460	pgd = pgd_offset(&init_mm, base);	460	pgd = pgd_offset(&init_mm, base);
461	pmd = pmd_alloc(&init_mm, pgd, base);	461	pmd = pmd_alloc(&init_mm, pgd, base);
462	if (!pmd) {	462	if (!pmd) {
463	printk(KERN_ERR "%s: no pmd tables\n", __func__);	463	printk(KERN_ERR "%s: no pmd tables\n", __func__);
464	ret = -ENOMEM;	464	ret = -ENOMEM;
465	break;	465	break;
466	}	466	}
467	WARN_ON(!pmd_none(*pmd));	467	WARN_ON(!pmd_none(*pmd));
468		468
469	pte = pte_alloc_kernel(pmd, base);	469	pte = pte_alloc_kernel(pmd, base);
470	if (!pte) {	470	if (!pte) {
471	printk(KERN_ERR "%s: no pte tables\n", __func__);	471	printk(KERN_ERR "%s: no pte tables\n", __func__);
472	ret = -ENOMEM;	472	ret = -ENOMEM;
473	break;	473	break;
474	}	474	}
475		475
476	consistent_pte[i++] = pte;	476	consistent_pte[i++] = pte;
477	base += (1 << PGDIR_SHIFT);	477	base += (1 << PGDIR_SHIFT);
478	} while (base < CONSISTENT_END);	478	} while (base < CONSISTENT_END);
479		479
480	return ret;	480	return ret;
481	}	481	}
482		482
483	core_initcall(consistent_init);	483	core_initcall(consistent_init);
484		484
485	/*	485	/*
486	* Make an area consistent for devices.	486	* Make an area consistent for devices.
487	* Note: Drivers should NOT use this function directly, as it will break	487	* Note: Drivers should NOT use this function directly, as it will break
488	* platforms with CONFIG_DMABOUNCE.	488	* platforms with CONFIG_DMABOUNCE.
489	* Use the driver DMA support - see dma-mapping.h (dma_sync_*)	489	* Use the driver DMA support - see dma-mapping.h (dma_sync_*)
490	*/	490	*/
491	void dma_cache_maint(const void *start, size_t size, int direction)	491	void dma_cache_maint(const void *start, size_t size, int direction)
492	{	492	{
493	const void *end = start + size;	493	const void *end = start + size;
494		494
495	BUG_ON(!virt_addr_valid(start) \|\| !virt_addr_valid(end - 1));	495	BUG_ON(!virt_addr_valid(start) \|\| !virt_addr_valid(end - 1));
496		496
497	switch (direction) {	497	switch (direction) {
498	case DMA_FROM_DEVICE: /* invalidate only */	498	case DMA_FROM_DEVICE: /* invalidate only */
499	dmac_inv_range(start, end);	499	dmac_inv_range(start, end);
500	outer_inv_range(__pa(start), __pa(end));	500	outer_inv_range(__pa(start), __pa(end));
501	break;	501	break;
502	case DMA_TO_DEVICE: /* writeback only */	502	case DMA_TO_DEVICE: /* writeback only */
503	dmac_clean_range(start, end);	503	dmac_clean_range(start, end);
504	outer_clean_range(__pa(start), __pa(end));	504	outer_clean_range(__pa(start), __pa(end));
505	break;	505	break;
506	case DMA_BIDIRECTIONAL: /* writeback and invalidate */	506	case DMA_BIDIRECTIONAL: /* writeback and invalidate */
507	dmac_flush_range(start, end);	507	dmac_flush_range(start, end);
508	outer_flush_range(__pa(start), __pa(end));	508	outer_flush_range(__pa(start), __pa(end));
509	break;	509	break;
510	default:	510	default:
511	BUG();	511	BUG();
512	}	512	}
513	}	513	}
514	EXPORT_SYMBOL(dma_cache_maint);	514	EXPORT_SYMBOL(dma_cache_maint);
515		515
516	/**	516	/**
517	* dma_map_sg - map a set of SG buffers for streaming mode DMA	517	* dma_map_sg - map a set of SG buffers for streaming mode DMA
518	* @dev: valid struct device pointer, or NULL for ISA and EISA-like devices	518	* @dev: valid struct device pointer, or NULL for ISA and EISA-like devices
519	* @sg: list of buffers	519	* @sg: list of buffers
520	* @nents: number of buffers to map	520	* @nents: number of buffers to map
521	* @dir: DMA transfer direction	521	* @dir: DMA transfer direction
522	*	522	*
523	* Map a set of buffers described by scatterlist in streaming mode for DMA.	523	* Map a set of buffers described by scatterlist in streaming mode for DMA.
524	* This is the scatter-gather version of the dma_map_single interface.	524	* This is the scatter-gather version of the dma_map_single interface.
525	* Here the scatter gather list elements are each tagged with the	525	* Here the scatter gather list elements are each tagged with the
526	* appropriate dma address and length. They are obtained via	526	* appropriate dma address and length. They are obtained via
527	* sg_dma_{address,length}.	527	* sg_dma_{address,length}.
528	*	528	*
529	* Device ownership issues as mentioned for dma_map_single are the same	529	* Device ownership issues as mentioned for dma_map_single are the same
530	* here.	530	* here.
531	*/	531	*/
532	int dma_map_sg(struct device dev, struct scatterlist sg, int nents,	532	int dma_map_sg(struct device dev, struct scatterlist sg, int nents,
533	enum dma_data_direction dir)	533	enum dma_data_direction dir)
534	{	534	{
535	struct scatterlist *s;	535	struct scatterlist *s;
536	int i, j;	536	int i, j;
537		537
538	for_each_sg(sg, s, nents, i) {	538	for_each_sg(sg, s, nents, i) {
539	s->dma_address = dma_map_page(dev, sg_page(s), s->offset,	539	s->dma_address = dma_map_page(dev, sg_page(s), s->offset,
540	s->length, dir);	540	s->length, dir);
541	if (dma_mapping_error(dev, s->dma_address))	541	if (dma_mapping_error(dev, s->dma_address))
542	goto bad_mapping;	542	goto bad_mapping;
543	}	543	}
544	return nents;	544	return nents;
545		545
546	bad_mapping:	546	bad_mapping:
547	for_each_sg(sg, s, i, j)	547	for_each_sg(sg, s, i, j)
548	dma_unmap_page(dev, sg_dma_address(s), sg_dma_len(s), dir);	548	dma_unmap_page(dev, sg_dma_address(s), sg_dma_len(s), dir);
549	return 0;	549	return 0;
550	}	550	}
551	EXPORT_SYMBOL(dma_map_sg);	551	EXPORT_SYMBOL(dma_map_sg);
552		552
553	/**	553	/**
554	* dma_unmap_sg - unmap a set of SG buffers mapped by dma_map_sg	554	* dma_unmap_sg - unmap a set of SG buffers mapped by dma_map_sg
555	* @dev: valid struct device pointer, or NULL for ISA and EISA-like devices	555	* @dev: valid struct device pointer, or NULL for ISA and EISA-like devices
556	* @sg: list of buffers	556	* @sg: list of buffers
557	* @nents: number of buffers to unmap (returned from dma_map_sg)	557	* @nents: number of buffers to unmap (returned from dma_map_sg)
558	* @dir: DMA transfer direction (same as was passed to dma_map_sg)	558	* @dir: DMA transfer direction (same as was passed to dma_map_sg)
559	*	559	*
560	* Unmap a set of streaming mode DMA translations. Again, CPU access	560	* Unmap a set of streaming mode DMA translations. Again, CPU access
561	* rules concerning calls here are the same as for dma_unmap_single().	561	* rules concerning calls here are the same as for dma_unmap_single().
562	*/	562	*/
563	void dma_unmap_sg(struct device dev, struct scatterlist sg, int nents,	563	void dma_unmap_sg(struct device dev, struct scatterlist sg, int nents,
564	enum dma_data_direction dir)	564	enum dma_data_direction dir)
565	{	565	{
566	struct scatterlist *s;	566	struct scatterlist *s;
567	int i;	567	int i;
568		568
569	for_each_sg(sg, s, nents, i)	569	for_each_sg(sg, s, nents, i)
570	dma_unmap_page(dev, sg_dma_address(s), sg_dma_len(s), dir);	570	dma_unmap_page(dev, sg_dma_address(s), sg_dma_len(s), dir);
571	}	571	}
572	EXPORT_SYMBOL(dma_unmap_sg);	572	EXPORT_SYMBOL(dma_unmap_sg);
573		573
574	/**	574	/**
575	* dma_sync_sg_for_cpu	575	* dma_sync_sg_for_cpu
576	* @dev: valid struct device pointer, or NULL for ISA and EISA-like devices	576	* @dev: valid struct device pointer, or NULL for ISA and EISA-like devices
577	* @sg: list of buffers	577	* @sg: list of buffers
578	* @nents: number of buffers to map (returned from dma_map_sg)	578	* @nents: number of buffers to map (returned from dma_map_sg)
579	* @dir: DMA transfer direction (same as was passed to dma_map_sg)	579	* @dir: DMA transfer direction (same as was passed to dma_map_sg)
580	*/	580	*/
581	void dma_sync_sg_for_cpu(struct device dev, struct scatterlist sg,	581	void dma_sync_sg_for_cpu(struct device dev, struct scatterlist sg,
582	int nents, enum dma_data_direction dir)	582	int nents, enum dma_data_direction dir)
583	{	583	{
584	struct scatterlist *s;	584	struct scatterlist *s;
585	int i;	585	int i;
586		586
587	for_each_sg(sg, s, nents, i) {	587	for_each_sg(sg, s, nents, i) {
588	dmabounce_sync_for_cpu(dev, sg_dma_address(s), 0,	588	dmabounce_sync_for_cpu(dev, sg_dma_address(s), 0,
589	sg_dma_len(s), dir);	589	sg_dma_len(s), dir);
590	}	590	}
591	}	591	}
592	EXPORT_SYMBOL(dma_sync_sg_for_cpu);	592	EXPORT_SYMBOL(dma_sync_sg_for_cpu);
593		593
594	/**	594	/**
595	* dma_sync_sg_for_device	595	* dma_sync_sg_for_device
596	* @dev: valid struct device pointer, or NULL for ISA and EISA-like devices	596	* @dev: valid struct device pointer, or NULL for ISA and EISA-like devices
597	* @sg: list of buffers	597	* @sg: list of buffers
598	* @nents: number of buffers to map (returned from dma_map_sg)	598	* @nents: number of buffers to map (returned from dma_map_sg)
599	* @dir: DMA transfer direction (same as was passed to dma_map_sg)	599	* @dir: DMA transfer direction (same as was passed to dma_map_sg)
600	*/	600	*/
601	void dma_sync_sg_for_device(struct device dev, struct scatterlist sg,	601	void dma_sync_sg_for_device(struct device dev, struct scatterlist sg,
602	int nents, enum dma_data_direction dir)	602	int nents, enum dma_data_direction dir)
603	{	603	{
604	struct scatterlist *s;	604	struct scatterlist *s;
605	int i;	605	int i;
606		606
607	for_each_sg(sg, s, nents, i) {	607	for_each_sg(sg, s, nents, i) {
608	if (!dmabounce_sync_for_device(dev, sg_dma_address(s), 0,	608	if (!dmabounce_sync_for_device(dev, sg_dma_address(s), 0,
609	sg_dma_len(s), dir))	609	sg_dma_len(s), dir))
610	continue;	610	continue;
611		611
612	if (!arch_is_coherent())	612	if (!arch_is_coherent())
613	dma_cache_maint(sg_virt(s), s->length, dir);	613	dma_cache_maint(sg_virt(s), s->length, dir);
614	}	614	}
615	}	615	}
616	EXPORT_SYMBOL(dma_sync_sg_for_device);	616	EXPORT_SYMBOL(dma_sync_sg_for_device);
617		617

arch/blackfin/include/asm/mmu.h

Diff comments View file @ c40f6f8

 #ifndef __MMU_H
 #define __MMU_H
 /* Copyright (C) 2002, David McCullough <davidm@snapgear.com> */
 struct sram_list_struct {
 	struct sram_list_struct *next;
 	void *addr;
 	size_t length;
 };
 typedef struct {
-	struct vm_list_struct *vmlist;
 	unsigned long end_brk;
 	unsigned long stack_start;
 	/* Points to the location in SDRAM where the L1 stack is normally
 	   saved, or NULL if the stack is always in SDRAM.  */
 	void *l1_stack_save;
 	struct sram_list_struct *sram_list;
 #ifdef CONFIG_BINFMT_ELF_FDPIC
 	unsigned long	exec_fdpic_loadmap;
 	unsigned long	interp_fdpic_loadmap;
 #endif
 #ifdef CONFIG_MPU
 	unsigned long *page_rwx_mask;
 #endif
 } mm_context_t;
 #endif

arch/blackfin/kernel/ptrace.c

Diff comments View file @ c40f6f8

1	/*	1	/*
2	* File: arch/blackfin/kernel/ptrace.c	2	* File: arch/blackfin/kernel/ptrace.c
3	* Based on: Taken from linux/kernel/ptrace.c	3	* Based on: Taken from linux/kernel/ptrace.c
4	* Author: linux/kernel/ptrace.c is by Ross Biro 1/23/92, edited by Linus Torvalds	4	* Author: linux/kernel/ptrace.c is by Ross Biro 1/23/92, edited by Linus Torvalds
5	*	5	*
6	* Created: 1/23/92	6	* Created: 1/23/92
7	* Description:	7	* Description:
8	*	8	*
9	* Modified:	9	* Modified:
10	* Copyright 2004-2006 Analog Devices Inc.	10	* Copyright 2004-2006 Analog Devices Inc.
11	*	11	*
12	* Bugs: Enter bugs at http://blackfin.uclinux.org/	12	* Bugs: Enter bugs at http://blackfin.uclinux.org/
13	*	13	*
14	* This program is free software; you can redistribute it and/or modify	14	* This program is free software; you can redistribute it and/or modify
15	* it under the terms of the GNU General Public License as published by	15	* it under the terms of the GNU General Public License as published by
16	* the Free Software Foundation; either version 2 of the License, or	16	* the Free Software Foundation; either version 2 of the License, or
17	* (at your option) any later version.	17	* (at your option) any later version.
18	*	18	*
19	* This program is distributed in the hope that it will be useful,	19	* This program is distributed in the hope that it will be useful,
20	* but WITHOUT ANY WARRANTY; without even the implied warranty of	20	* but WITHOUT ANY WARRANTY; without even the implied warranty of
21	* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the	21	* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
22	* GNU General Public License for more details.	22	* GNU General Public License for more details.
23	*	23	*
24	* You should have received a copy of the GNU General Public License	24	* You should have received a copy of the GNU General Public License
25	* along with this program; if not, see the file COPYING, or write	25	* along with this program; if not, see the file COPYING, or write
26	* to the Free Software Foundation, Inc.,	26	* to the Free Software Foundation, Inc.,
27	* 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA	27	* 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
28	*/	28	*/
29		29
30	#include <linux/kernel.h>	30	#include <linux/kernel.h>
31	#include <linux/sched.h>	31	#include <linux/sched.h>
32	#include <linux/mm.h>	32	#include <linux/mm.h>
33	#include <linux/smp.h>	33	#include <linux/smp.h>
34	#include <linux/smp_lock.h>	34	#include <linux/smp_lock.h>
35	#include <linux/errno.h>	35	#include <linux/errno.h>
36	#include <linux/ptrace.h>	36	#include <linux/ptrace.h>
37	#include <linux/user.h>	37	#include <linux/user.h>
38	#include <linux/signal.h>	38	#include <linux/signal.h>
39	#include <linux/uaccess.h>	39	#include <linux/uaccess.h>
40		40
41	#include <asm/page.h>	41	#include <asm/page.h>
42	#include <asm/pgtable.h>	42	#include <asm/pgtable.h>
43	#include <asm/system.h>	43	#include <asm/system.h>
44	#include <asm/processor.h>	44	#include <asm/processor.h>
45	#include <asm/asm-offsets.h>	45	#include <asm/asm-offsets.h>
46	#include <asm/dma.h>	46	#include <asm/dma.h>
47	#include <asm/fixed_code.h>	47	#include <asm/fixed_code.h>
48	#include <asm/mem_map.h>	48	#include <asm/mem_map.h>
49		49
50	#define TEXT_OFFSET 0	50	#define TEXT_OFFSET 0
51	/*	51	/*
52	* does not yet catch signals sent when the child dies.	52	* does not yet catch signals sent when the child dies.
53	* in exit.c or in signal.c.	53	* in exit.c or in signal.c.
54	*/	54	*/
55		55
56	/* determines which bits in the SYSCFG reg the user has access to. */	56	/* determines which bits in the SYSCFG reg the user has access to. */
57	/* 1 = access 0 = no access */	57	/* 1 = access 0 = no access */
58	#define SYSCFG_MASK 0x0007 /* SYSCFG reg */	58	#define SYSCFG_MASK 0x0007 /* SYSCFG reg */
59	/* sets the trace bits. */	59	/* sets the trace bits. */
60	#define TRACE_BITS 0x0001	60	#define TRACE_BITS 0x0001
61		61
62	/* Find the stack offset for a register, relative to thread.esp0. */	62	/* Find the stack offset for a register, relative to thread.esp0. */
63	#define PT_REG(reg) ((long)&((struct pt_regs *)0)->reg)	63	#define PT_REG(reg) ((long)&((struct pt_regs *)0)->reg)
64		64
65	/*	65	/*
66	* Get the address of the live pt_regs for the specified task.	66	* Get the address of the live pt_regs for the specified task.
67	* These are saved onto the top kernel stack when the process	67	* These are saved onto the top kernel stack when the process
68	* is not running.	68	* is not running.
69	*	69	*
70	* Note: if a user thread is execve'd from kernel space, the	70	* Note: if a user thread is execve'd from kernel space, the
71	* kernel stack will not be empty on entry to the kernel, so	71	* kernel stack will not be empty on entry to the kernel, so
72	* ptracing these tasks will fail.	72	* ptracing these tasks will fail.
73	*/	73	*/
74	static inline struct pt_regs get_user_regs(struct task_struct task)	74	static inline struct pt_regs get_user_regs(struct task_struct task)
75	{	75	{
76	return (struct pt_regs *)	76	return (struct pt_regs *)
77	((unsigned long)task_stack_page(task) +	77	((unsigned long)task_stack_page(task) +
78	(THREAD_SIZE - sizeof(struct pt_regs)));	78	(THREAD_SIZE - sizeof(struct pt_regs)));
79	}	79	}
80		80
81	/*	81	/*
82	* Get all user integer registers.	82	* Get all user integer registers.
83	*/	83	*/
84	static inline int ptrace_getregs(struct task_struct tsk, void __user uregs)	84	static inline int ptrace_getregs(struct task_struct tsk, void __user uregs)
85	{	85	{
86	struct pt_regs regs;	86	struct pt_regs regs;
87	memcpy(&regs, get_user_regs(tsk), sizeof(regs));	87	memcpy(&regs, get_user_regs(tsk), sizeof(regs));
88	regs.usp = tsk->thread.usp;	88	regs.usp = tsk->thread.usp;
89	return copy_to_user(uregs, &regs, sizeof(struct pt_regs)) ? -EFAULT : 0;	89	return copy_to_user(uregs, &regs, sizeof(struct pt_regs)) ? -EFAULT : 0;
90	}	90	}
91		91
92	/* Mapping from PT_xxx to the stack offset at which the register is	92	/* Mapping from PT_xxx to the stack offset at which the register is
93	* saved. Notice that usp has no stack-slot and needs to be treated	93	* saved. Notice that usp has no stack-slot and needs to be treated
94	* specially (see get_reg/put_reg below).	94	* specially (see get_reg/put_reg below).
95	*/	95	*/
96		96
97	/*	97	/*
98	* Get contents of register REGNO in task TASK.	98	* Get contents of register REGNO in task TASK.
99	*/	99	*/
100	static inline long get_reg(struct task_struct *task, int regno)	100	static inline long get_reg(struct task_struct *task, int regno)
101	{	101	{
102	unsigned char *reg_ptr;	102	unsigned char *reg_ptr;
103		103
104	struct pt_regs *regs =	104	struct pt_regs *regs =
105	(struct pt_regs *)((unsigned long)task_stack_page(task) +	105	(struct pt_regs *)((unsigned long)task_stack_page(task) +
106	(THREAD_SIZE - sizeof(struct pt_regs)));	106	(THREAD_SIZE - sizeof(struct pt_regs)));
107	reg_ptr = (char *)regs;	107	reg_ptr = (char *)regs;
108		108
109	switch (regno) {	109	switch (regno) {
110	case PT_USP:	110	case PT_USP:
111	return task->thread.usp;	111	return task->thread.usp;
112	default:	112	default:
113	if (regno <= 216)	113	if (regno <= 216)
114	return (long )(reg_ptr + regno);	114	return (long )(reg_ptr + regno);
115	}	115	}
116	/* slight mystery ... never seems to come here but kernel misbehaves without this code! */	116	/* slight mystery ... never seems to come here but kernel misbehaves without this code! */
117		117
118	printk(KERN_WARNING "Request to get for unknown register %d\n", regno);	118	printk(KERN_WARNING "Request to get for unknown register %d\n", regno);
119	return 0;	119	return 0;
120	}	120	}
121		121
122	/*	122	/*
123	* Write contents of register REGNO in task TASK.	123	* Write contents of register REGNO in task TASK.
124	*/	124	*/
125	static inline int	125	static inline int
126	put_reg(struct task_struct *task, int regno, unsigned long data)	126	put_reg(struct task_struct *task, int regno, unsigned long data)
127	{	127	{
128	char *reg_ptr;	128	char *reg_ptr;
129		129
130	struct pt_regs *regs =	130	struct pt_regs *regs =
131	(struct pt_regs *)((unsigned long)task_stack_page(task) +	131	(struct pt_regs *)((unsigned long)task_stack_page(task) +
132	(THREAD_SIZE - sizeof(struct pt_regs)));	132	(THREAD_SIZE - sizeof(struct pt_regs)));
133	reg_ptr = (char *)regs;	133	reg_ptr = (char *)regs;
134		134
135	switch (regno) {	135	switch (regno) {
136	case PT_PC:	136	case PT_PC:
137	/*********************************************************************/	137	/*********************************************************************/
138	/* At this point the kernel is most likely in exception. */	138	/* At this point the kernel is most likely in exception. */
139	/* The RETX register will be used to populate the pc of the process. */	139	/* The RETX register will be used to populate the pc of the process. */
140	/*********************************************************************/	140	/*********************************************************************/
141	regs->retx = data;	141	regs->retx = data;
142	regs->pc = data;	142	regs->pc = data;
143	break;	143	break;
144	case PT_RETX:	144	case PT_RETX:
145	break; /* regs->retx = data; break; */	145	break; /* regs->retx = data; break; */
146	case PT_USP:	146	case PT_USP:
147	regs->usp = data;	147	regs->usp = data;
148	task->thread.usp = data;	148	task->thread.usp = data;
149	break;	149	break;
150	default:	150	default:
151	if (regno <= 216)	151	if (regno <= 216)
152	(long )(reg_ptr + regno) = data;	152	(long )(reg_ptr + regno) = data;
153	}	153	}
154	return 0;	154	return 0;
155	}	155	}
156		156
157	/*	157	/*
158	* check that an address falls within the bounds of the target process's memory mappings	158	* check that an address falls within the bounds of the target process's memory mappings
159	*/	159	*/
160	static inline int is_user_addr_valid(struct task_struct *child,	160	static inline int is_user_addr_valid(struct task_struct *child,
161	unsigned long start, unsigned long len)	161	unsigned long start, unsigned long len)
162	{	162	{
163	struct vm_list_struct *vml;	163	struct vm_area_struct *vma;
164	struct sram_list_struct *sraml;	164	struct sram_list_struct *sraml;
165		165
166	/* overflow */	166	/* overflow */
167	if (start + len < start)	167	if (start + len < start)
168	return -EIO;	168	return -EIO;
169		169
170	for (vml = child->mm->context.vmlist; vml; vml = vml->next)	170	vma = find_vma(child->mm, start);
171	if (start >= vml->vma->vm_start && start + len < vml->vma->vm_end)	171	if (vma && start >= vma->vm_start && start + len <= vma->vm_end)
172	return 0;	172	return 0;
173		173
174	for (sraml = child->mm->context.sram_list; sraml; sraml = sraml->next)	174	for (sraml = child->mm->context.sram_list; sraml; sraml = sraml->next)
175	if (start >= (unsigned long)sraml->addr	175	if (start >= (unsigned long)sraml->addr
176	&& start + len < (unsigned long)sraml->addr + sraml->length)	176	&& start + len < (unsigned long)sraml->addr + sraml->length)
177	return 0;	177	return 0;
178		178
179	if (start >= FIXED_CODE_START && start + len < FIXED_CODE_END)	179	if (start >= FIXED_CODE_START && start + len < FIXED_CODE_END)
180	return 0;	180	return 0;
181		181
182	return -EIO;	182	return -EIO;
183	}	183	}
184		184
185	void ptrace_enable(struct task_struct *child)	185	void ptrace_enable(struct task_struct *child)
186	{	186	{
187	unsigned long tmp;	187	unsigned long tmp;
188	tmp = get_reg(child, PT_SYSCFG) \| (TRACE_BITS);	188	tmp = get_reg(child, PT_SYSCFG) \| (TRACE_BITS);
189	put_reg(child, PT_SYSCFG, tmp);	189	put_reg(child, PT_SYSCFG, tmp);
190	}	190	}
191		191
192	/*	192	/*
193	* Called by kernel/ptrace.c when detaching..	193	* Called by kernel/ptrace.c when detaching..
194	*	194	*
195	* Make sure the single step bit is not set.	195	* Make sure the single step bit is not set.
196	*/	196	*/
197	void ptrace_disable(struct task_struct *child)	197	void ptrace_disable(struct task_struct *child)
198	{	198	{
199	unsigned long tmp;	199	unsigned long tmp;
200	/* make sure the single step bit is not set. */	200	/* make sure the single step bit is not set. */
201	tmp = get_reg(child, PT_SYSCFG) & ~TRACE_BITS;	201	tmp = get_reg(child, PT_SYSCFG) & ~TRACE_BITS;
202	put_reg(child, PT_SYSCFG, tmp);	202	put_reg(child, PT_SYSCFG, tmp);
203	}	203	}
204		204
205	long arch_ptrace(struct task_struct *child, long request, long addr, long data)	205	long arch_ptrace(struct task_struct *child, long request, long addr, long data)
206	{	206	{
207	int ret;	207	int ret;
208	unsigned long __user datap = (unsigned long __user )data;	208	unsigned long __user datap = (unsigned long __user )data;
209		209
210	switch (request) {	210	switch (request) {
211	/* when I and D space are separate, these will need to be fixed. */	211	/* when I and D space are separate, these will need to be fixed. */
212	case PTRACE_PEEKDATA:	212	case PTRACE_PEEKDATA:
213	pr_debug("ptrace: PEEKDATA\n");	213	pr_debug("ptrace: PEEKDATA\n");
214	/* fall through */	214	/* fall through */
215	case PTRACE_PEEKTEXT: /* read word at location addr. */	215	case PTRACE_PEEKTEXT: /* read word at location addr. */
216	{	216	{
217	unsigned long tmp = 0;	217	unsigned long tmp = 0;
218	int copied;	218	int copied;
219		219
220	ret = -EIO;	220	ret = -EIO;
221	pr_debug("ptrace: PEEKTEXT at addr 0x%08lx + %ld\n", addr, sizeof(data));	221	pr_debug("ptrace: PEEKTEXT at addr 0x%08lx + %ld\n", addr, sizeof(data));
222	if (is_user_addr_valid(child, addr, sizeof(tmp)) < 0)	222	if (is_user_addr_valid(child, addr, sizeof(tmp)) < 0)
223	break;	223	break;
224	pr_debug("ptrace: user address is valid\n");	224	pr_debug("ptrace: user address is valid\n");
225		225
226	if (L1_CODE_LENGTH != 0 && addr >= get_l1_code_start()	226	if (L1_CODE_LENGTH != 0 && addr >= get_l1_code_start()
227	&& addr + sizeof(tmp) <= get_l1_code_start() + L1_CODE_LENGTH) {	227	&& addr + sizeof(tmp) <= get_l1_code_start() + L1_CODE_LENGTH) {
228	safe_dma_memcpy (&tmp, (const void *)(addr), sizeof(tmp));	228	safe_dma_memcpy (&tmp, (const void *)(addr), sizeof(tmp));
229	copied = sizeof(tmp);	229	copied = sizeof(tmp);
230		230
231	} else if (L1_DATA_A_LENGTH != 0 && addr >= L1_DATA_A_START	231	} else if (L1_DATA_A_LENGTH != 0 && addr >= L1_DATA_A_START
232	&& addr + sizeof(tmp) <= L1_DATA_A_START + L1_DATA_A_LENGTH) {	232	&& addr + sizeof(tmp) <= L1_DATA_A_START + L1_DATA_A_LENGTH) {
233	memcpy(&tmp, (const void *)(addr), sizeof(tmp));	233	memcpy(&tmp, (const void *)(addr), sizeof(tmp));
234	copied = sizeof(tmp);	234	copied = sizeof(tmp);
235		235
236	} else if (L1_DATA_B_LENGTH != 0 && addr >= L1_DATA_B_START	236	} else if (L1_DATA_B_LENGTH != 0 && addr >= L1_DATA_B_START
237	&& addr + sizeof(tmp) <= L1_DATA_B_START + L1_DATA_B_LENGTH) {	237	&& addr + sizeof(tmp) <= L1_DATA_B_START + L1_DATA_B_LENGTH) {
238	memcpy(&tmp, (const void *)(addr), sizeof(tmp));	238	memcpy(&tmp, (const void *)(addr), sizeof(tmp));
239	copied = sizeof(tmp);	239	copied = sizeof(tmp);
240		240
241	} else if (addr >= FIXED_CODE_START	241	} else if (addr >= FIXED_CODE_START
242	&& addr + sizeof(tmp) <= FIXED_CODE_END) {	242	&& addr + sizeof(tmp) <= FIXED_CODE_END) {
243	memcpy(&tmp, (const void *)(addr), sizeof(tmp));	243	memcpy(&tmp, (const void *)(addr), sizeof(tmp));
244	copied = sizeof(tmp);	244	copied = sizeof(tmp);
245		245
246	} else	246	} else
247	copied = access_process_vm(child, addr, &tmp,	247	copied = access_process_vm(child, addr, &tmp,
248	sizeof(tmp), 0);	248	sizeof(tmp), 0);
249		249
250	pr_debug("ptrace: copied size %d [0x%08lx]\n", copied, tmp);	250	pr_debug("ptrace: copied size %d [0x%08lx]\n", copied, tmp);
251	if (copied != sizeof(tmp))	251	if (copied != sizeof(tmp))
252	break;	252	break;
253	ret = put_user(tmp, datap);	253	ret = put_user(tmp, datap);
254	break;	254	break;
255	}	255	}
256		256
257	/* read the word at location addr in the USER area. */	257	/* read the word at location addr in the USER area. */
258	case PTRACE_PEEKUSR:	258	case PTRACE_PEEKUSR:
259	{	259	{
260	unsigned long tmp;	260	unsigned long tmp;
261	ret = -EIO;	261	ret = -EIO;
262	tmp = 0;	262	tmp = 0;
263	if ((addr & 3) \|\| (addr > (sizeof(struct pt_regs) + 16))) {	263	if ((addr & 3) \|\| (addr > (sizeof(struct pt_regs) + 16))) {
264	printk(KERN_WARNING "ptrace error : PEEKUSR : temporarily returning "	264	printk(KERN_WARNING "ptrace error : PEEKUSR : temporarily returning "
265	"0 - %x sizeof(pt_regs) is %lx\n",	265	"0 - %x sizeof(pt_regs) is %lx\n",
266	(int)addr, sizeof(struct pt_regs));	266	(int)addr, sizeof(struct pt_regs));
267	break;	267	break;
268	}	268	}
269	if (addr == sizeof(struct pt_regs)) {	269	if (addr == sizeof(struct pt_regs)) {
270	/* PT_TEXT_ADDR */	270	/* PT_TEXT_ADDR */
271	tmp = child->mm->start_code + TEXT_OFFSET;	271	tmp = child->mm->start_code + TEXT_OFFSET;
272	} else if (addr == (sizeof(struct pt_regs) + 4)) {	272	} else if (addr == (sizeof(struct pt_regs) + 4)) {
273	/* PT_TEXT_END_ADDR */	273	/* PT_TEXT_END_ADDR */
274	tmp = child->mm->end_code;	274	tmp = child->mm->end_code;
275	} else if (addr == (sizeof(struct pt_regs) + 8)) {	275	} else if (addr == (sizeof(struct pt_regs) + 8)) {
276	/* PT_DATA_ADDR */	276	/* PT_DATA_ADDR */
277	tmp = child->mm->start_data;	277	tmp = child->mm->start_data;
278	#ifdef CONFIG_BINFMT_ELF_FDPIC	278	#ifdef CONFIG_BINFMT_ELF_FDPIC
279	} else if (addr == (sizeof(struct pt_regs) + 12)) {	279	} else if (addr == (sizeof(struct pt_regs) + 12)) {
280	tmp = child->mm->context.exec_fdpic_loadmap;	280	tmp = child->mm->context.exec_fdpic_loadmap;
281	} else if (addr == (sizeof(struct pt_regs) + 16)) {	281	} else if (addr == (sizeof(struct pt_regs) + 16)) {
282	tmp = child->mm->context.interp_fdpic_loadmap;	282	tmp = child->mm->context.interp_fdpic_loadmap;
283	#endif	283	#endif
284	} else {	284	} else {
285	tmp = get_reg(child, addr);	285	tmp = get_reg(child, addr);
286	}	286	}
287	ret = put_user(tmp, datap);	287	ret = put_user(tmp, datap);
288	break;	288	break;
289	}	289	}
290		290
291	/* when I and D space are separate, this will have to be fixed. */	291	/* when I and D space are separate, this will have to be fixed. */
292	case PTRACE_POKEDATA:	292	case PTRACE_POKEDATA:
293	pr_debug("ptrace: PTRACE_PEEKDATA\n");	293	pr_debug("ptrace: PTRACE_PEEKDATA\n");
294	/* fall through */	294	/* fall through */
295	case PTRACE_POKETEXT: /* write the word at location addr. */	295	case PTRACE_POKETEXT: /* write the word at location addr. */
296	{	296	{
297	int copied;	297	int copied;
298		298
299	ret = -EIO;	299	ret = -EIO;
300	pr_debug("ptrace: POKETEXT at addr 0x%08lx + %ld bytes %lx\n",	300	pr_debug("ptrace: POKETEXT at addr 0x%08lx + %ld bytes %lx\n",
301	addr, sizeof(data), data);	301	addr, sizeof(data), data);
302	if (is_user_addr_valid(child, addr, sizeof(data)) < 0)	302	if (is_user_addr_valid(child, addr, sizeof(data)) < 0)
303	break;	303	break;
304	pr_debug("ptrace: user address is valid\n");	304	pr_debug("ptrace: user address is valid\n");
305		305
306	if (L1_CODE_LENGTH != 0 && addr >= get_l1_code_start()	306	if (L1_CODE_LENGTH != 0 && addr >= get_l1_code_start()
307	&& addr + sizeof(data) <= get_l1_code_start() + L1_CODE_LENGTH) {	307	&& addr + sizeof(data) <= get_l1_code_start() + L1_CODE_LENGTH) {
308	safe_dma_memcpy ((void *)(addr), &data, sizeof(data));	308	safe_dma_memcpy ((void *)(addr), &data, sizeof(data));
309	copied = sizeof(data);	309	copied = sizeof(data);
310		310
311	} else if (L1_DATA_A_LENGTH != 0 && addr >= L1_DATA_A_START	311	} else if (L1_DATA_A_LENGTH != 0 && addr >= L1_DATA_A_START
312	&& addr + sizeof(data) <= L1_DATA_A_START + L1_DATA_A_LENGTH) {	312	&& addr + sizeof(data) <= L1_DATA_A_START + L1_DATA_A_LENGTH) {
313	memcpy((void *)(addr), &data, sizeof(data));	313	memcpy((void *)(addr), &data, sizeof(data));
314	copied = sizeof(data);	314	copied = sizeof(data);
315		315
316	} else if (L1_DATA_B_LENGTH != 0 && addr >= L1_DATA_B_START	316	} else if (L1_DATA_B_LENGTH != 0 && addr >= L1_DATA_B_START
317	&& addr + sizeof(data) <= L1_DATA_B_START + L1_DATA_B_LENGTH) {	317	&& addr + sizeof(data) <= L1_DATA_B_START + L1_DATA_B_LENGTH) {
318	memcpy((void *)(addr), &data, sizeof(data));	318	memcpy((void *)(addr), &data, sizeof(data));
319	copied = sizeof(data);	319	copied = sizeof(data);
320		320
321	} else if (addr >= FIXED_CODE_START	321	} else if (addr >= FIXED_CODE_START
322	&& addr + sizeof(data) <= FIXED_CODE_END) {	322	&& addr + sizeof(data) <= FIXED_CODE_END) {
323	memcpy((void *)(addr), &data, sizeof(data));	323	memcpy((void *)(addr), &data, sizeof(data));
324	copied = sizeof(data);	324	copied = sizeof(data);
325		325
326	} else	326	} else
327	copied = access_process_vm(child, addr, &data,	327	copied = access_process_vm(child, addr, &data,
328	sizeof(data), 1);	328	sizeof(data), 1);
329		329
330	pr_debug("ptrace: copied size %d\n", copied);	330	pr_debug("ptrace: copied size %d\n", copied);
331	if (copied != sizeof(data))	331	if (copied != sizeof(data))
332	break;	332	break;
333	ret = 0;	333	ret = 0;
334	break;	334	break;
335	}	335	}
336		336
337	case PTRACE_POKEUSR: /* write the word at location addr in the USER area */	337	case PTRACE_POKEUSR: /* write the word at location addr in the USER area */
338	ret = -EIO;	338	ret = -EIO;
339	if ((addr & 3) \|\| (addr > (sizeof(struct pt_regs) + 16))) {	339	if ((addr & 3) \|\| (addr > (sizeof(struct pt_regs) + 16))) {
340	printk(KERN_WARNING "ptrace error : POKEUSR: temporarily returning 0\n");	340	printk(KERN_WARNING "ptrace error : POKEUSR: temporarily returning 0\n");
341	break;	341	break;
342	}	342	}
343		343
344	if (addr >= (sizeof(struct pt_regs))) {	344	if (addr >= (sizeof(struct pt_regs))) {
345	ret = 0;	345	ret = 0;
346	break;	346	break;
347	}	347	}
348	if (addr == PT_SYSCFG) {	348	if (addr == PT_SYSCFG) {
349	data &= SYSCFG_MASK;	349	data &= SYSCFG_MASK;
350	data \|= get_reg(child, PT_SYSCFG);	350	data \|= get_reg(child, PT_SYSCFG);
351	}	351	}
352	ret = put_reg(child, addr, data);	352	ret = put_reg(child, addr, data);
353	break;	353	break;
354		354
355	case PTRACE_SYSCALL: /* continue and stop at next (return from) syscall */	355	case PTRACE_SYSCALL: /* continue and stop at next (return from) syscall */
356	case PTRACE_CONT: /* restart after signal. */	356	case PTRACE_CONT: /* restart after signal. */
357	pr_debug("ptrace: syscall/cont\n");	357	pr_debug("ptrace: syscall/cont\n");
358		358
359	ret = -EIO;	359	ret = -EIO;
360	if (!valid_signal(data))	360	if (!valid_signal(data))
361	break;	361	break;
362	if (request == PTRACE_SYSCALL)	362	if (request == PTRACE_SYSCALL)
363	set_tsk_thread_flag(child, TIF_SYSCALL_TRACE);	363	set_tsk_thread_flag(child, TIF_SYSCALL_TRACE);
364	else	364	else
365	clear_tsk_thread_flag(child, TIF_SYSCALL_TRACE);	365	clear_tsk_thread_flag(child, TIF_SYSCALL_TRACE);
366	child->exit_code = data;	366	child->exit_code = data;
367	ptrace_disable(child);	367	ptrace_disable(child);
368	pr_debug("ptrace: before wake_up_process\n");	368	pr_debug("ptrace: before wake_up_process\n");
369	wake_up_process(child);	369	wake_up_process(child);
370	ret = 0;	370	ret = 0;
371	break;	371	break;
372		372
373	/*	373	/*
374	* make the child exit. Best I can do is send it a sigkill.	374	* make the child exit. Best I can do is send it a sigkill.
375	* perhaps it should be put in the status that it wants to	375	* perhaps it should be put in the status that it wants to
376	* exit.	376	* exit.
377	*/	377	*/
378	case PTRACE_KILL:	378	case PTRACE_KILL:
379	ret = 0;	379	ret = 0;
380	if (child->exit_state == EXIT_ZOMBIE) /* already dead */	380	if (child->exit_state == EXIT_ZOMBIE) /* already dead */
381	break;	381	break;
382	child->exit_code = SIGKILL;	382	child->exit_code = SIGKILL;
383	ptrace_disable(child);	383	ptrace_disable(child);
384	wake_up_process(child);	384	wake_up_process(child);
385	break;	385	break;
386		386
387	case PTRACE_SINGLESTEP: /* set the trap flag. */	387	case PTRACE_SINGLESTEP: /* set the trap flag. */
388	pr_debug("ptrace: single step\n");	388	pr_debug("ptrace: single step\n");
389	ret = -EIO;	389	ret = -EIO;
390	if (!valid_signal(data))	390	if (!valid_signal(data))
391	break;	391	break;
392	clear_tsk_thread_flag(child, TIF_SYSCALL_TRACE);	392	clear_tsk_thread_flag(child, TIF_SYSCALL_TRACE);
393	ptrace_enable(child);	393	ptrace_enable(child);
394	child->exit_code = data;	394	child->exit_code = data;
395	wake_up_process(child);	395	wake_up_process(child);
396	ret = 0;	396	ret = 0;
397	break;	397	break;
398		398
399	case PTRACE_GETREGS:	399	case PTRACE_GETREGS:
400	/* Get all gp regs from the child. */	400	/* Get all gp regs from the child. */
401	ret = ptrace_getregs(child, datap);	401	ret = ptrace_getregs(child, datap);
402	break;	402	break;
403		403
404	case PTRACE_SETREGS:	404	case PTRACE_SETREGS:
405	printk(KERN_WARNING "ptrace: SETREGS: ** NOT IMPLEMENTED *\n");	405	printk(KERN_WARNING "ptrace: SETREGS: ** NOT IMPLEMENTED *\n");
406	/* Set all gp regs in the child. */	406	/* Set all gp regs in the child. */
407	ret = 0;	407	ret = 0;
408	break;	408	break;
409		409
410	default:	410	default:
411	ret = ptrace_request(child, request, addr, data);	411	ret = ptrace_request(child, request, addr, data);
412	break;	412	break;
413	}	413	}
414		414
415	return ret;	415	return ret;
416	}	416	}
417		417
418	asmlinkage void syscall_trace(void)	418	asmlinkage void syscall_trace(void)
419	{	419	{
420	if (!test_thread_flag(TIF_SYSCALL_TRACE))	420	if (!test_thread_flag(TIF_SYSCALL_TRACE))
421	return;	421	return;
422		422
423	if (!(current->ptrace & PT_PTRACED))	423	if (!(current->ptrace & PT_PTRACED))
424	return;	424	return;
425		425
426	/* the 0x80 provides a way for the tracing parent to distinguish	426	/* the 0x80 provides a way for the tracing parent to distinguish
427	* between a syscall stop and SIGTRAP delivery	427	* between a syscall stop and SIGTRAP delivery
428	*/	428	*/
429	ptrace_notify(SIGTRAP \| ((current->ptrace & PT_TRACESYSGOOD)	429	ptrace_notify(SIGTRAP \| ((current->ptrace & PT_TRACESYSGOOD)
430	? 0x80 : 0));	430	? 0x80 : 0));
431		431
432	/*	432	/*
433	* this isn't the same as continuing with a signal, but it will do	433	* this isn't the same as continuing with a signal, but it will do
434	* for normal use. strace only continues with a signal if the	434	* for normal use. strace only continues with a signal if the
435	* stopping signal is not SIGTRAP. -brl	435	* stopping signal is not SIGTRAP. -brl
436	*/	436	*/
437	if (current->exit_code) {	437	if (current->exit_code) {
438	send_sig(current->exit_code, current, 1);	438	send_sig(current->exit_code, current, 1);
439	current->exit_code = 0;	439	current->exit_code = 0;
440	}	440	}
441	}	441	}
442		442

arch/blackfin/kernel/traps.c

Diff comments View file @ c40f6f8

 /*
  * File:         arch/blackfin/kernel/traps.c
  * Based on:
  * Author:       Hamish Macdonald
  *
  * Created:
  * Description:  uses S/W interrupt 15 for the system calls
  *
  * Modified:
  *               Copyright 2004-2006 Analog Devices Inc.
  *
  * Bugs:         Enter bugs at http://blackfin.uclinux.org/
  *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License as published by
  * the Free Software Foundation; either version 2 of the License, or
  * (at your option) any later version.
  *
  * This program is distributed in the hope that it will be useful,
  * but WITHOUT ANY WARRANTY; without even the implied warranty of
  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  * GNU General Public License for more details.
  *
  * You should have received a copy of the GNU General Public License
  * along with this program; if not, see the file COPYING, or write
  * to the Free Software Foundation, Inc.,
  * 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
  */
 #include <linux/uaccess.h>
 #include <linux/interrupt.h>
 #include <linux/module.h>
 #include <linux/kallsyms.h>
 #include <linux/fs.h>
+#include <linux/rbtree.h>
 #include <asm/traps.h>
 #include <asm/cacheflush.h>
 #include <asm/cplb.h>
 #include <asm/blackfin.h>
 #include <asm/irq_handler.h>
 #include <linux/irq.h>
 #include <asm/trace.h>
 #include <asm/fixed_code.h>
 #ifdef CONFIG_KGDB
 # include <linux/kgdb.h>
 # define CHK_DEBUGGER_TRAP() \
 	do { \
 		kgdb_handle_exception(trapnr, sig, info.si_code, fp); \
 	} while (0)
 # define CHK_DEBUGGER_TRAP_MAYBE() \
 	do { \
 		if (kgdb_connected) \
 			CHK_DEBUGGER_TRAP(); \
 	} while (0)
 #else
 # define CHK_DEBUGGER_TRAP() do { } while (0)
 # define CHK_DEBUGGER_TRAP_MAYBE() do { } while (0)
 #endif
 #ifdef CONFIG_DEBUG_VERBOSE
 #define verbose_printk(fmt, arg...) \
 	printk(fmt, ##arg)
 #else
 #define verbose_printk(fmt, arg...) \
 	({ if (0) printk(fmt, ##arg); 0; })
 #endif
 /* Initiate the event table handler */
 void __init trap_init(void)
 {
 	CSYNC();
 	bfin_write_EVT3(trap);
 	CSYNC();
 }
 static void decode_address(char *buf, unsigned long address)
 {
 #ifdef CONFIG_DEBUG_VERBOSE
 	struct vm_list_struct *vml;
 	struct task_struct *p;
 	struct mm_struct *mm;
 	unsigned long flags, offset;
 	unsigned char in_atomic = (bfin_read_IPEND() & 0x10) || in_atomic();
+	struct rb_node *n;
 #ifdef CONFIG_KALLSYMS
 	unsigned long symsize;
 	const char *symname;
 	char *modname;
 	char *delim = ":";
 	char namebuf[128];
 	/* look up the address and see if we are in kernel space */
 	symname = kallsyms_lookup(address, &symsize, &offset, &modname, namebuf);
 	if (symname) {
 		/* yeah! kernel space! */
 		if (!modname)
 			modname = delim = "";
 		sprintf(buf, "<0x%p> { %s%s%s%s + 0x%lx }",
 		              (void *)address, delim, modname, delim, symname,
 		              (unsigned long)offset);
 		return;
 	}
 #endif
 	/* Problem in fixed code section? */
 	if (address >= FIXED_CODE_START && address < FIXED_CODE_END) {
 		sprintf(buf, "<0x%p> /* Maybe fixed code section */", (void *)address);
 		return;
 	}
 	/* Problem somewhere before the kernel start address */
 	if (address < CONFIG_BOOT_LOAD) {
 		sprintf(buf, "<0x%p> /* Maybe null pointer? */", (void *)address);
 		return;
 	}
 	/* looks like we're off in user-land, so let's walk all the
 	 * mappings of all our processes and see if we can't be a whee
 	 * bit more specific
 	 */
 	write_lock_irqsave(&tasklist_lock, flags);
 	for_each_process(p) {
 		mm = (in_atomic ? p->mm : get_task_mm(p));
 		if (!mm)
 			continue;
-		vml = mm->context.vmlist;
+		for (n = rb_first(&mm->mm_rb); n; n = rb_next(n)) {
-		while (vml) {
+			struct vm_area_struct *vma;
-			struct vm_area_struct *vma = vml->vma;
+			vma = rb_entry(n, struct vm_area_struct, vm_rb);
 			if (address >= vma->vm_start && address < vma->vm_end) {
 				char _tmpbuf[256];
 				char *name = p->comm;
 				struct file *file = vma->vm_file;
 				if (file) {
 					char *d_name = d_path(&file->f_path, _tmpbuf,
 						      sizeof(_tmpbuf));
 					if (!IS_ERR(d_name))
 						name = d_name;
 				}
 				/* FLAT does not have its text aligned to the start of
 				 * the map while FDPIC ELF does ...
 				 */
 				/* before we can check flat/fdpic, we need to
 				 * make sure current is valid
 				 */
 				if ((unsigned long)current >= FIXED_CODE_START &&
 				    !((unsigned long)current & 0x3)) {
 					if (current->mm &&
 					    (address > current->mm->start_code) &&
 					    (address < current->mm->end_code))
 						offset = address - current->mm->start_code;
 					else
 						offset = (address - vma->vm_start) +
 							 (vma->vm_pgoff << PAGE_SHIFT);
 					sprintf(buf, "<0x%p> [ %s + 0x%lx ]",
 						(void *)address, name, offset);
 				} else
 					sprintf(buf, "<0x%p> [ %s vma:0x%lx-0x%lx]",
 						(void *)address, name,
 						vma->vm_start, vma->vm_end);
 				if (!in_atomic)
 					mmput(mm);
 				if (!strlen(buf))
 					sprintf(buf, "<0x%p> [ %s ] dynamic memory", (void *)address, name);
 				goto done;
 			}
-			vml = vml->next;
 		}
 		if (!in_atomic)
 			mmput(mm);
 	}
 	/* we were unable to find this address anywhere */
 	sprintf(buf, "<0x%p> /* kernel dynamic memory */", (void *)address);
 done:
 	write_unlock_irqrestore(&tasklist_lock, flags);
 #else
 	sprintf(buf, " ");
 #endif
 }
 asmlinkage void double_fault_c(struct pt_regs *fp)
 {
 	console_verbose();
 	oops_in_progress = 1;
 #ifdef CONFIG_DEBUG_VERBOSE
 	printk(KERN_EMERG "\n" KERN_EMERG "Double Fault\n");
 #ifdef CONFIG_DEBUG_DOUBLEFAULT_PRINT
 	if (((long)fp->seqstat &  SEQSTAT_EXCAUSE) == VEC_UNCOV) {
 		unsigned int cpu = smp_processor_id();
 		char buf[150];
 		decode_address(buf, cpu_pda[cpu].retx);
 		printk(KERN_EMERG "While handling exception (EXCAUSE = 0x%x) at %s:\n",
 			(unsigned int)cpu_pda[cpu].seqstat & SEQSTAT_EXCAUSE, buf);
 		decode_address(buf, cpu_pda[cpu].dcplb_fault_addr);
 		printk(KERN_NOTICE "   DCPLB_FAULT_ADDR: %s\n", buf);
 		decode_address(buf, cpu_pda[cpu].icplb_fault_addr);
 		printk(KERN_NOTICE "   ICPLB_FAULT_ADDR: %s\n", buf);
 		decode_address(buf, fp->retx);
 		printk(KERN_NOTICE "The instruction at %s caused a double exception\n", buf);
 	} else
 #endif
 	{
 		dump_bfin_process(fp);
 		dump_bfin_mem(fp);
 		show_regs(fp);
 	}
 #endif
 	panic("Double Fault - unrecoverable event\n");
 }
 asmlinkage void trap_c(struct pt_regs *fp)
 {
 #ifdef CONFIG_DEBUG_BFIN_HWTRACE_ON
 	int j;
 #endif
 #ifdef CONFIG_DEBUG_HUNT_FOR_ZERO
 	unsigned int cpu = smp_processor_id();
 #endif
 	int sig = 0;
 	siginfo_t info;
 	unsigned long trapnr = fp->seqstat & SEQSTAT_EXCAUSE;
 	trace_buffer_save(j);
 	/* Important - be very careful dereferncing pointers - will lead to
 	 * double faults if the stack has become corrupt
 	 */
 	/* If the fault was caused by a kernel thread, or interrupt handler
 	 * we will kernel panic, so the system reboots.
 	 * If KGDB is enabled, don't set this for kernel breakpoints
 	*/
 	/* TODO: check to see if we are in some sort of deferred HWERR
 	 * that we should be able to recover from, not kernel panic
 	 */
 	if ((bfin_read_IPEND() & 0xFFC0) && (trapnr != VEC_STEP)
 #ifdef CONFIG_KGDB
 		&& (trapnr != VEC_EXCPT02)
 #endif
 	){
 		console_verbose();
 		oops_in_progress = 1;
 	} else if (current) {
 		if (current->mm == NULL) {
 			console_verbose();
 			oops_in_progress = 1;
 		}
 	}
 	/* trap_c() will be called for exceptions. During exceptions
 	 * processing, the pc value should be set with retx value.
 	 * With this change we can cleanup some code in signal.c- TODO
 	 */
 	fp->orig_pc = fp->retx;
 	/* printk("exception: 0x%x, ipend=%x, reti=%x, retx=%x\n",
 		trapnr, fp->ipend, fp->pc, fp->retx); */
 	/* send the appropriate signal to the user program */
 	switch (trapnr) {
 	/* This table works in conjuction with the one in ./mach-common/entry.S
 	 * Some exceptions are handled there (in assembly, in exception space)
 	 * Some are handled here, (in C, in interrupt space)
 	 * Some, like CPLB, are handled in both, where the normal path is
 	 * handled in assembly/exception space, and the error path is handled
 	 * here
 	 */
 	/* 0x00 - Linux Syscall, getting here is an error */
 	/* 0x01 - userspace gdb breakpoint, handled here */
 	case VEC_EXCPT01:
 		info.si_code = TRAP_ILLTRAP;
 		sig = SIGTRAP;
 		CHK_DEBUGGER_TRAP_MAYBE();
 		/* Check if this is a breakpoint in kernel space */
 		if (fp->ipend & 0xffc0)
 			return;
 		else
 			break;
 	/* 0x03 - User Defined, userspace stack overflow */
 	case VEC_EXCPT03:
 		info.si_code = SEGV_STACKFLOW;
 		sig = SIGSEGV;
 		verbose_printk(KERN_NOTICE EXC_0x03(KERN_NOTICE));
 		CHK_DEBUGGER_TRAP_MAYBE();
 		break;
 	/* 0x02 - KGDB initial connection and break signal trap */
 	case VEC_EXCPT02:
 #ifdef CONFIG_KGDB
 		info.si_code = TRAP_ILLTRAP;
 		sig = SIGTRAP;
 		CHK_DEBUGGER_TRAP();
 		return;
 #endif
 	/* 0x04 - User Defined */
 	/* 0x05 - User Defined */
 	/* 0x06 - User Defined */
 	/* 0x07 - User Defined */
 	/* 0x08 - User Defined */
 	/* 0x09 - User Defined */
 	/* 0x0A - User Defined */
 	/* 0x0B - User Defined */
 	/* 0x0C - User Defined */
 	/* 0x0D - User Defined */
 	/* 0x0E - User Defined */
 	/* 0x0F - User Defined */
 	/* If we got here, it is most likely that someone was trying to use a
 	 * custom exception handler, and it is not actually installed properly
 	 */
 	case VEC_EXCPT04 ... VEC_EXCPT15:
 		info.si_code = ILL_ILLPARAOP;
 		sig = SIGILL;
 		verbose_printk(KERN_NOTICE EXC_0x04(KERN_NOTICE));
 		CHK_DEBUGGER_TRAP_MAYBE();
 		break;
 	/* 0x10 HW Single step, handled here */
 	case VEC_STEP:
 		info.si_code = TRAP_STEP;
 		sig = SIGTRAP;
 		CHK_DEBUGGER_TRAP_MAYBE();
 		/* Check if this is a single step in kernel space */
 		if (fp->ipend & 0xffc0)
 			return;
 		else
 			break;
 	/* 0x11 - Trace Buffer Full, handled here */
 	case VEC_OVFLOW:
 		info.si_code = TRAP_TRACEFLOW;
 		sig = SIGTRAP;
 		verbose_printk(KERN_NOTICE EXC_0x11(KERN_NOTICE));
 		CHK_DEBUGGER_TRAP_MAYBE();
 		break;
 	/* 0x12 - Reserved, Caught by default */
 	/* 0x13 - Reserved, Caught by default */
 	/* 0x14 - Reserved, Caught by default */
 	/* 0x15 - Reserved, Caught by default */
 	/* 0x16 - Reserved, Caught by default */
 	/* 0x17 - Reserved, Caught by default */
 	/* 0x18 - Reserved, Caught by default */
 	/* 0x19 - Reserved, Caught by default */
 	/* 0x1A - Reserved, Caught by default */
 	/* 0x1B - Reserved, Caught by default */
 	/* 0x1C - Reserved, Caught by default */
 	/* 0x1D - Reserved, Caught by default */
 	/* 0x1E - Reserved, Caught by default */
 	/* 0x1F - Reserved, Caught by default */
 	/* 0x20 - Reserved, Caught by default */
 	/* 0x21 - Undefined Instruction, handled here */
 	case VEC_UNDEF_I:
 		info.si_code = ILL_ILLOPC;
 		sig = SIGILL;
 		verbose_printk(KERN_NOTICE EXC_0x21(KERN_NOTICE));
 		CHK_DEBUGGER_TRAP_MAYBE();
 		break;
 	/* 0x22 - Illegal Instruction Combination, handled here */
 	case VEC_ILGAL_I:
 		info.si_code = ILL_ILLPARAOP;
 		sig = SIGILL;
 		verbose_printk(KERN_NOTICE EXC_0x22(KERN_NOTICE));
 		CHK_DEBUGGER_TRAP_MAYBE();
 		break;
 	/* 0x23 - Data CPLB protection violation, handled here */
 	case VEC_CPLB_VL:
 		info.si_code = ILL_CPLB_VI;
 		sig = SIGBUS;
 		verbose_printk(KERN_NOTICE EXC_0x23(KERN_NOTICE));
 		CHK_DEBUGGER_TRAP_MAYBE();
 		break;
 	/* 0x24 - Data access misaligned, handled here */
 	case VEC_MISALI_D:
 		info.si_code = BUS_ADRALN;
 		sig = SIGBUS;
 		verbose_printk(KERN_NOTICE EXC_0x24(KERN_NOTICE));
 		CHK_DEBUGGER_TRAP_MAYBE();
 		break;
 	/* 0x25 - Unrecoverable Event, handled here */
 	case VEC_UNCOV:
 		info.si_code = ILL_ILLEXCPT;
 		sig = SIGILL;
 		verbose_printk(KERN_NOTICE EXC_0x25(KERN_NOTICE));
 		CHK_DEBUGGER_TRAP_MAYBE();
 		break;
 	/* 0x26 - Data CPLB Miss, normal case is handled in _cplb_hdr,
 		error case is handled here */
 	case VEC_CPLB_M:
 		info.si_code = BUS_ADRALN;
 		sig = SIGBUS;
 		verbose_printk(KERN_NOTICE EXC_0x26(KERN_NOTICE));
 		break;
 	/* 0x27 - Data CPLB Multiple Hits - Linux Trap Zero, handled here */
 	case VEC_CPLB_MHIT:
 		info.si_code = ILL_CPLB_MULHIT;
 		sig = SIGSEGV;
 #ifdef CONFIG_DEBUG_HUNT_FOR_ZERO
 		if (cpu_pda[cpu].dcplb_fault_addr < FIXED_CODE_START)
 			verbose_printk(KERN_NOTICE "NULL pointer access\n");
 		else
 #endif
 			verbose_printk(KERN_NOTICE EXC_0x27(KERN_NOTICE));
 		CHK_DEBUGGER_TRAP_MAYBE();
 		break;
 	/* 0x28 - Emulation Watchpoint, handled here */
 	case VEC_WATCH:
 		info.si_code = TRAP_WATCHPT;
 		sig = SIGTRAP;
 		pr_debug(EXC_0x28(KERN_DEBUG));
 		CHK_DEBUGGER_TRAP_MAYBE();
 		/* Check if this is a watchpoint in kernel space */
 		if (fp->ipend & 0xffc0)
 			return;
 		else
 			break;
 #ifdef CONFIG_BF535
 	/* 0x29 - Instruction fetch access error (535 only) */
 	case VEC_ISTRU_VL:      /* ADSP-BF535 only (MH) */
 		info.si_code = BUS_OPFETCH;
 		sig = SIGBUS;
 		verbose_printk(KERN_NOTICE "BF535: VEC_ISTRU_VL\n");
 		CHK_DEBUGGER_TRAP_MAYBE();
 		break;
 #else
 	/* 0x29 - Reserved, Caught by default */
 #endif
 	/* 0x2A - Instruction fetch misaligned, handled here */
 	case VEC_MISALI_I:
 		info.si_code = BUS_ADRALN;
 		sig = SIGBUS;
 		verbose_printk(KERN_NOTICE EXC_0x2A(KERN_NOTICE));
 		CHK_DEBUGGER_TRAP_MAYBE();
 		break;
 	/* 0x2B - Instruction CPLB protection violation, handled here */
 	case VEC_CPLB_I_VL:
 		info.si_code = ILL_CPLB_VI;
 		sig = SIGBUS;
 		verbose_printk(KERN_NOTICE EXC_0x2B(KERN_NOTICE));
 		CHK_DEBUGGER_TRAP_MAYBE();
 		break;
 	/* 0x2C - Instruction CPLB miss, handled in _cplb_hdr */
 	case VEC_CPLB_I_M:
 		info.si_code = ILL_CPLB_MISS;
 		sig = SIGBUS;
 		verbose_printk(KERN_NOTICE EXC_0x2C(KERN_NOTICE));
 		break;
 	/* 0x2D - Instruction CPLB Multiple Hits, handled here */
 	case VEC_CPLB_I_MHIT:
 		info.si_code = ILL_CPLB_MULHIT;
 		sig = SIGSEGV;
 #ifdef CONFIG_DEBUG_HUNT_FOR_ZERO
 		if (cpu_pda[cpu].icplb_fault_addr < FIXED_CODE_START)
 			verbose_printk(KERN_NOTICE "Jump to NULL address\n");
 		else
 #endif
 			verbose_printk(KERN_NOTICE EXC_0x2D(KERN_NOTICE));
 		CHK_DEBUGGER_TRAP_MAYBE();
 		break;
 	/* 0x2E - Illegal use of Supervisor Resource, handled here */
 	case VEC_ILL_RES:
 		info.si_code = ILL_PRVOPC;
 		sig = SIGILL;
 		verbose_printk(KERN_NOTICE EXC_0x2E(KERN_NOTICE));
 		CHK_DEBUGGER_TRAP_MAYBE();
 		break;
 	/* 0x2F - Reserved, Caught by default */
 	/* 0x30 - Reserved, Caught by default */
 	/* 0x31 - Reserved, Caught by default */
 	/* 0x32 - Reserved, Caught by default */
 	/* 0x33 - Reserved, Caught by default */
 	/* 0x34 - Reserved, Caught by default */
 	/* 0x35 - Reserved, Caught by default */
 	/* 0x36 - Reserved, Caught by default */
 	/* 0x37 - Reserved, Caught by default */
 	/* 0x38 - Reserved, Caught by default */
 	/* 0x39 - Reserved, Caught by default */
 	/* 0x3A - Reserved, Caught by default */
 	/* 0x3B - Reserved, Caught by default */
 	/* 0x3C - Reserved, Caught by default */
 	/* 0x3D - Reserved, Caught by default */
 	/* 0x3E - Reserved, Caught by default */
 	/* 0x3F - Reserved, Caught by default */
 	case VEC_HWERR:
 		info.si_code = BUS_ADRALN;
 		sig = SIGBUS;
 		switch (fp->seqstat & SEQSTAT_HWERRCAUSE) {
 		/* System MMR Error */
 		case (SEQSTAT_HWERRCAUSE_SYSTEM_MMR):
 			info.si_code = BUS_ADRALN;
 			sig = SIGBUS;
 			verbose_printk(KERN_NOTICE HWC_x2(KERN_NOTICE));
 			break;
 		/* External Memory Addressing Error */
 		case (SEQSTAT_HWERRCAUSE_EXTERN_ADDR):
 			info.si_code = BUS_ADRERR;
 			sig = SIGBUS;
 			verbose_printk(KERN_NOTICE HWC_x3(KERN_NOTICE));
 			break;
 		/* Performance Monitor Overflow */
 		case (SEQSTAT_HWERRCAUSE_PERF_FLOW):
 			verbose_printk(KERN_NOTICE HWC_x12(KERN_NOTICE));
 			break;
 		/* RAISE 5 instruction */
 		case (SEQSTAT_HWERRCAUSE_RAISE_5):
 			printk(KERN_NOTICE HWC_x18(KERN_NOTICE));
 			break;
 		default:        /* Reserved */
 			printk(KERN_NOTICE HWC_default(KERN_NOTICE));
 			break;
 		}
 		CHK_DEBUGGER_TRAP_MAYBE();
 		break;
 	/*
 	 * We should be handling all known exception types above,
 	 * if we get here we hit a reserved one, so panic
 	 */
 	default:
 		oops_in_progress = 1;
 		info.si_code = ILL_ILLPARAOP;
 		sig = SIGILL;
 		verbose_printk(KERN_EMERG "Caught Unhandled Exception, code = %08lx\n",
 			(fp->seqstat & SEQSTAT_EXCAUSE));
 		CHK_DEBUGGER_TRAP_MAYBE();
 		break;
 	}
 	BUG_ON(sig == 0);
 	if (sig != SIGTRAP) {
 		dump_bfin_process(fp);
 		dump_bfin_mem(fp);
 		show_regs(fp);
 		/* Print out the trace buffer if it makes sense */
 #ifndef CONFIG_DEBUG_BFIN_NO_KERN_HWTRACE
 		if (trapnr == VEC_CPLB_I_M || trapnr == VEC_CPLB_M)
 			verbose_printk(KERN_NOTICE "No trace since you do not have "
 				"CONFIG_DEBUG_BFIN_NO_KERN_HWTRACE enabled\n"
 				KERN_NOTICE "\n");
 		else
 #endif
 			dump_bfin_trace_buffer();
 		if (oops_in_progress) {
 			/* Dump the current kernel stack */
 			verbose_printk(KERN_NOTICE "\n" KERN_NOTICE "Kernel Stack\n");
 			show_stack(current, NULL);
 			print_modules();
 #ifndef CONFIG_ACCESS_CHECK
 			verbose_printk(KERN_EMERG "Please turn on "
 			       "CONFIG_ACCESS_CHECK\n");
 #endif
 			panic("Kernel exception");
 		} else {
 #ifdef CONFIG_DEBUG_VERBOSE
 			unsigned long *stack;
 			/* Dump the user space stack */
 			stack = (unsigned long *)rdusp();
 			verbose_printk(KERN_NOTICE "Userspace Stack\n");
 			show_stack(NULL, stack);
 #endif
 		}
 	}
 #ifdef CONFIG_IPIPE
 	if (!ipipe_trap_notify(fp->seqstat & 0x3f, fp))
 #endif
 	{
 		info.si_signo = sig;
 		info.si_errno = 0;
 		info.si_addr = (void __user *)fp->pc;
 		force_sig_info(sig, &info, current);
 	}
 	trace_buffer_restore(j);
 	return;
 }
 /* Typical exception handling routines	*/
 #define EXPAND_LEN ((1 << CONFIG_DEBUG_BFIN_HWTRACE_EXPAND_LEN) * 256 - 1)
 /*
  * Similar to get_user, do some address checking, then dereference
  * Return true on sucess, false on bad address
  */
 static bool get_instruction(unsigned short *val, unsigned short *address)
 {
 	unsigned long addr;
 	addr = (unsigned long)address;
 	/* Check for odd addresses */
 	if (addr & 0x1)
 		return false;
 	/* Check that things do not wrap around */
 	if (addr > (addr + 2))
 		return false;
 	/*
 	 * Since we are in exception context, we need to do a little address checking
 	 * We need to make sure we are only accessing valid memory, and
 	 * we don't read something in the async space that can hang forever
 	 */
 	if ((addr >= FIXED_CODE_START && (addr + 2) <= physical_mem_end) ||
 #if L2_LENGTH != 0
 	    (addr >= L2_START && (addr + 2) <= (L2_START + L2_LENGTH)) ||
 #endif
 	    (addr >= BOOT_ROM_START && (addr + 2) <= (BOOT_ROM_START + BOOT_ROM_LENGTH)) ||
 #if L1_DATA_A_LENGTH != 0
 	    (addr >= L1_DATA_A_START && (addr + 2) <= (L1_DATA_A_START + L1_DATA_A_LENGTH)) ||
 #endif
 #if L1_DATA_B_LENGTH != 0
 	    (addr >= L1_DATA_B_START && (addr + 2) <= (L1_DATA_B_START + L1_DATA_B_LENGTH)) ||
 #endif
 	    (addr >= L1_SCRATCH_START && (addr + 2) <= (L1_SCRATCH_START + L1_SCRATCH_LENGTH)) ||
 	    (!(bfin_read_EBIU_AMBCTL0() & B0RDYEN) &&
 	       addr >= ASYNC_BANK0_BASE && (addr + 2) <= (ASYNC_BANK0_BASE + ASYNC_BANK0_SIZE)) ||
 	    (!(bfin_read_EBIU_AMBCTL0() & B1RDYEN) &&
 	       addr >= ASYNC_BANK1_BASE && (addr + 2) <= (ASYNC_BANK1_BASE + ASYNC_BANK1_SIZE)) ||
 	    (!(bfin_read_EBIU_AMBCTL1() & B2RDYEN) &&
 	       addr >= ASYNC_BANK2_BASE && (addr + 2) <= (ASYNC_BANK2_BASE + ASYNC_BANK1_SIZE)) ||
 	    (!(bfin_read_EBIU_AMBCTL1() & B3RDYEN) &&
 	      addr >= ASYNC_BANK3_BASE && (addr + 2) <= (ASYNC_BANK3_BASE + ASYNC_BANK1_SIZE))) {
 		*val = *address;
 		return true;
 	}
 #if L1_CODE_LENGTH != 0
 	if (addr >= L1_CODE_START && (addr + 2) <= (L1_CODE_START + L1_CODE_LENGTH)) {
 		isram_memcpy(val, address, 2);
 		return true;
 	}
 #endif
 	return false;
 }
 /*
  * decode the instruction if we are printing out the trace, as it
  * makes things easier to follow, without running it through objdump
  * These are the normal instructions which cause change of flow, which
  * would be at the source of the trace buffer
  */
 #if defined(CONFIG_DEBUG_VERBOSE) && defined(CONFIG_DEBUG_BFIN_HWTRACE_ON)
 static void decode_instruction(unsigned short *address)
 {
 	unsigned short opcode;
 	if (get_instruction(&opcode, address)) {
 		if (opcode == 0x0010)
 			verbose_printk("RTS");
 		else if (opcode == 0x0011)
 			verbose_printk("RTI");
 		else if (opcode == 0x0012)
 			verbose_printk("RTX");
 		else if (opcode >= 0x0050 && opcode <= 0x0057)
 			verbose_printk("JUMP (P%i)", opcode & 7);
 		else if (opcode >= 0x0060 && opcode <= 0x0067)
 			verbose_printk("CALL (P%i)", opcode & 7);
 		else if (opcode >= 0x0070 && opcode <= 0x0077)
 			verbose_printk("CALL (PC+P%i)", opcode & 7);
 		else if (opcode >= 0x0080 && opcode <= 0x0087)
 			verbose_printk("JUMP (PC+P%i)", opcode & 7);
 		else if ((opcode >= 0x1000 && opcode <= 0x13FF) || (opcode >= 0x1800 && opcode <= 0x1BFF))
 			verbose_printk("IF !CC JUMP");
 		else if ((opcode >= 0x1400 && opcode <= 0x17ff) || (opcode >= 0x1c00 && opcode <= 0x1fff))
 			verbose_printk("IF CC JUMP");
 		else if (opcode >= 0x2000 && opcode <= 0x2fff)
 			verbose_printk("JUMP.S");
 		else if (opcode >= 0xe080 && opcode <= 0xe0ff)
 			verbose_printk("LSETUP");
 		else if (opcode >= 0xe200 && opcode <= 0xe2ff)
 			verbose_printk("JUMP.L");
 		else if (opcode >= 0xe300 && opcode <= 0xe3ff)
 			verbose_printk("CALL pcrel");
 		else
 			verbose_printk("0x%04x", opcode);
 	}
 }
 #endif
 void dump_bfin_trace_buffer(void)
 {
 #ifdef CONFIG_DEBUG_VERBOSE
 #ifdef CONFIG_DEBUG_BFIN_HWTRACE_ON
 	int tflags, i = 0;
 	char buf[150];
 	unsigned short *addr;
 #ifdef CONFIG_DEBUG_BFIN_HWTRACE_EXPAND
 	int j, index;
 #endif
 	trace_buffer_save(tflags);
 	printk(KERN_NOTICE "Hardware Trace:\n");
 #ifdef CONFIG_DEBUG_BFIN_HWTRACE_EXPAND
 	printk(KERN_NOTICE "WARNING: Expanded trace turned on - can not trace exceptions\n");
 #endif
 	if (likely(bfin_read_TBUFSTAT() & TBUFCNT)) {
 		for (; bfin_read_TBUFSTAT() & TBUFCNT; i++) {
 			decode_address(buf, (unsigned long)bfin_read_TBUF());
 			printk(KERN_NOTICE "%4i Target : %s\n", i, buf);
 			addr = (unsigned short *)bfin_read_TBUF();
 			decode_address(buf, (unsigned long)addr);
 			printk(KERN_NOTICE "     Source : %s ", buf);
 			decode_instruction(addr);
 			printk("\n");
 		}
 	}
 #ifdef CONFIG_DEBUG_BFIN_HWTRACE_EXPAND
 	if (trace_buff_offset)
 		index = trace_buff_offset / 4;
 	else
 		index = EXPAND_LEN;
 	j = (1 << CONFIG_DEBUG_BFIN_HWTRACE_EXPAND_LEN) * 128;
 	while (j) {
 		decode_address(buf, software_trace_buff[index]);
 		printk(KERN_NOTICE "%4i Target : %s\n", i, buf);
 		index -= 1;
 		if (index < 0 )
 			index = EXPAND_LEN;
 		decode_address(buf, software_trace_buff[index]);
 		printk(KERN_NOTICE "     Source : %s ", buf);
 		decode_instruction((unsigned short *)software_trace_buff[index]);
 		printk("\n");
 		index -= 1;
 		if (index < 0)
 			index = EXPAND_LEN;
 		j--;
 		i++;
 	}
 #endif
 	trace_buffer_restore(tflags);
 #endif
 #endif
 }
 EXPORT_SYMBOL(dump_bfin_trace_buffer);
 /*
  * Checks to see if the address pointed to is either a
  * 16-bit CALL instruction, or a 32-bit CALL instruction
  */
 static bool is_bfin_call(unsigned short *addr)
 {
 	unsigned short opcode = 0, *ins_addr;
 	ins_addr = (unsigned short *)addr;
 	if (!get_instruction(&opcode, ins_addr))
 		return false;
 	if ((opcode >= 0x0060 && opcode <= 0x0067) ||
 	    (opcode >= 0x0070 && opcode <= 0x0077))
 		return true;
 	ins_addr--;
 	if (!get_instruction(&opcode, ins_addr))
 		return false;
 	if (opcode >= 0xE300 && opcode <= 0xE3FF)
 		return true;
 	return false;
 }
 void show_stack(struct task_struct *task, unsigned long *stack)
 {
 #ifdef CONFIG_PRINTK
 	unsigned int *addr, *endstack, *fp = 0, *frame;
 	unsigned short *ins_addr;
 	char buf[150];
 	unsigned int i, j, ret_addr, frame_no = 0;
 	/*
 	 * If we have been passed a specific stack, use that one otherwise
 	 *    if we have been passed a task structure, use that, otherwise
 	 *    use the stack of where the variable "stack" exists
 	 */
 	if (stack == NULL) {
 		if (task) {
 			/* We know this is a kernel stack, so this is the start/end */
 			stack = (unsigned long *)task->thread.ksp;
 			endstack = (unsigned int *)(((unsigned int)(stack) & ~(THREAD_SIZE - 1)) + THREAD_SIZE);
 		} else {
 			/* print out the existing stack info */
 			stack = (unsigned long *)&stack;
 			endstack = (unsigned int *)PAGE_ALIGN((unsigned int)stack);
 		}
 	} else
 		endstack = (unsigned int *)PAGE_ALIGN((unsigned int)stack);
 	printk(KERN_NOTICE "Stack info:\n");
 	decode_address(buf, (unsigned int)stack);
 	printk(KERN_NOTICE " SP: [0x%p] %s\n", stack, buf);
 	addr = (unsigned int *)((unsigned int)stack & ~0x3F);
 	/* First thing is to look for a frame pointer */
 	for (addr = (unsigned int *)((unsigned int)stack & ~0xF), i = 0;
 		addr < endstack; addr++, i++) {
 		if (*addr & 0x1)
 			continue;
 		ins_addr = (unsigned short *)*addr;
 		ins_addr--;
 		if (is_bfin_call(ins_addr))
 			fp = addr - 1;
 		if (fp) {
 			/* Let's check to see if it is a frame pointer */
 			while (fp >= (addr - 1) && fp < endstack && fp)
 				fp = (unsigned int *)*fp;
 			if (fp == 0 || fp == endstack) {
 				fp = addr - 1;
 				break;
 			}
 			fp = 0;
 		}
 	}
 	if (fp) {
 		frame = fp;
 		printk(KERN_NOTICE " FP: (0x%p)\n", fp);
 	} else
 		frame = 0;
 	/*
 	 * Now that we think we know where things are, we
 	 * walk the stack again, this time printing things out
 	 * incase there is no frame pointer, we still look for
 	 * valid return addresses
 	 */
 	/* First time print out data, next time, print out symbols */
 	for (j = 0; j <= 1; j++) {
 		if (j)
 			printk(KERN_NOTICE "Return addresses in stack:\n");
 		else
 			printk(KERN_NOTICE " Memory from 0x%08lx to %p", ((long unsigned int)stack & ~0xF), endstack);
 		fp = frame;
 		frame_no = 0;
 		for (addr = (unsigned int *)((unsigned int)stack & ~0xF), i = 0;
 		     addr <= endstack; addr++, i++) {
 			ret_addr = 0;
 			if (!j && i % 8 == 0)
 				printk("\n" KERN_NOTICE "%p:",addr);
 			/* if it is an odd address, or zero, just skip it */
 			if (*addr & 0x1 || !*addr)
 				goto print;
 			ins_addr = (unsigned short *)*addr;
 			/* Go back one instruction, and see if it is a CALL */
 			ins_addr--;
 			ret_addr = is_bfin_call(ins_addr);
  print:
 			if (!j && stack == (unsigned long *)addr)
 				printk("[%08x]", *addr);
 			else if (ret_addr)
 				if (j) {
 					decode_address(buf, (unsigned int)*addr);
 					if (frame == addr) {
 						printk(KERN_NOTICE "   frame %2i : %s\n", frame_no, buf);
 						continue;
 					}
 					printk(KERN_NOTICE "    address : %s\n", buf);
 				} else
 					printk("<%08x>", *addr);
 			else if (fp == addr) {
 				if (j)
 					frame = addr+1;
 				else
 					printk("(%08x)", *addr);
 				fp = (unsigned int *)*addr;
 				frame_no++;
 			} else if (!j)
 				printk(" %08x ", *addr);
 		}
 		if (!j)
 			printk("\n");
 	}
 #endif
 }
 void dump_stack(void)
 {
 	unsigned long stack;
 #ifdef CONFIG_DEBUG_BFIN_HWTRACE_ON
 	int tflags;
 #endif
 	trace_buffer_save(tflags);
 	dump_bfin_trace_buffer();
 	show_stack(current, &stack);
 	trace_buffer_restore(tflags);
 }
 EXPORT_SYMBOL(dump_stack);
 void dump_bfin_process(struct pt_regs *fp)
 {
 #ifdef CONFIG_DEBUG_VERBOSE
 	/* We should be able to look at fp->ipend, but we don't push it on the
 	 * stack all the time, so do this until we fix that */
 	unsigned int context = bfin_read_IPEND();
 	if (oops_in_progress)
 		verbose_printk(KERN_EMERG "Kernel OOPS in progress\n");
 	if (context & 0x0020 && (fp->seqstat & SEQSTAT_EXCAUSE) == VEC_HWERR)
 		verbose_printk(KERN_NOTICE "HW Error context\n");
 	else if (context & 0x0020)
 		verbose_printk(KERN_NOTICE "Deferred Exception context\n");
 	else if (context & 0x3FC0)
 		verbose_printk(KERN_NOTICE "Interrupt context\n");
 	else if (context & 0x4000)
 		verbose_printk(KERN_NOTICE "Deferred Interrupt context\n");
 	else if (context & 0x8000)
 		verbose_printk(KERN_NOTICE "Kernel process context\n");
 	/* Because we are crashing, and pointers could be bad, we check things
 	 * pretty closely before we use them
 	 */
 	if ((unsigned long)current >= FIXED_CODE_START &&
 	    !((unsigned long)current & 0x3) && current->pid) {
 		verbose_printk(KERN_NOTICE "CURRENT PROCESS:\n");
 		if (current->comm >= (char *)FIXED_CODE_START)
 			verbose_printk(KERN_NOTICE "COMM=%s PID=%d\n",
 				current->comm, current->pid);
 		else
 			verbose_printk(KERN_NOTICE "COMM= invalid\n");
 		printk(KERN_NOTICE "CPU = %d\n", current_thread_info()->cpu);
 		if (!((unsigned long)current->mm & 0x3) && (unsigned long)current->mm >= FIXED_CODE_START)
 			verbose_printk(KERN_NOTICE  "TEXT = 0x%p-0x%p        DATA = 0x%p-0x%p\n"
 				KERN_NOTICE " BSS = 0x%p-0x%p  USER-STACK = 0x%p\n"
 				KERN_NOTICE "\n",
 				(void *)current->mm->start_code,
 				(void *)current->mm->end_code,
 				(void *)current->mm->start_data,
 				(void *)current->mm->end_data,
 				(void *)current->mm->end_data,
 				(void *)current->mm->brk,
 				(void *)current->mm->start_stack);
 		else
 			verbose_printk(KERN_NOTICE "invalid mm\n");
 	} else
 		verbose_printk(KERN_NOTICE "\n" KERN_NOTICE
 		     "No Valid process in current context\n");
 #endif
 }
 void dump_bfin_mem(struct pt_regs *fp)
 {
 #ifdef CONFIG_DEBUG_VERBOSE
 	unsigned short *addr, *erraddr, val = 0, err = 0;
 	char sti = 0, buf[6];
 	erraddr = (void *)fp->pc;
 	verbose_printk(KERN_NOTICE "return address: [0x%p]; contents of:", erraddr);
 	for (addr = (unsigned short *)((unsigned long)erraddr & ~0xF) - 0x10;
 	     addr < (unsigned short *)((unsigned long)erraddr & ~0xF) + 0x10;
 	     addr++) {
 		if (!((unsigned long)addr & 0xF))
 			verbose_printk("\n" KERN_NOTICE "0x%p: ", addr);
 		if (!get_instruction(&val, addr)) {
 				val = 0;
 				sprintf(buf, "????");
 		} else
 			sprintf(buf, "%04x", val);
 		if (addr == erraddr) {
 			verbose_printk("[%s]", buf);
 			err = val;
 		} else
 			verbose_printk(" %s ", buf);
 		/* Do any previous instructions turn on interrupts? */
 		if (addr <= erraddr &&				/* in the past */
 		    ((val >= 0x0040 && val <= 0x0047) ||	/* STI instruction */
 		      val == 0x017b))				/* [SP++] = RETI */
 			sti = 1;
 	}
 	verbose_printk("\n");
 	/* Hardware error interrupts can be deferred */
 	if (unlikely(sti && (fp->seqstat & SEQSTAT_EXCAUSE) == VEC_HWERR &&
 	    oops_in_progress)){
 		verbose_printk(KERN_NOTICE "Looks like this was a deferred error - sorry\n");
 #ifndef CONFIG_DEBUG_HWERR
 		verbose_printk(KERN_NOTICE "The remaining message may be meaningless\n"
 			KERN_NOTICE "You should enable CONFIG_DEBUG_HWERR to get a"
 			 " better idea where it came from\n");
 #else
 		/* If we are handling only one peripheral interrupt
 		 * and current mm and pid are valid, and the last error
 		 * was in that user space process's text area
 		 * print it out - because that is where the problem exists
 		 */
 		if ((!(((fp)->ipend & ~0x30) & (((fp)->ipend & ~0x30) - 1))) &&
 		     (current->pid && current->mm)) {
 			/* And the last RETI points to the current userspace context */
 			if ((fp + 1)->pc >= current->mm->start_code &&
 			    (fp + 1)->pc <= current->mm->end_code) {
 				verbose_printk(KERN_NOTICE "It might be better to look around here : \n");
 				verbose_printk(KERN_NOTICE "-------------------------------------------\n");
 				show_regs(fp + 1);
 				verbose_printk(KERN_NOTICE "-------------------------------------------\n");
 			}
 		}
 #endif
 	}
 #endif
 }
 void show_regs(struct pt_regs *fp)
 {
 #ifdef CONFIG_DEBUG_VERBOSE
 	char buf [150];
 	struct irqaction *action;
 	unsigned int i;
 	unsigned long flags;
 	unsigned int cpu = smp_processor_id();
 	verbose_printk(KERN_NOTICE "\n" KERN_NOTICE "SEQUENCER STATUS:\t\t%s\n", print_tainted());
 	verbose_printk(KERN_NOTICE " SEQSTAT: %08lx  IPEND: %04lx  SYSCFG: %04lx\n",
 		(long)fp->seqstat, fp->ipend, fp->syscfg);
 	if ((fp->seqstat & SEQSTAT_EXCAUSE) == VEC_HWERR) {
 		verbose_printk(KERN_NOTICE "  HWERRCAUSE: 0x%lx\n",
 			(fp->seqstat & SEQSTAT_HWERRCAUSE) >> 14);
 #ifdef EBIU_ERRMST
 		/* If the error was from the EBIU, print it out */
 		if (bfin_read_EBIU_ERRMST() & CORE_ERROR) {
 			verbose_printk(KERN_NOTICE "  EBIU Error Reason  : 0x%04x\n",
 				bfin_read_EBIU_ERRMST());
 			verbose_printk(KERN_NOTICE "  EBIU Error Address : 0x%08x\n",
 				bfin_read_EBIU_ERRADD());
 		}
 #endif
 	}
 	verbose_printk(KERN_NOTICE "  EXCAUSE   : 0x%lx\n",
 		fp->seqstat & SEQSTAT_EXCAUSE);
 	for (i = 6; i <= 15 ; i++) {
 		if (fp->ipend & (1 << i)) {
 			decode_address(buf, bfin_read32(EVT0 + 4*i));
 			verbose_printk(KERN_NOTICE "  physical IVG%i asserted : %s\n", i, buf);
 		}
 	}
 	/* if no interrupts are going off, don't print this out */
 	if (fp->ipend & ~0x3F) {
 		for (i = 0; i < (NR_IRQS - 1); i++) {
 			spin_lock_irqsave(&irq_desc[i].lock, flags);
 			action = irq_desc[i].action;
 			if (!action)
 				goto unlock;
 			decode_address(buf, (unsigned int)action->handler);
 			verbose_printk(KERN_NOTICE "  logical irq %3d mapped  : %s", i, buf);
 			for (action = action->next; action; action = action->next) {
 				decode_address(buf, (unsigned int)action->handler);
 				verbose_printk(", %s", buf);
 			}
 			verbose_printk("\n");
 unlock:
 			spin_unlock_irqrestore(&irq_desc[i].lock, flags);
 		}
 	}
 	decode_address(buf, fp->rete);
 	verbose_printk(KERN_NOTICE " RETE: %s\n", buf);
 	decode_address(buf, fp->retn);
 	verbose_printk(KERN_NOTICE " RETN: %s\n", buf);
 	decode_address(buf, fp->retx);
 	verbose_printk(KERN_NOTICE " RETX: %s\n", buf);
 	decode_address(buf, fp->rets);
 	verbose_printk(KERN_NOTICE " RETS: %s\n", buf);
 	decode_address(buf, fp->pc);
 	verbose_printk(KERN_NOTICE " PC  : %s\n", buf);
 	if (((long)fp->seqstat &  SEQSTAT_EXCAUSE) &&
 	    (((long)fp->seqstat & SEQSTAT_EXCAUSE) != VEC_HWERR)) {
 		decode_address(buf, cpu_pda[cpu].dcplb_fault_addr);
 		verbose_printk(KERN_NOTICE "DCPLB_FAULT_ADDR: %s\n", buf);
 		decode_address(buf, cpu_pda[cpu].icplb_fault_addr);
 		verbose_printk(KERN_NOTICE "ICPLB_FAULT_ADDR: %s\n", buf);
 	}
 	verbose_printk(KERN_NOTICE "\n" KERN_NOTICE "PROCESSOR STATE:\n");
 	verbose_printk(KERN_NOTICE " R0 : %08lx    R1 : %08lx    R2 : %08lx    R3 : %08lx\n",
 		fp->r0, fp->r1, fp->r2, fp->r3);
 	verbose_printk(KERN_NOTICE " R4 : %08lx    R5 : %08lx    R6 : %08lx    R7 : %08lx\n",
 		fp->r4, fp->r5, fp->r6, fp->r7);
 	verbose_printk(KERN_NOTICE " P0 : %08lx    P1 : %08lx    P2 : %08lx    P3 : %08lx\n",
 		fp->p0, fp->p1, fp->p2, fp->p3);
 	verbose_printk(KERN_NOTICE " P4 : %08lx    P5 : %08lx    FP : %08lx    SP : %08lx\n",
 		fp->p4, fp->p5, fp->fp, (long)fp);
 	verbose_printk(KERN_NOTICE " LB0: %08lx    LT0: %08lx    LC0: %08lx\n",
 		fp->lb0, fp->lt0, fp->lc0);
 	verbose_printk(KERN_NOTICE " LB1: %08lx    LT1: %08lx    LC1: %08lx\n",
 		fp->lb1, fp->lt1, fp->lc1);
 	verbose_printk(KERN_NOTICE " B0 : %08lx    L0 : %08lx    M0 : %08lx    I0 : %08lx\n",
 		fp->b0, fp->l0, fp->m0, fp->i0);
 	verbose_printk(KERN_NOTICE " B1 : %08lx    L1 : %08lx    M1 : %08lx    I1 : %08lx\n",
 		fp->b1, fp->l1, fp->m1, fp->i1);
 	verbose_printk(KERN_NOTICE " B2 : %08lx    L2 : %08lx    M2 : %08lx    I2 : %08lx\n",
 		fp->b2, fp->l2, fp->m2, fp->i2);
 	verbose_printk(KERN_NOTICE " B3 : %08lx    L3 : %08lx    M3 : %08lx    I3 : %08lx\n",
 		fp->b3, fp->l3, fp->m3, fp->i3);
 	verbose_printk(KERN_NOTICE "A0.w: %08lx   A0.x: %08lx   A1.w: %08lx   A1.x: %08lx\n",
 		fp->a0w, fp->a0x, fp->a1w, fp->a1x);
 	verbose_printk(KERN_NOTICE "USP : %08lx  ASTAT: %08lx\n",
 		rdusp(), fp->astat);
 	verbose_printk(KERN_NOTICE "\n");
 #endif
 }
 #ifdef CONFIG_SYS_BFIN_SPINLOCK_L1
 asmlinkage int sys_bfin_spinlock(int *spinlock)__attribute__((l1_text));
 #endif
 static DEFINE_SPINLOCK(bfin_spinlock_lock);
 asmlinkage int sys_bfin_spinlock(int *p)
 {
 	int ret, tmp = 0;
 	spin_lock(&bfin_spinlock_lock);	/* This would also hold kernel preemption. */
 	ret = get_user(tmp, p);
 	if (likely(ret == 0)) {
 		if (unlikely(tmp))
 			ret = 1;
 		else
 			put_user(1, p);
 	}
 	spin_unlock(&bfin_spinlock_lock);
 	return ret;
 }
 int bfin_request_exception(unsigned int exception, void (*handler)(void))
 {
 	void (*curr_handler)(void);
 	if (exception > 0x3F)
 		return -EINVAL;
 	curr_handler = ex_table[exception];
 	if (curr_handler != ex_replaceable)
 		return -EBUSY;
 	ex_table[exception] = handler;
 	return 0;
 }
 EXPORT_SYMBOL(bfin_request_exception);
 int bfin_free_exception(unsigned int exception, void (*handler)(void))
 {
 	void (*curr_handler)(void);
 	if (exception > 0x3F)
 		return -EINVAL;
 	curr_handler = ex_table[exception];
 	if (curr_handler != handler)
 		return -EBUSY;
 	ex_table[exception] = ex_replaceable;
 	return 0;
 }
 EXPORT_SYMBOL(bfin_free_exception);
 void panic_cplb_error(int cplb_panic, struct pt_regs *fp)
 {
 	switch (cplb_panic) {
 	case CPLB_NO_UNLOCKED:
 		printk(KERN_EMERG "All CPLBs are locked\n");
 		break;
 	case CPLB_PROT_VIOL:
 		return;
 	case CPLB_NO_ADDR_MATCH:
 		return;
 	case CPLB_UNKNOWN_ERR:
 		printk(KERN_EMERG "Unknown CPLB Exception\n");
 		break;
 	}
 	oops_in_progress = 1;
 	dump_bfin_process(fp);
 	dump_bfin_mem(fp);
 	show_regs(fp);
 	dump_stack();

arch/frv/kernel/ptrace.c

Diff comments View file @ c40f6f8

 /* ptrace.c: FRV specific parts of process tracing
  *
  * Copyright (C) 2003-5 Red Hat, Inc. All Rights Reserved.
  * Written by David Howells (dhowells@redhat.com)
  * - Derived from arch/m68k/kernel/ptrace.c
  *
  * This program is free software; you can redistribute it and/or
  * modify it under the terms of the GNU General Public License
  * as published by the Free Software Foundation; either version
  * 2 of the License, or (at your option) any later version.
  */
 #include <linux/kernel.h>
 #include <linux/sched.h>
 #include <linux/mm.h>
 #include <linux/smp.h>
 #include <linux/errno.h>
 #include <linux/ptrace.h>
 #include <linux/user.h>
 #include <linux/security.h>
 #include <linux/signal.h>
 #include <asm/uaccess.h>
 #include <asm/page.h>
 #include <asm/pgtable.h>
 #include <asm/system.h>
 #include <asm/processor.h>
 #include <asm/unistd.h>
 /*
  * does not yet catch signals sent when the child dies.
  * in exit.c or in signal.c.
  */
 /*
  * Get contents of register REGNO in task TASK.
  */
 static inline long get_reg(struct task_struct *task, int regno)
 {
 	struct user_context *user = task->thread.user;
 	if (regno < 0 || regno >= PT__END)
 		return 0;
 	return ((unsigned long *) user)[regno];
 }
 /*
  * Write contents of register REGNO in task TASK.
  */
 static inline int put_reg(struct task_struct *task, int regno,
 			  unsigned long data)
 {
 	struct user_context *user = task->thread.user;
 	if (regno < 0 || regno >= PT__END)
 		return -EIO;
 	switch (regno) {
 	case PT_GR(0):
 		return 0;
 	case PT_PSR:
 	case PT__STATUS:
 		return -EIO;
 	default:
 		((unsigned long *) user)[regno] = data;
 		return 0;
 	}
 }
 /*
- * check that an address falls within the bounds of the target process's memory mappings
+ * check that an address falls within the bounds of the target process's memory
+ * mappings
  */
 static inline int is_user_addr_valid(struct task_struct *child,
 				     unsigned long start, unsigned long len)
 {
 #ifdef CONFIG_MMU
 	if (start >= PAGE_OFFSET || len > PAGE_OFFSET - start)
 		return -EIO;
 	return 0;
 #else
-	struct vm_list_struct *vml;
+	struct vm_area_struct *vma;
-	for (vml = child->mm->context.vmlist; vml; vml = vml->next)
+	vma = find_vma(child->mm, start);
-		if (start >= vml->vma->vm_start && start + len <= vml->vma->vm_end)
+	if (vma && start >= vma->vm_start && start + len <= vma->vm_end)
-			return 0;
+		return 0;
 	return -EIO;
 #endif
 }
 /*
  * Called by kernel/ptrace.c when detaching..
  *
  * Control h/w single stepping
  */
 void ptrace_disable(struct task_struct *child)
 {
 	child->thread.frame0->__status &= ~REG__STATUS_STEP;
 }
 void ptrace_enable(struct task_struct *child)
 {
 	child->thread.frame0->__status |= REG__STATUS_STEP;
 }
 long arch_ptrace(struct task_struct *child, long request, long addr, long data)
 {
 	unsigned long tmp;
 	int ret;
 	switch (request) {
 		/* when I and D space are separate, these will need to be fixed. */
 	case PTRACE_PEEKTEXT: /* read word at location addr. */
 	case PTRACE_PEEKDATA:
 		ret = -EIO;
 		if (is_user_addr_valid(child, addr, sizeof(tmp)) < 0)
 			break;
 		ret = generic_ptrace_peekdata(child, addr, data);
 		break;
 		/* read the word at location addr in the USER area. */
 	case PTRACE_PEEKUSR: {
 		tmp = 0;
 		ret = -EIO;
 		if ((addr & 3) || addr < 0)
 			break;
 		ret = 0;
 		switch (addr >> 2) {
 		case 0 ... PT__END - 1:
 			tmp = get_reg(child, addr >> 2);
 			break;
 		case PT__END + 0:
 			tmp = child->mm->end_code - child->mm->start_code;
 			break;
 		case PT__END + 1:
 			tmp = child->mm->end_data - child->mm->start_data;
 			break;
 		case PT__END + 2:
 			tmp = child->mm->start_stack - child->mm->start_brk;
 			break;
 		case PT__END + 3:
 			tmp = child->mm->start_code;
 			break;
 		case PT__END + 4:
 			tmp = child->mm->start_stack;
 			break;
 		default:
 			ret = -EIO;
 			break;
 		}
 		if (ret == 0)
 			ret = put_user(tmp, (unsigned long *) data);
 		break;
 	}
 		/* when I and D space are separate, this will have to be fixed. */
 	case PTRACE_POKETEXT: /* write the word at location addr. */
 	case PTRACE_POKEDATA:
 		ret = -EIO;
 		if (is_user_addr_valid(child, addr, sizeof(tmp)) < 0)
 			break;
 		ret = generic_ptrace_pokedata(child, addr, data);
 		break;
 	case PTRACE_POKEUSR: /* write the word at location addr in the USER area */
 		ret = -EIO;
 		if ((addr & 3) || addr < 0)
 			break;
 		ret = 0;
 		switch (addr >> 2) {
 		case 0 ... PT__END-1:
 			ret = put_reg(child, addr >> 2, data);
 			break;
 		default:
 			ret = -EIO;
 			break;
 		}
 		break;
 	case PTRACE_SYSCALL: /* continue and stop at next (return from) syscall */
 	case PTRACE_CONT: /* restart after signal. */
 		ret = -EIO;
 		if (!valid_signal(data))
 			break;
 		if (request == PTRACE_SYSCALL)
 			set_tsk_thread_flag(child, TIF_SYSCALL_TRACE);
 		else
 			clear_tsk_thread_flag(child, TIF_SYSCALL_TRACE);
 		child->exit_code = data;
 		ptrace_disable(child);
 		wake_up_process(child);
 		ret = 0;
 		break;
 		/* make the child exit.  Best I can do is send it a sigkill.
 		 * perhaps it should be put in the status that it wants to
 		 * exit.
 		 */
 	case PTRACE_KILL:
 		ret = 0;
 		if (child->exit_state == EXIT_ZOMBIE)	/* already dead */
 			break;
 		child->exit_code = SIGKILL;
 		clear_tsk_thread_flag(child, TIF_SINGLESTEP);
 		ptrace_disable(child);
 		wake_up_process(child);
 		break;
 	case PTRACE_SINGLESTEP:  /* set the trap flag. */
 		ret = -EIO;
 		if (!valid_signal(data))
 			break;
 		clear_tsk_thread_flag(child, TIF_SYSCALL_TRACE);
 		ptrace_enable(child);
 		child->exit_code = data;
 		wake_up_process(child);
 		ret = 0;
 		break;
 	case PTRACE_DETACH:	/* detach a process that was attached. */
 		ret = ptrace_detach(child, data);
 		break;
 	case PTRACE_GETREGS: { /* Get all integer regs from the child. */
 		int i;
 		for (i = 0; i < PT__GPEND; i++) {
 			tmp = get_reg(child, i);
 			if (put_user(tmp, (unsigned long *) data)) {
 				ret = -EFAULT;
 				break;
 			}
 			data += sizeof(long);
 		}
 		ret = 0;
 		break;
 	}
 	case PTRACE_SETREGS: { /* Set all integer regs in the child. */
 		int i;
 		for (i = 0; i < PT__GPEND; i++) {
 			if (get_user(tmp, (unsigned long *) data)) {
 				ret = -EFAULT;
 				break;
 			}
 			put_reg(child, i, tmp);
 			data += sizeof(long);
 		}
 		ret = 0;
 		break;
 	}
 	case PTRACE_GETFPREGS: { /* Get the child FP/Media state. */
 		ret = 0;
 		if (copy_to_user((void *) data,
 				 &child->thread.user->f,
 				 sizeof(child->thread.user->f)))
 			ret = -EFAULT;
 		break;
 	}
 	case PTRACE_SETFPREGS: { /* Set the child FP/Media state. */
 		ret = 0;
 		if (copy_from_user(&child->thread.user->f,
 				   (void *) data,
 				   sizeof(child->thread.user->f)))
 			ret = -EFAULT;
 		break;
 	}
 	case PTRACE_GETFDPIC:
 		tmp = 0;
 		switch (addr) {
 		case PTRACE_GETFDPIC_EXEC:
 			tmp = child->mm->context.exec_fdpic_loadmap;
 			break;
 		case PTRACE_GETFDPIC_INTERP:
 			tmp = child->mm->context.interp_fdpic_loadmap;
 			break;
 		default:
 			break;
 		}
 		ret = 0;
 		if (put_user(tmp, (unsigned long *) data)) {
 			ret = -EFAULT;
 			break;
 		}
 		break;
 	default:
 		ret = -EIO;
 		break;
 	}
 	return ret;
 }
 int __nongprelbss kstrace;
 static const struct {
 	const char	*name;
 	unsigned	argmask;
 } __syscall_name_table[NR_syscalls] = {
 	[0]	= { "restart_syscall"			},
 	[1]	= { "exit",		0x000001	},
 	[2]	= { "fork",		0xffffff	},
 	[3]	= { "read",		0x000141	},
 	[4]	= { "write",		0x000141	},
 	[5]	= { "open",		0x000235	},
 	[6]	= { "close",		0x000001	},
 	[7]	= { "waitpid",		0x000141	},
 	[8]	= { "creat",		0x000025	},
 	[9]	= { "link",		0x000055	},
 	[10]	= { "unlink",		0x000005	},
 	[11]	= { "execve",		0x000445	},
 	[12]	= { "chdir",		0x000005	},
 	[13]	= { "time",		0x000004	},
 	[14]	= { "mknod",		0x000325	},
 	[15]	= { "chmod",		0x000025	},
 	[16]	= { "lchown",		0x000025	},
 	[17]	= { "break" },
 	[18]	= { "oldstat",		0x000045	},
 	[19]	= { "lseek",		0x000131	},
 	[20]	= { "getpid",		0xffffff	},
 	[21]	= { "mount",		0x043555	},
 	[22]	= { "umount",		0x000005	},
 	[23]	= { "setuid",		0x000001	},
 	[24]	= { "getuid",		0xffffff	},
 	[25]	= { "stime",		0x000004	},
 	[26]	= { "ptrace",		0x004413	},
 	[27]	= { "alarm",		0x000001	},
 	[28]	= { "oldfstat",		0x000041	},
 	[29]	= { "pause",		0xffffff	},
 	[30]	= { "utime",		0x000045	},
 	[31]	= { "stty" },
 	[32]	= { "gtty" },
 	[33]	= { "access",		0x000025	},
 	[34]	= { "nice",		0x000001	},
 	[35]	= { "ftime" },
 	[36]	= { "sync",		0xffffff	},
 	[37]	= { "kill",		0x000011	},
 	[38]	= { "rename",		0x000055	},
 	[39]	= { "mkdir",		0x000025	},
 	[40]	= { "rmdir",		0x000005	},
 	[41]	= { "dup",		0x000001	},
 	[42]	= { "pipe",		0x000004	},
 	[43]	= { "times",		0x000004	},
 	[44]	= { "prof" },
 	[45]	= { "brk",		0x000004	},
 	[46]	= { "setgid",		0x000001	},
 	[47]	= { "getgid",		0xffffff	},
 	[48]	= { "signal",		0x000041	},
 	[49]	= { "geteuid",		0xffffff	},
 	[50]	= { "getegid",		0xffffff	},
 	[51]	= { "acct",		0x000005	},
 	[52]	= { "umount2",		0x000035	},
 	[53]	= { "lock" },
 	[54]	= { "ioctl",		0x000331	},
 	[55]	= { "fcntl",		0x000331	},
 	[56]	= { "mpx" },
 	[57]	= { "setpgid",		0x000011	},
 	[58]	= { "ulimit" },
 	[60]	= { "umask",		0x000002	},
 	[61]	= { "chroot",		0x000005	},
 	[62]	= { "ustat",		0x000043	},
 	[63]	= { "dup2",		0x000011	},
 	[64]	= { "getppid",		0xffffff	},
 	[65]	= { "getpgrp",		0xffffff	},
 	[66]	= { "setsid",		0xffffff	},
 	[67]	= { "sigaction" },
 	[68]	= { "sgetmask" },
 	[69]	= { "ssetmask" },
 	[70]	= { "setreuid" },
 	[71]	= { "setregid" },
 	[72]	= { "sigsuspend" },
 	[73]	= { "sigpending" },
 	[74]	= { "sethostname" },
 	[75]	= { "setrlimit" },
 	[76]	= { "getrlimit" },
 	[77]	= { "getrusage" },
 	[78]	= { "gettimeofday" },
 	[79]	= { "settimeofday" },
 	[80]	= { "getgroups" },
 	[81]	= { "setgroups" },
 	[82]	= { "select" },
 	[83]	= { "symlink" },
 	[84]	= { "oldlstat" },
 	[85]	= { "readlink" },
 	[86]	= { "uselib" },
 	[87]	= { "swapon" },
 	[88]	= { "reboot" },
 	[89]	= { "readdir" },
 	[91]	= { "munmap",		0x000034	},
 	[92]	= { "truncate" },
 	[93]	= { "ftruncate" },
 	[94]	= { "fchmod" },
 	[95]	= { "fchown" },
 	[96]	= { "getpriority" },
 	[97]	= { "setpriority" },
 	[99]	= { "statfs" },
 	[100]	= { "fstatfs" },
 	[102]	= { "socketcall" },
 	[103]	= { "syslog" },
 	[104]	= { "setitimer" },
 	[105]	= { "getitimer" },
 	[106]	= { "stat" },
 	[107]	= { "lstat" },
 	[108]	= { "fstat" },
 	[111]	= { "vhangup" },
 	[114]	= { "wait4" },
 	[115]	= { "swapoff" },
 	[116]	= { "sysinfo" },
 	[117]	= { "ipc" },
 	[118]	= { "fsync" },
 	[119]	= { "sigreturn" },
 	[120]	= { "clone" },
 	[121]	= { "setdomainname" },
 	[122]	= { "uname" },
 	[123]	= { "modify_ldt" },
 	[123]	= { "cacheflush" },
 	[124]	= { "adjtimex" },
 	[125]	= { "mprotect" },
 	[126]	= { "sigprocmask" },
 	[127]	= { "create_module" },
 	[128]	= { "init_module" },
 	[129]	= { "delete_module" },
 	[130]	= { "get_kernel_syms" },
 	[131]	= { "quotactl" },
 	[132]	= { "getpgid" },
 	[133]	= { "fchdir" },
 	[134]	= { "bdflush" },
 	[135]	= { "sysfs" },
 	[136]	= { "personality" },
 	[137]	= { "afs_syscall" },
 	[138]	= { "setfsuid" },
 	[139]	= { "setfsgid" },
 	[140]	= { "_llseek",			0x014331	},
 	[141]	= { "getdents" },
 	[142]	= { "_newselect",		0x000141	},
 	[143]	= { "flock" },
 	[144]	= { "msync" },
 	[145]	= { "readv" },
 	[146]	= { "writev" },
 	[147]	= { "getsid",			0x000001	},
 	[148]	= { "fdatasync",		0x000001	},
 	[149]	= { "_sysctl",			0x000004	},
 	[150]	= { "mlock" },
 	[151]	= { "munlock" },
 	[152]	= { "mlockall" },
 	[153]	= { "munlockall" },
 	[154]	= { "sched_setparam" },
 	[155]	= { "sched_getparam" },
 	[156]	= { "sched_setscheduler" },
 	[157]	= { "sched_getscheduler" },
 	[158]	= { "sched_yield" },
 	[159]	= { "sched_get_priority_max" },
 	[160]	= { "sched_get_priority_min" },
 	[161]	= { "sched_rr_get_interval" },
 	[162]	= { "nanosleep",		0x000044	},
 	[163]	= { "mremap" },
 	[164]	= { "setresuid" },
 	[165]	= { "getresuid" },
 	[166]	= { "vm86" },
 	[167]	= { "query_module" },
 	[168]	= { "poll" },
 	[169]	= { "nfsservctl" },
 	[170]	= { "setresgid" },
 	[171]	= { "getresgid" },
 	[172]	= { "prctl",			0x333331	},
 	[173]	= { "rt_sigreturn",		0xffffff	},
 	[174]	= { "rt_sigaction",		0x001441	},
 	[175]	= { "rt_sigprocmask",		0x001441	},
 	[176]	= { "rt_sigpending",		0x000014	},
 	[177]	= { "rt_sigtimedwait",		0x001444	},
 	[178]	= { "rt_sigqueueinfo",		0x000411	},
 	[179]	= { "rt_sigsuspend",		0x000014	},
 	[180]	= { "pread",			0x003341	},
 	[181]	= { "pwrite",			0x003341	},
 	[182]	= { "chown",			0x000115	},
 	[183]	= { "getcwd" },
 	[184]	= { "capget" },
 	[185]	= { "capset" },
 	[186]	= { "sigaltstack" },
 	[187]	= { "sendfile" },
 	[188]	= { "getpmsg" },
 	[189]	= { "putpmsg" },
 	[190]	= { "vfork",			0xffffff	},
 	[191]	= { "ugetrlimit" },
 	[192]	= { "mmap2",			0x313314	},
 	[193]	= { "truncate64" },
 	[194]	= { "ftruncate64" },
 	[195]	= { "stat64",			0x000045	},
 	[196]	= { "lstat64",			0x000045	},
 	[197]	= { "fstat64",			0x000041	},
 	[198]	= { "lchown32" },
 	[199]	= { "getuid32",			0xffffff	},
 	[200]	= { "getgid32",			0xffffff	},
 	[201]	= { "geteuid32",		0xffffff	},
 	[202]	= { "getegid32",		0xffffff	},
 	[203]	= { "setreuid32" },
 	[204]	= { "setregid32" },
 	[205]	= { "getgroups32" },
 	[206]	= { "setgroups32" },
 	[207]	= { "fchown32" },
 	[208]	= { "setresuid32" },
 	[209]	= { "getresuid32" },
 	[210]	= { "setresgid32" },
 	[211]	= { "getresgid32" },
 	[212]	= { "chown32" },
 	[213]	= { "setuid32" },
 	[214]	= { "setgid32" },
 	[215]	= { "setfsuid32" },
 	[216]	= { "setfsgid32" },
 	[217]	= { "pivot_root" },
 	[218]	= { "mincore" },
 	[219]	= { "madvise" },
 	[220]	= { "getdents64" },
 	[221]	= { "fcntl64" },
 	[223]	= { "security" },
 	[224]	= { "gettid" },
 	[225]	= { "readahead" },
 	[226]	= { "setxattr" },
 	[227]	= { "lsetxattr" },
 	[228]	= { "fsetxattr" },
 	[229]	= { "getxattr" },
 	[230]	= { "lgetxattr" },
 	[231]	= { "fgetxattr" },
 	[232]	= { "listxattr" },
 	[233]	= { "llistxattr" },
 	[234]	= { "flistxattr" },
 	[235]	= { "removexattr" },
 	[236]	= { "lremovexattr" },
 	[237]	= { "fremovexattr" },
 	[238]	= { "tkill" },
 	[239]	= { "sendfile64" },
 	[240]	= { "futex" },
 	[241]	= { "sched_setaffinity" },
 	[242]	= { "sched_getaffinity" },
 	[243]	= { "set_thread_area" },
 	[244]	= { "get_thread_area" },
 	[245]	= { "io_setup" },
 	[246]	= { "io_destroy" },
 	[247]	= { "io_getevents" },
 	[248]	= { "io_submit" },
 	[249]	= { "io_cancel" },
 	[250]	= { "fadvise64" },
 	[252]	= { "exit_group",		0x000001	},
 	[253]	= { "lookup_dcookie" },
 	[254]	= { "epoll_create" },
 	[255]	= { "epoll_ctl" },
 	[256]	= { "epoll_wait" },
 	[257]	= { "remap_file_pages" },
 	[258]	= { "set_tid_address" },
 	[259]	= { "timer_create" },
 	[260]	= { "timer_settime" },
 	[261]	= { "timer_gettime" },
 	[262]	= { "timer_getoverrun" },
 	[263]	= { "timer_delete" },
 	[264]	= { "clock_settime" },
 	[265]	= { "clock_gettime" },
 	[266]	= { "clock_getres" },
 	[267]	= { "clock_nanosleep" },
 	[268]	= { "statfs64" },
 	[269]	= { "fstatfs64" },
 	[270]	= { "tgkill" },
 	[271]	= { "utimes" },
 	[272]	= { "fadvise64_64" },
 	[273]	= { "vserver" },
 	[274]	= { "mbind" },
 	[275]	= { "get_mempolicy" },
 	[276]	= { "set_mempolicy" },
 	[277]	= { "mq_open" },
 	[278]	= { "mq_unlink" },
 	[279]	= { "mq_timedsend" },
 	[280]	= { "mq_timedreceive" },
 	[281]	= { "mq_notify" },
 	[282]	= { "mq_getsetattr" },
 	[283]	= { "sys_kexec_load" },
 };
 asmlinkage void do_syscall_trace(int leaving)
 {
 #if 0
 	unsigned long *argp;
 	const char *name;
 	unsigned argmask;
 	char buffer[16];
 	if (!kstrace)
 		return;
 	if (!current->mm)
 		return;
 	if (__frame->gr7 == __NR_close)
 		return;
 #if 0
 	if (__frame->gr7 != __NR_mmap2 &&
 	    __frame->gr7 != __NR_vfork &&
 	    __frame->gr7 != __NR_execve &&
 	    __frame->gr7 != __NR_exit)
 		return;
 #endif
 	argmask = 0;
 	name = NULL;
 	if (__frame->gr7 < NR_syscalls) {
 		name = __syscall_name_table[__frame->gr7].name;
 		argmask = __syscall_name_table[__frame->gr7].argmask;
 	}
 	if (!name) {
 		sprintf(buffer, "sys_%lx", __frame->gr7);
 		name = buffer;
 	}
 	if (!leaving) {
 		if (!argmask) {
 			printk(KERN_CRIT "[%d] %s(%lx,%lx,%lx,%lx,%lx,%lx)\n",
 			       current->pid,
 			       name,
 			       __frame->gr8,
 			       __frame->gr9,
 			       __frame->gr10,
 			       __frame->gr11,
 			       __frame->gr12,
 			       __frame->gr13);
 		}
 		else if (argmask == 0xffffff) {
 			printk(KERN_CRIT "[%d] %s()\n",
 			       current->pid,
 			       name);
 		}
 		else {
 			printk(KERN_CRIT "[%d] %s(",
 			       current->pid,
 			       name);
 			argp = &__frame->gr8;
 			do {
 				switch (argmask & 0xf) {
 				case 1:
 					printk("%ld", (long) *argp);
 					break;
 				case 2:
 					printk("%lo", *argp);
 					break;
 				case 3:
 					printk("%lx", *argp);
 					break;
 				case 4:
 					printk("%p", (void *) *argp);
 					break;
 				case 5:
 					printk("\"%s\"", (char *) *argp);
 					break;
 				}
 				argp++;
 				argmask >>= 4;
 				if (argmask)
 					printk(",");
 			} while (argmask);
 			printk(")\n");
 		}
 	}
 	else {
 		if ((int)__frame->gr8 > -4096 && (int)__frame->gr8 < 4096)
 			printk(KERN_CRIT "[%d] %s() = %ld\n", current->pid, name, __frame->gr8);
 		else
 			printk(KERN_CRIT "[%d] %s() = %lx\n", current->pid, name, __frame->gr8);
 	}
 	return;
 #endif
 	if (!test_thread_flag(TIF_SYSCALL_TRACE))
 		return;
 	if (!(current->ptrace & PT_PTRACED))
 		return;
 	/* we need to indicate entry or exit to strace */
 	if (leaving)
 		__frame->__status |= REG__STATUS_SYSC_EXIT;
 	else
 		__frame->__status |= REG__STATUS_SYSC_ENTRY;
 	ptrace_notify(SIGTRAP);
 	/*
 	 * this isn't the same as continuing with a signal, but it will do
 	 * for normal use.  strace only continues with a signal if the
 	 * stopping signal is not SIGTRAP.  -brl
 	 */
 	if (current->exit_code) {
 		send_sig(current->exit_code, current, 1);
 		current->exit_code = 0;
 	}
 }

arch/h8300/include/asm/mmu.h

Diff comments View file @ c40f6f8

 #ifndef __MMU_H
 #define __MMU_H
 /* Copyright (C) 2002, David McCullough <davidm@snapgear.com> */
 typedef struct {
-	struct vm_list_struct	*vmlist;
 	unsigned long		end_brk;
 } mm_context_t;
 #endif

arch/m68knommu/include/asm/mmu.h

Diff comments View file @ c40f6f8

 #ifndef __M68KNOMMU_MMU_H
 #define __M68KNOMMU_MMU_H
 /* Copyright (C) 2002, David McCullough <davidm@snapgear.com> */
 typedef struct {
-	struct vm_list_struct	*vmlist;
 	unsigned long		end_brk;
 } mm_context_t;
 #endif /* __M68KNOMMU_MMU_H */

arch/sh/include/asm/mmu.h

Diff comments View file @ c40f6f8

 #ifndef __MMU_H
 #define __MMU_H
 /* Default "unsigned long" context */
 typedef unsigned long mm_context_id_t[NR_CPUS];
 typedef struct {
 #ifdef CONFIG_MMU
 	mm_context_id_t		id;
 	void			*vdso;
 #else
-	struct vm_list_struct	*vmlist;
 	unsigned long		end_brk;
 #endif
 #ifdef CONFIG_BINFMT_ELF_FDPIC
 	unsigned long		exec_fdpic_loadmap;
 	unsigned long		interp_fdpic_loadmap;
 #endif
 } mm_context_t;
 /*
  * Privileged Space Mapping Buffer (PMB) definitions
  */
 #define PMB_PASCR		0xff000070
 #define PMB_IRMCR		0xff000078
 #define PMB_ADDR		0xf6100000
 #define PMB_DATA		0xf7100000
 #define PMB_ENTRY_MAX		16
 #define PMB_E_MASK		0x0000000f
 #define PMB_E_SHIFT		8
 #define PMB_SZ_16M		0x00000000
 #define PMB_SZ_64M		0x00000010
 #define PMB_SZ_128M		0x00000080
 #define PMB_SZ_512M		0x00000090
 #define PMB_SZ_MASK		PMB_SZ_512M
 #define PMB_C			0x00000008
 #define PMB_WT			0x00000001
 #define PMB_UB			0x00000200
 #define PMB_V			0x00000100
 #define PMB_NO_ENTRY		(-1)
 struct pmb_entry;
 struct pmb_entry {
 	unsigned long vpn;
 	unsigned long ppn;
 	unsigned long flags;
 	/*
 	 * 0 .. NR_PMB_ENTRIES for specific entry selection, or
 	 * PMB_NO_ENTRY to search for a free one
 	 */
 	int entry;
 	struct pmb_entry *next;
 	/* Adjacent entry link for contiguous multi-entry mappings */
 	struct pmb_entry *link;
 };
 /* arch/sh/mm/pmb.c */
 int __set_pmb_entry(unsigned long vpn, unsigned long ppn,
 		    unsigned long flags, int *entry);
 int set_pmb_entry(struct pmb_entry *pmbe);
 void clear_pmb_entry(struct pmb_entry *pmbe);
 struct pmb_entry *pmb_alloc(unsigned long vpn, unsigned long ppn,
 			    unsigned long flags);
 void pmb_free(struct pmb_entry *pmbe);
 long pmb_remap(unsigned long virt, unsigned long phys,
 	       unsigned long size, unsigned long flags);
 void pmb_unmap(unsigned long addr);
 #endif /* __MMU_H */

fs/binfmt_elf_fdpic.c

Diff comments View file @ c40f6f8

 /* binfmt_elf_fdpic.c: FDPIC ELF binary format
  *
  * Copyright (C) 2003, 2004, 2006 Red Hat, Inc. All Rights Reserved.
  * Written by David Howells (dhowells@redhat.com)
  * Derived from binfmt_elf.c
  *
  * This program is free software; you can redistribute it and/or
  * modify it under the terms of the GNU General Public License
  * as published by the Free Software Foundation; either version
  * 2 of the License, or (at your option) any later version.
  */
 #include <linux/module.h>
 #include <linux/fs.h>
 #include <linux/stat.h>
 #include <linux/sched.h>
 #include <linux/mm.h>
 #include <linux/mman.h>
 #include <linux/errno.h>
 #include <linux/signal.h>
 #include <linux/binfmts.h>
 #include <linux/string.h>
 #include <linux/file.h>
 #include <linux/fcntl.h>
 #include <linux/slab.h>
 #include <linux/pagemap.h>
 #include <linux/security.h>
 #include <linux/highmem.h>
 #include <linux/highuid.h>
 #include <linux/personality.h>
 #include <linux/ptrace.h>
 #include <linux/init.h>
 #include <linux/elf.h>
 #include <linux/elf-fdpic.h>
 #include <linux/elfcore.h>
 #include <asm/uaccess.h>
 #include <asm/param.h>
 #include <asm/pgalloc.h>
 typedef char *elf_caddr_t;
 #if 0
 #define kdebug(fmt, ...) printk("FDPIC "fmt"\n" ,##__VA_ARGS__ )
 #else
 #define kdebug(fmt, ...) do {} while(0)
 #endif
 #if 0
 #define kdcore(fmt, ...) printk("FDPIC "fmt"\n" ,##__VA_ARGS__ )
 #else
 #define kdcore(fmt, ...) do {} while(0)
 #endif
 MODULE_LICENSE("GPL");
 static int load_elf_fdpic_binary(struct linux_binprm *, struct pt_regs *);
 static int elf_fdpic_fetch_phdrs(struct elf_fdpic_params *, struct file *);
 static int elf_fdpic_map_file(struct elf_fdpic_params *, struct file *,
 			      struct mm_struct *, const char *);
 static int create_elf_fdpic_tables(struct linux_binprm *, struct mm_struct *,
 				   struct elf_fdpic_params *,
 				   struct elf_fdpic_params *);
 #ifndef CONFIG_MMU
 static int elf_fdpic_transfer_args_to_stack(struct linux_binprm *,
 					    unsigned long *);
 static int elf_fdpic_map_file_constdisp_on_uclinux(struct elf_fdpic_params *,
 						   struct file *,
 						   struct mm_struct *);
 #endif
 static int elf_fdpic_map_file_by_direct_mmap(struct elf_fdpic_params *,
 					     struct file *, struct mm_struct *);
 #if defined(USE_ELF_CORE_DUMP) && defined(CONFIG_ELF_CORE)
 static int elf_fdpic_core_dump(long, struct pt_regs *, struct file *, unsigned long limit);
 #endif
 static struct linux_binfmt elf_fdpic_format = {
 	.module		= THIS_MODULE,
 	.load_binary	= load_elf_fdpic_binary,
 #if defined(USE_ELF_CORE_DUMP) && defined(CONFIG_ELF_CORE)
 	.core_dump	= elf_fdpic_core_dump,
 #endif
 	.min_coredump	= ELF_EXEC_PAGESIZE,
 };
 static int __init init_elf_fdpic_binfmt(void)
 {
 	return register_binfmt(&elf_fdpic_format);
 }
 static void __exit exit_elf_fdpic_binfmt(void)
 {
 	unregister_binfmt(&elf_fdpic_format);
 }
 core_initcall(init_elf_fdpic_binfmt);
 module_exit(exit_elf_fdpic_binfmt);
 static int is_elf_fdpic(struct elfhdr *hdr, struct file *file)
 {
 	if (memcmp(hdr->e_ident, ELFMAG, SELFMAG) != 0)
 		return 0;
 	if (hdr->e_type != ET_EXEC && hdr->e_type != ET_DYN)
 		return 0;
 	if (!elf_check_arch(hdr) || !elf_check_fdpic(hdr))
 		return 0;
 	if (!file->f_op || !file->f_op->mmap)
 		return 0;
 	return 1;
 }
 /*****************************************************************************/
 /*
  * read the program headers table into memory
  */
 static int elf_fdpic_fetch_phdrs(struct elf_fdpic_params *params,
 				 struct file *file)
 {
 	struct elf32_phdr *phdr;
 	unsigned long size;
 	int retval, loop;
 	if (params->hdr.e_phentsize != sizeof(struct elf_phdr))
 		return -ENOMEM;
 	if (params->hdr.e_phnum > 65536U / sizeof(struct elf_phdr))
 		return -ENOMEM;
 	size = params->hdr.e_phnum * sizeof(struct elf_phdr);
 	params->phdrs = kmalloc(size, GFP_KERNEL);
 	if (!params->phdrs)
 		return -ENOMEM;
 	retval = kernel_read(file, params->hdr.e_phoff,
 			     (char *) params->phdrs, size);
 	if (unlikely(retval != size))
 		return retval < 0 ? retval : -ENOEXEC;
 	/* determine stack size for this binary */
 	phdr = params->phdrs;
 	for (loop = 0; loop < params->hdr.e_phnum; loop++, phdr++) {
 		if (phdr->p_type != PT_GNU_STACK)
 			continue;
 		if (phdr->p_flags & PF_X)
 			params->flags |= ELF_FDPIC_FLAG_EXEC_STACK;
 		else
 			params->flags |= ELF_FDPIC_FLAG_NOEXEC_STACK;
 		params->stack_size = phdr->p_memsz;
 		break;
 	}
 	return 0;
 }
 /*****************************************************************************/
 /*
  * load an fdpic binary into various bits of memory
  */
 static int load_elf_fdpic_binary(struct linux_binprm *bprm,
 				 struct pt_regs *regs)
 {
 	struct elf_fdpic_params exec_params, interp_params;
 	struct elf_phdr *phdr;
 	unsigned long stack_size, entryaddr;
-#ifndef CONFIG_MMU
-	unsigned long fullsize;
-#endif
 #ifdef ELF_FDPIC_PLAT_INIT
 	unsigned long dynaddr;
 #endif
 	struct file *interpreter = NULL; /* to shut gcc up */
 	char *interpreter_name = NULL;
 	int executable_stack;
 	int retval, i;
 	kdebug("____ LOAD %d ____", current->pid);
 	memset(&exec_params, 0, sizeof(exec_params));
 	memset(&interp_params, 0, sizeof(interp_params));
 	exec_params.hdr = *(struct elfhdr *) bprm->buf;
 	exec_params.flags = ELF_FDPIC_FLAG_PRESENT | ELF_FDPIC_FLAG_EXECUTABLE;
 	/* check that this is a binary we know how to deal with */
 	retval = -ENOEXEC;
 	if (!is_elf_fdpic(&exec_params.hdr, bprm->file))
 		goto error;
 	/* read the program header table */
 	retval = elf_fdpic_fetch_phdrs(&exec_params, bprm->file);
 	if (retval < 0)
 		goto error;
 	/* scan for a program header that specifies an interpreter */
 	phdr = exec_params.phdrs;
 	for (i = 0; i < exec_params.hdr.e_phnum; i++, phdr++) {
 		switch (phdr->p_type) {
 		case PT_INTERP:
 			retval = -ENOMEM;
 			if (phdr->p_filesz > PATH_MAX)
 				goto error;
 			retval = -ENOENT;
 			if (phdr->p_filesz < 2)
 				goto error;
 			/* read the name of the interpreter into memory */
 			interpreter_name = kmalloc(phdr->p_filesz, GFP_KERNEL);
 			if (!interpreter_name)
 				goto error;
 			retval = kernel_read(bprm->file,
 					     phdr->p_offset,
 					     interpreter_name,
 					     phdr->p_filesz);
 			if (unlikely(retval != phdr->p_filesz)) {
 				if (retval >= 0)
 					retval = -ENOEXEC;
 				goto error;
 			}
 			retval = -ENOENT;
 			if (interpreter_name[phdr->p_filesz - 1] != '\0')
 				goto error;
 			kdebug("Using ELF interpreter %s", interpreter_name);
 			/* replace the program with the interpreter */
 			interpreter = open_exec(interpreter_name);
 			retval = PTR_ERR(interpreter);
 			if (IS_ERR(interpreter)) {
 				interpreter = NULL;
 				goto error;
 			}
 			/*
 			 * If the binary is not readable then enforce
 			 * mm->dumpable = 0 regardless of the interpreter's
 			 * permissions.
 			 */
 			if (file_permission(interpreter, MAY_READ) < 0)
 				bprm->interp_flags |= BINPRM_FLAGS_ENFORCE_NONDUMP;
 			retval = kernel_read(interpreter, 0, bprm->buf,
 					     BINPRM_BUF_SIZE);
 			if (unlikely(retval != BINPRM_BUF_SIZE)) {
 				if (retval >= 0)
 					retval = -ENOEXEC;
 				goto error;
 			}
 			interp_params.hdr = *((struct elfhdr *) bprm->buf);
 			break;
 		case PT_LOAD:
 #ifdef CONFIG_MMU
 			if (exec_params.load_addr == 0)
 				exec_params.load_addr = phdr->p_vaddr;
 #endif
 			break;
 		}
 	}
 	if (elf_check_const_displacement(&exec_params.hdr))
 		exec_params.flags |= ELF_FDPIC_FLAG_CONSTDISP;
 	/* perform insanity checks on the interpreter */
 	if (interpreter_name) {
 		retval = -ELIBBAD;
 		if (!is_elf_fdpic(&interp_params.hdr, interpreter))
 			goto error;
 		interp_params.flags = ELF_FDPIC_FLAG_PRESENT;
 		/* read the interpreter's program header table */
 		retval = elf_fdpic_fetch_phdrs(&interp_params, interpreter);
 		if (retval < 0)
 			goto error;
 	}
 	stack_size = exec_params.stack_size;
 	if (stack_size < interp_params.stack_size)
 		stack_size = interp_params.stack_size;
 	if (exec_params.flags & ELF_FDPIC_FLAG_EXEC_STACK)
 		executable_stack = EXSTACK_ENABLE_X;
 	else if (exec_params.flags & ELF_FDPIC_FLAG_NOEXEC_STACK)
 		executable_stack = EXSTACK_DISABLE_X;
 	else if (interp_params.flags & ELF_FDPIC_FLAG_EXEC_STACK)
 		executable_stack = EXSTACK_ENABLE_X;
 	else if (interp_params.flags & ELF_FDPIC_FLAG_NOEXEC_STACK)
 		executable_stack = EXSTACK_DISABLE_X;
 	else
 		executable_stack = EXSTACK_DEFAULT;
 	retval = -ENOEXEC;
 	if (stack_size == 0)
 		goto error;
 	if (elf_check_const_displacement(&interp_params.hdr))
 		interp_params.flags |= ELF_FDPIC_FLAG_CONSTDISP;
 	/* flush all traces of the currently running executable */
 	retval = flush_old_exec(bprm);
 	if (retval)
 		goto error;
 	/* there's now no turning back... the old userspace image is dead,
 	 * defunct, deceased, etc. after this point we have to exit via
 	 * error_kill */
 	set_personality(PER_LINUX_FDPIC);
 	set_binfmt(&elf_fdpic_format);
 	current->mm->start_code = 0;
 	current->mm->end_code = 0;
 	current->mm->start_stack = 0;
 	current->mm->start_data = 0;
 	current->mm->end_data = 0;
 	current->mm->context.exec_fdpic_loadmap = 0;
 	current->mm->context.interp_fdpic_loadmap = 0;
 	current->flags &= ~PF_FORKNOEXEC;
 #ifdef CONFIG_MMU
 	elf_fdpic_arch_lay_out_mm(&exec_params,
 				  &interp_params,
 				  &current->mm->start_stack,
 				  &current->mm->start_brk);
 	retval = setup_arg_pages(bprm, current->mm->start_stack,
 				 executable_stack);
 	if (retval < 0) {
 		send_sig(SIGKILL, current, 0);
 		goto error_kill;
 	}
 #endif
 	/* load the executable and interpreter into memory */
 	retval = elf_fdpic_map_file(&exec_params, bprm->file, current->mm,
 				    "executable");
 	if (retval < 0)
 		goto error_kill;
 	if (interpreter_name) {
 		retval = elf_fdpic_map_file(&interp_params, interpreter,
 					    current->mm, "interpreter");
 		if (retval < 0) {
 			printk(KERN_ERR "Unable to load interpreter\n");
 			goto error_kill;
 		}
 		allow_write_access(interpreter);
 		fput(interpreter);
 		interpreter = NULL;
 	}
 #ifdef CONFIG_MMU
 	if (!current->mm->start_brk)
 		current->mm->start_brk = current->mm->end_data;
 	current->mm->brk = current->mm->start_brk =
 		PAGE_ALIGN(current->mm->start_brk);
 #else
 	/* create a stack and brk area big enough for everyone
 	 * - the brk heap starts at the bottom and works up
 	 * - the stack starts at the top and works down
 	 */
 	stack_size = (stack_size + PAGE_SIZE - 1) & PAGE_MASK;
 	if (stack_size < PAGE_SIZE * 2)
 		stack_size = PAGE_SIZE * 2;
 	down_write(&current->mm->mmap_sem);
 	current->mm->start_brk = do_mmap(NULL, 0, stack_size,
 					 PROT_READ | PROT_WRITE | PROT_EXEC,
 					 MAP_PRIVATE | MAP_ANONYMOUS | MAP_GROWSDOWN,
 					 0);
 	if (IS_ERR_VALUE(current->mm->start_brk)) {
 		up_write(&current->mm->mmap_sem);
 		retval = current->mm->start_brk;
 		current->mm->start_brk = 0;
 		goto error_kill;
 	}
-	/* expand the stack mapping to use up the entire allocation granule */
-	fullsize = kobjsize((char *) current->mm->start_brk);
-	if (!IS_ERR_VALUE(do_mremap(current->mm->start_brk, stack_size,
-				    fullsize, 0, 0)))
-		stack_size = fullsize;
 	up_write(&current->mm->mmap_sem);
 	current->mm->brk = current->mm->start_brk;
 	current->mm->context.end_brk = current->mm->start_brk;
 	current->mm->context.end_brk +=
 		(stack_size > PAGE_SIZE) ? (stack_size - PAGE_SIZE) : 0;
 	current->mm->start_stack = current->mm->start_brk + stack_size;
 #endif
 	install_exec_creds(bprm);
 	current->flags &= ~PF_FORKNOEXEC;
 	if (create_elf_fdpic_tables(bprm, current->mm,
 				    &exec_params, &interp_params) < 0)
 		goto error_kill;
 	kdebug("- start_code  %lx", current->mm->start_code);
 	kdebug("- end_code    %lx", current->mm->end_code);
 	kdebug("- start_data  %lx", current->mm->start_data);
 	kdebug("- end_data    %lx", current->mm->end_data);
 	kdebug("- start_brk   %lx", current->mm->start_brk);
 	kdebug("- brk         %lx", current->mm->brk);
 	kdebug("- start_stack %lx", current->mm->start_stack);
 #ifdef ELF_FDPIC_PLAT_INIT
 	/*
 	 * The ABI may specify that certain registers be set up in special
 	 * ways (on i386 %edx is the address of a DT_FINI function, for
 	 * example.  This macro performs whatever initialization to
 	 * the regs structure is required.
 	 */
 	dynaddr = interp_params.dynamic_addr ?: exec_params.dynamic_addr;
 	ELF_FDPIC_PLAT_INIT(regs, exec_params.map_addr, interp_params.map_addr,
 			    dynaddr);
 #endif
 	/* everything is now ready... get the userspace context ready to roll */
 	entryaddr = interp_params.entry_addr ?: exec_params.entry_addr;
 	start_thread(regs, entryaddr, current->mm->start_stack);
 	retval = 0;
 error:
 	if (interpreter) {
 		allow_write_access(interpreter);
 		fput(interpreter);
 	}
 	kfree(interpreter_name);
 	kfree(exec_params.phdrs);
 	kfree(exec_params.loadmap);
 	kfree(interp_params.phdrs);
 	kfree(interp_params.loadmap);
 	return retval;
 	/* unrecoverable error - kill the process */
 error_kill:
 	send_sig(SIGSEGV, current, 0);
 	goto error;
 }
 /*****************************************************************************/
 #ifndef ELF_BASE_PLATFORM
 /*
  * AT_BASE_PLATFORM indicates the "real" hardware/microarchitecture.
  * If the arch defines ELF_BASE_PLATFORM (in asm/elf.h), the value
  * will be copied to the user stack in the same manner as AT_PLATFORM.
  */
 #define ELF_BASE_PLATFORM NULL
 #endif
 /*
  * present useful information to the program by shovelling it onto the new
  * process's stack
  */
 static int create_elf_fdpic_tables(struct linux_binprm *bprm,
 				   struct mm_struct *mm,
 				   struct elf_fdpic_params *exec_params,
 				   struct elf_fdpic_params *interp_params)
 {
 	const struct cred *cred = current_cred();
 	unsigned long sp, csp, nitems;
 	elf_caddr_t __user *argv, *envp;
 	size_t platform_len = 0, len;
 	char *k_platform, *k_base_platform;
 	char __user *u_platform, *u_base_platform, *p;
 	long hwcap;
 	int loop;
 	int nr;	/* reset for each csp adjustment */
 #ifdef CONFIG_MMU
 	/* In some cases (e.g. Hyper-Threading), we want to avoid L1 evictions
 	 * by the processes running on the same package. One thing we can do is
 	 * to shuffle the initial stack for them, so we give the architecture
 	 * an opportunity to do so here.
 	 */
 	sp = arch_align_stack(bprm->p);
 #else
 	sp = mm->start_stack;
 	/* stack the program arguments and environment */
 	if (elf_fdpic_transfer_args_to_stack(bprm, &sp) < 0)
 		return -EFAULT;
 #endif
 	hwcap = ELF_HWCAP;
 	/*
 	 * If this architecture has a platform capability string, copy it
 	 * to userspace.  In some cases (Sparc), this info is impossible
 	 * for userspace to get any other way, in others (i386) it is
 	 * merely difficult.
 	 */
 	k_platform = ELF_PLATFORM;
 	u_platform = NULL;
 	if (k_platform) {
 		platform_len = strlen(k_platform) + 1;
 		sp -= platform_len;
 		u_platform = (char __user *) sp;
 		if (__copy_to_user(u_platform, k_platform, platform_len) != 0)
 			return -EFAULT;
 	}
 	/*
 	 * If this architecture has a "base" platform capability
 	 * string, copy it to userspace.
 	 */
 	k_base_platform = ELF_BASE_PLATFORM;
 	u_base_platform = NULL;
 	if (k_base_platform) {
 		platform_len = strlen(k_base_platform) + 1;
 		sp -= platform_len;
 		u_base_platform = (char __user *) sp;
 		if (__copy_to_user(u_base_platform, k_base_platform, platform_len) != 0)
 			return -EFAULT;
 	}
 	sp &= ~7UL;
 	/* stack the load map(s) */
 	len = sizeof(struct elf32_fdpic_loadmap);
 	len += sizeof(struct elf32_fdpic_loadseg) * exec_params->loadmap->nsegs;
 	sp = (sp - len) & ~7UL;
 	exec_params->map_addr = sp;
 	if (copy_to_user((void __user *) sp, exec_params->loadmap, len) != 0)
 		return -EFAULT;
 	current->mm->context.exec_fdpic_loadmap = (unsigned long) sp;
 	if (interp_params->loadmap) {
 		len = sizeof(struct elf32_fdpic_loadmap);
 		len += sizeof(struct elf32_fdpic_loadseg) *
 			interp_params->loadmap->nsegs;
 		sp = (sp - len) & ~7UL;
 		interp_params->map_addr = sp;
 		if (copy_to_user((void __user *) sp, interp_params->loadmap,
 				 len) != 0)
 			return -EFAULT;
 		current->mm->context.interp_fdpic_loadmap = (unsigned long) sp;
 	}
 	/* force 16 byte _final_ alignment here for generality */
 #define DLINFO_ITEMS 15
 	nitems = 1 + DLINFO_ITEMS + (k_platform ? 1 : 0) +
 		(k_base_platform ? 1 : 0) + AT_VECTOR_SIZE_ARCH;
 	if (bprm->interp_flags & BINPRM_FLAGS_EXECFD)
 		nitems++;
 	csp = sp;
 	sp -= nitems * 2 * sizeof(unsigned long);
 	sp -= (bprm->envc + 1) * sizeof(char *);	/* envv[] */
 	sp -= (bprm->argc + 1) * sizeof(char *);	/* argv[] */
 	sp -= 1 * sizeof(unsigned long);		/* argc */
 	csp -= sp & 15UL;
 	sp -= sp & 15UL;
 	/* put the ELF interpreter info on the stack */
 #define NEW_AUX_ENT(id, val)						\
 	do {								\
 		struct { unsigned long _id, _val; } __user *ent;	\
 									\
 		ent = (void __user *) csp;				\
 		__put_user((id), &ent[nr]._id);				\
 		__put_user((val), &ent[nr]._val);			\
 		nr++;							\
 	} while (0)
 	nr = 0;
 	csp -= 2 * sizeof(unsigned long);
 	NEW_AUX_ENT(AT_NULL, 0);
 	if (k_platform) {
 		nr = 0;
 		csp -= 2 * sizeof(unsigned long);
 		NEW_AUX_ENT(AT_PLATFORM,
 			    (elf_addr_t) (unsigned long) u_platform);
 	}
 	if (k_base_platform) {
 		nr = 0;
 		csp -= 2 * sizeof(unsigned long);
 		NEW_AUX_ENT(AT_BASE_PLATFORM,
 			    (elf_addr_t) (unsigned long) u_base_platform);
 	}
 	if (bprm->interp_flags & BINPRM_FLAGS_EXECFD) {
 		nr = 0;
 		csp -= 2 * sizeof(unsigned long);
 		NEW_AUX_ENT(AT_EXECFD, bprm->interp_data);
 	}
 	nr = 0;
 	csp -= DLINFO_ITEMS * 2 * sizeof(unsigned long);
 	NEW_AUX_ENT(AT_HWCAP,	hwcap);
 	NEW_AUX_ENT(AT_PAGESZ,	PAGE_SIZE);
 	NEW_AUX_ENT(AT_CLKTCK,	CLOCKS_PER_SEC);
 	NEW_AUX_ENT(AT_PHDR,	exec_params->ph_addr);
 	NEW_AUX_ENT(AT_PHENT,	sizeof(struct elf_phdr));
 	NEW_AUX_ENT(AT_PHNUM,	exec_params->hdr.e_phnum);
 	NEW_AUX_ENT(AT_BASE,	interp_params->elfhdr_addr);
 	NEW_AUX_ENT(AT_FLAGS,	0);
 	NEW_AUX_ENT(AT_ENTRY,	exec_params->entry_addr);
 	NEW_AUX_ENT(AT_UID,	(elf_addr_t) cred->uid);
 	NEW_AUX_ENT(AT_EUID,	(elf_addr_t) cred->euid);
 	NEW_AUX_ENT(AT_GID,	(elf_addr_t) cred->gid);
 	NEW_AUX_ENT(AT_EGID,	(elf_addr_t) cred->egid);
 	NEW_AUX_ENT(AT_SECURE,	security_bprm_secureexec(bprm));
 	NEW_AUX_ENT(AT_EXECFN,	bprm->exec);
 #ifdef ARCH_DLINFO
 	nr = 0;
 	csp -= AT_VECTOR_SIZE_ARCH * 2 * sizeof(unsigned long);
 	/* ARCH_DLINFO must come last so platform specific code can enforce
 	 * special alignment requirements on the AUXV if necessary (eg. PPC).
 	 */
 	ARCH_DLINFO;
 #endif
 #undef NEW_AUX_ENT
 	/* allocate room for argv[] and envv[] */
 	csp -= (bprm->envc + 1) * sizeof(elf_caddr_t);
 	envp = (elf_caddr_t __user *) csp;
 	csp -= (bprm->argc + 1) * sizeof(elf_caddr_t);
 	argv = (elf_caddr_t __user *) csp;
 	/* stack argc */
 	csp -= sizeof(unsigned long);
 	__put_user(bprm->argc, (unsigned long __user *) csp);
 	BUG_ON(csp != sp);
 	/* fill in the argv[] array */
 #ifdef CONFIG_MMU
 	current->mm->arg_start = bprm->p;
 #else
 	current->mm->arg_start = current->mm->start_stack -
 		(MAX_ARG_PAGES * PAGE_SIZE - bprm->p);
 #endif
 	p = (char __user *) current->mm->arg_start;
 	for (loop = bprm->argc; loop > 0; loop--) {
 		__put_user((elf_caddr_t) p, argv++);
 		len = strnlen_user(p, MAX_ARG_STRLEN);
 		if (!len || len > MAX_ARG_STRLEN)
 			return -EINVAL;
 		p += len;
 	}
 	__put_user(NULL, argv);
 	current->mm->arg_end = (unsigned long) p;
 	/* fill in the envv[] array */
 	current->mm->env_start = (unsigned long) p;
 	for (loop = bprm->envc; loop > 0; loop--) {
 		__put_user((elf_caddr_t)(unsigned long) p, envp++);
 		len = strnlen_user(p, MAX_ARG_STRLEN);
 		if (!len || len > MAX_ARG_STRLEN)
 			return -EINVAL;
 		p += len;
 	}
 	__put_user(NULL, envp);
 	current->mm->env_end = (unsigned long) p;
 	mm->start_stack = (unsigned long) sp;
 	return 0;
 }
 /*****************************************************************************/
 /*
  * transfer the program arguments and environment from the holding pages onto
  * the stack
  */
 #ifndef CONFIG_MMU
 static int elf_fdpic_transfer_args_to_stack(struct linux_binprm *bprm,
 					    unsigned long *_sp)
 {
 	unsigned long index, stop, sp;
 	char *src;
 	int ret = 0;
 	stop = bprm->p >> PAGE_SHIFT;
 	sp = *_sp;
 	for (index = MAX_ARG_PAGES - 1; index >= stop; index--) {
 		src = kmap(bprm->page[index]);
 		sp -= PAGE_SIZE;
 		if (copy_to_user((void *) sp, src, PAGE_SIZE) != 0)
 			ret = -EFAULT;
 		kunmap(bprm->page[index]);
 		if (ret < 0)
 			goto out;
 	}
 	*_sp = (*_sp - (MAX_ARG_PAGES * PAGE_SIZE - bprm->p)) & ~15;
 out:
 	return ret;
 }
 #endif
 /*****************************************************************************/
 /*
  * load the appropriate binary image (executable or interpreter) into memory
  * - we assume no MMU is available
  * - if no other PIC bits are set in params->hdr->e_flags
  *   - we assume that the LOADable segments in the binary are independently relocatable
  *   - we assume R/O executable segments are shareable
  * - else
  *   - we assume the loadable parts of the image to require fixed displacement
  *   - the image is not shareable
  */
 static int elf_fdpic_map_file(struct elf_fdpic_params *params,
 			      struct file *file,
 			      struct mm_struct *mm,
 			      const char *what)
 {
 	struct elf32_fdpic_loadmap *loadmap;
 #ifdef CONFIG_MMU
 	struct elf32_fdpic_loadseg *mseg;
 #endif
 	struct elf32_fdpic_loadseg *seg;
 	struct elf32_phdr *phdr;
 	unsigned long load_addr, stop;
 	unsigned nloads, tmp;
 	size_t size;
 	int loop, ret;
 	/* allocate a load map table */
 	nloads = 0;
 	for (loop = 0; loop < params->hdr.e_phnum; loop++)
 		if (params->phdrs[loop].p_type == PT_LOAD)
 			nloads++;
 	if (nloads == 0)
 		return -ELIBBAD;
 	size = sizeof(*loadmap) + nloads * sizeof(*seg);
 	loadmap = kzalloc(size, GFP_KERNEL);
 	if (!loadmap)
 		return -ENOMEM;
 	params->loadmap = loadmap;
 	loadmap->version = ELF32_FDPIC_LOADMAP_VERSION;
 	loadmap->nsegs = nloads;
 	load_addr = params->load_addr;
 	seg = loadmap->segs;
 	/* map the requested LOADs into the memory space */
 	switch (params->flags & ELF_FDPIC_FLAG_ARRANGEMENT) {
 	case ELF_FDPIC_FLAG_CONSTDISP:
 	case ELF_FDPIC_FLAG_CONTIGUOUS:
 #ifndef CONFIG_MMU
 		ret = elf_fdpic_map_file_constdisp_on_uclinux(params, file, mm);
 		if (ret < 0)
 			return ret;
 		break;
 #endif
 	default:
 		ret = elf_fdpic_map_file_by_direct_mmap(params, file, mm);
 		if (ret < 0)
 			return ret;
 		break;
 	}
 	/* map the entry point */
 	if (params->hdr.e_entry) {
 		seg = loadmap->segs;
 		for (loop = loadmap->nsegs; loop > 0; loop--, seg++) {
 			if (params->hdr.e_entry >= seg->p_vaddr &&
 			    params->hdr.e_entry < seg->p_vaddr + seg->p_memsz) {
 				params->entry_addr =
 					(params->hdr.e_entry - seg->p_vaddr) +
 					seg->addr;
 				break;
 			}
 		}
 	}
 	/* determine where the program header table has wound up if mapped */
 	stop = params->hdr.e_phoff;
 	stop += params->hdr.e_phnum * sizeof (struct elf_phdr);
 	phdr = params->phdrs;
 	for (loop = 0; loop < params->hdr.e_phnum; loop++, phdr++) {
 		if (phdr->p_type != PT_LOAD)
 			continue;
 		if (phdr->p_offset > params->hdr.e_phoff ||
 		    phdr->p_offset + phdr->p_filesz < stop)
 			continue;
 		seg = loadmap->segs;
 		for (loop = loadmap->nsegs; loop > 0; loop--, seg++) {
 			if (phdr->p_vaddr >= seg->p_vaddr &&
 			    phdr->p_vaddr + phdr->p_filesz <=
 			    seg->p_vaddr + seg->p_memsz) {
 				params->ph_addr =
 					(phdr->p_vaddr - seg->p_vaddr) +
 					seg->addr +
 					params->hdr.e_phoff - phdr->p_offset;
 				break;
 			}
 		}
 		break;
 	}
 	/* determine where the dynamic section has wound up if there is one */
 	phdr = params->phdrs;
 	for (loop = 0; loop < params->hdr.e_phnum; loop++, phdr++) {
 		if (phdr->p_type != PT_DYNAMIC)
 			continue;
 		seg = loadmap->segs;
 		for (loop = loadmap->nsegs; loop > 0; loop--, seg++) {
 			if (phdr->p_vaddr >= seg->p_vaddr &&
 			    phdr->p_vaddr + phdr->p_memsz <=
 			    seg->p_vaddr + seg->p_memsz) {
 				params->dynamic_addr =
 					(phdr->p_vaddr - seg->p_vaddr) +
 					seg->addr;
 				/* check the dynamic section contains at least
 				 * one item, and that the last item is a NULL
 				 * entry */
 				if (phdr->p_memsz == 0 ||
 				    phdr->p_memsz % sizeof(Elf32_Dyn) != 0)
 					goto dynamic_error;
 				tmp = phdr->p_memsz / sizeof(Elf32_Dyn);
 				if (((Elf32_Dyn *)
 				     params->dynamic_addr)[tmp - 1].d_tag != 0)
 					goto dynamic_error;
 				break;
 			}
 		}
 		break;
 	}
 	/* now elide adjacent segments in the load map on MMU linux
 	 * - on uClinux the holes between may actually be filled with system
 	 *   stuff or stuff from other processes
 	 */
 #ifdef CONFIG_MMU
 	nloads = loadmap->nsegs;
 	mseg = loadmap->segs;
 	seg = mseg + 1;
 	for (loop = 1; loop < nloads; loop++) {
 		/* see if we have a candidate for merging */
 		if (seg->p_vaddr - mseg->p_vaddr == seg->addr - mseg->addr) {
 			load_addr = PAGE_ALIGN(mseg->addr + mseg->p_memsz);
 			if (load_addr == (seg->addr & PAGE_MASK)) {
 				mseg->p_memsz +=
 					load_addr -
 					(mseg->addr + mseg->p_memsz);
 				mseg->p_memsz += seg->addr & ~PAGE_MASK;
 				mseg->p_memsz += seg->p_memsz;
 				loadmap->nsegs--;
 				continue;
 			}
 		}
 		mseg++;
 		if (mseg != seg)
 			*mseg = *seg;
 	}
 #endif
 	kdebug("Mapped Object [%s]:", what);
 	kdebug("- elfhdr   : %lx", params->elfhdr_addr);
 	kdebug("- entry    : %lx", params->entry_addr);
 	kdebug("- PHDR[]   : %lx", params->ph_addr);
 	kdebug("- DYNAMIC[]: %lx", params->dynamic_addr);
 	seg = loadmap->segs;
 	for (loop = 0; loop < loadmap->nsegs; loop++, seg++)
 		kdebug("- LOAD[%d] : %08x-%08x [va=%x ms=%x]",
 		       loop,
 		       seg->addr, seg->addr + seg->p_memsz - 1,
 		       seg->p_vaddr, seg->p_memsz);
 	return 0;
 dynamic_error:
 	printk("ELF FDPIC %s with invalid DYNAMIC section (inode=%lu)\n",
 	       what, file->f_path.dentry->d_inode->i_ino);
 	return -ELIBBAD;
 }
 /*****************************************************************************/
 /*
  * map a file with constant displacement under uClinux
  */
 #ifndef CONFIG_MMU
 static int elf_fdpic_map_file_constdisp_on_uclinux(
 	struct elf_fdpic_params *params,
 	struct file *file,
 	struct mm_struct *mm)
 {
 	struct elf32_fdpic_loadseg *seg;
 	struct elf32_phdr *phdr;
 	unsigned long load_addr, base = ULONG_MAX, top = 0, maddr = 0, mflags;
 	loff_t fpos;
 	int loop, ret;
 	load_addr = params->load_addr;
 	seg = params->loadmap->segs;
 	/* determine the bounds of the contiguous overall allocation we must
 	 * make */
 	phdr = params->phdrs;
 	for (loop = 0; loop < params->hdr.e_phnum; loop++, phdr++) {
 		if (params->phdrs[loop].p_type != PT_LOAD)
 			continue;
 		if (base > phdr->p_vaddr)
 			base = phdr->p_vaddr;
 		if (top < phdr->p_vaddr + phdr->p_memsz)
 			top = phdr->p_vaddr + phdr->p_memsz;
 	}
 	/* allocate one big anon block for everything */
 	mflags = MAP_PRIVATE;
 	if (params->flags & ELF_FDPIC_FLAG_EXECUTABLE)
 		mflags |= MAP_EXECUTABLE;
 	down_write(&mm->mmap_sem);
 	maddr = do_mmap(NULL, load_addr, top - base,
 			PROT_READ | PROT_WRITE | PROT_EXEC, mflags, 0);
 	up_write(&mm->mmap_sem);
 	if (IS_ERR_VALUE(maddr))
 		return (int) maddr;
 	if (load_addr != 0)
 		load_addr += PAGE_ALIGN(top - base);
 	/* and then load the file segments into it */
 	phdr = params->phdrs;
 	for (loop = 0; loop < params->hdr.e_phnum; loop++, phdr++) {
 		if (params->phdrs[loop].p_type != PT_LOAD)
 			continue;
 		fpos = phdr->p_offset;
 		seg->addr = maddr + (phdr->p_vaddr - base);
 		seg->p_vaddr = phdr->p_vaddr;
 		seg->p_memsz = phdr->p_memsz;
 		ret = file->f_op->read(file, (void *) seg->addr,
 				       phdr->p_filesz, &fpos);
 		if (ret < 0)
 			return ret;
 		/* map the ELF header address if in this segment */
 		if (phdr->p_offset == 0)
 			params->elfhdr_addr = seg->addr;
 		/* clear any space allocated but not loaded */
 		if (phdr->p_filesz < phdr->p_memsz)
 			clear_user((void *) (seg->addr + phdr->p_filesz),
 				   phdr->p_memsz - phdr->p_filesz);
 		if (mm) {
 			if (phdr->p_flags & PF_X) {
 				if (!mm->start_code) {
 					mm->start_code = seg->addr;
 					mm->end_code = seg->addr +
 						phdr->p_memsz;
 				}
 			} else if (!mm->start_data) {
 				mm->start_data = seg->addr;
 #ifndef CONFIG_MMU
 				mm->end_data = seg->addr + phdr->p_memsz;
 #endif
 			}
 #ifdef CONFIG_MMU
 			if (seg->addr + phdr->p_memsz > mm->end_data)
 				mm->end_data = seg->addr + phdr->p_memsz;
 #endif
 		}
 		seg++;
 	}
 	return 0;
 }
 #endif
 /*****************************************************************************/
 /*
  * map a binary by direct mmap() of the individual PT_LOAD segments
  */
 static int elf_fdpic_map_file_by_direct_mmap(struct elf_fdpic_params *params,
 					     struct file *file,
 					     struct mm_struct *mm)
 {
 	struct elf32_fdpic_loadseg *seg;
 	struct elf32_phdr *phdr;
 	unsigned long load_addr, delta_vaddr;
 	int loop, dvset;
 	load_addr = params->load_addr;
 	delta_vaddr = 0;
 	dvset = 0;
 	seg = params->loadmap->segs;
 	/* deal with each load segment separately */
 	phdr = params->phdrs;
 	for (loop = 0; loop < params->hdr.e_phnum; loop++, phdr++) {
 		unsigned long maddr, disp, excess, excess1;
 		int prot = 0, flags;
 		if (phdr->p_type != PT_LOAD)
 			continue;
 		kdebug("[LOAD] va=%lx of=%lx fs=%lx ms=%lx",
 		       (unsigned long) phdr->p_vaddr,
 		       (unsigned long) phdr->p_offset,
 		       (unsigned long) phdr->p_filesz,
 		       (unsigned long) phdr->p_memsz);
 		/* determine the mapping parameters */
 		if (phdr->p_flags & PF_R) prot |= PROT_READ;
 		if (phdr->p_flags & PF_W) prot |= PROT_WRITE;
 		if (phdr->p_flags & PF_X) prot |= PROT_EXEC;
 		flags = MAP_PRIVATE | MAP_DENYWRITE;
 		if (params->flags & ELF_FDPIC_FLAG_EXECUTABLE)
 			flags |= MAP_EXECUTABLE;
 		maddr = 0;
 		switch (params->flags & ELF_FDPIC_FLAG_ARRANGEMENT) {
 		case ELF_FDPIC_FLAG_INDEPENDENT:
 			/* PT_LOADs are independently locatable */
 			break;
 		case ELF_FDPIC_FLAG_HONOURVADDR:
 			/* the specified virtual address must be honoured */
 			maddr = phdr->p_vaddr;
 			flags |= MAP_FIXED;
 			break;
 		case ELF_FDPIC_FLAG_CONSTDISP:
 			/* constant displacement
 			 * - can be mapped anywhere, but must be mapped as a
 			 *   unit
 			 */
 			if (!dvset) {
 				maddr = load_addr;
 				delta_vaddr = phdr->p_vaddr;
 				dvset = 1;
 			} else {
 				maddr = load_addr + phdr->p_vaddr - delta_vaddr;
 				flags |= MAP_FIXED;
 			}
 			break;
 		case ELF_FDPIC_FLAG_CONTIGUOUS:
 			/* contiguity handled later */
 			break;
 		default:
 			BUG();
 		}
 		maddr &= PAGE_MASK;
 		/* create the mapping */
 		disp = phdr->p_vaddr & ~PAGE_MASK;
 		down_write(&mm->mmap_sem);
 		maddr = do_mmap(file, maddr, phdr->p_memsz + disp, prot, flags,
 				phdr->p_offset - disp);
 		up_write(&mm->mmap_sem);
 		kdebug("mmap[%d] <file> sz=%lx pr=%x fl=%x of=%lx --> %08lx",
 		       loop, phdr->p_memsz + disp, prot, flags,
 		       phdr->p_offset - disp, maddr);
 		if (IS_ERR_VALUE(maddr))
 			return (int) maddr;
 		if ((params->flags & ELF_FDPIC_FLAG_ARRANGEMENT) ==
 		    ELF_FDPIC_FLAG_CONTIGUOUS)
 			load_addr += PAGE_ALIGN(phdr->p_memsz + disp);
 		seg->addr = maddr + disp;
 		seg->p_vaddr = phdr->p_vaddr;
 		seg->p_memsz = phdr->p_memsz;
 		/* map the ELF header address if in this segment */
 		if (phdr->p_offset == 0)
 			params->elfhdr_addr = seg->addr;
 		/* clear the bit between beginning of mapping and beginning of
 		 * PT_LOAD */
 		if (prot & PROT_WRITE && disp > 0) {
 			kdebug("clear[%d] ad=%lx sz=%lx", loop, maddr, disp);
 			clear_user((void __user *) maddr, disp);
 			maddr += disp;
 		}
 		/* clear any space allocated but not loaded
 		 * - on uClinux we can just clear the lot
 		 * - on MMU linux we'll get a SIGBUS beyond the last page
 		 *   extant in the file
 		 */
 		excess = phdr->p_memsz - phdr->p_filesz;
 		excess1 = PAGE_SIZE - ((maddr + phdr->p_filesz) & ~PAGE_MASK);
 #ifdef CONFIG_MMU
 		if (excess > excess1) {
 			unsigned long xaddr = maddr + phdr->p_filesz + excess1;
 			unsigned long xmaddr;
 			flags |= MAP_FIXED | MAP_ANONYMOUS;
 			down_write(&mm->mmap_sem);
 			xmaddr = do_mmap(NULL, xaddr, excess - excess1,
 					 prot, flags, 0);
 			up_write(&mm->mmap_sem);
 			kdebug("mmap[%d] <anon>"
 			       " ad=%lx sz=%lx pr=%x fl=%x of=0 --> %08lx",
 			       loop, xaddr, excess - excess1, prot, flags,
 			       xmaddr);
 			if (xmaddr != xaddr)
 				return -ENOMEM;
 		}
 		if (prot & PROT_WRITE && excess1 > 0) {
 			kdebug("clear[%d] ad=%lx sz=%lx",
 			       loop, maddr + phdr->p_filesz, excess1);
 			clear_user((void __user *) maddr + phdr->p_filesz,
 				   excess1);
 		}
 #else
 		if (excess > 0) {
 			kdebug("clear[%d] ad=%lx sz=%lx",
 			       loop, maddr + phdr->p_filesz, excess);
 			clear_user((void *) maddr + phdr->p_filesz, excess);
 		}
 #endif
 		if (mm) {
 			if (phdr->p_flags & PF_X) {
 				if (!mm->start_code) {
 					mm->start_code = maddr;
 					mm->end_code = maddr + phdr->p_memsz;
 				}
 			} else if (!mm->start_data) {
 				mm->start_data = maddr;
 				mm->end_data = maddr + phdr->p_memsz;
 			}
 		}
 		seg++;
 	}
 	return 0;
 }
 /*****************************************************************************/
 /*
  * ELF-FDPIC core dumper
  *
  * Modelled on fs/exec.c:aout_core_dump()
  * Jeremy Fitzhardinge <jeremy@sw.oz.au>
  *
  * Modelled on fs/binfmt_elf.c core dumper
  */
 #if defined(USE_ELF_CORE_DUMP) && defined(CONFIG_ELF_CORE)
 /*
  * These are the only things you should do on a core-file: use only these
  * functions to write out all the necessary info.
  */
 static int dump_write(struct file *file, const void *addr, int nr)
 {
 	return file->f_op->write(file, addr, nr, &file->f_pos) == nr;
 }
 static int dump_seek(struct file *file, loff_t off)
 {
 	if (file->f_op->llseek) {
 		if (file->f_op->llseek(file, off, SEEK_SET) != off)
 			return 0;
 	} else {
 		file->f_pos = off;
 	}
 	return 1;
 }
 /*
  * Decide whether a segment is worth dumping; default is yes to be
  * sure (missing info is worse than too much; etc).
  * Personally I'd include everything, and use the coredump limit...
  *
  * I think we should skip something. But I am not sure how. H.J.
  */
 static int maydump(struct vm_area_struct *vma, unsigned long mm_flags)
 {
 	int dump_ok;
 	/* Do not dump I/O mapped devices or special mappings */
 	if (vma->vm_flags & (VM_IO | VM_RESERVED)) {
 		kdcore("%08lx: %08lx: no (IO)", vma->vm_start, vma->vm_flags);
 		return 0;
 	}
 	/* If we may not read the contents, don't allow us to dump
 	 * them either. "dump_write()" can't handle it anyway.
 	 */
 	if (!(vma->vm_flags & VM_READ)) {
 		kdcore("%08lx: %08lx: no (!read)", vma->vm_start, vma->vm_flags);
 		return 0;
 	}
 	/* By default, dump shared memory if mapped from an anonymous file. */
 	if (vma->vm_flags & VM_SHARED) {
 		if (vma->vm_file->f_path.dentry->d_inode->i_nlink == 0) {
 			dump_ok = test_bit(MMF_DUMP_ANON_SHARED, &mm_flags);
 			kdcore("%08lx: %08lx: %s (share)", vma->vm_start,
 			       vma->vm_flags, dump_ok ? "yes" : "no");
 			return dump_ok;
 		}
 		dump_ok = test_bit(MMF_DUMP_MAPPED_SHARED, &mm_flags);
 		kdcore("%08lx: %08lx: %s (share)", vma->vm_start,
 		       vma->vm_flags, dump_ok ? "yes" : "no");
 		return dump_ok;
 	}
 #ifdef CONFIG_MMU
 	/* By default, if it hasn't been written to, don't write it out */
 	if (!vma->anon_vma) {
 		dump_ok = test_bit(MMF_DUMP_MAPPED_PRIVATE, &mm_flags);
 		kdcore("%08lx: %08lx: %s (!anon)", vma->vm_start,
 		       vma->vm_flags, dump_ok ? "yes" : "no");
 		return dump_ok;
 	}
 #endif
 	dump_ok = test_bit(MMF_DUMP_ANON_PRIVATE, &mm_flags);
 	kdcore("%08lx: %08lx: %s", vma->vm_start, vma->vm_flags,
 	       dump_ok ? "yes" : "no");
 	return dump_ok;
 }
 /* An ELF note in memory */
 struct memelfnote
 {
 	const char *name;
 	int type;
 	unsigned int datasz;
 	void *data;
 };
 static int notesize(struct memelfnote *en)
 {
 	int sz;
 	sz = sizeof(struct elf_note);
 	sz += roundup(strlen(en->name) + 1, 4);
 	sz += roundup(en->datasz, 4);
 	return sz;
 }
 /* #define DEBUG */
 #define DUMP_WRITE(addr, nr)	\
 	do { if (!dump_write(file, (addr), (nr))) return 0; } while(0)
 #define DUMP_SEEK(off)	\
 	do { if (!dump_seek(file, (off))) return 0; } while(0)
 static int writenote(struct memelfnote *men, struct file *file)
 {
 	struct elf_note en;
 	en.n_namesz = strlen(men->name) + 1;
 	en.n_descsz = men->datasz;
 	en.n_type = men->type;
 	DUMP_WRITE(&en, sizeof(en));
 	DUMP_WRITE(men->name, en.n_namesz);
 	/* XXX - cast from long long to long to avoid need for libgcc.a */
 	DUMP_SEEK(roundup((unsigned long)file->f_pos, 4));	/* XXX */
 	DUMP_WRITE(men->data, men->datasz);
 	DUMP_SEEK(roundup((unsigned long)file->f_pos, 4));	/* XXX */
 	return 1;
 }
 #undef DUMP_WRITE
 #undef DUMP_SEEK
 #define DUMP_WRITE(addr, nr)	\
 	if ((size += (nr)) > limit || !dump_write(file, (addr), (nr))) \
 		goto end_coredump;
 #define DUMP_SEEK(off)	\
 	if (!dump_seek(file, (off))) \
 		goto end_coredump;
 static inline void fill_elf_fdpic_header(struct elfhdr *elf, int segs)
 {
 	memcpy(elf->e_ident, ELFMAG, SELFMAG);
 	elf->e_ident[EI_CLASS] = ELF_CLASS;
 	elf->e_ident[EI_DATA] = ELF_DATA;
 	elf->e_ident[EI_VERSION] = EV_CURRENT;
 	elf->e_ident[EI_OSABI] = ELF_OSABI;
 	memset(elf->e_ident+EI_PAD, 0, EI_NIDENT-EI_PAD);
 	elf->e_type = ET_CORE;
 	elf->e_machine = ELF_ARCH;
 	elf->e_version = EV_CURRENT;
 	elf->e_entry = 0;
 	elf->e_phoff = sizeof(struct elfhdr);
 	elf->e_shoff = 0;
 	elf->e_flags = ELF_FDPIC_CORE_EFLAGS;
 	elf->e_ehsize = sizeof(struct elfhdr);
 	elf->e_phentsize = sizeof(struct elf_phdr);
 	elf->e_phnum = segs;
 	elf->e_shentsize = 0;
 	elf->e_shnum = 0;
 	elf->e_shstrndx = 0;
 	return;
 }
 static inline void fill_elf_note_phdr(struct elf_phdr *phdr, int sz, loff_t offset)
 {
 	phdr->p_type = PT_NOTE;
 	phdr->p_offset = offset;
 	phdr->p_vaddr = 0;
 	phdr->p_paddr = 0;
 	phdr->p_filesz = sz;
 	phdr->p_memsz = 0;
 	phdr->p_flags = 0;
 	phdr->p_align = 0;
 	return;
 }
 static inline void fill_note(struct memelfnote *note, const char *name, int type,
 		unsigned int sz, void *data)
 {
 	note->name = name;
 	note->type = type;
 	note->datasz = sz;
 	note->data = data;
 	return;
 }
 /*
  * fill up all the fields in prstatus from the given task struct, except
  * registers which need to be filled up seperately.
  */
 static void fill_prstatus(struct elf_prstatus *prstatus,
 			  struct task_struct *p, long signr)
 {
 	prstatus->pr_info.si_signo = prstatus->pr_cursig = signr;
 	prstatus->pr_sigpend = p->pending.signal.sig[0];
 	prstatus->pr_sighold = p->blocked.sig[0];
 	prstatus->pr_pid = task_pid_vnr(p);
 	prstatus->pr_ppid = task_pid_vnr(p->parent);
 	prstatus->pr_pgrp = task_pgrp_vnr(p);
 	prstatus->pr_sid = task_session_vnr(p);
 	if (thread_group_leader(p)) {
 		struct task_cputime cputime;
 		/*
 		 * This is the record for the group leader.  It shows the
 		 * group-wide total, not its individual thread total.
 		 */
 		thread_group_cputime(p, &cputime);
 		cputime_to_timeval(cputime.utime, &prstatus->pr_utime);
 		cputime_to_timeval(cputime.stime, &prstatus->pr_stime);
 	} else {
 		cputime_to_timeval(p->utime, &prstatus->pr_utime);
 		cputime_to_timeval(p->stime, &prstatus->pr_stime);
 	}
 	cputime_to_timeval(p->signal->cutime, &prstatus->pr_cutime);
 	cputime_to_timeval(p->signal->cstime, &prstatus->pr_cstime);
 	prstatus->pr_exec_fdpic_loadmap = p->mm->context.exec_fdpic_loadmap;
 	prstatus->pr_interp_fdpic_loadmap = p->mm->context.interp_fdpic_loadmap;
 }
 static int fill_psinfo(struct elf_prpsinfo *psinfo, struct task_struct *p,
 		       struct mm_struct *mm)
 {
 	const struct cred *cred;
 	unsigned int i, len;
 	/* first copy the parameters from user space */
 	memset(psinfo, 0, sizeof(struct elf_prpsinfo));
 	len = mm->arg_end - mm->arg_start;
 	if (len >= ELF_PRARGSZ)
 		len = ELF_PRARGSZ - 1;
 	if (copy_from_user(&psinfo->pr_psargs,
 		           (const char __user *) mm->arg_start, len))
 		return -EFAULT;
 	for (i = 0; i < len; i++)
 		if (psinfo->pr_psargs[i] == 0)
 			psinfo->pr_psargs[i] = ' ';
 	psinfo->pr_psargs[len] = 0;
 	psinfo->pr_pid = task_pid_vnr(p);
 	psinfo->pr_ppid = task_pid_vnr(p->parent);
 	psinfo->pr_pgrp = task_pgrp_vnr(p);
 	psinfo->pr_sid = task_session_vnr(p);
 	i = p->state ? ffz(~p->state) + 1 : 0;
 	psinfo->pr_state = i;
 	psinfo->pr_sname = (i > 5) ? '.' : "RSDTZW"[i];
 	psinfo->pr_zomb = psinfo->pr_sname == 'Z';
 	psinfo->pr_nice = task_nice(p);
 	psinfo->pr_flag = p->flags;
 	rcu_read_lock();
 	cred = __task_cred(p);
 	SET_UID(psinfo->pr_uid, cred->uid);
 	SET_GID(psinfo->pr_gid, cred->gid);
 	rcu_read_unlock();
 	strncpy(psinfo->pr_fname, p->comm, sizeof(psinfo->pr_fname));
 	return 0;
 }
 /* Here is the structure in which status of each thread is captured. */
 struct elf_thread_status
 {
 	struct list_head list;
 	struct elf_prstatus prstatus;	/* NT_PRSTATUS */
 	elf_fpregset_t fpu;		/* NT_PRFPREG */
 	struct task_struct *thread;
 #ifdef ELF_CORE_COPY_XFPREGS
 	elf_fpxregset_t xfpu;		/* ELF_CORE_XFPREG_TYPE */
 #endif
 	struct memelfnote notes[3];
 	int num_notes;
 };
 /*
  * In order to add the specific thread information for the elf file format,
  * we need to keep a linked list of every thread's pr_status and then create
  * a single section for them in the final core file.
  */
 static int elf_dump_thread_status(long signr, struct elf_thread_status *t)
 {
 	struct task_struct *p = t->thread;
 	int sz = 0;
 	t->num_notes = 0;
 	fill_prstatus(&t->prstatus, p, signr);
 	elf_core_copy_task_regs(p, &t->prstatus.pr_reg);
 	fill_note(&t->notes[0], "CORE", NT_PRSTATUS, sizeof(t->prstatus),
 		  &t->prstatus);
 	t->num_notes++;
 	sz += notesize(&t->notes[0]);
 	t->prstatus.pr_fpvalid = elf_core_copy_task_fpregs(p, NULL, &t->fpu);
 	if (t->prstatus.pr_fpvalid) {
 		fill_note(&t->notes[1], "CORE", NT_PRFPREG, sizeof(t->fpu),
 			  &t->fpu);
 		t->num_notes++;
 		sz += notesize(&t->notes[1]);
 	}
 #ifdef ELF_CORE_COPY_XFPREGS
 	if (elf_core_copy_task_xfpregs(p, &t->xfpu)) {
 		fill_note(&t->notes[2], "LINUX", ELF_CORE_XFPREG_TYPE,
 			  sizeof(t->xfpu), &t->xfpu);
 		t->num_notes++;
 		sz += notesize(&t->notes[2]);
 	}
 #endif
 	return sz;
 }
 /*
  * dump the segments for an MMU process
  */
 #ifdef CONFIG_MMU
 static int elf_fdpic_dump_segments(struct file *file, size_t *size,
 			   unsigned long *limit, unsigned long mm_flags)
 {
 	struct vm_area_struct *vma;
 	for (vma = current->mm->mmap; vma; vma = vma->vm_next) {
 		unsigned long addr;
 		if (!maydump(vma, mm_flags))
 			continue;
 		for (addr = vma->vm_start;
 		     addr < vma->vm_end;
 		     addr += PAGE_SIZE
 		     ) {
 			struct vm_area_struct *vma;
 			struct page *page;
 			if (get_user_pages(current, current->mm, addr, 1, 0, 1,
 					   &page, &vma) <= 0) {
 				DUMP_SEEK(file->f_pos + PAGE_SIZE);
 			}
 			else if (page == ZERO_PAGE(0)) {
 				page_cache_release(page);
 				DUMP_SEEK(file->f_pos + PAGE_SIZE);
 			}
 			else {
 				void *kaddr;
 				flush_cache_page(vma, addr, page_to_pfn(page));
 				kaddr = kmap(page);
 				if ((*size += PAGE_SIZE) > *limit ||
 				    !dump_write(file, kaddr, PAGE_SIZE)
 				    ) {
 					kunmap(page);
 					page_cache_release(page);
 					return -EIO;
 				}
 				kunmap(page);
 				page_cache_release(page);
 			}
 		}
 	}
 	return 0;
 end_coredump:
 	return -EFBIG;
 }
 #endif
 /*
  * dump the segments for a NOMMU process
  */
 #ifndef CONFIG_MMU
 static int elf_fdpic_dump_segments(struct file *file, size_t *size,
 			   unsigned long *limit, unsigned long mm_flags)
 {
-	struct vm_list_struct *vml;
+	struct vm_area_struct *vma;
-	for (vml = current->mm->context.vmlist; vml; vml = vml->next) {
+	for (vma = current->mm->mmap; vma; vma = vma->vm_next) {
-	struct vm_area_struct *vma = vml->vma;
 		if (!maydump(vma, mm_flags))
 			continue;
 		if ((*size += PAGE_SIZE) > *limit)
 			return -EFBIG;
 		if (!dump_write(file, (void *) vma->vm_start,
 				vma->vm_end - vma->vm_start))
 			return -EIO;
 	}
 	return 0;
 }
 #endif
 /*
  * Actual dumper
  *
  * This is a two-pass process; first we find the offsets of the bits,
  * and then they are actually written out.  If we run out of core limit
  * we just truncate.
  */
 static int elf_fdpic_core_dump(long signr, struct pt_regs *regs,
 			       struct file *file, unsigned long limit)
 {
 #define	NUM_NOTES	6
 	int has_dumped = 0;
 	mm_segment_t fs;
 	int segs;
 	size_t size = 0;
 	int i;
 	struct vm_area_struct *vma;
 	struct elfhdr *elf = NULL;
 	loff_t offset = 0, dataoff;
 	int numnote;
 	struct memelfnote *notes = NULL;
 	struct elf_prstatus *prstatus = NULL;	/* NT_PRSTATUS */
 	struct elf_prpsinfo *psinfo = NULL;	/* NT_PRPSINFO */
  	LIST_HEAD(thread_list);
  	struct list_head *t;
 	elf_fpregset_t *fpu = NULL;
 #ifdef ELF_CORE_COPY_XFPREGS
 	elf_fpxregset_t *xfpu = NULL;
 #endif
 	int thread_status_size = 0;
-#ifndef CONFIG_MMU
-	struct vm_list_struct *vml;
-#endif
 	elf_addr_t *auxv;
 	unsigned long mm_flags;
 	/*
 	 * We no longer stop all VM operations.
 	 *
 	 * This is because those proceses that could possibly change map_count
 	 * or the mmap / vma pages are now blocked in do_exit on current
 	 * finishing this core dump.
 	 *
 	 * Only ptrace can touch these memory addresses, but it doesn't change
 	 * the map_count or the pages allocated. So no possibility of crashing
 	 * exists while dumping the mm->vm_next areas to the core file.
 	 */
 	/* alloc memory for large data structures: too large to be on stack */
 	elf = kmalloc(sizeof(*elf), GFP_KERNEL);
 	if (!elf)
 		goto cleanup;
 	prstatus = kzalloc(sizeof(*prstatus), GFP_KERNEL);
 	if (!prstatus)
 		goto cleanup;
 	psinfo = kmalloc(sizeof(*psinfo), GFP_KERNEL);
 	if (!psinfo)
 		goto cleanup;
 	notes = kmalloc(NUM_NOTES * sizeof(struct memelfnote), GFP_KERNEL);
 	if (!notes)
 		goto cleanup;
 	fpu = kmalloc(sizeof(*fpu), GFP_KERNEL);
 	if (!fpu)
 		goto cleanup;
 #ifdef ELF_CORE_COPY_XFPREGS
 	xfpu = kmalloc(sizeof(*xfpu), GFP_KERNEL);
 	if (!xfpu)
 		goto cleanup;
 #endif
 	if (signr) {
 		struct core_thread *ct;
 		struct elf_thread_status *tmp;
 		for (ct = current->mm->core_state->dumper.next;
 						ct; ct = ct->next) {
 			tmp = kzalloc(sizeof(*tmp), GFP_KERNEL);
 			if (!tmp)
 				goto cleanup;
 			tmp->thread = ct->task;
 			list_add(&tmp->list, &thread_list);
 		}
 		list_for_each(t, &thread_list) {
 			struct elf_thread_status *tmp;
 			int sz;
 			tmp = list_entry(t, struct elf_thread_status, list);
 			sz = elf_dump_thread_status(signr, tmp);
 			thread_status_size += sz;
 		}
 	}
 	/* now collect the dump for the current */
 	fill_prstatus(prstatus, current, signr);
 	elf_core_copy_regs(&prstatus->pr_reg, regs);
-#ifdef CONFIG_MMU
 	segs = current->mm->map_count;
-#else
-	segs = 0;
-	for (vml = current->mm->context.vmlist; vml; vml = vml->next)
-	    segs++;
-#endif
 #ifdef ELF_CORE_EXTRA_PHDRS
 	segs += ELF_CORE_EXTRA_PHDRS;
 #endif
 	/* Set up header */
 	fill_elf_fdpic_header(elf, segs + 1);	/* including notes section */
 	has_dumped = 1;
 	current->flags |= PF_DUMPCORE;
 	/*
 	 * Set up the notes in similar form to SVR4 core dumps made
 	 * with info from their /proc.
 	 */
 	fill_note(notes + 0, "CORE", NT_PRSTATUS, sizeof(*prstatus), prstatus);
 	fill_psinfo(psinfo, current->group_leader, current->mm);
 	fill_note(notes + 1, "CORE", NT_PRPSINFO, sizeof(*psinfo), psinfo);
 	numnote = 2;
 	auxv = (elf_addr_t *) current->mm->saved_auxv;
 	i = 0;
 	do
 		i += 2;
 	while (auxv[i - 2] != AT_NULL);
 	fill_note(&notes[numnote++], "CORE", NT_AUXV,
 		  i * sizeof(elf_addr_t), auxv);
   	/* Try to dump the FPU. */
 	if ((prstatus->pr_fpvalid =
 	     elf_core_copy_task_fpregs(current, regs, fpu)))
 		fill_note(notes + numnote++,
 			  "CORE", NT_PRFPREG, sizeof(*fpu), fpu);
 #ifdef ELF_CORE_COPY_XFPREGS
 	if (elf_core_copy_task_xfpregs(current, xfpu))
 		fill_note(notes + numnote++,
 			  "LINUX", ELF_CORE_XFPREG_TYPE, sizeof(*xfpu), xfpu);
 #endif
 	fs = get_fs();
 	set_fs(KERNEL_DS);
 	DUMP_WRITE(elf, sizeof(*elf));
 	offset += sizeof(*elf);				/* Elf header */
 	offset += (segs+1) * sizeof(struct elf_phdr);	/* Program headers */
 	/* Write notes phdr entry */
 	{
 		struct elf_phdr phdr;
 		int sz = 0;
 		for (i = 0; i < numnote; i++)
 			sz += notesize(notes + i);
 		sz += thread_status_size;
 		fill_elf_note_phdr(&phdr, sz, offset);
 		offset += sz;
 		DUMP_WRITE(&phdr, sizeof(phdr));
 	}
 	/* Page-align dumped data */
 	dataoff = offset = roundup(offset, ELF_EXEC_PAGESIZE);
 	/*
 	 * We must use the same mm->flags while dumping core to avoid
 	 * inconsistency between the program headers and bodies, otherwise an
 	 * unusable core file can be generated.
 	 */
 	mm_flags = current->mm->flags;
 	/* write program headers for segments dump */
-	for (
+	for (vma = current->mm->mmap; vma; vma = vma->vm_next) {
-#ifdef CONFIG_MMU
-		vma = current->mm->mmap; vma; vma = vma->vm_next
-#else
-			vml = current->mm->context.vmlist; vml; vml = vml->next
-#endif
-	     ) {
 		struct elf_phdr phdr;
 		size_t sz;
-#ifndef CONFIG_MMU
-		vma = vml->vma;
-#endif
 		sz = vma->vm_end - vma->vm_start;
 		phdr.p_type = PT_LOAD;
 		phdr.p_offset = offset;
 		phdr.p_vaddr = vma->vm_start;
 		phdr.p_paddr = 0;
 		phdr.p_filesz = maydump(vma, mm_flags) ? sz : 0;
 		phdr.p_memsz = sz;
 		offset += phdr.p_filesz;
 		phdr.p_flags = vma->vm_flags & VM_READ ? PF_R : 0;
 		if (vma->vm_flags & VM_WRITE)
 			phdr.p_flags |= PF_W;
 		if (vma->vm_flags & VM_EXEC)
 			phdr.p_flags |= PF_X;
 		phdr.p_align = ELF_EXEC_PAGESIZE;
 		DUMP_WRITE(&phdr, sizeof(phdr));
 	}
 #ifdef ELF_CORE_WRITE_EXTRA_PHDRS
 	ELF_CORE_WRITE_EXTRA_PHDRS;
 #endif
  	/* write out the notes section */
 	for (i = 0; i < numnote; i++)
 		if (!writenote(notes + i, file))
 			goto end_coredump;
 	/* write out the thread status notes section */
 	list_for_each(t, &thread_list) {
 		struct elf_thread_status *tmp =
 				list_entry(t, struct elf_thread_status, list);
 		for (i = 0; i < tmp->num_notes; i++)
 			if (!writenote(&tmp->notes[i], file))
 				goto end_coredump;
 	}
 	DUMP_SEEK(dataoff);
 	if (elf_fdpic_dump_segments(file, &size, &limit, mm_flags) < 0)
 		goto end_coredump;
 #ifdef ELF_CORE_WRITE_EXTRA_DATA
 	ELF_CORE_WRITE_EXTRA_DATA;
 #endif
 	if (file->f_pos != offset) {
 		/* Sanity check */
 		printk(KERN_WARNING
 		       "elf_core_dump: file->f_pos (%lld) != offset (%lld)\n",
 		       file->f_pos, offset);
 	}
 end_coredump:
 	set_fs(fs);
 cleanup:
 	while (!list_empty(&thread_list)) {
 		struct list_head *tmp = thread_list.next;
 		list_del(tmp);
 		kfree(list_entry(tmp, struct elf_thread_status, list));
 	}
 	kfree(elf);
 	kfree(prstatus);
 	kfree(psinfo);
 	kfree(notes);
 	kfree(fpu);
 #ifdef ELF_CORE_COPY_XFPREGS
 	kfree(xfpu);
 #endif
 	return has_dumped;
 #undef NUM_NOTES
 }
 #endif		/* USE_ELF_CORE_DUMP */

fs/binfmt_flat.c

Diff comments View file @ c40f6f8

1	/****************************************************************************/	1	/****************************************************************************/
2	/*	2	/*
3	* linux/fs/binfmt_flat.c	3	* linux/fs/binfmt_flat.c
4	*	4	*
5	* Copyright (C) 2000-2003 David McCullough <davidm@snapgear.com>	5	* Copyright (C) 2000-2003 David McCullough <davidm@snapgear.com>
6	* Copyright (C) 2002 Greg Ungerer <gerg@snapgear.com>	6	* Copyright (C) 2002 Greg Ungerer <gerg@snapgear.com>
7	* Copyright (C) 2002 SnapGear, by Paul Dale <pauli@snapgear.com>	7	* Copyright (C) 2002 SnapGear, by Paul Dale <pauli@snapgear.com>
8	* Copyright (C) 2000, 2001 Lineo, by David McCullough <davidm@lineo.com>	8	* Copyright (C) 2000, 2001 Lineo, by David McCullough <davidm@lineo.com>
9	* based heavily on:	9	* based heavily on:
10	*	10	*
11	* linux/fs/binfmt_aout.c:	11	* linux/fs/binfmt_aout.c:
12	* Copyright (C) 1991, 1992, 1996 Linus Torvalds	12	* Copyright (C) 1991, 1992, 1996 Linus Torvalds
13	* linux/fs/binfmt_flat.c for 2.0 kernel	13	* linux/fs/binfmt_flat.c for 2.0 kernel
14	* Copyright (C) 1998 Kenneth Albanowski <kjahds@kjahds.com>	14	* Copyright (C) 1998 Kenneth Albanowski <kjahds@kjahds.com>
15	* JAN/99 -- coded full program relocation (gerg@snapgear.com)	15	* JAN/99 -- coded full program relocation (gerg@snapgear.com)
16	*/	16	*/
17		17
18	#include <linux/module.h>	18	#include <linux/module.h>
19	#include <linux/kernel.h>	19	#include <linux/kernel.h>
20	#include <linux/sched.h>	20	#include <linux/sched.h>
21	#include <linux/mm.h>	21	#include <linux/mm.h>
22	#include <linux/mman.h>	22	#include <linux/mman.h>
23	#include <linux/errno.h>	23	#include <linux/errno.h>
24	#include <linux/signal.h>	24	#include <linux/signal.h>
25	#include <linux/string.h>	25	#include <linux/string.h>
26	#include <linux/fs.h>	26	#include <linux/fs.h>
27	#include <linux/file.h>	27	#include <linux/file.h>
28	#include <linux/stat.h>	28	#include <linux/stat.h>
29	#include <linux/fcntl.h>	29	#include <linux/fcntl.h>
30	#include <linux/ptrace.h>	30	#include <linux/ptrace.h>
31	#include <linux/user.h>	31	#include <linux/user.h>
32	#include <linux/slab.h>	32	#include <linux/slab.h>
33	#include <linux/binfmts.h>	33	#include <linux/binfmts.h>
34	#include <linux/personality.h>	34	#include <linux/personality.h>
35	#include <linux/init.h>	35	#include <linux/init.h>
36	#include <linux/flat.h>	36	#include <linux/flat.h>
37	#include <linux/syscalls.h>	37	#include <linux/syscalls.h>
38		38
39	#include <asm/byteorder.h>	39	#include <asm/byteorder.h>
40	#include <asm/system.h>	40	#include <asm/system.h>
41	#include <asm/uaccess.h>	41	#include <asm/uaccess.h>
42	#include <asm/unaligned.h>	42	#include <asm/unaligned.h>
43	#include <asm/cacheflush.h>	43	#include <asm/cacheflush.h>
44		44
45	/****************************************************************************/	45	/****************************************************************************/
46		46
47	#if 0	47	#if 0
48	#define DEBUG 1	48	#define DEBUG 1
49	#endif	49	#endif
50		50
51	#ifdef DEBUG	51	#ifdef DEBUG
52	#define DBG_FLT(a...) printk(a)	52	#define DBG_FLT(a...) printk(a)
53	#else	53	#else
54	#define DBG_FLT(a...)	54	#define DBG_FLT(a...)
55	#endif	55	#endif
56		56
57	#define RELOC_FAILED 0xff00ff01 /* Relocation incorrect somewhere */	57	#define RELOC_FAILED 0xff00ff01 /* Relocation incorrect somewhere */
58	#define UNLOADED_LIB 0x7ff000ff /* Placeholder for unused library */	58	#define UNLOADED_LIB 0x7ff000ff /* Placeholder for unused library */
59		59
60	struct lib_info {	60	struct lib_info {
61	struct {	61	struct {
62	unsigned long start_code; /* Start of text segment */	62	unsigned long start_code; /* Start of text segment */
63	unsigned long start_data; /* Start of data segment */	63	unsigned long start_data; /* Start of data segment */
64	unsigned long start_brk; /* End of data segment */	64	unsigned long start_brk; /* End of data segment */
65	unsigned long text_len; /* Length of text segment */	65	unsigned long text_len; /* Length of text segment */
66	unsigned long entry; /* Start address for this module */	66	unsigned long entry; /* Start address for this module */
67	unsigned long build_date; /* When this one was compiled */	67	unsigned long build_date; /* When this one was compiled */
68	short loaded; /* Has this library been loaded? */	68	short loaded; /* Has this library been loaded? */
69	} lib_list[MAX_SHARED_LIBS];	69	} lib_list[MAX_SHARED_LIBS];
70	};	70	};
71		71
72	#ifdef CONFIG_BINFMT_SHARED_FLAT	72	#ifdef CONFIG_BINFMT_SHARED_FLAT
73	static int load_flat_shared_library(int id, struct lib_info *p);	73	static int load_flat_shared_library(int id, struct lib_info *p);
74	#endif	74	#endif
75		75
76	static int load_flat_binary(struct linux_binprm , struct pt_regs regs);	76	static int load_flat_binary(struct linux_binprm , struct pt_regs regs);
77	static int flat_core_dump(long signr, struct pt_regs regs, struct file file, unsigned long limit);	77	static int flat_core_dump(long signr, struct pt_regs regs, struct file file, unsigned long limit);
78		78
79	static struct linux_binfmt flat_format = {	79	static struct linux_binfmt flat_format = {
80	.module = THIS_MODULE,	80	.module = THIS_MODULE,
81	.load_binary = load_flat_binary,	81	.load_binary = load_flat_binary,
82	.core_dump = flat_core_dump,	82	.core_dump = flat_core_dump,
83	.min_coredump = PAGE_SIZE	83	.min_coredump = PAGE_SIZE
84	};	84	};
85		85
86	/****************************************************************************/	86	/****************************************************************************/
87	/*	87	/*
88	* Routine writes a core dump image in the current directory.	88	* Routine writes a core dump image in the current directory.
89	* Currently only a stub-function.	89	* Currently only a stub-function.
90	*/	90	*/
91		91
92	static int flat_core_dump(long signr, struct pt_regs regs, struct file file, unsigned long limit)	92	static int flat_core_dump(long signr, struct pt_regs regs, struct file file, unsigned long limit)
93	{	93	{
94	printk("Process %s:%d received signr %d and should have core dumped\n",	94	printk("Process %s:%d received signr %d and should have core dumped\n",
95	current->comm, current->pid, (int) signr);	95	current->comm, current->pid, (int) signr);
96	return(1);	96	return(1);
97	}	97	}
98		98
99	/****************************************************************************/	99	/****************************************************************************/
100	/*	100	/*
101	* create_flat_tables() parses the env- and arg-strings in new user	101	* create_flat_tables() parses the env- and arg-strings in new user
102	* memory and creates the pointer tables from them, and puts their	102	* memory and creates the pointer tables from them, and puts their
103	* addresses on the "stack", returning the new stack pointer value.	103	* addresses on the "stack", returning the new stack pointer value.
104	*/	104	*/
105		105
106	static unsigned long create_flat_tables(	106	static unsigned long create_flat_tables(
107	unsigned long pp,	107	unsigned long pp,
108	struct linux_binprm * bprm)	108	struct linux_binprm * bprm)
109	{	109	{
110	unsigned long argv,envp;	110	unsigned long argv,envp;
111	unsigned long * sp;	111	unsigned long * sp;
112	char * p = (char*)pp;	112	char * p = (char*)pp;
113	int argc = bprm->argc;	113	int argc = bprm->argc;
114	int envc = bprm->envc;	114	int envc = bprm->envc;
115	char uninitialized_var(dummy);	115	char uninitialized_var(dummy);
116		116
117	sp = (unsigned long ) ((-(unsigned long)sizeof(char ))&(unsigned long) p);	117	sp = (unsigned long ) ((-(unsigned long)sizeof(char ))&(unsigned long) p);
118		118
119	sp -= envc+1;	119	sp -= envc+1;
120	envp = sp;	120	envp = sp;
121	sp -= argc+1;	121	sp -= argc+1;
122	argv = sp;	122	argv = sp;
123		123
124	flat_stack_align(sp);	124	flat_stack_align(sp);
125	if (flat_argvp_envp_on_stack()) {	125	if (flat_argvp_envp_on_stack()) {
126	--sp; put_user((unsigned long) envp, sp);	126	--sp; put_user((unsigned long) envp, sp);
127	--sp; put_user((unsigned long) argv, sp);	127	--sp; put_user((unsigned long) argv, sp);
128	}	128	}
129		129
130	put_user(argc,--sp);	130	put_user(argc,--sp);
131	current->mm->arg_start = (unsigned long) p;	131	current->mm->arg_start = (unsigned long) p;
132	while (argc-->0) {	132	while (argc-->0) {
133	put_user((unsigned long) p, argv++);	133	put_user((unsigned long) p, argv++);
134	do {	134	do {
135	get_user(dummy, p); p++;	135	get_user(dummy, p); p++;
136	} while (dummy);	136	} while (dummy);
137	}	137	}
138	put_user((unsigned long) NULL, argv);	138	put_user((unsigned long) NULL, argv);
139	current->mm->arg_end = current->mm->env_start = (unsigned long) p;	139	current->mm->arg_end = current->mm->env_start = (unsigned long) p;
140	while (envc-->0) {	140	while (envc-->0) {
141	put_user((unsigned long)p, envp); envp++;	141	put_user((unsigned long)p, envp); envp++;
142	do {	142	do {
143	get_user(dummy, p); p++;	143	get_user(dummy, p); p++;
144	} while (dummy);	144	} while (dummy);
145	}	145	}
146	put_user((unsigned long) NULL, envp);	146	put_user((unsigned long) NULL, envp);
147	current->mm->env_end = (unsigned long) p;	147	current->mm->env_end = (unsigned long) p;
148	return (unsigned long)sp;	148	return (unsigned long)sp;
149	}	149	}
150		150
151	/****************************************************************************/	151	/****************************************************************************/
152		152
153	#ifdef CONFIG_BINFMT_ZFLAT	153	#ifdef CONFIG_BINFMT_ZFLAT
154		154
155	#include <linux/zlib.h>	155	#include <linux/zlib.h>
156		156
157	#define LBUFSIZE 4000	157	#define LBUFSIZE 4000
158		158
159	/* gzip flag byte */	159	/* gzip flag byte */
160	#define ASCII_FLAG 0x01 /* bit 0 set: file probably ASCII text */	160	#define ASCII_FLAG 0x01 /* bit 0 set: file probably ASCII text */
161	#define CONTINUATION 0x02 /* bit 1 set: continuation of multi-part gzip file */	161	#define CONTINUATION 0x02 /* bit 1 set: continuation of multi-part gzip file */
162	#define EXTRA_FIELD 0x04 /* bit 2 set: extra field present */	162	#define EXTRA_FIELD 0x04 /* bit 2 set: extra field present */
163	#define ORIG_NAME 0x08 /* bit 3 set: original file name present */	163	#define ORIG_NAME 0x08 /* bit 3 set: original file name present */
164	#define COMMENT 0x10 /* bit 4 set: file comment present */	164	#define COMMENT 0x10 /* bit 4 set: file comment present */
165	#define ENCRYPTED 0x20 /* bit 5 set: file is encrypted */	165	#define ENCRYPTED 0x20 /* bit 5 set: file is encrypted */
166	#define RESERVED 0xC0 /* bit 6,7: reserved */	166	#define RESERVED 0xC0 /* bit 6,7: reserved */
167		167
168	static int decompress_exec(	168	static int decompress_exec(
169	struct linux_binprm *bprm,	169	struct linux_binprm *bprm,
170	unsigned long offset,	170	unsigned long offset,
171	char *dst,	171	char *dst,
172	long len,	172	long len,
173	int fd)	173	int fd)
174	{	174	{
175	unsigned char *buf;	175	unsigned char *buf;
176	z_stream strm;	176	z_stream strm;
177	loff_t fpos;	177	loff_t fpos;
178	int ret, retval;	178	int ret, retval;
179		179
180	DBG_FLT("decompress_exec(offset=%x,buf=%x,len=%x)\n",(int)offset, (int)dst, (int)len);	180	DBG_FLT("decompress_exec(offset=%x,buf=%x,len=%x)\n",(int)offset, (int)dst, (int)len);
181		181
182	memset(&strm, 0, sizeof(strm));	182	memset(&strm, 0, sizeof(strm));
183	strm.workspace = kmalloc(zlib_inflate_workspacesize(), GFP_KERNEL);	183	strm.workspace = kmalloc(zlib_inflate_workspacesize(), GFP_KERNEL);
184	if (strm.workspace == NULL) {	184	if (strm.workspace == NULL) {
185	DBG_FLT("binfmt_flat: no memory for decompress workspace\n");	185	DBG_FLT("binfmt_flat: no memory for decompress workspace\n");
186	return -ENOMEM;	186	return -ENOMEM;
187	}	187	}
188	buf = kmalloc(LBUFSIZE, GFP_KERNEL);	188	buf = kmalloc(LBUFSIZE, GFP_KERNEL);
189	if (buf == NULL) {	189	if (buf == NULL) {
190	DBG_FLT("binfmt_flat: no memory for read buffer\n");	190	DBG_FLT("binfmt_flat: no memory for read buffer\n");
191	retval = -ENOMEM;	191	retval = -ENOMEM;
192	goto out_free;	192	goto out_free;
193	}	193	}
194		194
195	/* Read in first chunk of data and parse gzip header. */	195	/* Read in first chunk of data and parse gzip header. */
196	fpos = offset;	196	fpos = offset;
197	ret = bprm->file->f_op->read(bprm->file, buf, LBUFSIZE, &fpos);	197	ret = bprm->file->f_op->read(bprm->file, buf, LBUFSIZE, &fpos);
198		198
199	strm.next_in = buf;	199	strm.next_in = buf;
200	strm.avail_in = ret;	200	strm.avail_in = ret;
201	strm.total_in = 0;	201	strm.total_in = 0;
202		202
203	retval = -ENOEXEC;	203	retval = -ENOEXEC;
204		204
205	/* Check minimum size -- gzip header */	205	/* Check minimum size -- gzip header */
206	if (ret < 10) {	206	if (ret < 10) {
207	DBG_FLT("binfmt_flat: file too small?\n");	207	DBG_FLT("binfmt_flat: file too small?\n");
208	goto out_free_buf;	208	goto out_free_buf;
209	}	209	}
210		210
211	/* Check gzip magic number */	211	/* Check gzip magic number */
212	if ((buf[0] != 037) \|\| ((buf[1] != 0213) && (buf[1] != 0236))) {	212	if ((buf[0] != 037) \|\| ((buf[1] != 0213) && (buf[1] != 0236))) {
213	DBG_FLT("binfmt_flat: unknown compression magic?\n");	213	DBG_FLT("binfmt_flat: unknown compression magic?\n");
214	goto out_free_buf;	214	goto out_free_buf;
215	}	215	}
216		216
217	/* Check gzip method */	217	/* Check gzip method */
218	if (buf[2] != 8) {	218	if (buf[2] != 8) {
219	DBG_FLT("binfmt_flat: unknown compression method?\n");	219	DBG_FLT("binfmt_flat: unknown compression method?\n");
220	goto out_free_buf;	220	goto out_free_buf;
221	}	221	}
222	/* Check gzip flags */	222	/* Check gzip flags */
223	if ((buf[3] & ENCRYPTED) \|\| (buf[3] & CONTINUATION) \|\|	223	if ((buf[3] & ENCRYPTED) \|\| (buf[3] & CONTINUATION) \|\|
224	(buf[3] & RESERVED)) {	224	(buf[3] & RESERVED)) {
225	DBG_FLT("binfmt_flat: unknown flags?\n");	225	DBG_FLT("binfmt_flat: unknown flags?\n");
226	goto out_free_buf;	226	goto out_free_buf;
227	}	227	}
228		228
229	ret = 10;	229	ret = 10;
230	if (buf[3] & EXTRA_FIELD) {	230	if (buf[3] & EXTRA_FIELD) {
231	ret += 2 + buf[10] + (buf[11] << 8);	231	ret += 2 + buf[10] + (buf[11] << 8);
232	if (unlikely(LBUFSIZE <= ret)) {	232	if (unlikely(LBUFSIZE <= ret)) {
233	DBG_FLT("binfmt_flat: buffer overflow (EXTRA)?\n");	233	DBG_FLT("binfmt_flat: buffer overflow (EXTRA)?\n");
234	goto out_free_buf;	234	goto out_free_buf;
235	}	235	}
236	}	236	}
237	if (buf[3] & ORIG_NAME) {	237	if (buf[3] & ORIG_NAME) {
238	while (ret < LBUFSIZE && buf[ret++] != 0)	238	while (ret < LBUFSIZE && buf[ret++] != 0)
239	;	239	;
240	if (unlikely(LBUFSIZE == ret)) {	240	if (unlikely(LBUFSIZE == ret)) {
241	DBG_FLT("binfmt_flat: buffer overflow (ORIG_NAME)?\n");	241	DBG_FLT("binfmt_flat: buffer overflow (ORIG_NAME)?\n");
242	goto out_free_buf;	242	goto out_free_buf;
243	}	243	}
244	}	244	}
245	if (buf[3] & COMMENT) {	245	if (buf[3] & COMMENT) {
246	while (ret < LBUFSIZE && buf[ret++] != 0)	246	while (ret < LBUFSIZE && buf[ret++] != 0)
247	;	247	;
248	if (unlikely(LBUFSIZE == ret)) {	248	if (unlikely(LBUFSIZE == ret)) {
249	DBG_FLT("binfmt_flat: buffer overflow (COMMENT)?\n");	249	DBG_FLT("binfmt_flat: buffer overflow (COMMENT)?\n");
250	goto out_free_buf;	250	goto out_free_buf;
251	}	251	}
252	}	252	}
253		253
254	strm.next_in += ret;	254	strm.next_in += ret;
255	strm.avail_in -= ret;	255	strm.avail_in -= ret;
256		256
257	strm.next_out = dst;	257	strm.next_out = dst;
258	strm.avail_out = len;	258	strm.avail_out = len;
259	strm.total_out = 0;	259	strm.total_out = 0;
260		260
261	if (zlib_inflateInit2(&strm, -MAX_WBITS) != Z_OK) {	261	if (zlib_inflateInit2(&strm, -MAX_WBITS) != Z_OK) {
262	DBG_FLT("binfmt_flat: zlib init failed?\n");	262	DBG_FLT("binfmt_flat: zlib init failed?\n");
263	goto out_free_buf;	263	goto out_free_buf;
264	}	264	}
265		265
266	while ((ret = zlib_inflate(&strm, Z_NO_FLUSH)) == Z_OK) {	266	while ((ret = zlib_inflate(&strm, Z_NO_FLUSH)) == Z_OK) {
267	ret = bprm->file->f_op->read(bprm->file, buf, LBUFSIZE, &fpos);	267	ret = bprm->file->f_op->read(bprm->file, buf, LBUFSIZE, &fpos);
268	if (ret <= 0)	268	if (ret <= 0)
269	break;	269	break;
270	if (ret >= (unsigned long) -4096)	270	if (ret >= (unsigned long) -4096)
271	break;	271	break;
272	len -= ret;	272	len -= ret;
273		273
274	strm.next_in = buf;	274	strm.next_in = buf;
275	strm.avail_in = ret;	275	strm.avail_in = ret;
276	strm.total_in = 0;	276	strm.total_in = 0;
277	}	277	}
278		278
279	if (ret < 0) {	279	if (ret < 0) {
280	DBG_FLT("binfmt_flat: decompression failed (%d), %s\n",	280	DBG_FLT("binfmt_flat: decompression failed (%d), %s\n",
281	ret, strm.msg);	281	ret, strm.msg);
282	goto out_zlib;	282	goto out_zlib;
283	}	283	}
284		284
285	retval = 0;	285	retval = 0;
286	out_zlib:	286	out_zlib:
287	zlib_inflateEnd(&strm);	287	zlib_inflateEnd(&strm);
288	out_free_buf:	288	out_free_buf:
289	kfree(buf);	289	kfree(buf);
290	out_free:	290	out_free:
291	kfree(strm.workspace);	291	kfree(strm.workspace);
292	return retval;	292	return retval;
293	}	293	}
294		294
295	#endif /* CONFIG_BINFMT_ZFLAT */	295	#endif /* CONFIG_BINFMT_ZFLAT */
296		296
297	/****************************************************************************/	297	/****************************************************************************/
298		298
299	static unsigned long	299	static unsigned long
300	calc_reloc(unsigned long r, struct lib_info *p, int curid, int internalp)	300	calc_reloc(unsigned long r, struct lib_info *p, int curid, int internalp)
301	{	301	{
302	unsigned long addr;	302	unsigned long addr;
303	int id;	303	int id;
304	unsigned long start_brk;	304	unsigned long start_brk;
305	unsigned long start_data;	305	unsigned long start_data;
306	unsigned long text_len;	306	unsigned long text_len;
307	unsigned long start_code;	307	unsigned long start_code;
308		308
309	#ifdef CONFIG_BINFMT_SHARED_FLAT	309	#ifdef CONFIG_BINFMT_SHARED_FLAT
310	if (r == 0)	310	if (r == 0)
311	id = curid; /* Relocs of 0 are always self referring */	311	id = curid; /* Relocs of 0 are always self referring */
312	else {	312	else {
313	id = (r >> 24) & 0xff; /* Find ID for this reloc */	313	id = (r >> 24) & 0xff; /* Find ID for this reloc */
314	r &= 0x00ffffff; /* Trim ID off here */	314	r &= 0x00ffffff; /* Trim ID off here */
315	}	315	}
316	if (id >= MAX_SHARED_LIBS) {	316	if (id >= MAX_SHARED_LIBS) {
317	printk("BINFMT_FLAT: reference 0x%x to shared library %d",	317	printk("BINFMT_FLAT: reference 0x%x to shared library %d",
318	(unsigned) r, id);	318	(unsigned) r, id);
319	goto failed;	319	goto failed;
320	}	320	}
321	if (curid != id) {	321	if (curid != id) {
322	if (internalp) {	322	if (internalp) {
323	printk("BINFMT_FLAT: reloc address 0x%x not in same module "	323	printk("BINFMT_FLAT: reloc address 0x%x not in same module "
324	"(%d != %d)", (unsigned) r, curid, id);	324	"(%d != %d)", (unsigned) r, curid, id);
325	goto failed;	325	goto failed;
326	} else if ( ! p->lib_list[id].loaded &&	326	} else if ( ! p->lib_list[id].loaded &&
327	load_flat_shared_library(id, p) > (unsigned long) -4096) {	327	load_flat_shared_library(id, p) > (unsigned long) -4096) {
328	printk("BINFMT_FLAT: failed to load library %d", id);	328	printk("BINFMT_FLAT: failed to load library %d", id);
329	goto failed;	329	goto failed;
330	}	330	}
331	/* Check versioning information (i.e. time stamps) */	331	/* Check versioning information (i.e. time stamps) */
332	if (p->lib_list[id].build_date && p->lib_list[curid].build_date &&	332	if (p->lib_list[id].build_date && p->lib_list[curid].build_date &&
333	p->lib_list[curid].build_date < p->lib_list[id].build_date) {	333	p->lib_list[curid].build_date < p->lib_list[id].build_date) {
334	printk("BINFMT_FLAT: library %d is younger than %d", id, curid);	334	printk("BINFMT_FLAT: library %d is younger than %d", id, curid);
335	goto failed;	335	goto failed;
336	}	336	}
337	}	337	}
338	#else	338	#else
339	id = 0;	339	id = 0;
340	#endif	340	#endif
341		341
342	start_brk = p->lib_list[id].start_brk;	342	start_brk = p->lib_list[id].start_brk;
343	start_data = p->lib_list[id].start_data;	343	start_data = p->lib_list[id].start_data;
344	start_code = p->lib_list[id].start_code;	344	start_code = p->lib_list[id].start_code;
345	text_len = p->lib_list[id].text_len;	345	text_len = p->lib_list[id].text_len;
346		346
347	if (!flat_reloc_valid(r, start_brk - start_data + text_len)) {	347	if (!flat_reloc_valid(r, start_brk - start_data + text_len)) {
348	printk("BINFMT_FLAT: reloc outside program 0x%x (0 - 0x%x/0x%x)",	348	printk("BINFMT_FLAT: reloc outside program 0x%x (0 - 0x%x/0x%x)",
349	(int) r,(int)(start_brk-start_code),(int)text_len);	349	(int) r,(int)(start_brk-start_code),(int)text_len);
350	goto failed;	350	goto failed;
351	}	351	}
352		352
353	if (r < text_len) /* In text segment */	353	if (r < text_len) /* In text segment */
354	addr = r + start_code;	354	addr = r + start_code;
355	else /* In data segment */	355	else /* In data segment */
356	addr = r - text_len + start_data;	356	addr = r - text_len + start_data;
357		357
358	/* Range checked already above so doing the range tests is redundant...*/	358	/* Range checked already above so doing the range tests is redundant...*/
359	return(addr);	359	return(addr);
360		360
361	failed:	361	failed:
362	printk(", killing %s!\n", current->comm);	362	printk(", killing %s!\n", current->comm);
363	send_sig(SIGSEGV, current, 0);	363	send_sig(SIGSEGV, current, 0);
364		364
365	return RELOC_FAILED;	365	return RELOC_FAILED;
366	}	366	}
367		367
368	/****************************************************************************/	368	/****************************************************************************/
369		369
370	void old_reloc(unsigned long rl)	370	void old_reloc(unsigned long rl)
371	{	371	{
372	#ifdef DEBUG	372	#ifdef DEBUG
373	char segment[] = { "TEXT", "DATA", "BSS", "UNKNOWN*" };	373	char segment[] = { "TEXT", "DATA", "BSS", "UNKNOWN*" };
374	#endif	374	#endif
375	flat_v2_reloc_t r;	375	flat_v2_reloc_t r;
376	unsigned long *ptr;	376	unsigned long *ptr;
377		377
378	r.value = rl;	378	r.value = rl;
379	#if defined(CONFIG_COLDFIRE)	379	#if defined(CONFIG_COLDFIRE)
380	ptr = (unsigned long *) (current->mm->start_code + r.reloc.offset);	380	ptr = (unsigned long *) (current->mm->start_code + r.reloc.offset);
381	#else	381	#else
382	ptr = (unsigned long *) (current->mm->start_data + r.reloc.offset);	382	ptr = (unsigned long *) (current->mm->start_data + r.reloc.offset);
383	#endif	383	#endif
384		384
385	#ifdef DEBUG	385	#ifdef DEBUG
386	printk("Relocation of variable at DATASEG+%x "	386	printk("Relocation of variable at DATASEG+%x "
387	"(address %p, currently %x) into segment %s\n",	387	"(address %p, currently %x) into segment %s\n",
388	r.reloc.offset, ptr, (int)*ptr, segment[r.reloc.type]);	388	r.reloc.offset, ptr, (int)*ptr, segment[r.reloc.type]);
389	#endif	389	#endif
390		390
391	switch (r.reloc.type) {	391	switch (r.reloc.type) {
392	case OLD_FLAT_RELOC_TYPE_TEXT:	392	case OLD_FLAT_RELOC_TYPE_TEXT:
393	*ptr += current->mm->start_code;	393	*ptr += current->mm->start_code;
394	break;	394	break;
395	case OLD_FLAT_RELOC_TYPE_DATA:	395	case OLD_FLAT_RELOC_TYPE_DATA:
396	*ptr += current->mm->start_data;	396	*ptr += current->mm->start_data;
397	break;	397	break;
398	case OLD_FLAT_RELOC_TYPE_BSS:	398	case OLD_FLAT_RELOC_TYPE_BSS:
399	*ptr += current->mm->end_data;	399	*ptr += current->mm->end_data;
400	break;	400	break;
401	default:	401	default:
402	printk("BINFMT_FLAT: Unknown relocation type=%x\n", r.reloc.type);	402	printk("BINFMT_FLAT: Unknown relocation type=%x\n", r.reloc.type);
403	break;	403	break;
404	}	404	}
405		405
406	#ifdef DEBUG	406	#ifdef DEBUG
407	printk("Relocation became %x\n", (int)*ptr);	407	printk("Relocation became %x\n", (int)*ptr);
408	#endif	408	#endif
409	}	409	}
410		410
411	/****************************************************************************/	411	/****************************************************************************/
412		412
413	static int load_flat_file(struct linux_binprm * bprm,	413	static int load_flat_file(struct linux_binprm * bprm,
414	struct lib_info libinfo, int id, unsigned long extra_stack)	414	struct lib_info libinfo, int id, unsigned long extra_stack)
415	{	415	{
416	struct flat_hdr * hdr;	416	struct flat_hdr * hdr;
417	unsigned long textpos = 0, datapos = 0, result;	417	unsigned long textpos = 0, datapos = 0, result;
418	unsigned long realdatastart = 0;	418	unsigned long realdatastart = 0;
419	unsigned long text_len, data_len, bss_len, stack_len, flags;	419	unsigned long text_len, data_len, bss_len, stack_len, flags;
420	unsigned long len, reallen, memp = 0;	420	unsigned long len, memp = 0;
421	unsigned long extra, rlim;	421	unsigned long memp_size, extra, rlim;
422	unsigned long reloc = 0, rp;	422	unsigned long reloc = 0, rp;
423	struct inode *inode;	423	struct inode *inode;
424	int i, rev, relocs = 0;	424	int i, rev, relocs = 0;
425	loff_t fpos;	425	loff_t fpos;
426	unsigned long start_code, end_code;	426	unsigned long start_code, end_code;
427	int ret;	427	int ret;
428		428
429	hdr = ((struct flat_hdr ) bprm->buf); / exec-header */	429	hdr = ((struct flat_hdr ) bprm->buf); / exec-header */
430	inode = bprm->file->f_path.dentry->d_inode;	430	inode = bprm->file->f_path.dentry->d_inode;
431		431
432	text_len = ntohl(hdr->data_start);	432	text_len = ntohl(hdr->data_start);
433	data_len = ntohl(hdr->data_end) - ntohl(hdr->data_start);	433	data_len = ntohl(hdr->data_end) - ntohl(hdr->data_start);
434	bss_len = ntohl(hdr->bss_end) - ntohl(hdr->data_end);	434	bss_len = ntohl(hdr->bss_end) - ntohl(hdr->data_end);
435	stack_len = ntohl(hdr->stack_size);	435	stack_len = ntohl(hdr->stack_size);
436	if (extra_stack) {	436	if (extra_stack) {
437	stack_len += *extra_stack;	437	stack_len += *extra_stack;
438	*extra_stack = stack_len;	438	*extra_stack = stack_len;
439	}	439	}
440	relocs = ntohl(hdr->reloc_count);	440	relocs = ntohl(hdr->reloc_count);
441	flags = ntohl(hdr->flags);	441	flags = ntohl(hdr->flags);
442	rev = ntohl(hdr->rev);	442	rev = ntohl(hdr->rev);
443		443
444	if (strncmp(hdr->magic, "bFLT", 4)) {	444	if (strncmp(hdr->magic, "bFLT", 4)) {
445	/*	445	/*
446	* Previously, here was a printk to tell people	446	* Previously, here was a printk to tell people
447	* "BINFMT_FLAT: bad header magic".	447	* "BINFMT_FLAT: bad header magic".
448	* But for the kernel which also use ELF FD-PIC format, this	448	* But for the kernel which also use ELF FD-PIC format, this
449	* error message is confusing.	449	* error message is confusing.
450	* because a lot of people do not manage to produce good	450	* because a lot of people do not manage to produce good
451	*/	451	*/
452	ret = -ENOEXEC;	452	ret = -ENOEXEC;
453	goto err;	453	goto err;
454	}	454	}
455		455
456	if (flags & FLAT_FLAG_KTRACE)	456	if (flags & FLAT_FLAG_KTRACE)
457	printk("BINFMT_FLAT: Loading file: %s\n", bprm->filename);	457	printk("BINFMT_FLAT: Loading file: %s\n", bprm->filename);
458		458
459	if (rev != FLAT_VERSION && rev != OLD_FLAT_VERSION) {	459	if (rev != FLAT_VERSION && rev != OLD_FLAT_VERSION) {
460	printk("BINFMT_FLAT: bad flat file version 0x%x (supported "	460	printk("BINFMT_FLAT: bad flat file version 0x%x (supported "
461	"0x%lx and 0x%lx)\n",	461	"0x%lx and 0x%lx)\n",
462	rev, FLAT_VERSION, OLD_FLAT_VERSION);	462	rev, FLAT_VERSION, OLD_FLAT_VERSION);
463	ret = -ENOEXEC;	463	ret = -ENOEXEC;
464	goto err;	464	goto err;
465	}	465	}
466		466
467	/* Don't allow old format executables to use shared libraries */	467	/* Don't allow old format executables to use shared libraries */
468	if (rev == OLD_FLAT_VERSION && id != 0) {	468	if (rev == OLD_FLAT_VERSION && id != 0) {
469	printk("BINFMT_FLAT: shared libraries are not available before rev 0x%x\n",	469	printk("BINFMT_FLAT: shared libraries are not available before rev 0x%x\n",
470	(int) FLAT_VERSION);	470	(int) FLAT_VERSION);
471	ret = -ENOEXEC;	471	ret = -ENOEXEC;
472	goto err;	472	goto err;
473	}	473	}
474		474
475	/*	475	/*
476	* fix up the flags for the older format, there were all kinds	476	* fix up the flags for the older format, there were all kinds
477	* of endian hacks, this only works for the simple cases	477	* of endian hacks, this only works for the simple cases
478	*/	478	*/
479	if (rev == OLD_FLAT_VERSION && flat_old_ram_flag(flags))	479	if (rev == OLD_FLAT_VERSION && flat_old_ram_flag(flags))
480	flags = FLAT_FLAG_RAM;	480	flags = FLAT_FLAG_RAM;
481		481
482	#ifndef CONFIG_BINFMT_ZFLAT	482	#ifndef CONFIG_BINFMT_ZFLAT
483	if (flags & (FLAT_FLAG_GZIP\|FLAT_FLAG_GZDATA)) {	483	if (flags & (FLAT_FLAG_GZIP\|FLAT_FLAG_GZDATA)) {
484	printk("Support for ZFLAT executables is not enabled.\n");	484	printk("Support for ZFLAT executables is not enabled.\n");
485	ret = -ENOEXEC;	485	ret = -ENOEXEC;
486	goto err;	486	goto err;
487	}	487	}
488	#endif	488	#endif
489		489
490	/*	490	/*
491	* Check initial limits. This avoids letting people circumvent	491	* Check initial limits. This avoids letting people circumvent
492	* size limits imposed on them by creating programs with large	492	* size limits imposed on them by creating programs with large
493	* arrays in the data or bss.	493	* arrays in the data or bss.
494	*/	494	*/
495	rlim = current->signal->rlim[RLIMIT_DATA].rlim_cur;	495	rlim = current->signal->rlim[RLIMIT_DATA].rlim_cur;
496	if (rlim >= RLIM_INFINITY)	496	if (rlim >= RLIM_INFINITY)
497	rlim = ~0;	497	rlim = ~0;
498	if (data_len + bss_len > rlim) {	498	if (data_len + bss_len > rlim) {
499	ret = -ENOMEM;	499	ret = -ENOMEM;
500	goto err;	500	goto err;
501	}	501	}
502		502
503	/* Flush all traces of the currently running executable */	503	/* Flush all traces of the currently running executable */
504	if (id == 0) {	504	if (id == 0) {
505	result = flush_old_exec(bprm);	505	result = flush_old_exec(bprm);
506	if (result) {	506	if (result) {
507	ret = result;	507	ret = result;
508	goto err;	508	goto err;
509	}	509	}
510		510
511	/* OK, This is the point of no return */	511	/* OK, This is the point of no return */
512	set_personality(PER_LINUX_32BIT);	512	set_personality(PER_LINUX_32BIT);
513	}	513	}
514		514
515	/*	515	/*
516	* calculate the extra space we need to map in	516	* calculate the extra space we need to map in
517	*/	517	*/
518	extra = max_t(unsigned long, bss_len + stack_len,	518	extra = max_t(unsigned long, bss_len + stack_len,
519	relocs * sizeof(unsigned long));	519	relocs * sizeof(unsigned long));
520		520
521	/*	521	/*
522	* there are a couple of cases here, the separate code/data	522	* there are a couple of cases here, the separate code/data
523	* case, and then the fully copied to RAM case which lumps	523	* case, and then the fully copied to RAM case which lumps
524	* it all together.	524	* it all together.
525	*/	525	*/
526	if ((flags & (FLAT_FLAG_RAM\|FLAT_FLAG_GZIP)) == 0) {	526	if ((flags & (FLAT_FLAG_RAM\|FLAT_FLAG_GZIP)) == 0) {
527	/*	527	/*
528	* this should give us a ROM ptr, but if it doesn't we don't	528	* this should give us a ROM ptr, but if it doesn't we don't
529	* really care	529	* really care
530	*/	530	*/
531	DBG_FLT("BINFMT_FLAT: ROM mapping of file (we hope)\n");	531	DBG_FLT("BINFMT_FLAT: ROM mapping of file (we hope)\n");
532		532
533	down_write(&current->mm->mmap_sem);	533	down_write(&current->mm->mmap_sem);
534	textpos = do_mmap(bprm->file, 0, text_len, PROT_READ\|PROT_EXEC,	534	textpos = do_mmap(bprm->file, 0, text_len, PROT_READ\|PROT_EXEC,
535	MAP_PRIVATE\|MAP_EXECUTABLE, 0);	535	MAP_PRIVATE\|MAP_EXECUTABLE, 0);
536	up_write(&current->mm->mmap_sem);	536	up_write(&current->mm->mmap_sem);
537	if (!textpos \|\| textpos >= (unsigned long) -4096) {	537	if (!textpos \|\| textpos >= (unsigned long) -4096) {
538	if (!textpos)	538	if (!textpos)
539	textpos = (unsigned long) -ENOMEM;	539	textpos = (unsigned long) -ENOMEM;
540	printk("Unable to mmap process text, errno %d\n", (int)-textpos);	540	printk("Unable to mmap process text, errno %d\n", (int)-textpos);
541	ret = textpos;	541	ret = textpos;
542	goto err;	542	goto err;
543	}	543	}
544		544
545	len = data_len + extra + MAX_SHARED_LIBS * sizeof(unsigned long);	545	len = data_len + extra + MAX_SHARED_LIBS * sizeof(unsigned long);
		546	len = PAGE_ALIGN(len);
546	down_write(&current->mm->mmap_sem);	547	down_write(&current->mm->mmap_sem);
547	realdatastart = do_mmap(0, 0, len,	548	realdatastart = do_mmap(0, 0, len,
548	PROT_READ\|PROT_WRITE\|PROT_EXEC, MAP_PRIVATE, 0);	549	PROT_READ\|PROT_WRITE\|PROT_EXEC, MAP_PRIVATE, 0);
549	/* Remap to use all availabe slack region space */
550	if (realdatastart && (realdatastart < (unsigned long)-4096)) {
551	reallen = kobjsize((void *)realdatastart);
552	if (reallen > len) {
553	realdatastart = do_mremap(realdatastart, len,
554	reallen, MREMAP_FIXED, realdatastart);
555	}
556	}
557	up_write(&current->mm->mmap_sem);	550	up_write(&current->mm->mmap_sem);
558		551
559	if (realdatastart == 0 \|\| realdatastart >= (unsigned long)-4096) {	552	if (realdatastart == 0 \|\| realdatastart >= (unsigned long)-4096) {
560	if (!realdatastart)	553	if (!realdatastart)
561	realdatastart = (unsigned long) -ENOMEM;	554	realdatastart = (unsigned long) -ENOMEM;
562	printk("Unable to allocate RAM for process data, errno %d\n",	555	printk("Unable to allocate RAM for process data, errno %d\n",
563	(int)-realdatastart);	556	(int)-realdatastart);
564	do_munmap(current->mm, textpos, text_len);	557	do_munmap(current->mm, textpos, text_len);
565	ret = realdatastart;	558	ret = realdatastart;
566	goto err;	559	goto err;
567	}	560	}
568	datapos = realdatastart + MAX_SHARED_LIBS * sizeof(unsigned long);	561	datapos = realdatastart + MAX_SHARED_LIBS * sizeof(unsigned long);
569		562
570	DBG_FLT("BINFMT_FLAT: Allocated data+bss+stack (%d bytes): %x\n",	563	DBG_FLT("BINFMT_FLAT: Allocated data+bss+stack (%d bytes): %x\n",
571	(int)(data_len + bss_len + stack_len), (int)datapos);	564	(int)(data_len + bss_len + stack_len), (int)datapos);
572		565
573	fpos = ntohl(hdr->data_start);	566	fpos = ntohl(hdr->data_start);
574	#ifdef CONFIG_BINFMT_ZFLAT	567	#ifdef CONFIG_BINFMT_ZFLAT
575	if (flags & FLAT_FLAG_GZDATA) {	568	if (flags & FLAT_FLAG_GZDATA) {
576	result = decompress_exec(bprm, fpos, (char *) datapos,	569	result = decompress_exec(bprm, fpos, (char *) datapos,
577	data_len + (relocs * sizeof(unsigned long)), 0);	570	data_len + (relocs * sizeof(unsigned long)), 0);
578	} else	571	} else
579	#endif	572	#endif
580	{	573	{
581	result = bprm->file->f_op->read(bprm->file, (char *) datapos,	574	result = bprm->file->f_op->read(bprm->file, (char *) datapos,
582	data_len + (relocs * sizeof(unsigned long)), &fpos);	575	data_len + (relocs * sizeof(unsigned long)), &fpos);
583	}	576	}
584	if (result >= (unsigned long)-4096) {	577	if (result >= (unsigned long)-4096) {
585	printk("Unable to read data+bss, errno %d\n", (int)-result);	578	printk("Unable to read data+bss, errno %d\n", (int)-result);
586	do_munmap(current->mm, textpos, text_len);	579	do_munmap(current->mm, textpos, text_len);
587	do_munmap(current->mm, realdatastart, data_len + extra);	580	do_munmap(current->mm, realdatastart, data_len + extra);
588	ret = result;	581	ret = result;
589	goto err;	582	goto err;
590	}	583	}
591		584
592	reloc = (unsigned long *) (datapos+(ntohl(hdr->reloc_start)-text_len));	585	reloc = (unsigned long *) (datapos+(ntohl(hdr->reloc_start)-text_len));
593	memp = realdatastart;	586	memp = realdatastart;
594		587	memp_size = len;
595	} else {	588	} else {
596		589
597	len = text_len + data_len + extra + MAX_SHARED_LIBS * sizeof(unsigned long);	590	len = text_len + data_len + extra + MAX_SHARED_LIBS * sizeof(unsigned long);
		591	len = PAGE_ALIGN(len);
598	down_write(&current->mm->mmap_sem);	592	down_write(&current->mm->mmap_sem);
599	textpos = do_mmap(0, 0, len,	593	textpos = do_mmap(0, 0, len,
600	PROT_READ \| PROT_EXEC \| PROT_WRITE, MAP_PRIVATE, 0);	594	PROT_READ \| PROT_EXEC \| PROT_WRITE, MAP_PRIVATE, 0);
601	/* Remap to use all availabe slack region space */
602	if (textpos && (textpos < (unsigned long) -4096)) {
603	reallen = kobjsize((void *)textpos);
604	if (reallen > len) {
605	textpos = do_mremap(textpos, len, reallen,
606	MREMAP_FIXED, textpos);
607	}
608	}
609	up_write(&current->mm->mmap_sem);	595	up_write(&current->mm->mmap_sem);
610		596
611	if (!textpos \|\| textpos >= (unsigned long) -4096) {	597	if (!textpos \|\| textpos >= (unsigned long) -4096) {
612	if (!textpos)	598	if (!textpos)
613	textpos = (unsigned long) -ENOMEM;	599	textpos = (unsigned long) -ENOMEM;
614	printk("Unable to allocate RAM for process text/data, errno %d\n",	600	printk("Unable to allocate RAM for process text/data, errno %d\n",
615	(int)-textpos);	601	(int)-textpos);
616	ret = textpos;	602	ret = textpos;
617	goto err;	603	goto err;
618	}	604	}
619		605
620	realdatastart = textpos + ntohl(hdr->data_start);	606	realdatastart = textpos + ntohl(hdr->data_start);
621	datapos = realdatastart + MAX_SHARED_LIBS * sizeof(unsigned long);	607	datapos = realdatastart + MAX_SHARED_LIBS * sizeof(unsigned long);
622	reloc = (unsigned long *) (textpos + ntohl(hdr->reloc_start) +	608	reloc = (unsigned long *) (textpos + ntohl(hdr->reloc_start) +
623	MAX_SHARED_LIBS * sizeof(unsigned long));	609	MAX_SHARED_LIBS * sizeof(unsigned long));
624	memp = textpos;	610	memp = textpos;
625		611	memp_size = len;
626	#ifdef CONFIG_BINFMT_ZFLAT	612	#ifdef CONFIG_BINFMT_ZFLAT
627	/*	613	/*
628	* load it all in and treat it like a RAM load from now on	614	* load it all in and treat it like a RAM load from now on
629	*/	615	*/
630	if (flags & FLAT_FLAG_GZIP) {	616	if (flags & FLAT_FLAG_GZIP) {
631	result = decompress_exec(bprm, sizeof (struct flat_hdr),	617	result = decompress_exec(bprm, sizeof (struct flat_hdr),
632	(((char *) textpos) + sizeof (struct flat_hdr)),	618	(((char *) textpos) + sizeof (struct flat_hdr)),
633	(text_len + data_len + (relocs * sizeof(unsigned long))	619	(text_len + data_len + (relocs * sizeof(unsigned long))
634	- sizeof (struct flat_hdr)),	620	- sizeof (struct flat_hdr)),
635	0);	621	0);
636	memmove((void ) datapos, (void ) realdatastart,	622	memmove((void ) datapos, (void ) realdatastart,
637	data_len + (relocs * sizeof(unsigned long)));	623	data_len + (relocs * sizeof(unsigned long)));
638	} else if (flags & FLAT_FLAG_GZDATA) {	624	} else if (flags & FLAT_FLAG_GZDATA) {
639	fpos = 0;	625	fpos = 0;
640	result = bprm->file->f_op->read(bprm->file,	626	result = bprm->file->f_op->read(bprm->file,
641	(char *) textpos, text_len, &fpos);	627	(char *) textpos, text_len, &fpos);
642	if (result < (unsigned long) -4096)	628	if (result < (unsigned long) -4096)
643	result = decompress_exec(bprm, text_len, (char *) datapos,	629	result = decompress_exec(bprm, text_len, (char *) datapos,
644	data_len + (relocs * sizeof(unsigned long)), 0);	630	data_len + (relocs * sizeof(unsigned long)), 0);
645	}	631	}
646	else	632	else
647	#endif	633	#endif
648	{	634	{
649	fpos = 0;	635	fpos = 0;
650	result = bprm->file->f_op->read(bprm->file,	636	result = bprm->file->f_op->read(bprm->file,
651	(char *) textpos, text_len, &fpos);	637	(char *) textpos, text_len, &fpos);
652	if (result < (unsigned long) -4096) {	638	if (result < (unsigned long) -4096) {
653	fpos = ntohl(hdr->data_start);	639	fpos = ntohl(hdr->data_start);
654	result = bprm->file->f_op->read(bprm->file, (char *) datapos,	640	result = bprm->file->f_op->read(bprm->file, (char *) datapos,
655	data_len + (relocs * sizeof(unsigned long)), &fpos);	641	data_len + (relocs * sizeof(unsigned long)), &fpos);
656	}	642	}
657	}	643	}
658	if (result >= (unsigned long)-4096) {	644	if (result >= (unsigned long)-4096) {
659	printk("Unable to read code+data+bss, errno %d\n",(int)-result);	645	printk("Unable to read code+data+bss, errno %d\n",(int)-result);
660	do_munmap(current->mm, textpos, text_len + data_len + extra +	646	do_munmap(current->mm, textpos, text_len + data_len + extra +
661	MAX_SHARED_LIBS * sizeof(unsigned long));	647	MAX_SHARED_LIBS * sizeof(unsigned long));
662	ret = result;	648	ret = result;
663	goto err;	649	goto err;
664	}	650	}
665	}	651	}
666		652
667	if (flags & FLAT_FLAG_KTRACE)	653	if (flags & FLAT_FLAG_KTRACE)
668	printk("Mapping is %x, Entry point is %x, data_start is %x\n",	654	printk("Mapping is %x, Entry point is %x, data_start is %x\n",
669	(int)textpos, 0x00ffffff&ntohl(hdr->entry), ntohl(hdr->data_start));	655	(int)textpos, 0x00ffffff&ntohl(hdr->entry), ntohl(hdr->data_start));
670		656
671	/* The main program needs a little extra setup in the task structure */	657	/* The main program needs a little extra setup in the task structure */
672	start_code = textpos + sizeof (struct flat_hdr);	658	start_code = textpos + sizeof (struct flat_hdr);
673	end_code = textpos + text_len;	659	end_code = textpos + text_len;
674	if (id == 0) {	660	if (id == 0) {
675	current->mm->start_code = start_code;	661	current->mm->start_code = start_code;
676	current->mm->end_code = end_code;	662	current->mm->end_code = end_code;
677	current->mm->start_data = datapos;	663	current->mm->start_data = datapos;
678	current->mm->end_data = datapos + data_len;	664	current->mm->end_data = datapos + data_len;
679	/*	665	/*
680	* set up the brk stuff, uses any slack left in data/bss/stack	666	* set up the brk stuff, uses any slack left in data/bss/stack
681	* allocation. We put the brk after the bss (between the bss	667	* allocation. We put the brk after the bss (between the bss
682	* and stack) like other platforms.	668	* and stack) like other platforms.
		669	* Userspace code relies on the stack pointer starting out at
		670	* an address right at the end of a page.
683	*/	671	*/
684	current->mm->start_brk = datapos + data_len + bss_len;	672	current->mm->start_brk = datapos + data_len + bss_len;
685	current->mm->brk = (current->mm->start_brk + 3) & ~3;	673	current->mm->brk = (current->mm->start_brk + 3) & ~3;
686	current->mm->context.end_brk = memp + kobjsize((void *) memp) - stack_len;	674	current->mm->context.end_brk = memp + memp_size - stack_len;
687	}	675	}
688		676
689	if (flags & FLAT_FLAG_KTRACE)	677	if (flags & FLAT_FLAG_KTRACE)
690	printk("%s %s: TEXT=%x-%x DATA=%x-%x BSS=%x-%x\n",	678	printk("%s %s: TEXT=%x-%x DATA=%x-%x BSS=%x-%x\n",
691	id ? "Lib" : "Load", bprm->filename,	679	id ? "Lib" : "Load", bprm->filename,
692	(int) start_code, (int) end_code,	680	(int) start_code, (int) end_code,
693	(int) datapos,	681	(int) datapos,
694	(int) (datapos + data_len),	682	(int) (datapos + data_len),
695	(int) (datapos + data_len),	683	(int) (datapos + data_len),
696	(int) (((datapos + data_len + bss_len) + 3) & ~3));	684	(int) (((datapos + data_len + bss_len) + 3) & ~3));
697		685
698	text_len -= sizeof(struct flat_hdr); /* the real code len */	686	text_len -= sizeof(struct flat_hdr); /* the real code len */
699		687
700	/* Store the current module values into the global library structure */	688	/* Store the current module values into the global library structure */
701	libinfo->lib_list[id].start_code = start_code;	689	libinfo->lib_list[id].start_code = start_code;
702	libinfo->lib_list[id].start_data = datapos;	690	libinfo->lib_list[id].start_data = datapos;
703	libinfo->lib_list[id].start_brk = datapos + data_len + bss_len;	691	libinfo->lib_list[id].start_brk = datapos + data_len + bss_len;
704	libinfo->lib_list[id].text_len = text_len;	692	libinfo->lib_list[id].text_len = text_len;
705	libinfo->lib_list[id].loaded = 1;	693	libinfo->lib_list[id].loaded = 1;
706	libinfo->lib_list[id].entry = (0x00ffffff & ntohl(hdr->entry)) + textpos;	694	libinfo->lib_list[id].entry = (0x00ffffff & ntohl(hdr->entry)) + textpos;
707	libinfo->lib_list[id].build_date = ntohl(hdr->build_date);	695	libinfo->lib_list[id].build_date = ntohl(hdr->build_date);
708		696
709	/*	697	/*
710	* We just load the allocations into some temporary memory to	698	* We just load the allocations into some temporary memory to
711	* help simplify all this mumbo jumbo	699	* help simplify all this mumbo jumbo
712	*	700	*
713	* We've got two different sections of relocation entries.	701	* We've got two different sections of relocation entries.
714	* The first is the GOT which resides at the begining of the data segment	702	* The first is the GOT which resides at the begining of the data segment
715	* and is terminated with a -1. This one can be relocated in place.	703	* and is terminated with a -1. This one can be relocated in place.
716	* The second is the extra relocation entries tacked after the image's	704	* The second is the extra relocation entries tacked after the image's
717	* data segment. These require a little more processing as the entry is	705	* data segment. These require a little more processing as the entry is
718	* really an offset into the image which contains an offset into the	706	* really an offset into the image which contains an offset into the
719	* image.	707	* image.
720	*/	708	*/
721	if (flags & FLAT_FLAG_GOTPIC) {	709	if (flags & FLAT_FLAG_GOTPIC) {
722	for (rp = (unsigned long )datapos; rp != 0xffffffff; rp++) {	710	for (rp = (unsigned long )datapos; rp != 0xffffffff; rp++) {
723	unsigned long addr;	711	unsigned long addr;
724	if (*rp) {	712	if (*rp) {
725	addr = calc_reloc(*rp, libinfo, id, 0);	713	addr = calc_reloc(*rp, libinfo, id, 0);
726	if (addr == RELOC_FAILED) {	714	if (addr == RELOC_FAILED) {
727	ret = -ENOEXEC;	715	ret = -ENOEXEC;
728	goto err;	716	goto err;
729	}	717	}
730	*rp = addr;	718	*rp = addr;
731	}	719	}
732	}	720	}
733	}	721	}
734		722
735	/*	723	/*
736	* Now run through the relocation entries.	724	* Now run through the relocation entries.
737	* We've got to be careful here as C++ produces relocatable zero	725	* We've got to be careful here as C++ produces relocatable zero
738	* entries in the constructor and destructor tables which are then	726	* entries in the constructor and destructor tables which are then
739	* tested for being not zero (which will always occur unless we're	727	* tested for being not zero (which will always occur unless we're
740	* based from address zero). This causes an endless loop as __start	728	* based from address zero). This causes an endless loop as __start
741	* is at zero. The solution used is to not relocate zero addresses.	729	* is at zero. The solution used is to not relocate zero addresses.
742	* This has the negative side effect of not allowing a global data	730	* This has the negative side effect of not allowing a global data
743	* reference to be statically initialised to _stext (I've moved	731	* reference to be statically initialised to _stext (I've moved
744	* __start to address 4 so that is okay).	732	* __start to address 4 so that is okay).
745	*/	733	*/
746	if (rev > OLD_FLAT_VERSION) {	734	if (rev > OLD_FLAT_VERSION) {
747	unsigned long persistent = 0;	735	unsigned long persistent = 0;
748	for (i=0; i < relocs; i++) {	736	for (i=0; i < relocs; i++) {
749	unsigned long addr, relval;	737	unsigned long addr, relval;
750		738
751	/* Get the address of the pointer to be	739	/* Get the address of the pointer to be
752	relocated (of course, the address has to be	740	relocated (of course, the address has to be
753	relocated first). */	741	relocated first). */
754	relval = ntohl(reloc[i]);	742	relval = ntohl(reloc[i]);
755	if (flat_set_persistent (relval, &persistent))	743	if (flat_set_persistent (relval, &persistent))
756	continue;	744	continue;
757	addr = flat_get_relocate_addr(relval);	745	addr = flat_get_relocate_addr(relval);
758	rp = (unsigned long *) calc_reloc(addr, libinfo, id, 1);	746	rp = (unsigned long *) calc_reloc(addr, libinfo, id, 1);
759	if (rp == (unsigned long *)RELOC_FAILED) {	747	if (rp == (unsigned long *)RELOC_FAILED) {
760	ret = -ENOEXEC;	748	ret = -ENOEXEC;
761	goto err;	749	goto err;
762	}	750	}
763		751
764	/* Get the pointer's value. */	752	/* Get the pointer's value. */
765	addr = flat_get_addr_from_rp(rp, relval, flags,	753	addr = flat_get_addr_from_rp(rp, relval, flags,
766	&persistent);	754	&persistent);
767	if (addr != 0) {	755	if (addr != 0) {
768	/*	756	/*
769	* Do the relocation. PIC relocs in the data section are	757	* Do the relocation. PIC relocs in the data section are
770	* already in target order	758	* already in target order
771	*/	759	*/
772	if ((flags & FLAT_FLAG_GOTPIC) == 0)	760	if ((flags & FLAT_FLAG_GOTPIC) == 0)
773	addr = ntohl(addr);	761	addr = ntohl(addr);
774	addr = calc_reloc(addr, libinfo, id, 0);	762	addr = calc_reloc(addr, libinfo, id, 0);
775	if (addr == RELOC_FAILED) {	763	if (addr == RELOC_FAILED) {
776	ret = -ENOEXEC;	764	ret = -ENOEXEC;
777	goto err;	765	goto err;
778	}	766	}
779		767
780	/* Write back the relocated pointer. */	768	/* Write back the relocated pointer. */
781	flat_put_addr_at_rp(rp, addr, relval);	769	flat_put_addr_at_rp(rp, addr, relval);
782	}	770	}
783	}	771	}
784	} else {	772	} else {
785	for (i=0; i < relocs; i++)	773	for (i=0; i < relocs; i++)
786	old_reloc(ntohl(reloc[i]));	774	old_reloc(ntohl(reloc[i]));
787	}	775	}
788		776
789	flush_icache_range(start_code, end_code);	777	flush_icache_range(start_code, end_code);
790		778
791	/* zero the BSS, BRK and stack areas */	779	/* zero the BSS, BRK and stack areas */
792	memset((void*)(datapos + data_len), 0, bss_len +	780	memset((void*)(datapos + data_len), 0, bss_len +
793	(memp + kobjsize((void ) memp) - stack_len - / end brk */	781	(memp + memp_size - stack_len - /* end brk */
794	libinfo->lib_list[id].start_brk) + /* start brk */	782	libinfo->lib_list[id].start_brk) + /* start brk */
795	stack_len);	783	stack_len);
796		784
797	return 0;	785	return 0;
798	err:	786	err:
799	return ret;	787	return ret;
800	}	788	}
801		789
802		790
803	/****************************************************************************/	791	/****************************************************************************/
804	#ifdef CONFIG_BINFMT_SHARED_FLAT	792	#ifdef CONFIG_BINFMT_SHARED_FLAT
805		793
806	/*	794	/*
807	* Load a shared library into memory. The library gets its own data	795	* Load a shared library into memory. The library gets its own data
808	* segment (including bss) but not argv/argc/environ.	796	* segment (including bss) but not argv/argc/environ.
809	*/	797	*/
810		798
811	static int load_flat_shared_library(int id, struct lib_info *libs)	799	static int load_flat_shared_library(int id, struct lib_info *libs)
812	{	800	{
813	struct linux_binprm bprm;	801	struct linux_binprm bprm;
814	int res;	802	int res;
815	char buf[16];	803	char buf[16];
816		804
817	/* Create the file name */	805	/* Create the file name */
818	sprintf(buf, "/lib/lib%d.so", id);	806	sprintf(buf, "/lib/lib%d.so", id);
819		807
820	/* Open the file up */	808	/* Open the file up */
821	bprm.filename = buf;	809	bprm.filename = buf;
822	bprm.file = open_exec(bprm.filename);	810	bprm.file = open_exec(bprm.filename);
823	res = PTR_ERR(bprm.file);	811	res = PTR_ERR(bprm.file);
824	if (IS_ERR(bprm.file))	812	if (IS_ERR(bprm.file))
825	return res;	813	return res;
826		814
827	res = prepare_binprm(&bprm);	815	res = prepare_binprm(&bprm);
828		816
829	if (res <= (unsigned long)-4096)	817	if (res <= (unsigned long)-4096)
830	res = load_flat_file(&bprm, libs, id, NULL);	818	res = load_flat_file(&bprm, libs, id, NULL);
831	if (bprm.file) {	819	if (bprm.file) {
832	allow_write_access(bprm.file);	820	allow_write_access(bprm.file);
833	fput(bprm.file);	821	fput(bprm.file);
834	bprm.file = NULL;	822	bprm.file = NULL;
835	}	823	}
836	return(res);	824	return(res);
837	}	825	}
838		826
839	#endif /* CONFIG_BINFMT_SHARED_FLAT */	827	#endif /* CONFIG_BINFMT_SHARED_FLAT */
840	/****************************************************************************/	828	/****************************************************************************/
841		829
842	/*	830	/*
843	* These are the functions used to load flat style executables and shared	831	* These are the functions used to load flat style executables and shared
844	* libraries. There is no binary dependent code anywhere else.	832	* libraries. There is no binary dependent code anywhere else.
845	*/	833	*/
846		834
847	static int load_flat_binary(struct linux_binprm * bprm, struct pt_regs * regs)	835	static int load_flat_binary(struct linux_binprm * bprm, struct pt_regs * regs)
848	{	836	{
849	struct lib_info libinfo;	837	struct lib_info libinfo;
850	unsigned long p = bprm->p;	838	unsigned long p = bprm->p;
851	unsigned long stack_len;	839	unsigned long stack_len;
852	unsigned long start_addr;	840	unsigned long start_addr;
853	unsigned long *sp;	841	unsigned long *sp;
854	int res;	842	int res;
855	int i, j;	843	int i, j;
856		844
857	memset(&libinfo, 0, sizeof(libinfo));	845	memset(&libinfo, 0, sizeof(libinfo));
858	/*	846	/*
859	* We have to add the size of our arguments to our stack size	847	* We have to add the size of our arguments to our stack size
860	* otherwise it's too easy for users to create stack overflows	848	* otherwise it's too easy for users to create stack overflows
861	* by passing in a huge argument list. And yes, we have to be	849	* by passing in a huge argument list. And yes, we have to be
862	* pedantic and include space for the argv/envp array as it may have	850	* pedantic and include space for the argv/envp array as it may have
863	* a lot of entries.	851	* a lot of entries.
864	*/	852	*/
865	#define TOP_OF_ARGS (PAGE_SIZE * MAX_ARG_PAGES - sizeof(void *))	853	#define TOP_OF_ARGS (PAGE_SIZE * MAX_ARG_PAGES - sizeof(void *))
866	stack_len = TOP_OF_ARGS - bprm->p; /* the strings */	854	stack_len = TOP_OF_ARGS - bprm->p; /* the strings */
867	stack_len += (bprm->argc + 1) * sizeof(char ); / the argv array */	855	stack_len += (bprm->argc + 1) * sizeof(char ); / the argv array */
868	stack_len += (bprm->envc + 1) * sizeof(char ); / the envp array */	856	stack_len += (bprm->envc + 1) * sizeof(char ); / the envp array */
869		857
870		858
871	res = load_flat_file(bprm, &libinfo, 0, &stack_len);	859	res = load_flat_file(bprm, &libinfo, 0, &stack_len);
872	if (res > (unsigned long)-4096)	860	if (res > (unsigned long)-4096)
873	return res;	861	return res;
874		862
875	/* Update data segment pointers for all libraries */	863	/* Update data segment pointers for all libraries */
876	for (i=0; i<MAX_SHARED_LIBS; i++)	864	for (i=0; i<MAX_SHARED_LIBS; i++)
877	if (libinfo.lib_list[i].loaded)	865	if (libinfo.lib_list[i].loaded)
878	for (j=0; j<MAX_SHARED_LIBS; j++)	866	for (j=0; j<MAX_SHARED_LIBS; j++)
879	(-(j+1))[(unsigned long *)(libinfo.lib_list[i].start_data)] =	867	(-(j+1))[(unsigned long *)(libinfo.lib_list[i].start_data)] =
880	(libinfo.lib_list[j].loaded)?	868	(libinfo.lib_list[j].loaded)?
881	libinfo.lib_list[j].start_data:UNLOADED_LIB;	869	libinfo.lib_list[j].start_data:UNLOADED_LIB;
882		870
883	install_exec_creds(bprm);	871	install_exec_creds(bprm);
884	current->flags &= ~PF_FORKNOEXEC;	872	current->flags &= ~PF_FORKNOEXEC;
885		873
886	set_binfmt(&flat_format);	874	set_binfmt(&flat_format);
887		875
888	p = ((current->mm->context.end_brk + stack_len + 3) & ~3) - 4;	876	p = ((current->mm->context.end_brk + stack_len + 3) & ~3) - 4;
889	DBG_FLT("p=%x\n", (int)p);	877	DBG_FLT("p=%x\n", (int)p);
890		878
891	/* copy the arg pages onto the stack, this could be more efficient :-) */	879	/* copy the arg pages onto the stack, this could be more efficient :-) */
892	for (i = TOP_OF_ARGS - 1; i >= bprm->p; i--)	880	for (i = TOP_OF_ARGS - 1; i >= bprm->p; i--)
893	* (char *) --p =	881	* (char *) --p =
894	((char *) page_address(bprm->page[i/PAGE_SIZE]))[i % PAGE_SIZE];	882	((char *) page_address(bprm->page[i/PAGE_SIZE]))[i % PAGE_SIZE];
895		883
896	sp = (unsigned long *) create_flat_tables(p, bprm);	884	sp = (unsigned long *) create_flat_tables(p, bprm);
897		885
898	/* Fake some return addresses to ensure the call chain will	886	/* Fake some return addresses to ensure the call chain will
899	* initialise library in order for us. We are required to call	887	* initialise library in order for us. We are required to call
900	* lib 1 first, then 2, ... and finally the main program (id 0).	888	* lib 1 first, then 2, ... and finally the main program (id 0).
901	*/	889	*/
902	start_addr = libinfo.lib_list[0].entry;	890	start_addr = libinfo.lib_list[0].entry;
903		891
904	#ifdef CONFIG_BINFMT_SHARED_FLAT	892	#ifdef CONFIG_BINFMT_SHARED_FLAT
905	for (i = MAX_SHARED_LIBS-1; i>0; i--) {	893	for (i = MAX_SHARED_LIBS-1; i>0; i--) {
906	if (libinfo.lib_list[i].loaded) {	894	if (libinfo.lib_list[i].loaded) {
907	/* Push previos first to call address */	895	/* Push previos first to call address */
908	--sp; put_user(start_addr, sp);	896	--sp; put_user(start_addr, sp);
909	start_addr = libinfo.lib_list[i].entry;	897	start_addr = libinfo.lib_list[i].entry;
910	}	898	}
911	}	899	}
912	#endif	900	#endif
913		901
914	/* Stash our initial stack pointer into the mm structure */	902	/* Stash our initial stack pointer into the mm structure */
915	current->mm->start_stack = (unsigned long )sp;	903	current->mm->start_stack = (unsigned long )sp;
916		904
917	#ifdef FLAT_PLAT_INIT	905	#ifdef FLAT_PLAT_INIT
918	FLAT_PLAT_INIT(regs);	906	FLAT_PLAT_INIT(regs);
919	#endif	907	#endif
920	DBG_FLT("start_thread(regs=0x%x, entry=0x%x, start_stack=0x%x)\n",	908	DBG_FLT("start_thread(regs=0x%x, entry=0x%x, start_stack=0x%x)\n",
921	(int)regs, (int)start_addr, (int)current->mm->start_stack);	909	(int)regs, (int)start_addr, (int)current->mm->start_stack);
922		910
923	start_thread(regs, start_addr, current->mm->start_stack);	911	start_thread(regs, start_addr, current->mm->start_stack);
924		912
925	return 0;	913	return 0;
926	}	914	}
927		915
928	/****************************************************************************/	916	/****************************************************************************/
929		917
930	static int __init init_flat_binfmt(void)	918	static int __init init_flat_binfmt(void)
931	{	919	{
932	return register_binfmt(&flat_format);	920	return register_binfmt(&flat_format);
933	}	921	}
934		922
935	/****************************************************************************/	923	/****************************************************************************/
936		924

fs/proc/internal.h

Diff comments View file @ c40f6f8

 /* internal.h: internal procfs definitions
  *
  * Copyright (C) 2004 Red Hat, Inc. All Rights Reserved.
  * Written by David Howells (dhowells@redhat.com)
  *
  * This program is free software; you can redistribute it and/or
  * modify it under the terms of the GNU General Public License
  * as published by the Free Software Foundation; either version
  * 2 of the License, or (at your option) any later version.
  */
 #include <linux/proc_fs.h>
 extern struct proc_dir_entry proc_root;
 #ifdef CONFIG_PROC_SYSCTL
 extern int proc_sys_init(void);
 #else
 static inline void proc_sys_init(void) { }
 #endif
 #ifdef CONFIG_NET
 extern int proc_net_init(void);
 #else
 static inline int proc_net_init(void) { return 0; }
 #endif
 struct vmalloc_info {
 	unsigned long	used;
 	unsigned long	largest_chunk;
 };
 extern struct mm_struct *mm_for_maps(struct task_struct *);
 #ifdef CONFIG_MMU
 #define VMALLOC_TOTAL (VMALLOC_END - VMALLOC_START)
 extern void get_vmalloc_info(struct vmalloc_info *vmi);
 #else
 #define VMALLOC_TOTAL 0UL
 #define get_vmalloc_info(vmi)			\
 do {						\
 	(vmi)->used = 0;			\
 	(vmi)->largest_chunk = 0;		\
 } while(0)
-extern int nommu_vma_show(struct seq_file *, struct vm_area_struct *);
 #endif
 extern int proc_tid_stat(struct seq_file *m, struct pid_namespace *ns,
 				struct pid *pid, struct task_struct *task);
 extern int proc_tgid_stat(struct seq_file *m, struct pid_namespace *ns,
 				struct pid *pid, struct task_struct *task);
 extern int proc_pid_status(struct seq_file *m, struct pid_namespace *ns,
 				struct pid *pid, struct task_struct *task);
 extern int proc_pid_statm(struct seq_file *m, struct pid_namespace *ns,
 				struct pid *pid, struct task_struct *task);
 extern loff_t mem_lseek(struct file *file, loff_t offset, int orig);
 extern const struct file_operations proc_maps_operations;
 extern const struct file_operations proc_numa_maps_operations;
 extern const struct file_operations proc_smaps_operations;
 extern const struct file_operations proc_clear_refs_operations;
 extern const struct file_operations proc_pagemap_operations;
 extern const struct file_operations proc_net_operations;
 extern const struct inode_operations proc_net_inode_operations;
 void free_proc_entry(struct proc_dir_entry *de);
 void proc_init_inodecache(void);
 static inline struct pid *proc_pid(struct inode *inode)
 {
 	return PROC_I(inode)->pid;
 }
 static inline struct task_struct *get_proc_task(struct inode *inode)
 {
 	return get_pid_task(proc_pid(inode), PIDTYPE_PID);
 }
 static inline int proc_fd(struct inode *inode)
 {
 	return PROC_I(inode)->fd;
 }
 struct dentry *proc_lookup_de(struct proc_dir_entry *de, struct inode *ino,
 		struct dentry *dentry);
 int proc_readdir_de(struct proc_dir_entry *de, struct file *filp, void *dirent,
 		filldir_t filldir);
 struct pde_opener {
 	struct inode *inode;
 	struct file *file;
 	int (*release)(struct inode *, struct file *);
 	struct list_head lh;
 };

fs/proc/meminfo.c

Diff comments View file @ c40f6f8

 #include <linux/fs.h>
 #include <linux/hugetlb.h>
 #include <linux/init.h>
 #include <linux/kernel.h>
 #include <linux/mm.h>
 #include <linux/mman.h>
 #include <linux/mmzone.h>
 #include <linux/proc_fs.h>
 #include <linux/quicklist.h>
 #include <linux/seq_file.h>
 #include <linux/swap.h>
 #include <linux/vmstat.h>
 #include <asm/atomic.h>
 #include <asm/page.h>
 #include <asm/pgtable.h>
 #include "internal.h"
 void __attribute__((weak)) arch_report_meminfo(struct seq_file *m)
 {
 }
 static int meminfo_proc_show(struct seq_file *m, void *v)
 {
 	struct sysinfo i;
 	unsigned long committed;
 	unsigned long allowed;
 	struct vmalloc_info vmi;
 	long cached;
 	unsigned long pages[NR_LRU_LISTS];
 	int lru;
 /*
  * display in kilobytes.
  */
 #define K(x) ((x) << (PAGE_SHIFT - 10))
 	si_meminfo(&i);
 	si_swapinfo(&i);
 	committed = atomic_long_read(&vm_committed_space);
 	allowed = ((totalram_pages - hugetlb_total_pages())
 		* sysctl_overcommit_ratio / 100) + total_swap_pages;
 	cached = global_page_state(NR_FILE_PAGES) -
 			total_swapcache_pages - i.bufferram;
 	if (cached < 0)
 		cached = 0;
 	get_vmalloc_info(&vmi);
 	for (lru = LRU_BASE; lru < NR_LRU_LISTS; lru++)
 		pages[lru] = global_page_state(NR_LRU_BASE + lru);
 	/*
 	 * Tagged format, for easy grepping and expansion.
 	 */
 	seq_printf(m,
 		"MemTotal:       %8lu kB\n"
 		"MemFree:        %8lu kB\n"
 		"Buffers:        %8lu kB\n"
 		"Cached:         %8lu kB\n"
 		"SwapCached:     %8lu kB\n"
 		"Active:         %8lu kB\n"
 		"Inactive:       %8lu kB\n"
 		"Active(anon):   %8lu kB\n"
 		"Inactive(anon): %8lu kB\n"
 		"Active(file):   %8lu kB\n"
 		"Inactive(file): %8lu kB\n"
 #ifdef CONFIG_UNEVICTABLE_LRU
 		"Unevictable:    %8lu kB\n"
 		"Mlocked:        %8lu kB\n"
 #endif
 #ifdef CONFIG_HIGHMEM
 		"HighTotal:      %8lu kB\n"
 		"HighFree:       %8lu kB\n"
 		"LowTotal:       %8lu kB\n"
 		"LowFree:        %8lu kB\n"
 #endif
+#ifndef CONFIG_MMU
+		"MmapCopy:       %8lu kB\n"
+#endif
 		"SwapTotal:      %8lu kB\n"
 		"SwapFree:       %8lu kB\n"
 		"Dirty:          %8lu kB\n"
 		"Writeback:      %8lu kB\n"
 		"AnonPages:      %8lu kB\n"
 		"Mapped:         %8lu kB\n"
 		"Slab:           %8lu kB\n"
 		"SReclaimable:   %8lu kB\n"
 		"SUnreclaim:     %8lu kB\n"
 		"PageTables:     %8lu kB\n"
 #ifdef CONFIG_QUICKLIST
 		"Quicklists:     %8lu kB\n"
 #endif
 		"NFS_Unstable:   %8lu kB\n"
 		"Bounce:         %8lu kB\n"
 		"WritebackTmp:   %8lu kB\n"
 		"CommitLimit:    %8lu kB\n"
 		"Committed_AS:   %8lu kB\n"
 		"VmallocTotal:   %8lu kB\n"
 		"VmallocUsed:    %8lu kB\n"
 		"VmallocChunk:   %8lu kB\n",
 		K(i.totalram),
 		K(i.freeram),
 		K(i.bufferram),
 		K(cached),
 		K(total_swapcache_pages),
 		K(pages[LRU_ACTIVE_ANON]   + pages[LRU_ACTIVE_FILE]),
 		K(pages[LRU_INACTIVE_ANON] + pages[LRU_INACTIVE_FILE]),
 		K(pages[LRU_ACTIVE_ANON]),
 		K(pages[LRU_INACTIVE_ANON]),
 		K(pages[LRU_ACTIVE_FILE]),
 		K(pages[LRU_INACTIVE_FILE]),
 #ifdef CONFIG_UNEVICTABLE_LRU
 		K(pages[LRU_UNEVICTABLE]),
 		K(global_page_state(NR_MLOCK)),
 #endif
 #ifdef CONFIG_HIGHMEM
 		K(i.totalhigh),
 		K(i.freehigh),
 		K(i.totalram-i.totalhigh),
 		K(i.freeram-i.freehigh),
+#endif
+#ifndef CONFIG_MMU
+		K((unsigned long) atomic_read(&mmap_pages_allocated)),
 #endif
 		K(i.totalswap),
 		K(i.freeswap),
 		K(global_page_state(NR_FILE_DIRTY)),
 		K(global_page_state(NR_WRITEBACK)),
 		K(global_page_state(NR_ANON_PAGES)),
 		K(global_page_state(NR_FILE_MAPPED)),
 		K(global_page_state(NR_SLAB_RECLAIMABLE) +
 				global_page_state(NR_SLAB_UNRECLAIMABLE)),
 		K(global_page_state(NR_SLAB_RECLAIMABLE)),
 		K(global_page_state(NR_SLAB_UNRECLAIMABLE)),
 		K(global_page_state(NR_PAGETABLE)),
 #ifdef CONFIG_QUICKLIST
 		K(quicklist_total_size()),
 #endif
 		K(global_page_state(NR_UNSTABLE_NFS)),
 		K(global_page_state(NR_BOUNCE)),
 		K(global_page_state(NR_WRITEBACK_TEMP)),
 		K(allowed),
 		K(committed),
 		(unsigned long)VMALLOC_TOTAL >> 10,
 		vmi.used >> 10,
 		vmi.largest_chunk >> 10
 		);
 	hugetlb_report_meminfo(m);
 	arch_report_meminfo(m);
 	return 0;
 #undef K
 }
 static int meminfo_proc_open(struct inode *inode, struct file *file)
 {
 	return single_open(file, meminfo_proc_show, NULL);
 }
 static const struct file_operations meminfo_proc_fops = {
 	.open		= meminfo_proc_open,
 	.read		= seq_read,
 	.llseek		= seq_lseek,
 	.release	= single_release,
 };
 static int __init proc_meminfo_init(void)
 {
 	proc_create("meminfo", 0, NULL, &meminfo_proc_fops);
 	return 0;
 }
 module_init(proc_meminfo_init);

fs/proc/nommu.c

Diff comments View file @ c40f6f8

 /* nommu.c: mmu-less memory info files
  *
  * Copyright (C) 2004 Red Hat, Inc. All Rights Reserved.
  * Written by David Howells (dhowells@redhat.com)
  *
  * This program is free software; you can redistribute it and/or
  * modify it under the terms of the GNU General Public License
  * as published by the Free Software Foundation; either version
  * 2 of the License, or (at your option) any later version.
  */
 #include <linux/init.h>
 #include <linux/module.h>
 #include <linux/errno.h>
 #include <linux/time.h>
 #include <linux/kernel.h>
 #include <linux/string.h>
 #include <linux/mman.h>
 #include <linux/proc_fs.h>
 #include <linux/mm.h>
 #include <linux/mmzone.h>
 #include <linux/pagemap.h>
 #include <linux/swap.h>
 #include <linux/slab.h>
 #include <linux/smp.h>
 #include <linux/seq_file.h>
 #include <linux/hugetlb.h>
 #include <linux/vmalloc.h>
 #include <asm/uaccess.h>
 #include <asm/pgtable.h>
 #include <asm/tlb.h>
 #include <asm/div64.h>
 #include "internal.h"
 /*
- * display a single VMA to a sequenced file
+ * display a single region to a sequenced file
  */
-int nommu_vma_show(struct seq_file *m, struct vm_area_struct *vma)
+static int nommu_region_show(struct seq_file *m, struct vm_region *region)
 {
 	unsigned long ino = 0;
 	struct file *file;
 	dev_t dev = 0;
 	int flags, len;
-	flags = vma->vm_flags;
+	flags = region->vm_flags;
-	file = vma->vm_file;
+	file = region->vm_file;
 	if (file) {
-		struct inode *inode = vma->vm_file->f_path.dentry->d_inode;
+		struct inode *inode = region->vm_file->f_path.dentry->d_inode;
 		dev = inode->i_sb->s_dev;
 		ino = inode->i_ino;
 	}
 	seq_printf(m,
 		   "%08lx-%08lx %c%c%c%c %08llx %02x:%02x %lu %n",
-		   vma->vm_start,
+		   region->vm_start,
-		   vma->vm_end,
+		   region->vm_end,
 		   flags & VM_READ ? 'r' : '-',
 		   flags & VM_WRITE ? 'w' : '-',
 		   flags & VM_EXEC ? 'x' : '-',
 		   flags & VM_MAYSHARE ? flags & VM_SHARED ? 'S' : 's' : 'p',
-		   ((loff_t)vma->vm_pgoff) << PAGE_SHIFT,
+		   ((loff_t)region->vm_pgoff) << PAGE_SHIFT,
 		   MAJOR(dev), MINOR(dev), ino, &len);
 	if (file) {
 		len = 25 + sizeof(void *) * 6 - len;
 		if (len < 1)
 			len = 1;
 		seq_printf(m, "%*c", len, ' ');
 		seq_path(m, &file->f_path, "");
 	}
 	seq_putc(m, '\n');
 	return 0;
 }
 /*
- * display a list of all the VMAs the kernel knows about
+ * display a list of all the REGIONs the kernel knows about
  * - nommu kernals have a single flat list
  */
-static int nommu_vma_list_show(struct seq_file *m, void *v)
+static int nommu_region_list_show(struct seq_file *m, void *_p)
 {
-	struct vm_area_struct *vma;
+	struct rb_node *p = _p;
-	vma = rb_entry((struct rb_node *) v, struct vm_area_struct, vm_rb);
+	return nommu_region_show(m, rb_entry(p, struct vm_region, vm_rb));
-	return nommu_vma_show(m, vma);
 }
-static void *nommu_vma_list_start(struct seq_file *m, loff_t *_pos)
+static void *nommu_region_list_start(struct seq_file *m, loff_t *_pos)
 {
-	struct rb_node *_rb;
+	struct rb_node *p;
 	loff_t pos = *_pos;
-	void *next = NULL;
-	down_read(&nommu_vma_sem);
+	down_read(&nommu_region_sem);
-	for (_rb = rb_first(&nommu_vma_tree); _rb; _rb = rb_next(_rb)) {
+	for (p = rb_first(&nommu_region_tree); p; p = rb_next(p))
-		if (pos == 0) {
+		if (pos-- == 0)
-			next = _rb;
+			return p;
-			break;
+	return NULL;
-		}
-		pos--;
-	}
-	return next;
 }
-static void nommu_vma_list_stop(struct seq_file *m, void *v)
+static void nommu_region_list_stop(struct seq_file *m, void *v)
 {
-	up_read(&nommu_vma_sem);
+	up_read(&nommu_region_sem);
 }
-static void *nommu_vma_list_next(struct seq_file *m, void *v, loff_t *pos)
+static void *nommu_region_list_next(struct seq_file *m, void *v, loff_t *pos)
 {
 	(*pos)++;
 	return rb_next((struct rb_node *) v);
 }
-static const struct seq_operations proc_nommu_vma_list_seqop = {
+static struct seq_operations proc_nommu_region_list_seqop = {
-	.start	= nommu_vma_list_start,
+	.start	= nommu_region_list_start,
-	.next	= nommu_vma_list_next,
+	.next	= nommu_region_list_next,
-	.stop	= nommu_vma_list_stop,
+	.stop	= nommu_region_list_stop,
-	.show	= nommu_vma_list_show
+	.show	= nommu_region_list_show
 };
-static int proc_nommu_vma_list_open(struct inode *inode, struct file *file)
+static int proc_nommu_region_list_open(struct inode *inode, struct file *file)
 {
-	return seq_open(file, &proc_nommu_vma_list_seqop);
+	return seq_open(file, &proc_nommu_region_list_seqop);
 }
-static const struct file_operations proc_nommu_vma_list_operations = {
+static const struct file_operations proc_nommu_region_list_operations = {
-	.open    = proc_nommu_vma_list_open,
+	.open    = proc_nommu_region_list_open,
 	.read    = seq_read,
 	.llseek  = seq_lseek,
 	.release = seq_release,
 };
 static int __init proc_nommu_init(void)
 {
-	proc_create("maps", S_IRUGO, NULL, &proc_nommu_vma_list_operations);
+	proc_create("maps", S_IRUGO, NULL, &proc_nommu_region_list_operations);
 	return 0;
 }
 module_init(proc_nommu_init);

fs/proc/task_nommu.c

Diff comments View file @ c40f6f8

 #include <linux/mm.h>
 #include <linux/file.h>
 #include <linux/fdtable.h>
 #include <linux/mount.h>
 #include <linux/ptrace.h>
 #include <linux/seq_file.h>
 #include "internal.h"
 /*
  * Logic: we've got two memory sums for each process, "shared", and
  * "non-shared". Shared memory may get counted more than once, for
  * each process that owns it. Non-shared memory is counted
  * accurately.
  */
 void task_mem(struct seq_file *m, struct mm_struct *mm)
 {
-	struct vm_list_struct *vml;
+	struct vm_area_struct *vma;
-	unsigned long bytes = 0, sbytes = 0, slack = 0;
+	struct vm_region *region;
+	struct rb_node *p;
+	unsigned long bytes = 0, sbytes = 0, slack = 0, size;
 	down_read(&mm->mmap_sem);
-	for (vml = mm->context.vmlist; vml; vml = vml->next) {
+	for (p = rb_first(&mm->mm_rb); p; p = rb_next(p)) {
-		if (!vml->vma)
+		vma = rb_entry(p, struct vm_area_struct, vm_rb);
-			continue;
-		bytes += kobjsize(vml);
+		bytes += kobjsize(vma);
+		region = vma->vm_region;
+		if (region) {
+			size = kobjsize(region);
+			size += region->vm_end - region->vm_start;
+		} else {
+			size = vma->vm_end - vma->vm_start;
+		}
 		if (atomic_read(&mm->mm_count) > 1 ||
-		    atomic_read(&vml->vma->vm_usage) > 1
+		    vma->vm_flags & VM_MAYSHARE) {
-		    ) {
+			sbytes += size;
-			sbytes += kobjsize((void *) vml->vma->vm_start);
-			sbytes += kobjsize(vml->vma);
 		} else {
-			bytes += kobjsize((void *) vml->vma->vm_start);
+			bytes += size;
-			bytes += kobjsize(vml->vma);
+			if (region)
-			slack += kobjsize((void *) vml->vma->vm_start) -
+				slack = region->vm_end - vma->vm_end;
-				(vml->vma->vm_end - vml->vma->vm_start);
 		}
 	}
 	if (atomic_read(&mm->mm_count) > 1)
 		sbytes += kobjsize(mm);
 	else
 		bytes += kobjsize(mm);
 	if (current->fs && atomic_read(&current->fs->count) > 1)
 		sbytes += kobjsize(current->fs);
 	else
 		bytes += kobjsize(current->fs);
 	if (current->files && atomic_read(&current->files->count) > 1)
 		sbytes += kobjsize(current->files);
 	else
 		bytes += kobjsize(current->files);
 	if (current->sighand && atomic_read(&current->sighand->count) > 1)
 		sbytes += kobjsize(current->sighand);
 	else
 		bytes += kobjsize(current->sighand);
 	bytes += kobjsize(current); /* includes kernel stack */
 	seq_printf(m,
 		"Mem:\t%8lu bytes\n"
 		"Slack:\t%8lu bytes\n"
 		"Shared:\t%8lu bytes\n",
 		bytes, slack, sbytes);
 	up_read(&mm->mmap_sem);
 }
 unsigned long task_vsize(struct mm_struct *mm)
 {
-	struct vm_list_struct *tbp;
+	struct vm_area_struct *vma;
+	struct rb_node *p;
 	unsigned long vsize = 0;
 	down_read(&mm->mmap_sem);
-	for (tbp = mm->context.vmlist; tbp; tbp = tbp->next) {
+	for (p = rb_first(&mm->mm_rb); p; p = rb_next(p)) {
-		if (tbp->vma)
+		vma = rb_entry(p, struct vm_area_struct, vm_rb);
-			vsize += kobjsize((void *) tbp->vma->vm_start);
+		vsize += vma->vm_end - vma->vm_start;
 	}
 	up_read(&mm->mmap_sem);
 	return vsize;
 }
 int task_statm(struct mm_struct *mm, int *shared, int *text,
 	       int *data, int *resident)
 {
-	struct vm_list_struct *tbp;
+	struct vm_area_struct *vma;
+	struct vm_region *region;
+	struct rb_node *p;
 	int size = kobjsize(mm);
 	down_read(&mm->mmap_sem);
-	for (tbp = mm->context.vmlist; tbp; tbp = tbp->next) {
+	for (p = rb_first(&mm->mm_rb); p; p = rb_next(p)) {
-		size += kobjsize(tbp);
+		vma = rb_entry(p, struct vm_area_struct, vm_rb);
-		if (tbp->vma) {
+		size += kobjsize(vma);
-			size += kobjsize(tbp->vma);
+		region = vma->vm_region;
-			size += kobjsize((void *) tbp->vma->vm_start);
+		if (region) {
+			size += kobjsize(region);
+			size += region->vm_end - region->vm_start;
 		}
 	}
 	size += (*text = mm->end_code - mm->start_code);
 	size += (*data = mm->start_stack - mm->start_data);
 	up_read(&mm->mmap_sem);
 	*resident = size;
 	return size;
 }
 /*
+ * display a single VMA to a sequenced file
+ */
+static int nommu_vma_show(struct seq_file *m, struct vm_area_struct *vma)
+{
+	unsigned long ino = 0;
+	struct file *file;
+	dev_t dev = 0;
+	int flags, len;
+	flags = vma->vm_flags;
+	file = vma->vm_file;
+	if (file) {
+		struct inode *inode = vma->vm_file->f_path.dentry->d_inode;
+		dev = inode->i_sb->s_dev;
+		ino = inode->i_ino;
+	}
+	seq_printf(m,
+		   "%08lx-%08lx %c%c%c%c %08lx %02x:%02x %lu %n",
+		   vma->vm_start,
+		   vma->vm_end,
+		   flags & VM_READ ? 'r' : '-',
+		   flags & VM_WRITE ? 'w' : '-',
+		   flags & VM_EXEC ? 'x' : '-',
+		   flags & VM_MAYSHARE ? flags & VM_SHARED ? 'S' : 's' : 'p',
+		   vma->vm_pgoff << PAGE_SHIFT,
+		   MAJOR(dev), MINOR(dev), ino, &len);
+	if (file) {
+		len = 25 + sizeof(void *) * 6 - len;
+		if (len < 1)
+			len = 1;
+		seq_printf(m, "%*c", len, ' ');
+		seq_path(m, &file->f_path, "");
+	}
+	seq_putc(m, '\n');
+	return 0;
+}
+/*
  * display mapping lines for a particular process's /proc/pid/maps
  */
-static int show_map(struct seq_file *m, void *_vml)
+static int show_map(struct seq_file *m, void *_p)
 {
-	struct vm_list_struct *vml = _vml;
+	struct rb_node *p = _p;
-	return nommu_vma_show(m, vml->vma);
+	return nommu_vma_show(m, rb_entry(p, struct vm_area_struct, vm_rb));
 }
 static void *m_start(struct seq_file *m, loff_t *pos)
 {
 	struct proc_maps_private *priv = m->private;
-	struct vm_list_struct *vml;
 	struct mm_struct *mm;
+	struct rb_node *p;
 	loff_t n = *pos;
 	/* pin the task and mm whilst we play with them */
 	priv->task = get_pid_task(priv->pid, PIDTYPE_PID);
 	if (!priv->task)
 		return NULL;
 	mm = mm_for_maps(priv->task);
 	if (!mm) {
 		put_task_struct(priv->task);
 		priv->task = NULL;
 		return NULL;
 	}
 	/* start from the Nth VMA */
-	for (vml = mm->context.vmlist; vml; vml = vml->next)
+	for (p = rb_first(&mm->mm_rb); p; p = rb_next(p))
 		if (n-- == 0)
-			return vml;
+			return p;
 	return NULL;
 }
 static void m_stop(struct seq_file *m, void *_vml)
 {
 	struct proc_maps_private *priv = m->private;
 	if (priv->task) {
 		struct mm_struct *mm = priv->task->mm;
 		up_read(&mm->mmap_sem);
 		mmput(mm);
 		put_task_struct(priv->task);
 	}
 }
-static void *m_next(struct seq_file *m, void *_vml, loff_t *pos)
+static void *m_next(struct seq_file *m, void *_p, loff_t *pos)
 {
-	struct vm_list_struct *vml = _vml;
+	struct rb_node *p = _p;
 	(*pos)++;
-	return vml ? vml->next : NULL;
+	return p ? rb_next(p) : NULL;
 }
 static const struct seq_operations proc_pid_maps_ops = {
 	.start	= m_start,
 	.next	= m_next,
 	.stop	= m_stop,
 	.show	= show_map
 };
 static int maps_open(struct inode *inode, struct file *file)
 {
 	struct proc_maps_private *priv;
 	int ret = -ENOMEM;
 	priv = kzalloc(sizeof(*priv), GFP_KERNEL);
 	if (priv) {
 		priv->pid = proc_pid(inode);
 		ret = seq_open(file, &proc_pid_maps_ops);
 		if (!ret) {
 			struct seq_file *m = file->private_data;
 			m->private = priv;
 		} else {
 			kfree(priv);
 		}
 	}
 	return ret;
 }
 const struct file_operations proc_maps_operations = {
 	.open		= maps_open,
 	.read		= seq_read,

fs/ramfs/file-nommu.c

Diff comments View file @ c40f6f8

1	/* file-nommu.c: no-MMU version of ramfs	1	/* file-nommu.c: no-MMU version of ramfs
2	*	2	*
3	* Copyright (C) 2005 Red Hat, Inc. All Rights Reserved.	3	* Copyright (C) 2005 Red Hat, Inc. All Rights Reserved.
4	* Written by David Howells (dhowells@redhat.com)	4	* Written by David Howells (dhowells@redhat.com)
5	*	5	*
6	* This program is free software; you can redistribute it and/or	6	* This program is free software; you can redistribute it and/or
7	* modify it under the terms of the GNU General Public License	7	* modify it under the terms of the GNU General Public License
8	* as published by the Free Software Foundation; either version	8	* as published by the Free Software Foundation; either version
9	* 2 of the License, or (at your option) any later version.	9	* 2 of the License, or (at your option) any later version.
10	*/	10	*/
11		11
12	#include <linux/module.h>	12	#include <linux/module.h>
13	#include <linux/fs.h>	13	#include <linux/fs.h>
14	#include <linux/mm.h>	14	#include <linux/mm.h>
15	#include <linux/pagemap.h>	15	#include <linux/pagemap.h>
16	#include <linux/highmem.h>	16	#include <linux/highmem.h>
17	#include <linux/init.h>	17	#include <linux/init.h>
18	#include <linux/string.h>	18	#include <linux/string.h>
19	#include <linux/backing-dev.h>	19	#include <linux/backing-dev.h>
20	#include <linux/ramfs.h>	20	#include <linux/ramfs.h>
21	#include <linux/quotaops.h>	21	#include <linux/quotaops.h>
22	#include <linux/pagevec.h>	22	#include <linux/pagevec.h>
23	#include <linux/mman.h>	23	#include <linux/mman.h>
24		24
25	#include <asm/uaccess.h>	25	#include <asm/uaccess.h>
26	#include "internal.h"	26	#include "internal.h"
27		27
28	static int ramfs_nommu_setattr(struct dentry , struct iattr );	28	static int ramfs_nommu_setattr(struct dentry , struct iattr );
29		29
30	const struct address_space_operations ramfs_aops = {	30	const struct address_space_operations ramfs_aops = {
31	.readpage = simple_readpage,	31	.readpage = simple_readpage,
32	.write_begin = simple_write_begin,	32	.write_begin = simple_write_begin,
33	.write_end = simple_write_end,	33	.write_end = simple_write_end,
34	.set_page_dirty = __set_page_dirty_no_writeback,	34	.set_page_dirty = __set_page_dirty_no_writeback,
35	};	35	};
36		36
37	const struct file_operations ramfs_file_operations = {	37	const struct file_operations ramfs_file_operations = {
38	.mmap = ramfs_nommu_mmap,	38	.mmap = ramfs_nommu_mmap,
39	.get_unmapped_area = ramfs_nommu_get_unmapped_area,	39	.get_unmapped_area = ramfs_nommu_get_unmapped_area,
40	.read = do_sync_read,	40	.read = do_sync_read,
41	.aio_read = generic_file_aio_read,	41	.aio_read = generic_file_aio_read,
42	.write = do_sync_write,	42	.write = do_sync_write,
43	.aio_write = generic_file_aio_write,	43	.aio_write = generic_file_aio_write,
44	.fsync = simple_sync_file,	44	.fsync = simple_sync_file,
45	.splice_read = generic_file_splice_read,	45	.splice_read = generic_file_splice_read,
46	.splice_write = generic_file_splice_write,	46	.splice_write = generic_file_splice_write,
47	.llseek = generic_file_llseek,	47	.llseek = generic_file_llseek,
48	};	48	};
49		49
50	const struct inode_operations ramfs_file_inode_operations = {	50	const struct inode_operations ramfs_file_inode_operations = {
51	.setattr = ramfs_nommu_setattr,	51	.setattr = ramfs_nommu_setattr,
52	.getattr = simple_getattr,	52	.getattr = simple_getattr,
53	};	53	};
54		54
55	/*****************************************************************************/	55	/*****************************************************************************/
56	/*	56	/*
57	* add a contiguous set of pages into a ramfs inode when it's truncated from	57	* add a contiguous set of pages into a ramfs inode when it's truncated from
58	* size 0 on the assumption that it's going to be used for an mmap of shared	58	* size 0 on the assumption that it's going to be used for an mmap of shared
59	* memory	59	* memory
60	*/	60	*/
61	int ramfs_nommu_expand_for_mapping(struct inode *inode, size_t newsize)	61	int ramfs_nommu_expand_for_mapping(struct inode *inode, size_t newsize)
62	{	62	{
63	struct pagevec lru_pvec;	63	struct pagevec lru_pvec;
64	unsigned long npages, xpages, loop, limit;	64	unsigned long npages, xpages, loop, limit;
65	struct page *pages;	65	struct page *pages;
66	unsigned order;	66	unsigned order;
67	void *data;	67	void *data;
68	int ret;	68	int ret;
69		69
70	/* make various checks */	70	/* make various checks */
71	order = get_order(newsize);	71	order = get_order(newsize);
72	if (unlikely(order >= MAX_ORDER))	72	if (unlikely(order >= MAX_ORDER))
73	goto too_big;	73	goto too_big;
74		74
75	limit = current->signal->rlim[RLIMIT_FSIZE].rlim_cur;	75	limit = current->signal->rlim[RLIMIT_FSIZE].rlim_cur;
76	if (limit != RLIM_INFINITY && newsize > limit)	76	if (limit != RLIM_INFINITY && newsize > limit)
77	goto fsize_exceeded;	77	goto fsize_exceeded;
78		78
79	if (newsize > inode->i_sb->s_maxbytes)	79	if (newsize > inode->i_sb->s_maxbytes)
80	goto too_big;	80	goto too_big;
81		81
82	i_size_write(inode, newsize);	82	i_size_write(inode, newsize);
83		83
84	/* allocate enough contiguous pages to be able to satisfy the	84	/* allocate enough contiguous pages to be able to satisfy the
85	* request */	85	* request */
86	pages = alloc_pages(mapping_gfp_mask(inode->i_mapping), order);	86	pages = alloc_pages(mapping_gfp_mask(inode->i_mapping), order);
87	if (!pages)	87	if (!pages)
88	return -ENOMEM;	88	return -ENOMEM;
89		89
90	/* split the high-order page into an array of single pages */	90	/* split the high-order page into an array of single pages */
91	xpages = 1UL << order;	91	xpages = 1UL << order;
92	npages = (newsize + PAGE_SIZE - 1) >> PAGE_SHIFT;	92	npages = (newsize + PAGE_SIZE - 1) >> PAGE_SHIFT;
93		93
94	split_page(pages, order);	94	split_page(pages, order);
95		95
96	/* trim off any pages we don't actually require */	96	/* trim off any pages we don't actually require */
97	for (loop = npages; loop < xpages; loop++)	97	for (loop = npages; loop < xpages; loop++)
98	__free_page(pages + loop);	98	__free_page(pages + loop);
99		99
100	/* clear the memory we allocated */	100	/* clear the memory we allocated */
101	newsize = PAGE_SIZE * npages;	101	newsize = PAGE_SIZE * npages;
102	data = page_address(pages);	102	data = page_address(pages);
103	memset(data, 0, newsize);	103	memset(data, 0, newsize);
104		104
105	/* attach all the pages to the inode's address space */	105	/* attach all the pages to the inode's address space */
106	pagevec_init(&lru_pvec, 0);	106	pagevec_init(&lru_pvec, 0);
107	for (loop = 0; loop < npages; loop++) {	107	for (loop = 0; loop < npages; loop++) {
108	struct page *page = pages + loop;	108	struct page *page = pages + loop;
109		109
110	ret = add_to_page_cache(page, inode->i_mapping, loop, GFP_KERNEL);	110	ret = add_to_page_cache(page, inode->i_mapping, loop, GFP_KERNEL);
111	if (ret < 0)	111	if (ret < 0)
112	goto add_error;	112	goto add_error;
113		113
114	if (!pagevec_add(&lru_pvec, page))	114	if (!pagevec_add(&lru_pvec, page))
115	__pagevec_lru_add_file(&lru_pvec);	115	__pagevec_lru_add_file(&lru_pvec);
116		116
117	unlock_page(page);	117	unlock_page(page);
118	}	118	}
119		119
120	pagevec_lru_add_file(&lru_pvec);	120	pagevec_lru_add_file(&lru_pvec);
121	return 0;	121	return 0;
122		122
123	fsize_exceeded:	123	fsize_exceeded:
124	send_sig(SIGXFSZ, current, 0);	124	send_sig(SIGXFSZ, current, 0);
125	too_big:	125	too_big:
126	return -EFBIG;	126	return -EFBIG;
127		127
128	add_error:	128	add_error:
129	page_cache_release(pages + loop);	129	page_cache_release(pages + loop);
130	for (loop++; loop < npages; loop++)	130	for (loop++; loop < npages; loop++)
131	__free_page(pages + loop);	131	__free_page(pages + loop);
132	return ret;	132	return ret;
133	}	133	}
134		134
135	/*****************************************************************************/	135	/*****************************************************************************/
136	/*	136	/*
137	* check that file shrinkage doesn't leave any VMAs dangling in midair	137	* check that file shrinkage doesn't leave any VMAs dangling in midair
138	*/	138	*/
139	static int ramfs_nommu_check_mappings(struct inode *inode,	139	static int ramfs_nommu_check_mappings(struct inode *inode,
140	size_t newsize, size_t size)	140	size_t newsize, size_t size)
141	{	141	{
142	struct vm_area_struct *vma;	142	struct vm_area_struct *vma;
143	struct prio_tree_iter iter;	143	struct prio_tree_iter iter;
144		144
145	/* search for VMAs that fall within the dead zone */	145	/* search for VMAs that fall within the dead zone */
146	vma_prio_tree_foreach(vma, &iter, &inode->i_mapping->i_mmap,	146	vma_prio_tree_foreach(vma, &iter, &inode->i_mapping->i_mmap,
147	newsize >> PAGE_SHIFT,	147	newsize >> PAGE_SHIFT,
148	(size + PAGE_SIZE - 1) >> PAGE_SHIFT	148	(size + PAGE_SIZE - 1) >> PAGE_SHIFT
149	) {	149	) {
150	/* found one - only interested if it's shared out of the page	150	/* found one - only interested if it's shared out of the page
151	* cache */	151	* cache */
152	if (vma->vm_flags & VM_SHARED)	152	if (vma->vm_flags & VM_SHARED)
153	return -ETXTBSY; /* not quite true, but near enough */	153	return -ETXTBSY; /* not quite true, but near enough */
154	}	154	}
155		155
156	return 0;	156	return 0;
157	}	157	}
158		158
159	/*****************************************************************************/	159	/*****************************************************************************/
160	/*	160	/*
161	*	161	*
162	*/	162	*/
163	static int ramfs_nommu_resize(struct inode *inode, loff_t newsize, loff_t size)	163	static int ramfs_nommu_resize(struct inode *inode, loff_t newsize, loff_t size)
164	{	164	{
165	int ret;	165	int ret;
166		166
167	/* assume a truncate from zero size is going to be for the purposes of	167	/* assume a truncate from zero size is going to be for the purposes of
168	* shared mmap */	168	* shared mmap */
169	if (size == 0) {	169	if (size == 0) {
170	if (unlikely(newsize >> 32))	170	if (unlikely(newsize >> 32))
171	return -EFBIG;	171	return -EFBIG;
172		172
173	return ramfs_nommu_expand_for_mapping(inode, newsize);	173	return ramfs_nommu_expand_for_mapping(inode, newsize);
174	}	174	}
175		175
176	/* check that a decrease in size doesn't cut off any shared mappings */	176	/* check that a decrease in size doesn't cut off any shared mappings */
177	if (newsize < size) {	177	if (newsize < size) {
178	ret = ramfs_nommu_check_mappings(inode, newsize, size);	178	ret = ramfs_nommu_check_mappings(inode, newsize, size);
179	if (ret < 0)	179	if (ret < 0)
180	return ret;	180	return ret;
181	}	181	}
182		182
183	ret = vmtruncate(inode, newsize);	183	ret = vmtruncate(inode, newsize);
184		184
185	return ret;	185	return ret;
186	}	186	}
187		187
188	/*****************************************************************************/	188	/*****************************************************************************/
189	/*	189	/*
190	* handle a change of attributes	190	* handle a change of attributes
191	* - we're specifically interested in a change of size	191	* - we're specifically interested in a change of size
192	*/	192	*/
193	static int ramfs_nommu_setattr(struct dentry dentry, struct iattr ia)	193	static int ramfs_nommu_setattr(struct dentry dentry, struct iattr ia)
194	{	194	{
195	struct inode *inode = dentry->d_inode;	195	struct inode *inode = dentry->d_inode;
196	unsigned int old_ia_valid = ia->ia_valid;	196	unsigned int old_ia_valid = ia->ia_valid;
197	int ret = 0;	197	int ret = 0;
198		198
199	/* POSIX UID/GID verification for setting inode attributes */	199	/* POSIX UID/GID verification for setting inode attributes */
200	ret = inode_change_ok(inode, ia);	200	ret = inode_change_ok(inode, ia);
201	if (ret)	201	if (ret)
202	return ret;	202	return ret;
203		203
204	/* by providing our own setattr() method, we skip this quotaism */	204	/* by providing our own setattr() method, we skip this quotaism */
205	if ((old_ia_valid & ATTR_UID && ia->ia_uid != inode->i_uid) \|\|	205	if ((old_ia_valid & ATTR_UID && ia->ia_uid != inode->i_uid) \|\|
206	(old_ia_valid & ATTR_GID && ia->ia_gid != inode->i_gid))	206	(old_ia_valid & ATTR_GID && ia->ia_gid != inode->i_gid))
207	ret = DQUOT_TRANSFER(inode, ia) ? -EDQUOT : 0;	207	ret = DQUOT_TRANSFER(inode, ia) ? -EDQUOT : 0;
208		208
209	/* pick out size-changing events */	209	/* pick out size-changing events */
210	if (ia->ia_valid & ATTR_SIZE) {	210	if (ia->ia_valid & ATTR_SIZE) {
211	loff_t size = i_size_read(inode);	211	loff_t size = i_size_read(inode);
212	if (ia->ia_size != size) {	212	if (ia->ia_size != size) {
213	ret = ramfs_nommu_resize(inode, ia->ia_size, size);	213	ret = ramfs_nommu_resize(inode, ia->ia_size, size);
214	if (ret < 0 \|\| ia->ia_valid == ATTR_SIZE)	214	if (ret < 0 \|\| ia->ia_valid == ATTR_SIZE)
215	goto out;	215	goto out;
216	} else {	216	} else {
217	/* we skipped the truncate but must still update	217	/* we skipped the truncate but must still update
218	* timestamps	218	* timestamps
219	*/	219	*/
220	ia->ia_valid \|= ATTR_MTIME\|ATTR_CTIME;	220	ia->ia_valid \|= ATTR_MTIME\|ATTR_CTIME;
221	}	221	}
222	}	222	}
223		223
224	ret = inode_setattr(inode, ia);	224	ret = inode_setattr(inode, ia);
225	out:	225	out:
226	ia->ia_valid = old_ia_valid;	226	ia->ia_valid = old_ia_valid;
227	return ret;	227	return ret;
228	}	228	}
229		229
230	/*****************************************************************************/	230	/*****************************************************************************/
231	/*	231	/*
232	* try to determine where a shared mapping can be made	232	* try to determine where a shared mapping can be made
233	* - we require that:	233	* - we require that:
234	* - the pages to be mapped must exist	234	* - the pages to be mapped must exist
235	* - the pages be physically contiguous in sequence	235	* - the pages be physically contiguous in sequence
236	*/	236	*/
237	unsigned long ramfs_nommu_get_unmapped_area(struct file *file,	237	unsigned long ramfs_nommu_get_unmapped_area(struct file *file,
238	unsigned long addr, unsigned long len,	238	unsigned long addr, unsigned long len,
239	unsigned long pgoff, unsigned long flags)	239	unsigned long pgoff, unsigned long flags)
240	{	240	{
241	unsigned long maxpages, lpages, nr, loop, ret;	241	unsigned long maxpages, lpages, nr, loop, ret;
242	struct inode *inode = file->f_path.dentry->d_inode;	242	struct inode *inode = file->f_path.dentry->d_inode;
243	struct page pages = NULL, ptr, *page;	243	struct page pages = NULL, ptr, *page;
244	loff_t isize;	244	loff_t isize;
245		245
246	if (!(flags & MAP_SHARED))	246	if (!(flags & MAP_SHARED))
247	return addr;	247	return addr;
248		248
249	/* the mapping mustn't extend beyond the EOF */	249	/* the mapping mustn't extend beyond the EOF */
250	lpages = (len + PAGE_SIZE - 1) >> PAGE_SHIFT;	250	lpages = (len + PAGE_SIZE - 1) >> PAGE_SHIFT;
251	isize = i_size_read(inode);	251	isize = i_size_read(inode);
252		252
253	ret = -EINVAL;	253	ret = -EINVAL;
254	maxpages = (isize + PAGE_SIZE - 1) >> PAGE_SHIFT;	254	maxpages = (isize + PAGE_SIZE - 1) >> PAGE_SHIFT;
255	if (pgoff >= maxpages)	255	if (pgoff >= maxpages)
256	goto out;	256	goto out;
257		257
258	if (maxpages - pgoff < lpages)	258	if (maxpages - pgoff < lpages)
259	goto out;	259	goto out;
260		260
261	/* gang-find the pages */	261	/* gang-find the pages */
262	ret = -ENOMEM;	262	ret = -ENOMEM;
263	pages = kzalloc(lpages * sizeof(struct page *), GFP_KERNEL);	263	pages = kzalloc(lpages * sizeof(struct page *), GFP_KERNEL);
264	if (!pages)	264	if (!pages)
265	goto out;	265	goto out_free;
266		266
267	nr = find_get_pages(inode->i_mapping, pgoff, lpages, pages);	267	nr = find_get_pages(inode->i_mapping, pgoff, lpages, pages);
268	if (nr != lpages)	268	if (nr != lpages)
269	goto out; /* leave if some pages were missing */	269	goto out_free_pages; /* leave if some pages were missing */
270		270
271	/* check the pages for physical adjacency */	271	/* check the pages for physical adjacency */
272	ptr = pages;	272	ptr = pages;
273	page = *ptr++;	273	page = *ptr++;
274	page++;	274	page++;
275	for (loop = lpages; loop > 1; loop--)	275	for (loop = lpages; loop > 1; loop--)
276	if (*ptr++ != page++)	276	if (*ptr++ != page++)
277	goto out;	277	goto out_free_pages;
278		278
279	/* okay - all conditions fulfilled */	279	/* okay - all conditions fulfilled */
280	ret = (unsigned long) page_address(pages[0]);	280	ret = (unsigned long) page_address(pages[0]);
281		281
282	out:	282	out_free_pages:
283	if (pages) {	283	ptr = pages;
284	ptr = pages;	284	for (loop = nr; loop > 0; loop--)
285	for (loop = lpages; loop > 0; loop--)	285	put_page(*ptr++);
286	put_page(*ptr++);	286	out_free:
287	kfree(pages);	287	kfree(pages);
288	}	288	out:
289
290	return ret;	289	return ret;
291	}	290	}
292		291
293	/*****************************************************************************/	292	/*****************************************************************************/
294	/*	293	/*
295	* set up a mapping for shared memory segments	294	* set up a mapping for shared memory segments
296	*/	295	*/
297	int ramfs_nommu_mmap(struct file file, struct vm_area_struct vma)	296	int ramfs_nommu_mmap(struct file file, struct vm_area_struct vma)
298	{	297	{
299	if (!(vma->vm_flags & VM_SHARED))	298	if (!(vma->vm_flags & VM_SHARED))
300	return -ENOSYS;	299	return -ENOSYS;
301		300
302	file_accessed(file);	301	file_accessed(file);
303	vma->vm_ops = &generic_file_vm_ops;	302	vma->vm_ops = &generic_file_vm_ops;
304	return 0;	303	return 0;
305	}	304	}
306		305

include/asm-frv/mmu.h

Diff comments View file @ c40f6f8

 /* mmu.h: memory management context for FR-V with or without MMU support
  *
  * Copyright (C) 2004 Red Hat, Inc. All Rights Reserved.
  * Written by David Howells (dhowells@redhat.com)
  *
  * This program is free software; you can redistribute it and/or
  * modify it under the terms of the GNU General Public License
  * as published by the Free Software Foundation; either version
  * 2 of the License, or (at your option) any later version.
  */
 #ifndef _ASM_MMU_H
 #define _ASM_MMU_H
 typedef struct {
 #ifdef CONFIG_MMU
 	struct list_head id_link;		/* link in list of context ID owners */
 	unsigned short	id;			/* MMU context ID */
 	unsigned short	id_busy;		/* true if ID is in CXNR */
 	unsigned long	itlb_cached_pge;	/* [SCR0] PGE cached for insn TLB handler */
 	unsigned long	itlb_ptd_mapping;	/* [DAMR4] PTD mapping for itlb cached PGE */
 	unsigned long	dtlb_cached_pge;	/* [SCR1] PGE cached for data TLB handler */
 	unsigned long	dtlb_ptd_mapping;	/* [DAMR5] PTD mapping for dtlb cached PGE */
 #else
-	struct vm_list_struct	*vmlist;
 	unsigned long		end_brk;
 #endif
 #ifdef CONFIG_BINFMT_ELF_FDPIC
 	unsigned long	exec_fdpic_loadmap;
 	unsigned long	interp_fdpic_loadmap;
 #endif
 } mm_context_t;
 #ifdef CONFIG_MMU
 extern int __nongpreldata cxn_pinned;
 extern int cxn_pin_by_pid(pid_t pid);
 #endif
 #endif /* _ASM_MMU_H */

include/asm-m32r/mmu.h

Diff comments View file @ c40f6f8

 #ifndef _ASM_M32R_MMU_H
 #define _ASM_M32R_MMU_H
 #if !defined(CONFIG_MMU)
 typedef struct {
-	struct vm_list_struct	*vmlist;
 	unsigned long		end_brk;
 } mm_context_t;
 #else /* CONFIG_MMU */
 /* Default "unsigned long" context */
 #ifndef CONFIG_SMP
 typedef unsigned long mm_context_t;
 #else
 typedef unsigned long mm_context_t[NR_CPUS];
 #endif
 #endif /* CONFIG_MMU */
 #endif /* _ASM_M32R_MMU_H */

include/linux/mm.h

Diff comments View file @ c40f6f8

 #ifndef _LINUX_MM_H
 #define _LINUX_MM_H
 #include <linux/errno.h>
 #ifdef __KERNEL__
 #include <linux/gfp.h>
 #include <linux/list.h>
 #include <linux/mmdebug.h>
 #include <linux/mmzone.h>
 #include <linux/rbtree.h>
 #include <linux/prio_tree.h>
 #include <linux/debug_locks.h>
 #include <linux/mm_types.h>
 struct mempolicy;
 struct anon_vma;
 struct file_ra_state;
 struct user_struct;
 struct writeback_control;
 #ifndef CONFIG_DISCONTIGMEM          /* Don't use mapnrs, do it properly */
 extern unsigned long max_mapnr;
 #endif
 extern unsigned long num_physpages;
 extern void * high_memory;
 extern int page_cluster;
 #ifdef CONFIG_SYSCTL
 extern int sysctl_legacy_va_layout;
 #else
 #define sysctl_legacy_va_layout 0
 #endif
 extern unsigned long mmap_min_addr;
 #include <asm/page.h>
 #include <asm/pgtable.h>
 #include <asm/processor.h>
 #define nth_page(page,n) pfn_to_page(page_to_pfn((page)) + (n))
 /* to align the pointer to the (next) page boundary */
 #define PAGE_ALIGN(addr) ALIGN(addr, PAGE_SIZE)
 /*
  * Linux kernel virtual memory manager primitives.
  * The idea being to have a "virtual" mm in the same way
  * we have a virtual fs - giving a cleaner interface to the
  * mm details, and allowing different kinds of memory mappings
  * (from shared memory to executable loading to arbitrary
  * mmap() functions).
  */
 extern struct kmem_cache *vm_area_cachep;
-/*
- * This struct defines the per-mm list of VMAs for uClinux. If CONFIG_MMU is
- * disabled, then there's a single shared list of VMAs maintained by the
- * system, and mm's subscribe to these individually
- */
-struct vm_list_struct {
-	struct vm_list_struct	*next;
-	struct vm_area_struct	*vma;
-};
 #ifndef CONFIG_MMU
-extern struct rb_root nommu_vma_tree;
+extern struct rb_root nommu_region_tree;
-extern struct rw_semaphore nommu_vma_sem;
+extern struct rw_semaphore nommu_region_sem;
 extern unsigned int kobjsize(const void *objp);
 #endif
 /*
  * vm_flags in vm_area_struct, see mm_types.h.
  */
 #define VM_READ		0x00000001	/* currently active flags */
 #define VM_WRITE	0x00000002
 #define VM_EXEC		0x00000004
 #define VM_SHARED	0x00000008
 /* mprotect() hardcodes VM_MAYREAD >> 4 == VM_READ, and so for r/w/x bits. */
 #define VM_MAYREAD	0x00000010	/* limits for mprotect() etc */
 #define VM_MAYWRITE	0x00000020
 #define VM_MAYEXEC	0x00000040
 #define VM_MAYSHARE	0x00000080
 #define VM_GROWSDOWN	0x00000100	/* general info on the segment */
 #define VM_GROWSUP	0x00000200
 #define VM_PFNMAP	0x00000400	/* Page-ranges managed without "struct page", just pure PFN */
 #define VM_DENYWRITE	0x00000800	/* ETXTBSY on write attempts.. */
 #define VM_EXECUTABLE	0x00001000
 #define VM_LOCKED	0x00002000
 #define VM_IO           0x00004000	/* Memory mapped I/O or similar */
 					/* Used by sys_madvise() */
 #define VM_SEQ_READ	0x00008000	/* App will access data sequentially */
 #define VM_RAND_READ	0x00010000	/* App will not benefit from clustered reads */
 #define VM_DONTCOPY	0x00020000      /* Do not copy this vma on fork */
 #define VM_DONTEXPAND	0x00040000	/* Cannot expand with mremap() */
 #define VM_RESERVED	0x00080000	/* Count as reserved_vm like IO */
 #define VM_ACCOUNT	0x00100000	/* Is a VM accounted object */
 #define VM_NORESERVE	0x00200000	/* should the VM suppress accounting */
 #define VM_HUGETLB	0x00400000	/* Huge TLB Page VM */
 #define VM_NONLINEAR	0x00800000	/* Is non-linear (remap_file_pages) */
 #define VM_MAPPED_COPY	0x01000000	/* T if mapped copy of data (nommu mmap) */
 #define VM_INSERTPAGE	0x02000000	/* The vma has had "vm_insert_page()" done on it */
 #define VM_ALWAYSDUMP	0x04000000	/* Always include in core dumps */
 #define VM_CAN_NONLINEAR 0x08000000	/* Has ->fault & does nonlinear pages */
 #define VM_MIXEDMAP	0x10000000	/* Can contain "struct page" and pure PFN pages */
 #define VM_SAO		0x20000000	/* Strong Access Ordering (powerpc) */
 #ifndef VM_STACK_DEFAULT_FLAGS		/* arch can override this */
 #define VM_STACK_DEFAULT_FLAGS VM_DATA_DEFAULT_FLAGS
 #endif
 #ifdef CONFIG_STACK_GROWSUP
 #define VM_STACK_FLAGS	(VM_GROWSUP | VM_STACK_DEFAULT_FLAGS | VM_ACCOUNT)
 #else
 #define VM_STACK_FLAGS	(VM_GROWSDOWN | VM_STACK_DEFAULT_FLAGS | VM_ACCOUNT)
 #endif
 #define VM_READHINTMASK			(VM_SEQ_READ | VM_RAND_READ)
 #define VM_ClearReadHint(v)		(v)->vm_flags &= ~VM_READHINTMASK
 #define VM_NormalReadHint(v)		(!((v)->vm_flags & VM_READHINTMASK))
 #define VM_SequentialReadHint(v)	((v)->vm_flags & VM_SEQ_READ)
 #define VM_RandomReadHint(v)		((v)->vm_flags & VM_RAND_READ)
 /*
  * special vmas that are non-mergable, non-mlock()able
  */
 #define VM_SPECIAL (VM_IO | VM_DONTEXPAND | VM_RESERVED | VM_PFNMAP)
 /*
  * mapping from the currently active vm_flags protection bits (the
  * low four bits) to a page protection mask..
  */
 extern pgprot_t protection_map[16];
 #define FAULT_FLAG_WRITE	0x01	/* Fault was a write access */
 #define FAULT_FLAG_NONLINEAR	0x02	/* Fault was via a nonlinear mapping */
 /*
  * This interface is used by x86 PAT code to identify a pfn mapping that is
  * linear over entire vma. This is to optimize PAT code that deals with
  * marking the physical region with a particular prot. This is not for generic
  * mm use. Note also that this check will not work if the pfn mapping is
  * linear for a vma starting at physical address 0. In which case PAT code
  * falls back to slow path of reserving physical range page by page.
  */
 static inline int is_linear_pfn_mapping(struct vm_area_struct *vma)
 {
 	return ((vma->vm_flags & VM_PFNMAP) && vma->vm_pgoff);
 }
 static inline int is_pfn_mapping(struct vm_area_struct *vma)
 {
 	return (vma->vm_flags & VM_PFNMAP);
 }
 /*
  * vm_fault is filled by the the pagefault handler and passed to the vma's
  * ->fault function. The vma's ->fault is responsible for returning a bitmask
  * of VM_FAULT_xxx flags that give details about how the fault was handled.
  *
  * pgoff should be used in favour of virtual_address, if possible. If pgoff
  * is used, one may set VM_CAN_NONLINEAR in the vma->vm_flags to get nonlinear
  * mapping support.
  */
 struct vm_fault {
 	unsigned int flags;		/* FAULT_FLAG_xxx flags */
 	pgoff_t pgoff;			/* Logical page offset based on vma */
 	void __user *virtual_address;	/* Faulting virtual address */
 	struct page *page;		/* ->fault handlers should return a
 					 * page here, unless VM_FAULT_NOPAGE
 					 * is set (which is also implied by
 					 * VM_FAULT_ERROR).
 					 */
 };
 /*
  * These are the virtual MM functions - opening of an area, closing and
  * unmapping it (needed to keep files on disk up-to-date etc), pointer
  * to the functions called when a no-page or a wp-page exception occurs.
  */
 struct vm_operations_struct {
 	void (*open)(struct vm_area_struct * area);
 	void (*close)(struct vm_area_struct * area);
 	int (*fault)(struct vm_area_struct *vma, struct vm_fault *vmf);
 	/* notification that a previously read-only page is about to become
 	 * writable, if an error is returned it will cause a SIGBUS */
 	int (*page_mkwrite)(struct vm_area_struct *vma, struct page *page);
 	/* called by access_process_vm when get_user_pages() fails, typically
 	 * for use by special VMAs that can switch between memory and hardware
 	 */
 	int (*access)(struct vm_area_struct *vma, unsigned long addr,
 		      void *buf, int len, int write);
 #ifdef CONFIG_NUMA
 	/*
 	 * set_policy() op must add a reference to any non-NULL @new mempolicy
 	 * to hold the policy upon return.  Caller should pass NULL @new to
 	 * remove a policy and fall back to surrounding context--i.e. do not
 	 * install a MPOL_DEFAULT policy, nor the task or system default
 	 * mempolicy.
 	 */
 	int (*set_policy)(struct vm_area_struct *vma, struct mempolicy *new);
 	/*
 	 * get_policy() op must add reference [mpol_get()] to any policy at
 	 * (vma,addr) marked as MPOL_SHARED.  The shared policy infrastructure
 	 * in mm/mempolicy.c will do this automatically.
 	 * get_policy() must NOT add a ref if the policy at (vma,addr) is not
 	 * marked as MPOL_SHARED. vma policies are protected by the mmap_sem.
 	 * If no [shared/vma] mempolicy exists at the addr, get_policy() op
 	 * must return NULL--i.e., do not "fallback" to task or system default
 	 * policy.
 	 */
 	struct mempolicy *(*get_policy)(struct vm_area_struct *vma,
 					unsigned long addr);
 	int (*migrate)(struct vm_area_struct *vma, const nodemask_t *from,
 		const nodemask_t *to, unsigned long flags);
 #endif
 };
 struct mmu_gather;
 struct inode;
 #define page_private(page)		((page)->private)
 #define set_page_private(page, v)	((page)->private = (v))
 /*
  * FIXME: take this include out, include page-flags.h in
  * files which need it (119 of them)
  */
 #include <linux/page-flags.h>
 /*
  * Methods to modify the page usage count.
  *
  * What counts for a page usage:
  * - cache mapping   (page->mapping)
  * - private data    (page->private)
  * - page mapped in a task's page tables, each mapping
  *   is counted separately
  *
  * Also, many kernel routines increase the page count before a critical
  * routine so they can be sure the page doesn't go away from under them.
  */
 /*
  * Drop a ref, return true if the refcount fell to zero (the page has no users)
  */
 static inline int put_page_testzero(struct page *page)
 {
 	VM_BUG_ON(atomic_read(&page->_count) == 0);
 	return atomic_dec_and_test(&page->_count);
 }
 /*
  * Try to grab a ref unless the page has a refcount of zero, return false if
  * that is the case.
  */
 static inline int get_page_unless_zero(struct page *page)
 {
 	VM_BUG_ON(PageTail(page));
 	return atomic_inc_not_zero(&page->_count);
 }
 /* Support for virtually mapped pages */
 struct page *vmalloc_to_page(const void *addr);
 unsigned long vmalloc_to_pfn(const void *addr);
 /*
  * Determine if an address is within the vmalloc range
  *
  * On nommu, vmalloc/vfree wrap through kmalloc/kfree directly, so there
  * is no special casing required.
  */
 static inline int is_vmalloc_addr(const void *x)
 {
 #ifdef CONFIG_MMU
 	unsigned long addr = (unsigned long)x;
 	return addr >= VMALLOC_START && addr < VMALLOC_END;
 #else
 	return 0;
 #endif
 }
 static inline struct page *compound_head(struct page *page)
 {
 	if (unlikely(PageTail(page)))
 		return page->first_page;
 	return page;
 }
 static inline int page_count(struct page *page)
 {
 	return atomic_read(&compound_head(page)->_count);
 }
 static inline void get_page(struct page *page)
 {
 	page = compound_head(page);
 	VM_BUG_ON(atomic_read(&page->_count) == 0);
 	atomic_inc(&page->_count);
 }
 static inline struct page *virt_to_head_page(const void *x)
 {
 	struct page *page = virt_to_page(x);
 	return compound_head(page);
 }
 /*
  * Setup the page count before being freed into the page allocator for
  * the first time (boot or memory hotplug)
  */
 static inline void init_page_count(struct page *page)
 {
 	atomic_set(&page->_count, 1);
 }
 void put_page(struct page *page);
 void put_pages_list(struct list_head *pages);
 void split_page(struct page *page, unsigned int order);
 /*
  * Compound pages have a destructor function.  Provide a
  * prototype for that function and accessor functions.
  * These are _only_ valid on the head of a PG_compound page.
  */
 typedef void compound_page_dtor(struct page *);
 static inline void set_compound_page_dtor(struct page *page,
 						compound_page_dtor *dtor)
 {
 	page[1].lru.next = (void *)dtor;
 }
 static inline compound_page_dtor *get_compound_page_dtor(struct page *page)
 {
 	return (compound_page_dtor *)page[1].lru.next;
 }
 static inline int compound_order(struct page *page)
 {
 	if (!PageHead(page))
 		return 0;
 	return (unsigned long)page[1].lru.prev;
 }
 static inline void set_compound_order(struct page *page, unsigned long order)
 {
 	page[1].lru.prev = (void *)order;
 }
 /*
  * Multiple processes may "see" the same page. E.g. for untouched
  * mappings of /dev/null, all processes see the same page full of
  * zeroes, and text pages of executables and shared libraries have
  * only one copy in memory, at most, normally.
  *
  * For the non-reserved pages, page_count(page) denotes a reference count.
  *   page_count() == 0 means the page is free. page->lru is then used for
  *   freelist management in the buddy allocator.
  *   page_count() > 0  means the page has been allocated.
  *
  * Pages are allocated by the slab allocator in order to provide memory
  * to kmalloc and kmem_cache_alloc. In this case, the management of the
  * page, and the fields in 'struct page' are the responsibility of mm/slab.c
  * unless a particular usage is carefully commented. (the responsibility of
  * freeing the kmalloc memory is the caller's, of course).
  *
  * A page may be used by anyone else who does a __get_free_page().
  * In this case, page_count still tracks the references, and should only
  * be used through the normal accessor functions. The top bits of page->flags
  * and page->virtual store page management information, but all other fields
  * are unused and could be used privately, carefully. The management of this
  * page is the responsibility of the one who allocated it, and those who have
  * subsequently been given references to it.
  *
  * The other pages (we may call them "pagecache pages") are completely
  * managed by the Linux memory manager: I/O, buffers, swapping etc.
  * The following discussion applies only to them.
  *
  * A pagecache page contains an opaque `private' member, which belongs to the
  * page's address_space. Usually, this is the address of a circular list of
  * the page's disk buffers. PG_private must be set to tell the VM to call
  * into the filesystem to release these pages.
  *
  * A page may belong to an inode's memory mapping. In this case, page->mapping
  * is the pointer to the inode, and page->index is the file offset of the page,
  * in units of PAGE_CACHE_SIZE.
  *
  * If pagecache pages are not associated with an inode, they are said to be
  * anonymous pages. These may become associated with the swapcache, and in that
  * case PG_swapcache is set, and page->private is an offset into the swapcache.
  *
  * In either case (swapcache or inode backed), the pagecache itself holds one
  * reference to the page. Setting PG_private should also increment the
  * refcount. The each user mapping also has a reference to the page.
  *
  * The pagecache pages are stored in a per-mapping radix tree, which is
  * rooted at mapping->page_tree, and indexed by offset.
  * Where 2.4 and early 2.6 kernels kept dirty/clean pages in per-address_space
  * lists, we instead now tag pages as dirty/writeback in the radix tree.
  *
  * All pagecache pages may be subject to I/O:
  * - inode pages may need to be read from disk,
  * - inode pages which have been modified and are MAP_SHARED may need
  *   to be written back to the inode on disk,
  * - anonymous pages (including MAP_PRIVATE file mappings) which have been
  *   modified may need to be swapped out to swap space and (later) to be read
  *   back into memory.
  */
 /*
  * The zone field is never updated after free_area_init_core()
  * sets it, so none of the operations on it need to be atomic.
  */
 /*
  * page->flags layout:
  *
  * There are three possibilities for how page->flags get
  * laid out.  The first is for the normal case, without
  * sparsemem.  The second is for sparsemem when there is
  * plenty of space for node and section.  The last is when
  * we have run out of space and have to fall back to an
  * alternate (slower) way of determining the node.
  *
  * No sparsemem or sparsemem vmemmap: |       NODE     | ZONE | ... | FLAGS |
  * classic sparse with space for node:| SECTION | NODE | ZONE | ... | FLAGS |
  * classic sparse no space for node:  | SECTION |     ZONE    | ... | FLAGS |
  */
 #if defined(CONFIG_SPARSEMEM) && !defined(CONFIG_SPARSEMEM_VMEMMAP)
 #define SECTIONS_WIDTH		SECTIONS_SHIFT
 #else
 #define SECTIONS_WIDTH		0
 #endif
 #define ZONES_WIDTH		ZONES_SHIFT
 #if SECTIONS_WIDTH+ZONES_WIDTH+NODES_SHIFT <= BITS_PER_LONG - NR_PAGEFLAGS
 #define NODES_WIDTH		NODES_SHIFT
 #else
 #ifdef CONFIG_SPARSEMEM_VMEMMAP
 #error "Vmemmap: No space for nodes field in page flags"
 #endif
 #define NODES_WIDTH		0
 #endif
 /* Page flags: | [SECTION] | [NODE] | ZONE | ... | FLAGS | */
 #define SECTIONS_PGOFF		((sizeof(unsigned long)*8) - SECTIONS_WIDTH)
 #define NODES_PGOFF		(SECTIONS_PGOFF - NODES_WIDTH)
 #define ZONES_PGOFF		(NODES_PGOFF - ZONES_WIDTH)
 /*
  * We are going to use the flags for the page to node mapping if its in
  * there.  This includes the case where there is no node, so it is implicit.
  */
 #if !(NODES_WIDTH > 0 || NODES_SHIFT == 0)
 #define NODE_NOT_IN_PAGE_FLAGS
 #endif
 #ifndef PFN_SECTION_SHIFT
 #define PFN_SECTION_SHIFT 0
 #endif
 /*
  * Define the bit shifts to access each section.  For non-existant
  * sections we define the shift as 0; that plus a 0 mask ensures
  * the compiler will optimise away reference to them.
  */
 #define SECTIONS_PGSHIFT	(SECTIONS_PGOFF * (SECTIONS_WIDTH != 0))
 #define NODES_PGSHIFT		(NODES_PGOFF * (NODES_WIDTH != 0))
 #define ZONES_PGSHIFT		(ZONES_PGOFF * (ZONES_WIDTH != 0))
 /* NODE:ZONE or SECTION:ZONE is used to ID a zone for the buddy allcator */
 #ifdef NODE_NOT_IN_PAGEFLAGS
 #define ZONEID_SHIFT		(SECTIONS_SHIFT + ZONES_SHIFT)
 #define ZONEID_PGOFF		((SECTIONS_PGOFF < ZONES_PGOFF)? \
 						SECTIONS_PGOFF : ZONES_PGOFF)
 #else
 #define ZONEID_SHIFT		(NODES_SHIFT + ZONES_SHIFT)
 #define ZONEID_PGOFF		((NODES_PGOFF < ZONES_PGOFF)? \
 						NODES_PGOFF : ZONES_PGOFF)
 #endif
 #define ZONEID_PGSHIFT		(ZONEID_PGOFF * (ZONEID_SHIFT != 0))
 #if SECTIONS_WIDTH+NODES_WIDTH+ZONES_WIDTH > BITS_PER_LONG - NR_PAGEFLAGS
 #error SECTIONS_WIDTH+NODES_WIDTH+ZONES_WIDTH > BITS_PER_LONG - NR_PAGEFLAGS
 #endif
 #define ZONES_MASK		((1UL << ZONES_WIDTH) - 1)
 #define NODES_MASK		((1UL << NODES_WIDTH) - 1)
 #define SECTIONS_MASK		((1UL << SECTIONS_WIDTH) - 1)
 #define ZONEID_MASK		((1UL << ZONEID_SHIFT) - 1)
 static inline enum zone_type page_zonenum(struct page *page)
 {
 	return (page->flags >> ZONES_PGSHIFT) & ZONES_MASK;
 }
 /*
  * The identification function is only used by the buddy allocator for
  * determining if two pages could be buddies. We are not really
  * identifying a zone since we could be using a the section number
  * id if we have not node id available in page flags.
  * We guarantee only that it will return the same value for two
  * combinable pages in a zone.
  */
 static inline int page_zone_id(struct page *page)
 {
 	return (page->flags >> ZONEID_PGSHIFT) & ZONEID_MASK;
 }
 static inline int zone_to_nid(struct zone *zone)
 {
 #ifdef CONFIG_NUMA
 	return zone->node;
 #else
 	return 0;
 #endif
 }
 #ifdef NODE_NOT_IN_PAGE_FLAGS
 extern int page_to_nid(struct page *page);
 #else
 static inline int page_to_nid(struct page *page)
 {
 	return (page->flags >> NODES_PGSHIFT) & NODES_MASK;
 }
 #endif
 static inline struct zone *page_zone(struct page *page)
 {
 	return &NODE_DATA(page_to_nid(page))->node_zones[page_zonenum(page)];
 }
 #if defined(CONFIG_SPARSEMEM) && !defined(CONFIG_SPARSEMEM_VMEMMAP)
 static inline unsigned long page_to_section(struct page *page)
 {
 	return (page->flags >> SECTIONS_PGSHIFT) & SECTIONS_MASK;
 }
 #endif
 static inline void set_page_zone(struct page *page, enum zone_type zone)
 {
 	page->flags &= ~(ZONES_MASK << ZONES_PGSHIFT);
 	page->flags |= (zone & ZONES_MASK) << ZONES_PGSHIFT;
 }
 static inline void set_page_node(struct page *page, unsigned long node)
 {
 	page->flags &= ~(NODES_MASK << NODES_PGSHIFT);
 	page->flags |= (node & NODES_MASK) << NODES_PGSHIFT;
 }
 static inline void set_page_section(struct page *page, unsigned long section)
 {
 	page->flags &= ~(SECTIONS_MASK << SECTIONS_PGSHIFT);
 	page->flags |= (section & SECTIONS_MASK) << SECTIONS_PGSHIFT;
 }
 static inline void set_page_links(struct page *page, enum zone_type zone,
 	unsigned long node, unsigned long pfn)
 {
 	set_page_zone(page, zone);
 	set_page_node(page, node);
 	set_page_section(page, pfn_to_section_nr(pfn));
 }
 /*
  * If a hint addr is less than mmap_min_addr change hint to be as
  * low as possible but still greater than mmap_min_addr
  */
 static inline unsigned long round_hint_to_min(unsigned long hint)
 {
 #ifdef CONFIG_SECURITY
 	hint &= PAGE_MASK;
 	if (((void *)hint != NULL) &&
 	    (hint < mmap_min_addr))
 		return PAGE_ALIGN(mmap_min_addr);
 #endif
 	return hint;
 }
 /*
  * Some inline functions in vmstat.h depend on page_zone()
  */
 #include <linux/vmstat.h>
 static __always_inline void *lowmem_page_address(struct page *page)
 {
 	return __va(page_to_pfn(page) << PAGE_SHIFT);
 }
 #if defined(CONFIG_HIGHMEM) && !defined(WANT_PAGE_VIRTUAL)
 #define HASHED_PAGE_VIRTUAL
 #endif
 #if defined(WANT_PAGE_VIRTUAL)
 #define page_address(page) ((page)->virtual)
 #define set_page_address(page, address)			\
 	do {						\
 		(page)->virtual = (address);		\
 	} while(0)
 #define page_address_init()  do { } while(0)
 #endif
 #if defined(HASHED_PAGE_VIRTUAL)
 void *page_address(struct page *page);
 void set_page_address(struct page *page, void *virtual);
 void page_address_init(void);
 #endif
 #if !defined(HASHED_PAGE_VIRTUAL) && !defined(WANT_PAGE_VIRTUAL)
 #define page_address(page) lowmem_page_address(page)
 #define set_page_address(page, address)  do { } while(0)
 #define page_address_init()  do { } while(0)
 #endif
 /*
  * On an anonymous page mapped into a user virtual memory area,
  * page->mapping points to its anon_vma, not to a struct address_space;
  * with the PAGE_MAPPING_ANON bit set to distinguish it.
  *
  * Please note that, confusingly, "page_mapping" refers to the inode
  * address_space which maps the page from disk; whereas "page_mapped"
  * refers to user virtual address space into which the page is mapped.
  */
 #define PAGE_MAPPING_ANON	1
 extern struct address_space swapper_space;
 static inline struct address_space *page_mapping(struct page *page)
 {
 	struct address_space *mapping = page->mapping;
 	VM_BUG_ON(PageSlab(page));
 #ifdef CONFIG_SWAP
 	if (unlikely(PageSwapCache(page)))
 		mapping = &swapper_space;
 	else
 #endif
 	if (unlikely((unsigned long)mapping & PAGE_MAPPING_ANON))
 		mapping = NULL;
 	return mapping;
 }
 static inline int PageAnon(struct page *page)
 {
 	return ((unsigned long)page->mapping & PAGE_MAPPING_ANON) != 0;
 }
 /*
  * Return the pagecache index of the passed page.  Regular pagecache pages
  * use ->index whereas swapcache pages use ->private
  */
 static inline pgoff_t page_index(struct page *page)
 {
 	if (unlikely(PageSwapCache(page)))
 		return page_private(page);
 	return page->index;
 }
 /*
  * The atomic page->_mapcount, like _count, starts from -1:
  * so that transitions both from it and to it can be tracked,
  * using atomic_inc_and_test and atomic_add_negative(-1).
  */
 static inline void reset_page_mapcount(struct page *page)
 {
 	atomic_set(&(page)->_mapcount, -1);
 }
 static inline int page_mapcount(struct page *page)
 {
 	return atomic_read(&(page)->_mapcount) + 1;
 }
 /*
  * Return true if this page is mapped into pagetables.
  */
 static inline int page_mapped(struct page *page)
 {
 	return atomic_read(&(page)->_mapcount) >= 0;
 }
 /*
  * Different kinds of faults, as returned by handle_mm_fault().
  * Used to decide whether a process gets delivered SIGBUS or
  * just gets major/minor fault counters bumped up.
  */
 #define VM_FAULT_MINOR	0 /* For backwards compat. Remove me quickly. */
 #define VM_FAULT_OOM	0x0001
 #define VM_FAULT_SIGBUS	0x0002
 #define VM_FAULT_MAJOR	0x0004
 #define VM_FAULT_WRITE	0x0008	/* Special case for get_user_pages */
 #define VM_FAULT_NOPAGE	0x0100	/* ->fault installed the pte, not return page */
 #define VM_FAULT_LOCKED	0x0200	/* ->fault locked the returned page */
 #define VM_FAULT_ERROR	(VM_FAULT_OOM | VM_FAULT_SIGBUS)
 /*
  * Can be called by the pagefault handler when it gets a VM_FAULT_OOM.
  */
 extern void pagefault_out_of_memory(void);
 #define offset_in_page(p)	((unsigned long)(p) & ~PAGE_MASK)
 extern void show_free_areas(void);
 #ifdef CONFIG_SHMEM
 extern int shmem_lock(struct file *file, int lock, struct user_struct *user);
 #else
 static inline int shmem_lock(struct file *file, int lock,
 			    struct user_struct *user)
 {
 	return 0;
 }
 #endif
 struct file *shmem_file_setup(char *name, loff_t size, unsigned long flags);
 int shmem_zero_setup(struct vm_area_struct *);
 #ifndef CONFIG_MMU
 extern unsigned long shmem_get_unmapped_area(struct file *file,
 					     unsigned long addr,
 					     unsigned long len,
 					     unsigned long pgoff,
 					     unsigned long flags);
 #endif
 extern int can_do_mlock(void);
 extern int user_shm_lock(size_t, struct user_struct *);
 extern void user_shm_unlock(size_t, struct user_struct *);
 /*
  * Parameter block passed down to zap_pte_range in exceptional cases.
  */
 struct zap_details {
 	struct vm_area_struct *nonlinear_vma;	/* Check page->index if set */
 	struct address_space *check_mapping;	/* Check page->mapping if set */
 	pgoff_t	first_index;			/* Lowest page->index to unmap */
 	pgoff_t last_index;			/* Highest page->index to unmap */
 	spinlock_t *i_mmap_lock;		/* For unmap_mapping_range: */
 	unsigned long truncate_count;		/* Compare vm_truncate_count */
 };
 struct page *vm_normal_page(struct vm_area_struct *vma, unsigned long addr,
 		pte_t pte);
 int zap_vma_ptes(struct vm_area_struct *vma, unsigned long address,
 		unsigned long size);
 unsigned long zap_page_range(struct vm_area_struct *vma, unsigned long address,
 		unsigned long size, struct zap_details *);
 unsigned long unmap_vmas(struct mmu_gather **tlb,
 		struct vm_area_struct *start_vma, unsigned long start_addr,
 		unsigned long end_addr, unsigned long *nr_accounted,
 		struct zap_details *);
 /**
  * mm_walk - callbacks for walk_page_range
  * @pgd_entry: if set, called for each non-empty PGD (top-level) entry
  * @pud_entry: if set, called for each non-empty PUD (2nd-level) entry
  * @pmd_entry: if set, called for each non-empty PMD (3rd-level) entry
  * @pte_entry: if set, called for each non-empty PTE (4th-level) entry
  * @pte_hole: if set, called for each hole at all levels
  *
  * (see walk_page_range for more details)
  */
 struct mm_walk {
 	int (*pgd_entry)(pgd_t *, unsigned long, unsigned long, struct mm_walk *);
 	int (*pud_entry)(pud_t *, unsigned long, unsigned long, struct mm_walk *);
 	int (*pmd_entry)(pmd_t *, unsigned long, unsigned long, struct mm_walk *);
 	int (*pte_entry)(pte_t *, unsigned long, unsigned long, struct mm_walk *);
 	int (*pte_hole)(unsigned long, unsigned long, struct mm_walk *);
 	struct mm_struct *mm;
 	void *private;
 };
 int walk_page_range(unsigned long addr, unsigned long end,
 		struct mm_walk *walk);
 void free_pgd_range(struct mmu_gather *tlb, unsigned long addr,
 		unsigned long end, unsigned long floor, unsigned long ceiling);
 int copy_page_range(struct mm_struct *dst, struct mm_struct *src,
 			struct vm_area_struct *vma);
 void unmap_mapping_range(struct address_space *mapping,
 		loff_t const holebegin, loff_t const holelen, int even_cows);
 int follow_phys(struct vm_area_struct *vma, unsigned long address,
 		unsigned int flags, unsigned long *prot, resource_size_t *phys);
 int generic_access_phys(struct vm_area_struct *vma, unsigned long addr,
 			void *buf, int len, int write);
 static inline void unmap_shared_mapping_range(struct address_space *mapping,
 		loff_t const holebegin, loff_t const holelen)
 {
 	unmap_mapping_range(mapping, holebegin, holelen, 0);
 }
 extern int vmtruncate(struct inode * inode, loff_t offset);
 extern int vmtruncate_range(struct inode * inode, loff_t offset, loff_t end);
 #ifdef CONFIG_MMU
 extern int handle_mm_fault(struct mm_struct *mm, struct vm_area_struct *vma,
 			unsigned long address, int write_access);
 #else
 static inline int handle_mm_fault(struct mm_struct *mm,
 			struct vm_area_struct *vma, unsigned long address,
 			int write_access)
 {
 	/* should never happen if there's no MMU */
 	BUG();
 	return VM_FAULT_SIGBUS;
 }
 #endif
 extern int make_pages_present(unsigned long addr, unsigned long end);
 extern int access_process_vm(struct task_struct *tsk, unsigned long addr, void *buf, int len, int write);
 int get_user_pages(struct task_struct *tsk, struct mm_struct *mm, unsigned long start,
 		int len, int write, int force, struct page **pages, struct vm_area_struct **vmas);
 extern int try_to_release_page(struct page * page, gfp_t gfp_mask);
 extern void do_invalidatepage(struct page *page, unsigned long offset);
 int __set_page_dirty_nobuffers(struct page *page);
 int __set_page_dirty_no_writeback(struct page *page);
 int redirty_page_for_writepage(struct writeback_control *wbc,
 				struct page *page);
 int set_page_dirty(struct page *page);
 int set_page_dirty_lock(struct page *page);
 int clear_page_dirty_for_io(struct page *page);
 extern unsigned long move_page_tables(struct vm_area_struct *vma,
 		unsigned long old_addr, struct vm_area_struct *new_vma,
 		unsigned long new_addr, unsigned long len);
 extern unsigned long do_mremap(unsigned long addr,
 			       unsigned long old_len, unsigned long new_len,
 			       unsigned long flags, unsigned long new_addr);
 extern int mprotect_fixup(struct vm_area_struct *vma,
 			  struct vm_area_struct **pprev, unsigned long start,
 			  unsigned long end, unsigned long newflags);
 /*
  * get_user_pages_fast provides equivalent functionality to get_user_pages,
  * operating on current and current->mm (force=0 and doesn't return any vmas).
  *
  * get_user_pages_fast may take mmap_sem and page tables, so no assumptions
  * can be made about locking. get_user_pages_fast is to be implemented in a
  * way that is advantageous (vs get_user_pages()) when the user memory area is
  * already faulted in and present in ptes. However if the pages have to be
  * faulted in, it may turn out to be slightly slower).
  */
 int get_user_pages_fast(unsigned long start, int nr_pages, int write,
 			struct page **pages);
 /*
  * A callback you can register to apply pressure to ageable caches.
  *
  * 'shrink' is passed a count 'nr_to_scan' and a 'gfpmask'.  It should
  * look through the least-recently-used 'nr_to_scan' entries and
  * attempt to free them up.  It should return the number of objects
  * which remain in the cache.  If it returns -1, it means it cannot do
  * any scanning at this time (eg. there is a risk of deadlock).
  *
  * The 'gfpmask' refers to the allocation we are currently trying to
  * fulfil.
  *
  * Note that 'shrink' will be passed nr_to_scan == 0 when the VM is
  * querying the cache size, so a fastpath for that case is appropriate.
  */
 struct shrinker {
 	int (*shrink)(int nr_to_scan, gfp_t gfp_mask);
 	int seeks;	/* seeks to recreate an obj */
 	/* These are for internal use */
 	struct list_head list;
 	long nr;	/* objs pending delete */
 };
 #define DEFAULT_SEEKS 2 /* A good number if you don't know better. */
 extern void register_shrinker(struct shrinker *);
 extern void unregister_shrinker(struct shrinker *);
 int vma_wants_writenotify(struct vm_area_struct *vma);
 extern pte_t *get_locked_pte(struct mm_struct *mm, unsigned long addr, spinlock_t **ptl);
 #ifdef __PAGETABLE_PUD_FOLDED
 static inline int __pud_alloc(struct mm_struct *mm, pgd_t *pgd,
 						unsigned long address)
 {
 	return 0;
 }
 #else
 int __pud_alloc(struct mm_struct *mm, pgd_t *pgd, unsigned long address);
 #endif
 #ifdef __PAGETABLE_PMD_FOLDED
 static inline int __pmd_alloc(struct mm_struct *mm, pud_t *pud,
 						unsigned long address)
 {
 	return 0;
 }
 #else
 int __pmd_alloc(struct mm_struct *mm, pud_t *pud, unsigned long address);
 #endif
 int __pte_alloc(struct mm_struct *mm, pmd_t *pmd, unsigned long address);
 int __pte_alloc_kernel(pmd_t *pmd, unsigned long address);
 /*
  * The following ifdef needed to get the 4level-fixup.h header to work.
  * Remove it when 4level-fixup.h has been removed.
  */
 #if defined(CONFIG_MMU) && !defined(__ARCH_HAS_4LEVEL_HACK)
 static inline pud_t *pud_alloc(struct mm_struct *mm, pgd_t *pgd, unsigned long address)
 {
 	return (unlikely(pgd_none(*pgd)) && __pud_alloc(mm, pgd, address))?
 		NULL: pud_offset(pgd, address);
 }
 static inline pmd_t *pmd_alloc(struct mm_struct *mm, pud_t *pud, unsigned long address)
 {
 	return (unlikely(pud_none(*pud)) && __pmd_alloc(mm, pud, address))?
 		NULL: pmd_offset(pud, address);
 }
 #endif /* CONFIG_MMU && !__ARCH_HAS_4LEVEL_HACK */
 #if USE_SPLIT_PTLOCKS
 /*
  * We tuck a spinlock to guard each pagetable page into its struct page,
  * at page->private, with BUILD_BUG_ON to make sure that this will not
  * overflow into the next struct page (as it might with DEBUG_SPINLOCK).
  * When freeing, reset page->mapping so free_pages_check won't complain.
  */
 #define __pte_lockptr(page)	&((page)->ptl)
 #define pte_lock_init(_page)	do {					\
 	spin_lock_init(__pte_lockptr(_page));				\
 } while (0)
 #define pte_lock_deinit(page)	((page)->mapping = NULL)
 #define pte_lockptr(mm, pmd)	({(void)(mm); __pte_lockptr(pmd_page(*(pmd)));})
 #else	/* !USE_SPLIT_PTLOCKS */
 /*
  * We use mm->page_table_lock to guard all pagetable pages of the mm.
  */
 #define pte_lock_init(page)	do {} while (0)
 #define pte_lock_deinit(page)	do {} while (0)
 #define pte_lockptr(mm, pmd)	({(void)(pmd); &(mm)->page_table_lock;})
 #endif /* USE_SPLIT_PTLOCKS */
 static inline void pgtable_page_ctor(struct page *page)
 {
 	pte_lock_init(page);
 	inc_zone_page_state(page, NR_PAGETABLE);
 }
 static inline void pgtable_page_dtor(struct page *page)
 {
 	pte_lock_deinit(page);
 	dec_zone_page_state(page, NR_PAGETABLE);
 }
 #define pte_offset_map_lock(mm, pmd, address, ptlp)	\
 ({							\
 	spinlock_t *__ptl = pte_lockptr(mm, pmd);	\
 	pte_t *__pte = pte_offset_map(pmd, address);	\
 	*(ptlp) = __ptl;				\
 	spin_lock(__ptl);				\
 	__pte;						\
 })
 #define pte_unmap_unlock(pte, ptl)	do {		\
 	spin_unlock(ptl);				\
 	pte_unmap(pte);					\
 } while (0)
 #define pte_alloc_map(mm, pmd, address)			\
 	((unlikely(!pmd_present(*(pmd))) && __pte_alloc(mm, pmd, address))? \
 		NULL: pte_offset_map(pmd, address))
 #define pte_alloc_map_lock(mm, pmd, address, ptlp)	\
 	((unlikely(!pmd_present(*(pmd))) && __pte_alloc(mm, pmd, address))? \
 		NULL: pte_offset_map_lock(mm, pmd, address, ptlp))
 #define pte_alloc_kernel(pmd, address)			\
 	((unlikely(!pmd_present(*(pmd))) && __pte_alloc_kernel(pmd, address))? \
 		NULL: pte_offset_kernel(pmd, address))
 extern void free_area_init(unsigned long * zones_size);
 extern void free_area_init_node(int nid, unsigned long * zones_size,
 		unsigned long zone_start_pfn, unsigned long *zholes_size);
 #ifdef CONFIG_ARCH_POPULATES_NODE_MAP
 /*
  * With CONFIG_ARCH_POPULATES_NODE_MAP set, an architecture may initialise its
  * zones, allocate the backing mem_map and account for memory holes in a more
  * architecture independent manner. This is a substitute for creating the
  * zone_sizes[] and zholes_size[] arrays and passing them to
  * free_area_init_node()
  *
  * An architecture is expected to register range of page frames backed by
  * physical memory with add_active_range() before calling
  * free_area_init_nodes() passing in the PFN each zone ends at. At a basic
  * usage, an architecture is expected to do something like
  *
  * unsigned long max_zone_pfns[MAX_NR_ZONES] = {max_dma, max_normal_pfn,
  * 							 max_highmem_pfn};
  * for_each_valid_physical_page_range()
  * 	add_active_range(node_id, start_pfn, end_pfn)
  * free_area_init_nodes(max_zone_pfns);
  *
  * If the architecture guarantees that there are no holes in the ranges
  * registered with add_active_range(), free_bootmem_active_regions()
  * will call free_bootmem_node() for each registered physical page range.
  * Similarly sparse_memory_present_with_active_regions() calls
  * memory_present() for each range when SPARSEMEM is enabled.
  *
  * See mm/page_alloc.c for more information on each function exposed by
  * CONFIG_ARCH_POPULATES_NODE_MAP
  */
 extern void free_area_init_nodes(unsigned long *max_zone_pfn);
 extern void add_active_range(unsigned int nid, unsigned long start_pfn,
 					unsigned long end_pfn);
 extern void remove_active_range(unsigned int nid, unsigned long start_pfn,
 					unsigned long end_pfn);
 extern void push_node_boundaries(unsigned int nid, unsigned long start_pfn,
 					unsigned long end_pfn);
 extern void remove_all_active_ranges(void);
 extern unsigned long absent_pages_in_range(unsigned long start_pfn,
 						unsigned long end_pfn);
 extern void get_pfn_range_for_nid(unsigned int nid,
 			unsigned long *start_pfn, unsigned long *end_pfn);
 extern unsigned long find_min_pfn_with_active_regions(void);
 extern void free_bootmem_with_active_regions(int nid,
 						unsigned long max_low_pfn);
 typedef int (*work_fn_t)(unsigned long, unsigned long, void *);
 extern void work_with_active_regions(int nid, work_fn_t work_fn, void *data);
 extern void sparse_memory_present_with_active_regions(int nid);
 #ifndef CONFIG_HAVE_ARCH_EARLY_PFN_TO_NID
 extern int early_pfn_to_nid(unsigned long pfn);
 #endif /* CONFIG_HAVE_ARCH_EARLY_PFN_TO_NID */
 #endif /* CONFIG_ARCH_POPULATES_NODE_MAP */
 extern void set_dma_reserve(unsigned long new_dma_reserve);
 extern void memmap_init_zone(unsigned long, int, unsigned long,
 				unsigned long, enum memmap_context);
 extern void setup_per_zone_pages_min(void);
 extern void mem_init(void);
+extern void __init mmap_init(void);
 extern void show_mem(void);
 extern void si_meminfo(struct sysinfo * val);
 extern void si_meminfo_node(struct sysinfo *val, int nid);
 extern int after_bootmem;
 #ifdef CONFIG_NUMA
 extern void setup_per_cpu_pageset(void);
 #else
 static inline void setup_per_cpu_pageset(void) {}
 #endif
+/* nommu.c */
+extern atomic_t mmap_pages_allocated;
 /* prio_tree.c */
 void vma_prio_tree_add(struct vm_area_struct *, struct vm_area_struct *old);
 void vma_prio_tree_insert(struct vm_area_struct *, struct prio_tree_root *);
 void vma_prio_tree_remove(struct vm_area_struct *, struct prio_tree_root *);
 struct vm_area_struct *vma_prio_tree_next(struct vm_area_struct *vma,
 	struct prio_tree_iter *iter);
 #define vma_prio_tree_foreach(vma, iter, root, begin, end)	\
 	for (prio_tree_iter_init(iter, root, begin, end), vma = NULL;	\
 		(vma = vma_prio_tree_next(vma, iter)); )
 static inline void vma_nonlinear_insert(struct vm_area_struct *vma,
 					struct list_head *list)
 {
 	vma->shared.vm_set.parent = NULL;
 	list_add_tail(&vma->shared.vm_set.list, list);
 }
 /* mmap.c */
 extern int __vm_enough_memory(struct mm_struct *mm, long pages, int cap_sys_admin);
 extern void vma_adjust(struct vm_area_struct *vma, unsigned long start,
 	unsigned long end, pgoff_t pgoff, struct vm_area_struct *insert);
 extern struct vm_area_struct *vma_merge(struct mm_struct *,
 	struct vm_area_struct *prev, unsigned long addr, unsigned long end,
 	unsigned long vm_flags, struct anon_vma *, struct file *, pgoff_t,
 	struct mempolicy *);
 extern struct anon_vma *find_mergeable_anon_vma(struct vm_area_struct *);
 extern int split_vma(struct mm_struct *,
 	struct vm_area_struct *, unsigned long addr, int new_below);
 extern int insert_vm_struct(struct mm_struct *, struct vm_area_struct *);
 extern void __vma_link_rb(struct mm_struct *, struct vm_area_struct *,
 	struct rb_node **, struct rb_node *);
 extern void unlink_file_vma(struct vm_area_struct *);
 extern struct vm_area_struct *copy_vma(struct vm_area_struct **,
 	unsigned long addr, unsigned long len, pgoff_t pgoff);
 extern void exit_mmap(struct mm_struct *);
 extern int mm_take_all_locks(struct mm_struct *mm);
 extern void mm_drop_all_locks(struct mm_struct *mm);
 #ifdef CONFIG_PROC_FS
 /* From fs/proc/base.c. callers must _not_ hold the mm's exe_file_lock */
 extern void added_exe_file_vma(struct mm_struct *mm);
 extern void removed_exe_file_vma(struct mm_struct *mm);
 #else
 static inline void added_exe_file_vma(struct mm_struct *mm)
 {}
 static inline void removed_exe_file_vma(struct mm_struct *mm)
 {}
 #endif /* CONFIG_PROC_FS */
 extern int may_expand_vm(struct mm_struct *mm, unsigned long npages);
 extern int install_special_mapping(struct mm_struct *mm,
 				   unsigned long addr, unsigned long len,
 				   unsigned long flags, struct page **pages);
 extern unsigned long get_unmapped_area(struct file *, unsigned long, unsigned long, unsigned long, unsigned long);
 extern unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
 	unsigned long len, unsigned long prot,
 	unsigned long flag, unsigned long pgoff);
 extern unsigned long mmap_region(struct file *file, unsigned long addr,
 	unsigned long len, unsigned long flags,
 	unsigned int vm_flags, unsigned long pgoff,
 	int accountable);
 static inline unsigned long do_mmap(struct file *file, unsigned long addr,
 	unsigned long len, unsigned long prot,
 	unsigned long flag, unsigned long offset)
 {
 	unsigned long ret = -EINVAL;
 	if ((offset + PAGE_ALIGN(len)) < offset)
 		goto out;
 	if (!(offset & ~PAGE_MASK))
 		ret = do_mmap_pgoff(file, addr, len, prot, flag, offset >> PAGE_SHIFT);
 out:
 	return ret;
 }
 extern int do_munmap(struct mm_struct *, unsigned long, size_t);
 extern unsigned long do_brk(unsigned long, unsigned long);
 /* filemap.c */
 extern unsigned long page_unuse(struct page *);
 extern void truncate_inode_pages(struct address_space *, loff_t);
 extern void truncate_inode_pages_range(struct address_space *,
 				       loff_t lstart, loff_t lend);
 /* generic vm_area_ops exported for stackable file systems */
 extern int filemap_fault(struct vm_area_struct *, struct vm_fault *);
 /* mm/page-writeback.c */
 int write_one_page(struct page *page, int wait);
 /* readahead.c */
 #define VM_MAX_READAHEAD	128	/* kbytes */
 #define VM_MIN_READAHEAD	16	/* kbytes (includes current page) */
 int do_page_cache_readahead(struct address_space *mapping, struct file *filp,
 			pgoff_t offset, unsigned long nr_to_read);
 int force_page_cache_readahead(struct address_space *mapping, struct file *filp,
 			pgoff_t offset, unsigned long nr_to_read);
 void page_cache_sync_readahead(struct address_space *mapping,
 			       struct file_ra_state *ra,
 			       struct file *filp,
 			       pgoff_t offset,
 			       unsigned long size);
 void page_cache_async_readahead(struct address_space *mapping,
 				struct file_ra_state *ra,
 				struct file *filp,
 				struct page *pg,
 				pgoff_t offset,
 				unsigned long size);
 unsigned long max_sane_readahead(unsigned long nr);
 /* Do stack extension */
 extern int expand_stack(struct vm_area_struct *vma, unsigned long address);
 #ifdef CONFIG_IA64
 extern int expand_upwards(struct vm_area_struct *vma, unsigned long address);
 #endif
 extern int expand_stack_downwards(struct vm_area_struct *vma,
 				  unsigned long address);
 /* Look up the first VMA which satisfies  addr < vm_end,  NULL if none. */
 extern struct vm_area_struct * find_vma(struct mm_struct * mm, unsigned long addr);
 extern struct vm_area_struct * find_vma_prev(struct mm_struct * mm, unsigned long addr,
 					     struct vm_area_struct **pprev);
 /* Look up the first VMA which intersects the interval start_addr..end_addr-1,
    NULL if none.  Assume start_addr < end_addr. */
 static inline struct vm_area_struct * find_vma_intersection(struct mm_struct * mm, unsigned long start_addr, unsigned long end_addr)
 {
 	struct vm_area_struct * vma = find_vma(mm,start_addr);
 	if (vma && end_addr <= vma->vm_start)
 		vma = NULL;
 	return vma;
 }
 static inline unsigned long vma_pages(struct vm_area_struct *vma)
 {
 	return (vma->vm_end - vma->vm_start) >> PAGE_SHIFT;
 }
 pgprot_t vm_get_page_prot(unsigned long vm_flags);
 struct vm_area_struct *find_extend_vma(struct mm_struct *, unsigned long addr);
 int remap_pfn_range(struct vm_area_struct *, unsigned long addr,
 			unsigned long pfn, unsigned long size, pgprot_t);
 int vm_insert_page(struct vm_area_struct *, unsigned long addr, struct page *);
 int vm_insert_pfn(struct vm_area_struct *vma, unsigned long addr,
 			unsigned long pfn);
 int vm_insert_mixed(struct vm_area_struct *vma, unsigned long addr,
 			unsigned long pfn);
 struct page *follow_page(struct vm_area_struct *, unsigned long address,
 			unsigned int foll_flags);
 #define FOLL_WRITE	0x01	/* check pte is writable */
 #define FOLL_TOUCH	0x02	/* mark page accessed */
 #define FOLL_GET	0x04	/* do get_page on page */
 #define FOLL_ANON	0x08	/* give ZERO_PAGE if no pgtable */
 typedef int (*pte_fn_t)(pte_t *pte, pgtable_t token, unsigned long addr,
 			void *data);
 extern int apply_to_page_range(struct mm_struct *mm, unsigned long address,
 			       unsigned long size, pte_fn_t fn, void *data);
 #ifdef CONFIG_PROC_FS
 void vm_stat_account(struct mm_struct *, unsigned long, struct file *, long);
 #else
 static inline void vm_stat_account(struct mm_struct *mm,
 			unsigned long flags, struct file *file, long pages)
 {
 }
 #endif /* CONFIG_PROC_FS */
 #ifdef CONFIG_DEBUG_PAGEALLOC
 extern int debug_pagealloc_enabled;
 extern void kernel_map_pages(struct page *page, int numpages, int enable);
 static inline void enable_debug_pagealloc(void)
 {
 	debug_pagealloc_enabled = 1;
 }
 #ifdef CONFIG_HIBERNATION
 extern bool kernel_page_present(struct page *page);
 #endif /* CONFIG_HIBERNATION */
 #else
 static inline void
 kernel_map_pages(struct page *page, int numpages, int enable) {}
 static inline void enable_debug_pagealloc(void)
 {
 }
 #ifdef CONFIG_HIBERNATION
 static inline bool kernel_page_present(struct page *page) { return true; }
 #endif /* CONFIG_HIBERNATION */
 #endif
 extern struct vm_area_struct *get_gate_vma(struct task_struct *tsk);
 #ifdef	__HAVE_ARCH_GATE_AREA
 int in_gate_area_no_task(unsigned long addr);
 int in_gate_area(struct task_struct *task, unsigned long addr);
 #else
 int in_gate_area_no_task(unsigned long addr);
 #define in_gate_area(task, addr) ({(void)task; in_gate_area_no_task(addr);})
 #endif	/* __HAVE_ARCH_GATE_AREA */
 int drop_caches_sysctl_handler(struct ctl_table *, int, struct file *,
 					void __user *, size_t *, loff_t *);
 unsigned long shrink_slab(unsigned long scanned, gfp_t gfp_mask,
 			unsigned long lru_pages);
 #ifndef CONFIG_MMU
 #define randomize_va_space 0
 #else
 extern int randomize_va_space;
 #endif
 const char * arch_vma_name(struct vm_area_struct *vma);
 void print_vma_addr(char *prefix, unsigned long rip);
 struct page *sparse_mem_map_populate(unsigned long pnum, int nid);
 pgd_t *vmemmap_pgd_populate(unsigned long addr, int node);
 pud_t *vmemmap_pud_populate(pgd_t *pgd, unsigned long addr, int node);
 pmd_t *vmemmap_pmd_populate(pud_t *pud, unsigned long addr, int node);
 pte_t *vmemmap_pte_populate(pmd_t *pmd, unsigned long addr, int node);
 void *vmemmap_alloc_block(unsigned long size, int node);
 void vmemmap_verify(pte_t *, int, unsigned long, unsigned long);
 int vmemmap_populate_basepages(struct page *start_page,
 						unsigned long pages, int node);
 int vmemmap_populate(struct page *start_page, unsigned long pages, int node);
 void vmemmap_populate_print_last(void);
 extern void *alloc_locked_buffer(size_t size);

include/linux/mm_types.h

Diff comments View file @ c40f6f8

 #ifndef _LINUX_MM_TYPES_H
 #define _LINUX_MM_TYPES_H
 #include <linux/auxvec.h>
 #include <linux/types.h>
 #include <linux/threads.h>
 #include <linux/list.h>
 #include <linux/spinlock.h>
 #include <linux/prio_tree.h>
 #include <linux/rbtree.h>
 #include <linux/rwsem.h>
 #include <linux/completion.h>
 #include <linux/cpumask.h>
 #include <asm/page.h>
 #include <asm/mmu.h>
 #ifndef AT_VECTOR_SIZE_ARCH
 #define AT_VECTOR_SIZE_ARCH 0
 #endif
 #define AT_VECTOR_SIZE (2*(AT_VECTOR_SIZE_ARCH + AT_VECTOR_SIZE_BASE + 1))
 struct address_space;
 #define USE_SPLIT_PTLOCKS	(NR_CPUS >= CONFIG_SPLIT_PTLOCK_CPUS)
 #if USE_SPLIT_PTLOCKS
 typedef atomic_long_t mm_counter_t;
 #else  /* !USE_SPLIT_PTLOCKS */
 typedef unsigned long mm_counter_t;
 #endif /* !USE_SPLIT_PTLOCKS */
 /*
  * Each physical page in the system has a struct page associated with
  * it to keep track of whatever it is we are using the page for at the
  * moment. Note that we have no way to track which tasks are using
  * a page, though if it is a pagecache page, rmap structures can tell us
  * who is mapping it.
  */
 struct page {
 	unsigned long flags;		/* Atomic flags, some possibly
 					 * updated asynchronously */
 	atomic_t _count;		/* Usage count, see below. */
 	union {
 		atomic_t _mapcount;	/* Count of ptes mapped in mms,
 					 * to show when page is mapped
 					 * & limit reverse map searches.
 					 */
 		struct {		/* SLUB */
 			u16 inuse;
 			u16 objects;
 		};
 	};
 	union {
 	    struct {
 		unsigned long private;		/* Mapping-private opaque data:
 					 	 * usually used for buffer_heads
 						 * if PagePrivate set; used for
 						 * swp_entry_t if PageSwapCache;
 						 * indicates order in the buddy
 						 * system if PG_buddy is set.
 						 */
 		struct address_space *mapping;	/* If low bit clear, points to
 						 * inode address_space, or NULL.
 						 * If page mapped as anonymous
 						 * memory, low bit is set, and
 						 * it points to anon_vma object:
 						 * see PAGE_MAPPING_ANON below.
 						 */
 	    };
 #if USE_SPLIT_PTLOCKS
 	    spinlock_t ptl;
 #endif
 	    struct kmem_cache *slab;	/* SLUB: Pointer to slab */
 	    struct page *first_page;	/* Compound tail pages */
 	};
 	union {
 		pgoff_t index;		/* Our offset within mapping. */
 		void *freelist;		/* SLUB: freelist req. slab lock */
 	};
 	struct list_head lru;		/* Pageout list, eg. active_list
 					 * protected by zone->lru_lock !
 					 */
 	/*
 	 * On machines where all RAM is mapped into kernel address space,
 	 * we can simply calculate the virtual address. On machines with
 	 * highmem some memory is mapped into kernel virtual memory
 	 * dynamically, so we need a place to store that address.
 	 * Note that this field could be 16 bits on x86 ... ;)
 	 *
 	 * Architectures with slow multiplication can define
 	 * WANT_PAGE_VIRTUAL in asm/page.h
 	 */
 #if defined(WANT_PAGE_VIRTUAL)
 	void *virtual;			/* Kernel virtual address (NULL if
 					   not kmapped, ie. highmem) */
 #endif /* WANT_PAGE_VIRTUAL */
 };
 /*
+ * A region containing a mapping of a non-memory backed file under NOMMU
+ * conditions.  These are held in a global tree and are pinned by the VMAs that
+ * map parts of them.
+ */
+struct vm_region {
+	struct rb_node	vm_rb;		/* link in global region tree */
+	unsigned long	vm_flags;	/* VMA vm_flags */
+	unsigned long	vm_start;	/* start address of region */
+	unsigned long	vm_end;		/* region initialised to here */
+	unsigned long	vm_top;		/* region allocated to here */
+	unsigned long	vm_pgoff;	/* the offset in vm_file corresponding to vm_start */
+	struct file	*vm_file;	/* the backing file or NULL */
+	atomic_t	vm_usage;	/* region usage count */
+};
+/*
  * This struct defines a memory VMM memory area. There is one of these
  * per VM-area/task.  A VM area is any part of the process virtual memory
  * space that has a special rule for the page-fault handlers (ie a shared
  * library, the executable area etc).
  */
 struct vm_area_struct {
 	struct mm_struct * vm_mm;	/* The address space we belong to. */
 	unsigned long vm_start;		/* Our start address within vm_mm. */
 	unsigned long vm_end;		/* The first byte after our end address
 					   within vm_mm. */
 	/* linked list of VM areas per task, sorted by address */
 	struct vm_area_struct *vm_next;
 	pgprot_t vm_page_prot;		/* Access permissions of this VMA. */
 	unsigned long vm_flags;		/* Flags, see mm.h. */
 	struct rb_node vm_rb;
 	/*
 	 * For areas with an address space and backing store,
 	 * linkage into the address_space->i_mmap prio tree, or
 	 * linkage to the list of like vmas hanging off its node, or
 	 * linkage of vma in the address_space->i_mmap_nonlinear list.
 	 */
 	union {
 		struct {
 			struct list_head list;
 			void *parent;	/* aligns with prio_tree_node parent */
 			struct vm_area_struct *head;
 		} vm_set;
 		struct raw_prio_tree_node prio_tree_node;
 	} shared;
 	/*
 	 * A file's MAP_PRIVATE vma can be in both i_mmap tree and anon_vma
 	 * list, after a COW of one of the file pages.	A MAP_SHARED vma
 	 * can only be in the i_mmap tree.  An anonymous MAP_PRIVATE, stack
 	 * or brk vma (with NULL file) can only be in an anon_vma list.
 	 */
 	struct list_head anon_vma_node;	/* Serialized by anon_vma->lock */
 	struct anon_vma *anon_vma;	/* Serialized by page_table_lock */
 	/* Function pointers to deal with this struct. */
 	struct vm_operations_struct * vm_ops;
 	/* Information about our backing store: */
 	unsigned long vm_pgoff;		/* Offset (within vm_file) in PAGE_SIZE
 					   units, *not* PAGE_CACHE_SIZE */
 	struct file * vm_file;		/* File we map to (can be NULL). */
 	void * vm_private_data;		/* was vm_pte (shared mem) */
 	unsigned long vm_truncate_count;/* truncate_count or restart_addr */
 #ifndef CONFIG_MMU
-	atomic_t vm_usage;		/* refcount (VMAs shared if !MMU) */
+	struct vm_region *vm_region;	/* NOMMU mapping region */
 #endif
 #ifdef CONFIG_NUMA
 	struct mempolicy *vm_policy;	/* NUMA policy for the VMA */
 #endif
 };
 struct core_thread {
 	struct task_struct *task;
 	struct core_thread *next;
 };
 struct core_state {
 	atomic_t nr_threads;
 	struct core_thread dumper;
 	struct completion startup;
 };
 struct mm_struct {
 	struct vm_area_struct * mmap;		/* list of VMAs */
 	struct rb_root mm_rb;
 	struct vm_area_struct * mmap_cache;	/* last find_vma result */
 	unsigned long (*get_unmapped_area) (struct file *filp,
 				unsigned long addr, unsigned long len,
 				unsigned long pgoff, unsigned long flags);
 	void (*unmap_area) (struct mm_struct *mm, unsigned long addr);
 	unsigned long mmap_base;		/* base of mmap area */
 	unsigned long task_size;		/* size of task vm space */
 	unsigned long cached_hole_size; 	/* if non-zero, the largest hole below free_area_cache */
 	unsigned long free_area_cache;		/* first hole of size cached_hole_size or larger */
 	pgd_t * pgd;
 	atomic_t mm_users;			/* How many users with user space? */
 	atomic_t mm_count;			/* How many references to "struct mm_struct" (users count as 1) */
 	int map_count;				/* number of VMAs */
 	struct rw_semaphore mmap_sem;
 	spinlock_t page_table_lock;		/* Protects page tables and some counters */
 	struct list_head mmlist;		/* List of maybe swapped mm's.	These are globally strung
 						 * together off init_mm.mmlist, and are protected
 						 * by mmlist_lock
 						 */
 	/* Special counters, in some configurations protected by the
 	 * page_table_lock, in other configurations by being atomic.
 	 */
 	mm_counter_t _file_rss;
 	mm_counter_t _anon_rss;
 	unsigned long hiwater_rss;	/* High-watermark of RSS usage */
 	unsigned long hiwater_vm;	/* High-water virtual memory usage */
 	unsigned long total_vm, locked_vm, shared_vm, exec_vm;
 	unsigned long stack_vm, reserved_vm, def_flags, nr_ptes;
 	unsigned long start_code, end_code, start_data, end_data;
 	unsigned long start_brk, brk, start_stack;
 	unsigned long arg_start, arg_end, env_start, env_end;
 	unsigned long saved_auxv[AT_VECTOR_SIZE]; /* for /proc/PID/auxv */
 	cpumask_t cpu_vm_mask;
 	/* Architecture-specific MM context */
 	mm_context_t context;
 	/* Swap token stuff */
 	/*
 	 * Last value of global fault stamp as seen by this process.
 	 * In other words, this value gives an indication of how long
 	 * it has been since this task got the token.
 	 * Look at mm/thrash.c
 	 */
 	unsigned int faultstamp;
 	unsigned int token_priority;
 	unsigned int last_interval;
 	unsigned long flags; /* Must use atomic bitops to access the bits */
 	struct core_state *core_state; /* coredumping support */
 	/* aio bits */
 	spinlock_t		ioctx_lock;
 	struct hlist_head	ioctx_list;
 #ifdef CONFIG_MM_OWNER
 	/*
 	 * "owner" points to a task that is regarded as the canonical
 	 * user/owner of this mm. All of the following must be true in
 	 * order for it to be changed:
 	 *
 	 * current == mm->owner
 	 * current->mm != mm
 	 * new_owner->mm == mm
 	 * new_owner->alloc_lock is held
 	 */
 	struct task_struct *owner;
 #endif
 #ifdef CONFIG_PROC_FS
 	/* store ref to file /proc/<pid>/exe symlink points to */
 	struct file *exe_file;
 	unsigned long num_exe_file_vmas;
 #endif
 #ifdef CONFIG_MMU_NOTIFIER
 	struct mmu_notifier_mm *mmu_notifier_mm;
 #endif
 };
 #endif /* _LINUX_MM_TYPES_H */

init/initramfs.c

Diff comments View file @ c40f6f8

 #include <linux/init.h>
 #include <linux/fs.h>
 #include <linux/slab.h>
 #include <linux/types.h>
 #include <linux/fcntl.h>
 #include <linux/delay.h>
 #include <linux/string.h>
 #include <linux/syscalls.h>
 #include <linux/utime.h>
 static __initdata char *message;
 static void __init error(char *x)
 {
 	if (!message)
 		message = x;
 }
 /* link hash */
 #define N_ALIGN(len) ((((len) + 1) & ~3) + 2)
 static __initdata struct hash {
 	int ino, minor, major;
 	mode_t mode;
 	struct hash *next;
 	char name[N_ALIGN(PATH_MAX)];
 } *head[32];
 static inline int hash(int major, int minor, int ino)
 {
 	unsigned long tmp = ino + minor + (major << 3);
 	tmp += tmp >> 5;
 	return tmp & 31;
 }
 static char __init *find_link(int major, int minor, int ino,
 			      mode_t mode, char *name)
 {
 	struct hash **p, *q;
 	for (p = head + hash(major, minor, ino); *p; p = &(*p)->next) {
 		if ((*p)->ino != ino)
 			continue;
 		if ((*p)->minor != minor)
 			continue;
 		if ((*p)->major != major)
 			continue;
 		if (((*p)->mode ^ mode) & S_IFMT)
 			continue;
 		return (*p)->name;
 	}
 	q = kmalloc(sizeof(struct hash), GFP_KERNEL);
 	if (!q)
 		panic("can't allocate link hash entry");
 	q->major = major;
 	q->minor = minor;
 	q->ino = ino;
 	q->mode = mode;
 	strcpy(q->name, name);
 	q->next = NULL;
 	*p = q;
 	return NULL;
 }
 static void __init free_hash(void)
 {
 	struct hash **p, *q;
 	for (p = head; p < head + 32; p++) {
 		while (*p) {
 			q = *p;
 			*p = q->next;
 			kfree(q);
 		}
 	}
 }
 static long __init do_utime(char __user *filename, time_t mtime)
 {
 	struct timespec t[2];
 	t[0].tv_sec = mtime;
 	t[0].tv_nsec = 0;
 	t[1].tv_sec = mtime;
 	t[1].tv_nsec = 0;
 	return do_utimes(AT_FDCWD, filename, t, AT_SYMLINK_NOFOLLOW);
 }
 static __initdata LIST_HEAD(dir_list);
 struct dir_entry {
 	struct list_head list;
 	char *name;
 	time_t mtime;
 };
 static void __init dir_add(const char *name, time_t mtime)
 {
 	struct dir_entry *de = kmalloc(sizeof(struct dir_entry), GFP_KERNEL);
 	if (!de)
 		panic("can't allocate dir_entry buffer");
 	INIT_LIST_HEAD(&de->list);
 	de->name = kstrdup(name, GFP_KERNEL);
 	de->mtime = mtime;
 	list_add(&de->list, &dir_list);
 }
 static void __init dir_utime(void)
 {
 	struct dir_entry *de, *tmp;
 	list_for_each_entry_safe(de, tmp, &dir_list, list) {
 		list_del(&de->list);
 		do_utime(de->name, de->mtime);
 		kfree(de->name);
 		kfree(de);
 	}
 }
 static __initdata time_t mtime;
 /* cpio header parsing */
 static __initdata unsigned long ino, major, minor, nlink;
 static __initdata mode_t mode;
 static __initdata unsigned long body_len, name_len;
 static __initdata uid_t uid;
 static __initdata gid_t gid;
 static __initdata unsigned rdev;
 static void __init parse_header(char *s)
 {
 	unsigned long parsed[12];
 	char buf[9];
 	int i;
 	buf[8] = '\0';
 	for (i = 0, s += 6; i < 12; i++, s += 8) {
 		memcpy(buf, s, 8);
 		parsed[i] = simple_strtoul(buf, NULL, 16);
 	}
 	ino = parsed[0];
 	mode = parsed[1];
 	uid = parsed[2];
 	gid = parsed[3];
 	nlink = parsed[4];
 	mtime = parsed[5];
 	body_len = parsed[6];
 	major = parsed[7];
 	minor = parsed[8];
 	rdev = new_encode_dev(MKDEV(parsed[9], parsed[10]));
 	name_len = parsed[11];
 }
 /* FSM */
 static __initdata enum state {
 	Start,
 	Collect,
 	GotHeader,
 	SkipIt,
 	GotName,
 	CopyFile,
 	GotSymlink,
 	Reset
 } state, next_state;
 static __initdata char *victim;
 static __initdata unsigned count;
 static __initdata loff_t this_header, next_header;
 static __initdata int dry_run;
 static inline void __init eat(unsigned n)
 {
 	victim += n;
 	this_header += n;
 	count -= n;
 }
 static __initdata char *vcollected;
 static __initdata char *collected;
 static __initdata int remains;
 static __initdata char *collect;
 static void __init read_into(char *buf, unsigned size, enum state next)
 {
 	if (count >= size) {
 		collected = victim;
 		eat(size);
 		state = next;
 	} else {
 		collect = collected = buf;
 		remains = size;
 		next_state = next;
 		state = Collect;
 	}
 }
 static __initdata char *header_buf, *symlink_buf, *name_buf;
 static int __init do_start(void)
 {
 	read_into(header_buf, 110, GotHeader);
 	return 0;
 }
 static int __init do_collect(void)
 {
 	unsigned n = remains;
 	if (count < n)
 		n = count;
 	memcpy(collect, victim, n);
 	eat(n);
 	collect += n;
 	if ((remains -= n) != 0)
 		return 1;
 	state = next_state;
 	return 0;
 }
 static int __init do_header(void)
 {
 	if (memcmp(collected, "070707", 6)==0) {
 		error("incorrect cpio method used: use -H newc option");
 		return 1;
 	}
 	if (memcmp(collected, "070701", 6)) {
 		error("no cpio magic");
 		return 1;
 	}
 	parse_header(collected);
 	next_header = this_header + N_ALIGN(name_len) + body_len;
 	next_header = (next_header + 3) & ~3;
 	if (dry_run) {
 		read_into(name_buf, N_ALIGN(name_len), GotName);
 		return 0;
 	}
 	state = SkipIt;
 	if (name_len <= 0 || name_len > PATH_MAX)
 		return 0;
 	if (S_ISLNK(mode)) {
 		if (body_len > PATH_MAX)
 			return 0;
 		collect = collected = symlink_buf;
 		remains = N_ALIGN(name_len) + body_len;
 		next_state = GotSymlink;
 		state = Collect;
 		return 0;
 	}
 	if (S_ISREG(mode) || !body_len)
 		read_into(name_buf, N_ALIGN(name_len), GotName);
 	return 0;
 }
 static int __init do_skip(void)
 {
 	if (this_header + count < next_header) {
 		eat(count);
 		return 1;
 	} else {
 		eat(next_header - this_header);
 		state = next_state;
 		return 0;
 	}
 }
 static int __init do_reset(void)
 {
 	while(count && *victim == '\0')
 		eat(1);
 	if (count && (this_header & 3))
 		error("broken padding");
 	return 1;
 }
 static int __init maybe_link(void)
 {
 	if (nlink >= 2) {
 		char *old = find_link(major, minor, ino, mode, collected);
 		if (old)
 			return (sys_link(old, collected) < 0) ? -1 : 1;
 	}
 	return 0;
 }
 static void __init clean_path(char *path, mode_t mode)
 {
 	struct stat st;
 	if (!sys_newlstat(path, &st) && (st.st_mode^mode) & S_IFMT) {
 		if (S_ISDIR(st.st_mode))
 			sys_rmdir(path);
 		else
 			sys_unlink(path);
 	}
 }
 static __initdata int wfd;
 static int __init do_name(void)
 {
 	state = SkipIt;
 	next_state = Reset;
 	if (strcmp(collected, "TRAILER!!!") == 0) {
 		free_hash();
 		return 0;
 	}
 	if (dry_run)
 		return 0;
 	clean_path(collected, mode);
 	if (S_ISREG(mode)) {
 		int ml = maybe_link();
 		if (ml >= 0) {
 			int openflags = O_WRONLY|O_CREAT;
 			if (ml != 1)
 				openflags |= O_TRUNC;
 			wfd = sys_open(collected, openflags, mode);
 			if (wfd >= 0) {
 				sys_fchown(wfd, uid, gid);
 				sys_fchmod(wfd, mode);
+				sys_ftruncate(wfd, body_len);
 				vcollected = kstrdup(collected, GFP_KERNEL);
 				state = CopyFile;
 			}
 		}
 	} else if (S_ISDIR(mode)) {
 		sys_mkdir(collected, mode);
 		sys_chown(collected, uid, gid);
 		sys_chmod(collected, mode);
 		dir_add(collected, mtime);
 	} else if (S_ISBLK(mode) || S_ISCHR(mode) ||
 		   S_ISFIFO(mode) || S_ISSOCK(mode)) {
 		if (maybe_link() == 0) {
 			sys_mknod(collected, mode, rdev);
 			sys_chown(collected, uid, gid);
 			sys_chmod(collected, mode);
 			do_utime(collected, mtime);
 		}
 	}
 	return 0;
 }
 static int __init do_copy(void)
 {
 	if (count >= body_len) {
 		sys_write(wfd, victim, body_len);
 		sys_close(wfd);
 		do_utime(vcollected, mtime);
 		kfree(vcollected);
 		eat(body_len);
 		state = SkipIt;
 		return 0;
 	} else {
 		sys_write(wfd, victim, count);
 		body_len -= count;
 		eat(count);
 		return 1;
 	}
 }
 static int __init do_symlink(void)
 {
 	collected[N_ALIGN(name_len) + body_len] = '\0';
 	clean_path(collected, 0);
 	sys_symlink(collected + N_ALIGN(name_len), collected);
 	sys_lchown(collected, uid, gid);
 	do_utime(collected, mtime);
 	state = SkipIt;
 	next_state = Reset;
 	return 0;
 }
 static __initdata int (*actions[])(void) = {
 	[Start]		= do_start,
 	[Collect]	= do_collect,
 	[GotHeader]	= do_header,
 	[SkipIt]	= do_skip,
 	[GotName]	= do_name,
 	[CopyFile]	= do_copy,
 	[GotSymlink]	= do_symlink,
 	[Reset]		= do_reset,
 };
 static int __init write_buffer(char *buf, unsigned len)
 {
 	count = len;
 	victim = buf;
 	while (!actions[state]())
 		;
 	return len - count;
 }
 static void __init flush_buffer(char *buf, unsigned len)
 {
 	int written;
 	if (message)
 		return;
 	while ((written = write_buffer(buf, len)) < len && !message) {
 		char c = buf[written];
 		if (c == '0') {
 			buf += written;
 			len -= written;
 			state = Start;
 		} else if (c == 0) {
 			buf += written;
 			len -= written;
 			state = Reset;
 		} else
 			error("junk in compressed archive");
 	}
 }
 /*
  * gzip declarations
  */
 #define OF(args)  args
 #ifndef memzero
 #define memzero(s, n)     memset ((s), 0, (n))
 #endif
 typedef unsigned char  uch;
 typedef unsigned short ush;
 typedef unsigned long  ulg;
 #define WSIZE 0x8000    /* window size--must be a power of two, and */
 			/*  at least 32K for zip's deflate method */
 static uch *inbuf;
 static uch *window;
 static unsigned insize;  /* valid bytes in inbuf */
 static unsigned inptr;   /* index of next byte to be processed in inbuf */
 static unsigned outcnt;  /* bytes in output buffer */
 static long bytes_out;
 #define get_byte()  (inptr < insize ? inbuf[inptr++] : -1)
 /* Diagnostic functions (stubbed out) */
 #define Assert(cond,msg)
 #define Trace(x)
 #define Tracev(x)
 #define Tracevv(x)
 #define Tracec(c,x)
 #define Tracecv(c,x)
 #define STATIC static
 #define INIT __init
 static void __init flush_window(void);
 static void __init error(char *m);
 #define NO_INFLATE_MALLOC
 #include "../lib/inflate.c"
 /* ===========================================================================
  * Write the output window window[0..outcnt-1] and update crc and bytes_out.
  * (Used for the decompressed data only.)
  */
 static void __init flush_window(void)
 {
 	ulg c = crc;         /* temporary variable */
 	unsigned n;
 	uch *in, ch;
 	flush_buffer(window, outcnt);
 	in = window;
 	for (n = 0; n < outcnt; n++) {
 		ch = *in++;
 		c = crc_32_tab[((int)c ^ ch) & 0xff] ^ (c >> 8);
 	}
 	crc = c;
 	bytes_out += (ulg)outcnt;
 	outcnt = 0;
 }
 static char * __init unpack_to_rootfs(char *buf, unsigned len, int check_only)
 {
 	int written;
 	dry_run = check_only;
 	header_buf = kmalloc(110, GFP_KERNEL);
 	symlink_buf = kmalloc(PATH_MAX + N_ALIGN(PATH_MAX) + 1, GFP_KERNEL);
 	name_buf = kmalloc(N_ALIGN(PATH_MAX), GFP_KERNEL);
 	window = kmalloc(WSIZE, GFP_KERNEL);
 	if (!window || !header_buf || !symlink_buf || !name_buf)
 		panic("can't allocate buffers");
 	state = Start;
 	this_header = 0;
 	message = NULL;
 	while (!message && len) {
 		loff_t saved_offset = this_header;
 		if (*buf == '0' && !(this_header & 3)) {
 			state = Start;
 			written = write_buffer(buf, len);
 			buf += written;
 			len -= written;
 			continue;
 		}
 		if (!*buf) {
 			buf++;
 			len--;
 			this_header++;
 			continue;
 		}
 		this_header = 0;
 		insize = len;
 		inbuf = buf;
 		inptr = 0;
 		outcnt = 0;		/* bytes in output buffer */
 		bytes_out = 0;
 		crc = (ulg)0xffffffffL; /* shift register contents */
 		makecrc();
 		gunzip();
 		if (state != Reset)
 			error("junk in gzipped archive");
 		this_header = saved_offset + inptr;
 		buf += inptr;
 		len -= inptr;
 	}
 	dir_utime();
 	kfree(window);
 	kfree(name_buf);
 	kfree(symlink_buf);
 	kfree(header_buf);
 	return message;
 }
 static int __initdata do_retain_initrd;
 static int __init retain_initrd_param(char *str)
 {
 	if (*str)
 		return 0;
 	do_retain_initrd = 1;
 	return 1;
 }
 __setup("retain_initrd", retain_initrd_param);
 extern char __initramfs_start[], __initramfs_end[];
 #include <linux/initrd.h>
 #include <linux/kexec.h>
 static void __init free_initrd(void)
 {
 #ifdef CONFIG_KEXEC
 	unsigned long crashk_start = (unsigned long)__va(crashk_res.start);
 	unsigned long crashk_end   = (unsigned long)__va(crashk_res.end);
 #endif
 	if (do_retain_initrd)
 		goto skip;
 #ifdef CONFIG_KEXEC
 	/*
 	 * If the initrd region is overlapped with crashkernel reserved region,
 	 * free only memory that is not part of crashkernel region.
 	 */
 	if (initrd_start < crashk_end && initrd_end > crashk_start) {
 		/*
 		 * Initialize initrd memory region since the kexec boot does
 		 * not do.
 		 */
 		memset((void *)initrd_start, 0, initrd_end - initrd_start);
 		if (initrd_start < crashk_start)
 			free_initrd_mem(initrd_start, crashk_start);
 		if (initrd_end > crashk_end)
 			free_initrd_mem(crashk_end, initrd_end);
 	} else
 #endif
 		free_initrd_mem(initrd_start, initrd_end);
 skip:
 	initrd_start = 0;
 	initrd_end = 0;
 }
 static int __init populate_rootfs(void)
 {
 	char *err = unpack_to_rootfs(__initramfs_start,
 			 __initramfs_end - __initramfs_start, 0);
 	if (err)
 		panic(err);
 	if (initrd_start) {
 #ifdef CONFIG_BLK_DEV_RAM
 		int fd;
 		printk(KERN_INFO "checking if image is initramfs...");
 		err = unpack_to_rootfs((char *)initrd_start,
 			initrd_end - initrd_start, 1);
 		if (!err) {
 			printk(" it is\n");
 			unpack_to_rootfs((char *)initrd_start,
 				initrd_end - initrd_start, 0);
 			free_initrd();
 			return 0;
 		}
 		printk("it isn't (%s); looks like an initrd\n", err);
 		fd = sys_open("/initrd.image", O_WRONLY|O_CREAT, 0700);
 		if (fd >= 0) {
 			sys_write(fd, (char *)initrd_start,
 					initrd_end - initrd_start);
 			sys_close(fd);
 			free_initrd();
 		}
 #else
 		printk(KERN_INFO "Unpacking initramfs...");
 		err = unpack_to_rootfs((char *)initrd_start,
 			initrd_end - initrd_start, 0);
 		if (err)
 			panic(err);
 		printk(" done\n");
 		free_initrd();
 #endif
 	}
 	return 0;
 }
 rootfs_initcall(populate_rootfs);

ipc/shm.c

Diff comments View file @ c40f6f8

 /*
  * linux/ipc/shm.c
  * Copyright (C) 1992, 1993 Krishna Balasubramanian
  *	 Many improvements/fixes by Bruno Haible.
  * Replaced `struct shm_desc' by `struct vm_area_struct', July 1994.
  * Fixed the shm swap deallocation (shm_unuse()), August 1998 Andrea Arcangeli.
  *
  * /proc/sysvipc/shm support (c) 1999 Dragos Acostachioaie <dragos@iname.com>
  * BIGMEM support, Andrea Arcangeli <andrea@suse.de>
  * SMP thread shm, Jean-Luc Boyard <jean-luc.boyard@siemens.fr>
  * HIGHMEM support, Ingo Molnar <mingo@redhat.com>
  * Make shmmax, shmall, shmmni sysctl'able, Christoph Rohland <cr@sap.com>
  * Shared /dev/zero support, Kanoj Sarcar <kanoj@sgi.com>
  * Move the mm functionality over to mm/shmem.c, Christoph Rohland <cr@sap.com>
  *
  * support for audit of ipc object properties and permission changes
  * Dustin Kirkland <dustin.kirkland@us.ibm.com>
  *
  * namespaces support
  * OpenVZ, SWsoft Inc.
  * Pavel Emelianov <xemul@openvz.org>
  */
 #include <linux/slab.h>
 #include <linux/mm.h>
 #include <linux/hugetlb.h>
 #include <linux/shm.h>
 #include <linux/init.h>
 #include <linux/file.h>
 #include <linux/mman.h>
 #include <linux/shmem_fs.h>
 #include <linux/security.h>
 #include <linux/syscalls.h>
 #include <linux/audit.h>
 #include <linux/capability.h>
 #include <linux/ptrace.h>
 #include <linux/seq_file.h>
 #include <linux/rwsem.h>
 #include <linux/nsproxy.h>
 #include <linux/mount.h>
 #include <linux/ipc_namespace.h>
 #include <asm/uaccess.h>
 #include "util.h"
 struct shm_file_data {
 	int id;
 	struct ipc_namespace *ns;
 	struct file *file;
 	const struct vm_operations_struct *vm_ops;
 };
 #define shm_file_data(file) (*((struct shm_file_data **)&(file)->private_data))
 static const struct file_operations shm_file_operations;
 static struct vm_operations_struct shm_vm_ops;
 #define shm_ids(ns)	((ns)->ids[IPC_SHM_IDS])
 #define shm_unlock(shp)			\
 	ipc_unlock(&(shp)->shm_perm)
 static int newseg(struct ipc_namespace *, struct ipc_params *);
 static void shm_open(struct vm_area_struct *vma);
 static void shm_close(struct vm_area_struct *vma);
 static void shm_destroy (struct ipc_namespace *ns, struct shmid_kernel *shp);
 #ifdef CONFIG_PROC_FS
 static int sysvipc_shm_proc_show(struct seq_file *s, void *it);
 #endif
 void shm_init_ns(struct ipc_namespace *ns)
 {
 	ns->shm_ctlmax = SHMMAX;
 	ns->shm_ctlall = SHMALL;
 	ns->shm_ctlmni = SHMMNI;
 	ns->shm_tot = 0;
 	ipc_init_ids(&shm_ids(ns));
 }
 /*
  * Called with shm_ids.rw_mutex (writer) and the shp structure locked.
  * Only shm_ids.rw_mutex remains locked on exit.
  */
 static void do_shm_rmid(struct ipc_namespace *ns, struct kern_ipc_perm *ipcp)
 {
 	struct shmid_kernel *shp;
 	shp = container_of(ipcp, struct shmid_kernel, shm_perm);
 	if (shp->shm_nattch){
 		shp->shm_perm.mode |= SHM_DEST;
 		/* Do not find it any more */
 		shp->shm_perm.key = IPC_PRIVATE;
 		shm_unlock(shp);
 	} else
 		shm_destroy(ns, shp);
 }
 #ifdef CONFIG_IPC_NS
 void shm_exit_ns(struct ipc_namespace *ns)
 {
 	free_ipcs(ns, &shm_ids(ns), do_shm_rmid);
 }
 #endif
 void __init shm_init (void)
 {
 	shm_init_ns(&init_ipc_ns);
 	ipc_init_proc_interface("sysvipc/shm",
 				"       key      shmid perms       size  cpid  lpid nattch   uid   gid  cuid  cgid      atime      dtime      ctime\n",
 				IPC_SHM_IDS, sysvipc_shm_proc_show);
 }
 /*
  * shm_lock_(check_) routines are called in the paths where the rw_mutex
  * is not necessarily held.
  */
 static inline struct shmid_kernel *shm_lock(struct ipc_namespace *ns, int id)
 {
 	struct kern_ipc_perm *ipcp = ipc_lock(&shm_ids(ns), id);
 	if (IS_ERR(ipcp))
 		return (struct shmid_kernel *)ipcp;
 	return container_of(ipcp, struct shmid_kernel, shm_perm);
 }
 static inline struct shmid_kernel *shm_lock_check(struct ipc_namespace *ns,
 						int id)
 {
 	struct kern_ipc_perm *ipcp = ipc_lock_check(&shm_ids(ns), id);
 	if (IS_ERR(ipcp))
 		return (struct shmid_kernel *)ipcp;
 	return container_of(ipcp, struct shmid_kernel, shm_perm);
 }
 static inline void shm_rmid(struct ipc_namespace *ns, struct shmid_kernel *s)
 {
 	ipc_rmid(&shm_ids(ns), &s->shm_perm);
 }
 /* This is called by fork, once for every shm attach. */
 static void shm_open(struct vm_area_struct *vma)
 {
 	struct file *file = vma->vm_file;
 	struct shm_file_data *sfd = shm_file_data(file);
 	struct shmid_kernel *shp;
 	shp = shm_lock(sfd->ns, sfd->id);
 	BUG_ON(IS_ERR(shp));
 	shp->shm_atim = get_seconds();
 	shp->shm_lprid = task_tgid_vnr(current);
 	shp->shm_nattch++;
 	shm_unlock(shp);
 }
 /*
  * shm_destroy - free the struct shmid_kernel
  *
  * @ns: namespace
  * @shp: struct to free
  *
  * It has to be called with shp and shm_ids.rw_mutex (writer) locked,
  * but returns with shp unlocked and freed.
  */
 static void shm_destroy(struct ipc_namespace *ns, struct shmid_kernel *shp)
 {
 	ns->shm_tot -= (shp->shm_segsz + PAGE_SIZE - 1) >> PAGE_SHIFT;
 	shm_rmid(ns, shp);
 	shm_unlock(shp);
 	if (!is_file_hugepages(shp->shm_file))
 		shmem_lock(shp->shm_file, 0, shp->mlock_user);
 	else
 		user_shm_unlock(shp->shm_file->f_path.dentry->d_inode->i_size,
 						shp->mlock_user);
 	fput (shp->shm_file);
 	security_shm_free(shp);
 	ipc_rcu_putref(shp);
 }
 /*
  * remove the attach descriptor vma.
  * free memory for segment if it is marked destroyed.
  * The descriptor has already been removed from the current->mm->mmap list
  * and will later be kfree()d.
  */
 static void shm_close(struct vm_area_struct *vma)
 {
 	struct file * file = vma->vm_file;
 	struct shm_file_data *sfd = shm_file_data(file);
 	struct shmid_kernel *shp;
 	struct ipc_namespace *ns = sfd->ns;
 	down_write(&shm_ids(ns).rw_mutex);
 	/* remove from the list of attaches of the shm segment */
 	shp = shm_lock(ns, sfd->id);
 	BUG_ON(IS_ERR(shp));
 	shp->shm_lprid = task_tgid_vnr(current);
 	shp->shm_dtim = get_seconds();
 	shp->shm_nattch--;
 	if(shp->shm_nattch == 0 &&
 	   shp->shm_perm.mode & SHM_DEST)
 		shm_destroy(ns, shp);
 	else
 		shm_unlock(shp);
 	up_write(&shm_ids(ns).rw_mutex);
 }
 static int shm_fault(struct vm_area_struct *vma, struct vm_fault *vmf)
 {
 	struct file *file = vma->vm_file;
 	struct shm_file_data *sfd = shm_file_data(file);
 	return sfd->vm_ops->fault(vma, vmf);
 }
 #ifdef CONFIG_NUMA
 static int shm_set_policy(struct vm_area_struct *vma, struct mempolicy *new)
 {
 	struct file *file = vma->vm_file;
 	struct shm_file_data *sfd = shm_file_data(file);
 	int err = 0;
 	if (sfd->vm_ops->set_policy)
 		err = sfd->vm_ops->set_policy(vma, new);
 	return err;
 }
 static struct mempolicy *shm_get_policy(struct vm_area_struct *vma,
 					unsigned long addr)
 {
 	struct file *file = vma->vm_file;
 	struct shm_file_data *sfd = shm_file_data(file);
 	struct mempolicy *pol = NULL;
 	if (sfd->vm_ops->get_policy)
 		pol = sfd->vm_ops->get_policy(vma, addr);
 	else if (vma->vm_policy)
 		pol = vma->vm_policy;
 	return pol;
 }
 #endif
 static int shm_mmap(struct file * file, struct vm_area_struct * vma)
 {
 	struct shm_file_data *sfd = shm_file_data(file);
 	int ret;
 	ret = sfd->file->f_op->mmap(sfd->file, vma);
 	if (ret != 0)
 		return ret;
 	sfd->vm_ops = vma->vm_ops;
 #ifdef CONFIG_MMU
 	BUG_ON(!sfd->vm_ops->fault);
 #endif
 	vma->vm_ops = &shm_vm_ops;
 	shm_open(vma);
 	return ret;
 }
 static int shm_release(struct inode *ino, struct file *file)
 {
 	struct shm_file_data *sfd = shm_file_data(file);
 	put_ipc_ns(sfd->ns);
 	shm_file_data(file) = NULL;
 	kfree(sfd);
 	return 0;
 }
 static int shm_fsync(struct file *file, struct dentry *dentry, int datasync)
 {
 	int (*fsync) (struct file *, struct dentry *, int datasync);
 	struct shm_file_data *sfd = shm_file_data(file);
 	int ret = -EINVAL;
 	fsync = sfd->file->f_op->fsync;
 	if (fsync)
 		ret = fsync(sfd->file, sfd->file->f_path.dentry, datasync);
 	return ret;
 }
 static unsigned long shm_get_unmapped_area(struct file *file,
 	unsigned long addr, unsigned long len, unsigned long pgoff,
 	unsigned long flags)
 {
 	struct shm_file_data *sfd = shm_file_data(file);
 	return get_unmapped_area(sfd->file, addr, len, pgoff, flags);
 }
 int is_file_shm_hugepages(struct file *file)
 {
 	int ret = 0;
 	if (file->f_op == &shm_file_operations) {
 		struct shm_file_data *sfd;
 		sfd = shm_file_data(file);
 		ret = is_file_hugepages(sfd->file);
 	}
 	return ret;
 }
 static const struct file_operations shm_file_operations = {
 	.mmap		= shm_mmap,
 	.fsync		= shm_fsync,
 	.release	= shm_release,
 	.get_unmapped_area	= shm_get_unmapped_area,
 };
 static struct vm_operations_struct shm_vm_ops = {
 	.open	= shm_open,	/* callback for a new vm-area open */
 	.close	= shm_close,	/* callback for when the vm-area is released */
 	.fault	= shm_fault,
 #if defined(CONFIG_NUMA)
 	.set_policy = shm_set_policy,
 	.get_policy = shm_get_policy,
 #endif
 };
 /**
  * newseg - Create a new shared memory segment
  * @ns: namespace
  * @params: ptr to the structure that contains key, size and shmflg
  *
  * Called with shm_ids.rw_mutex held as a writer.
  */
 static int newseg(struct ipc_namespace *ns, struct ipc_params *params)
 {
 	key_t key = params->key;
 	int shmflg = params->flg;
 	size_t size = params->u.size;
 	int error;
 	struct shmid_kernel *shp;
 	int numpages = (size + PAGE_SIZE -1) >> PAGE_SHIFT;
 	struct file * file;
 	char name[13];
 	int id;
 	if (size < SHMMIN || size > ns->shm_ctlmax)
 		return -EINVAL;
 	if (ns->shm_tot + numpages > ns->shm_ctlall)
 		return -ENOSPC;
 	shp = ipc_rcu_alloc(sizeof(*shp));
 	if (!shp)
 		return -ENOMEM;
 	shp->shm_perm.key = key;
 	shp->shm_perm.mode = (shmflg & S_IRWXUGO);
 	shp->mlock_user = NULL;
 	shp->shm_perm.security = NULL;
 	error = security_shm_alloc(shp);
 	if (error) {
 		ipc_rcu_putref(shp);
 		return error;
 	}
 	sprintf (name, "SYSV%08x", key);
 	if (shmflg & SHM_HUGETLB) {
 		/* hugetlb_file_setup takes care of mlock user accounting */
 		file = hugetlb_file_setup(name, size);
 		shp->mlock_user = current_user();
 	} else {
 		int acctflag = VM_ACCOUNT;
 		/*
 		 * Do not allow no accounting for OVERCOMMIT_NEVER, even
 	 	 * if it's asked for.
 		 */
 		if  ((shmflg & SHM_NORESERVE) &&
 				sysctl_overcommit_memory != OVERCOMMIT_NEVER)
 			acctflag = 0;
 		file = shmem_file_setup(name, size, acctflag);
 	}
 	error = PTR_ERR(file);
 	if (IS_ERR(file))
 		goto no_file;
 	id = ipc_addid(&shm_ids(ns), &shp->shm_perm, ns->shm_ctlmni);
 	if (id < 0) {
 		error = id;
 		goto no_id;
 	}
 	shp->shm_cprid = task_tgid_vnr(current);
 	shp->shm_lprid = 0;
 	shp->shm_atim = shp->shm_dtim = 0;
 	shp->shm_ctim = get_seconds();
 	shp->shm_segsz = size;
 	shp->shm_nattch = 0;
 	shp->shm_file = file;
 	/*
 	 * shmid gets reported as "inode#" in /proc/pid/maps.
 	 * proc-ps tools use this. Changing this will break them.
 	 */
 	file->f_dentry->d_inode->i_ino = shp->shm_perm.id;
 	ns->shm_tot += numpages;
 	error = shp->shm_perm.id;
 	shm_unlock(shp);
 	return error;
 no_id:
 	fput(file);
 no_file:
 	security_shm_free(shp);
 	ipc_rcu_putref(shp);
 	return error;
 }
 /*
  * Called with shm_ids.rw_mutex and ipcp locked.
  */
 static inline int shm_security(struct kern_ipc_perm *ipcp, int shmflg)
 {
 	struct shmid_kernel *shp;
 	shp = container_of(ipcp, struct shmid_kernel, shm_perm);
 	return security_shm_associate(shp, shmflg);
 }
 /*
  * Called with shm_ids.rw_mutex and ipcp locked.
  */
 static inline int shm_more_checks(struct kern_ipc_perm *ipcp,
 				struct ipc_params *params)
 {
 	struct shmid_kernel *shp;
 	shp = container_of(ipcp, struct shmid_kernel, shm_perm);
 	if (shp->shm_segsz < params->u.size)
 		return -EINVAL;
 	return 0;
 }
 asmlinkage long sys_shmget (key_t key, size_t size, int shmflg)
 {
 	struct ipc_namespace *ns;
 	struct ipc_ops shm_ops;
 	struct ipc_params shm_params;
 	ns = current->nsproxy->ipc_ns;
 	shm_ops.getnew = newseg;
 	shm_ops.associate = shm_security;
 	shm_ops.more_checks = shm_more_checks;
 	shm_params.key = key;
 	shm_params.flg = shmflg;
 	shm_params.u.size = size;
 	return ipcget(ns, &shm_ids(ns), &shm_ops, &shm_params);
 }
 static inline unsigned long copy_shmid_to_user(void __user *buf, struct shmid64_ds *in, int version)
 {
 	switch(version) {
 	case IPC_64:
 		return copy_to_user(buf, in, sizeof(*in));
 	case IPC_OLD:
 	    {
 		struct shmid_ds out;
 		ipc64_perm_to_ipc_perm(&in->shm_perm, &out.shm_perm);
 		out.shm_segsz	= in->shm_segsz;
 		out.shm_atime	= in->shm_atime;
 		out.shm_dtime	= in->shm_dtime;
 		out.shm_ctime	= in->shm_ctime;
 		out.shm_cpid	= in->shm_cpid;
 		out.shm_lpid	= in->shm_lpid;
 		out.shm_nattch	= in->shm_nattch;
 		return copy_to_user(buf, &out, sizeof(out));
 	    }
 	default:
 		return -EINVAL;
 	}
 }
 static inline unsigned long
 copy_shmid_from_user(struct shmid64_ds *out, void __user *buf, int version)
 {
 	switch(version) {
 	case IPC_64:
 		if (copy_from_user(out, buf, sizeof(*out)))
 			return -EFAULT;
 		return 0;
 	case IPC_OLD:
 	    {
 		struct shmid_ds tbuf_old;
 		if (copy_from_user(&tbuf_old, buf, sizeof(tbuf_old)))
 			return -EFAULT;
 		out->shm_perm.uid	= tbuf_old.shm_perm.uid;
 		out->shm_perm.gid	= tbuf_old.shm_perm.gid;
 		out->shm_perm.mode	= tbuf_old.shm_perm.mode;
 		return 0;
 	    }
 	default:
 		return -EINVAL;
 	}
 }
 static inline unsigned long copy_shminfo_to_user(void __user *buf, struct shminfo64 *in, int version)
 {
 	switch(version) {
 	case IPC_64:
 		return copy_to_user(buf, in, sizeof(*in));
 	case IPC_OLD:
 	    {
 		struct shminfo out;
 		if(in->shmmax > INT_MAX)
 			out.shmmax = INT_MAX;
 		else
 			out.shmmax = (int)in->shmmax;
 		out.shmmin	= in->shmmin;
 		out.shmmni	= in->shmmni;
 		out.shmseg	= in->shmseg;
 		out.shmall	= in->shmall;
 		return copy_to_user(buf, &out, sizeof(out));
 	    }
 	default:
 		return -EINVAL;
 	}
 }
 /*
  * Called with shm_ids.rw_mutex held as a reader
  */
 static void shm_get_stat(struct ipc_namespace *ns, unsigned long *rss,
 		unsigned long *swp)
 {
 	int next_id;
 	int total, in_use;
 	*rss = 0;
 	*swp = 0;
 	in_use = shm_ids(ns).in_use;
 	for (total = 0, next_id = 0; total < in_use; next_id++) {
 		struct shmid_kernel *shp;
 		struct inode *inode;
 		shp = idr_find(&shm_ids(ns).ipcs_idr, next_id);
 		if (shp == NULL)
 			continue;
 		inode = shp->shm_file->f_path.dentry->d_inode;
 		if (is_file_hugepages(shp->shm_file)) {
 			struct address_space *mapping = inode->i_mapping;
 			struct hstate *h = hstate_file(shp->shm_file);
 			*rss += pages_per_huge_page(h) * mapping->nrpages;
 		} else {
 			struct shmem_inode_info *info = SHMEM_I(inode);
 			spin_lock(&info->lock);
 			*rss += inode->i_mapping->nrpages;
 			*swp += info->swapped;
 			spin_unlock(&info->lock);
 		}
 		total++;
 	}
 }
 /*
  * This function handles some shmctl commands which require the rw_mutex
  * to be held in write mode.
  * NOTE: no locks must be held, the rw_mutex is taken inside this function.
  */
 static int shmctl_down(struct ipc_namespace *ns, int shmid, int cmd,
 		       struct shmid_ds __user *buf, int version)
 {
 	struct kern_ipc_perm *ipcp;
 	struct shmid64_ds shmid64;
 	struct shmid_kernel *shp;
 	int err;
 	if (cmd == IPC_SET) {
 		if (copy_shmid_from_user(&shmid64, buf, version))
 			return -EFAULT;
 	}
 	ipcp = ipcctl_pre_down(&shm_ids(ns), shmid, cmd, &shmid64.shm_perm, 0);
 	if (IS_ERR(ipcp))
 		return PTR_ERR(ipcp);
 	shp = container_of(ipcp, struct shmid_kernel, shm_perm);
 	err = security_shm_shmctl(shp, cmd);
 	if (err)
 		goto out_unlock;
 	switch (cmd) {
 	case IPC_RMID:
 		do_shm_rmid(ns, ipcp);
 		goto out_up;
 	case IPC_SET:
 		ipc_update_perm(&shmid64.shm_perm, ipcp);
 		shp->shm_ctim = get_seconds();
 		break;
 	default:
 		err = -EINVAL;
 	}
 out_unlock:
 	shm_unlock(shp);
 out_up:
 	up_write(&shm_ids(ns).rw_mutex);
 	return err;
 }
 asmlinkage long sys_shmctl(int shmid, int cmd, struct shmid_ds __user *buf)
 {
 	struct shmid_kernel *shp;
 	int err, version;
 	struct ipc_namespace *ns;
 	if (cmd < 0 || shmid < 0) {
 		err = -EINVAL;
 		goto out;
 	}
 	version = ipc_parse_version(&cmd);
 	ns = current->nsproxy->ipc_ns;
 	switch (cmd) { /* replace with proc interface ? */
 	case IPC_INFO:
 	{
 		struct shminfo64 shminfo;
 		err = security_shm_shmctl(NULL, cmd);
 		if (err)
 			return err;
 		memset(&shminfo, 0, sizeof(shminfo));
 		shminfo.shmmni = shminfo.shmseg = ns->shm_ctlmni;
 		shminfo.shmmax = ns->shm_ctlmax;
 		shminfo.shmall = ns->shm_ctlall;
 		shminfo.shmmin = SHMMIN;
 		if(copy_shminfo_to_user (buf, &shminfo, version))
 			return -EFAULT;
 		down_read(&shm_ids(ns).rw_mutex);
 		err = ipc_get_maxid(&shm_ids(ns));
 		up_read(&shm_ids(ns).rw_mutex);
 		if(err<0)
 			err = 0;
 		goto out;
 	}
 	case SHM_INFO:
 	{
 		struct shm_info shm_info;
 		err = security_shm_shmctl(NULL, cmd);
 		if (err)
 			return err;
 		memset(&shm_info, 0, sizeof(shm_info));
 		down_read(&shm_ids(ns).rw_mutex);
 		shm_info.used_ids = shm_ids(ns).in_use;
 		shm_get_stat (ns, &shm_info.shm_rss, &shm_info.shm_swp);
 		shm_info.shm_tot = ns->shm_tot;
 		shm_info.swap_attempts = 0;
 		shm_info.swap_successes = 0;
 		err = ipc_get_maxid(&shm_ids(ns));
 		up_read(&shm_ids(ns).rw_mutex);
 		if (copy_to_user(buf, &shm_info, sizeof(shm_info))) {
 			err = -EFAULT;
 			goto out;
 		}
 		err = err < 0 ? 0 : err;
 		goto out;
 	}
 	case SHM_STAT:
 	case IPC_STAT:
 	{
 		struct shmid64_ds tbuf;
 		int result;
 		if (cmd == SHM_STAT) {
 			shp = shm_lock(ns, shmid);
 			if (IS_ERR(shp)) {
 				err = PTR_ERR(shp);
 				goto out;
 			}
 			result = shp->shm_perm.id;
 		} else {
 			shp = shm_lock_check(ns, shmid);
 			if (IS_ERR(shp)) {
 				err = PTR_ERR(shp);
 				goto out;
 			}
 			result = 0;
 		}
 		err = -EACCES;
 		if (ipcperms (&shp->shm_perm, S_IRUGO))
 			goto out_unlock;
 		err = security_shm_shmctl(shp, cmd);
 		if (err)
 			goto out_unlock;
 		memset(&tbuf, 0, sizeof(tbuf));
 		kernel_to_ipc64_perm(&shp->shm_perm, &tbuf.shm_perm);
 		tbuf.shm_segsz	= shp->shm_segsz;
 		tbuf.shm_atime	= shp->shm_atim;
 		tbuf.shm_dtime	= shp->shm_dtim;
 		tbuf.shm_ctime	= shp->shm_ctim;
 		tbuf.shm_cpid	= shp->shm_cprid;
 		tbuf.shm_lpid	= shp->shm_lprid;
 		tbuf.shm_nattch	= shp->shm_nattch;
 		shm_unlock(shp);
 		if(copy_shmid_to_user (buf, &tbuf, version))
 			err = -EFAULT;
 		else
 			err = result;
 		goto out;
 	}
 	case SHM_LOCK:
 	case SHM_UNLOCK:
 	{
 		struct file *uninitialized_var(shm_file);
 		lru_add_drain_all();  /* drain pagevecs to lru lists */
 		shp = shm_lock_check(ns, shmid);
 		if (IS_ERR(shp)) {
 			err = PTR_ERR(shp);
 			goto out;
 		}
 		audit_ipc_obj(&(shp->shm_perm));
 		if (!capable(CAP_IPC_LOCK)) {
 			uid_t euid = current_euid();
 			err = -EPERM;
 			if (euid != shp->shm_perm.uid &&
 			    euid != shp->shm_perm.cuid)
 				goto out_unlock;
 			if (cmd == SHM_LOCK &&
 			    !current->signal->rlim[RLIMIT_MEMLOCK].rlim_cur)
 				goto out_unlock;
 		}
 		err = security_shm_shmctl(shp, cmd);
 		if (err)
 			goto out_unlock;
 		if(cmd==SHM_LOCK) {
 			struct user_struct *user = current_user();
 			if (!is_file_hugepages(shp->shm_file)) {
 				err = shmem_lock(shp->shm_file, 1, user);
 				if (!err && !(shp->shm_perm.mode & SHM_LOCKED)){
 					shp->shm_perm.mode |= SHM_LOCKED;
 					shp->mlock_user = user;
 				}
 			}
 		} else if (!is_file_hugepages(shp->shm_file)) {
 			shmem_lock(shp->shm_file, 0, shp->mlock_user);
 			shp->shm_perm.mode &= ~SHM_LOCKED;
 			shp->mlock_user = NULL;
 		}
 		shm_unlock(shp);
 		goto out;
 	}
 	case IPC_RMID:
 	case IPC_SET:
 		err = shmctl_down(ns, shmid, cmd, buf, version);
 		return err;
 	default:
 		return -EINVAL;
 	}
 out_unlock:
 	shm_unlock(shp);
 out:
 	return err;
 }
 /*
  * Fix shmaddr, allocate descriptor, map shm, add attach descriptor to lists.
  *
  * NOTE! Despite the name, this is NOT a direct system call entrypoint. The
  * "raddr" thing points to kernel space, and there has to be a wrapper around
  * this.
  */
 long do_shmat(int shmid, char __user *shmaddr, int shmflg, ulong *raddr)
 {
 	struct shmid_kernel *shp;
 	unsigned long addr;
 	unsigned long size;
 	struct file * file;
 	int    err;
 	unsigned long flags;
 	unsigned long prot;
 	int acc_mode;
 	unsigned long user_addr;
 	struct ipc_namespace *ns;
 	struct shm_file_data *sfd;
 	struct path path;
 	fmode_t f_mode;
 	err = -EINVAL;
 	if (shmid < 0)
 		goto out;
 	else if ((addr = (ulong)shmaddr)) {
 		if (addr & (SHMLBA-1)) {
 			if (shmflg & SHM_RND)
 				addr &= ~(SHMLBA-1);	   /* round down */
 			else
 #ifndef __ARCH_FORCE_SHMLBA
 				if (addr & ~PAGE_MASK)
 #endif
 					goto out;
 		}
 		flags = MAP_SHARED | MAP_FIXED;
 	} else {
 		if ((shmflg & SHM_REMAP))
 			goto out;
 		flags = MAP_SHARED;
 	}
 	if (shmflg & SHM_RDONLY) {
 		prot = PROT_READ;
 		acc_mode = S_IRUGO;
 		f_mode = FMODE_READ;
 	} else {
 		prot = PROT_READ | PROT_WRITE;
 		acc_mode = S_IRUGO | S_IWUGO;
 		f_mode = FMODE_READ | FMODE_WRITE;
 	}
 	if (shmflg & SHM_EXEC) {
 		prot |= PROT_EXEC;
 		acc_mode |= S_IXUGO;
 	}
 	/*
 	 * We cannot rely on the fs check since SYSV IPC does have an
 	 * additional creator id...
 	 */
 	ns = current->nsproxy->ipc_ns;
 	shp = shm_lock_check(ns, shmid);
 	if (IS_ERR(shp)) {
 		err = PTR_ERR(shp);
 		goto out;
 	}
 	err = -EACCES;
 	if (ipcperms(&shp->shm_perm, acc_mode))
 		goto out_unlock;
 	err = security_shm_shmat(shp, shmaddr, shmflg);
 	if (err)
 		goto out_unlock;
 	path.dentry = dget(shp->shm_file->f_path.dentry);
 	path.mnt    = shp->shm_file->f_path.mnt;
 	shp->shm_nattch++;
 	size = i_size_read(path.dentry->d_inode);
 	shm_unlock(shp);
 	err = -ENOMEM;
 	sfd = kzalloc(sizeof(*sfd), GFP_KERNEL);
 	if (!sfd)
 		goto out_put_dentry;
 	file = alloc_file(path.mnt, path.dentry, f_mode, &shm_file_operations);
 	if (!file)
 		goto out_free;
 	file->private_data = sfd;
 	file->f_mapping = shp->shm_file->f_mapping;
 	sfd->id = shp->shm_perm.id;
 	sfd->ns = get_ipc_ns(ns);
 	sfd->file = shp->shm_file;
 	sfd->vm_ops = NULL;
 	down_write(&current->mm->mmap_sem);
 	if (addr && !(shmflg & SHM_REMAP)) {
 		err = -EINVAL;
 		if (find_vma_intersection(current->mm, addr, addr + size))
 			goto invalid;
 		/*
 		 * If shm segment goes below stack, make sure there is some
 		 * space left for the stack to grow (at least 4 pages).
 		 */
 		if (addr < current->mm->start_stack &&
 		    addr > current->mm->start_stack - size - PAGE_SIZE * 5)
 			goto invalid;
 	}
 	user_addr = do_mmap (file, addr, size, prot, flags, 0);
 	*raddr = user_addr;
 	err = 0;
 	if (IS_ERR_VALUE(user_addr))
 		err = (long)user_addr;
 invalid:
 	up_write(&current->mm->mmap_sem);
 	fput(file);
 out_nattch:
 	down_write(&shm_ids(ns).rw_mutex);
 	shp = shm_lock(ns, shmid);
 	BUG_ON(IS_ERR(shp));
 	shp->shm_nattch--;
 	if(shp->shm_nattch == 0 &&
 	   shp->shm_perm.mode & SHM_DEST)
 		shm_destroy(ns, shp);
 	else
 		shm_unlock(shp);
 	up_write(&shm_ids(ns).rw_mutex);
 out:
 	return err;
 out_unlock:
 	shm_unlock(shp);
 	goto out;
 out_free:
 	kfree(sfd);
 out_put_dentry:
 	dput(path.dentry);
 	goto out_nattch;
 }
 asmlinkage long sys_shmat(int shmid, char __user *shmaddr, int shmflg)
 {
 	unsigned long ret;
 	long err;
 	err = do_shmat(shmid, shmaddr, shmflg, &ret);
 	if (err)
 		return err;
 	force_successful_syscall_return();
 	return (long)ret;
 }
 /*
  * detach and kill segment if marked destroyed.
  * The work is done in shm_close.
  */
 asmlinkage long sys_shmdt(char __user *shmaddr)
 {
 	struct mm_struct *mm = current->mm;
 	struct vm_area_struct *vma, *next;
 	unsigned long addr = (unsigned long)shmaddr;
 	loff_t size = 0;
 	int retval = -EINVAL;
 	if (addr & ~PAGE_MASK)
 		return retval;
 	down_write(&mm->mmap_sem);
 	/*
 	 * This function tries to be smart and unmap shm segments that
 	 * were modified by partial mlock or munmap calls:
 	 * - It first determines the size of the shm segment that should be
 	 *   unmapped: It searches for a vma that is backed by shm and that
 	 *   started at address shmaddr. It records it's size and then unmaps
 	 *   it.
 	 * - Then it unmaps all shm vmas that started at shmaddr and that
 	 *   are within the initially determined size.
 	 * Errors from do_munmap are ignored: the function only fails if
 	 * it's called with invalid parameters or if it's called to unmap
 	 * a part of a vma. Both calls in this function are for full vmas,
 	 * the parameters are directly copied from the vma itself and always
 	 * valid - therefore do_munmap cannot fail. (famous last words?)
 	 */
 	/*
 	 * If it had been mremap()'d, the starting address would not
 	 * match the usual checks anyway. So assume all vma's are
 	 * above the starting address given.
 	 */
 	vma = find_vma(mm, addr);
+#ifdef CONFIG_MMU
 	while (vma) {
 		next = vma->vm_next;
 		/*
 		 * Check if the starting address would match, i.e. it's
 		 * a fragment created by mprotect() and/or munmap(), or it
 		 * otherwise it starts at this address with no hassles.
 		 */
 		if ((vma->vm_ops == &shm_vm_ops) &&
 			(vma->vm_start - addr)/PAGE_SIZE == vma->vm_pgoff) {
 			size = vma->vm_file->f_path.dentry->d_inode->i_size;
 			do_munmap(mm, vma->vm_start, vma->vm_end - vma->vm_start);
 			/*
 			 * We discovered the size of the shm segment, so
 			 * break out of here and fall through to the next
 			 * loop that uses the size information to stop
 			 * searching for matching vma's.
 			 */
 			retval = 0;
 			vma = next;
 			break;
 		}
 		vma = next;
 	}
 	/*
 	 * We need look no further than the maximum address a fragment
 	 * could possibly have landed at. Also cast things to loff_t to
 	 * prevent overflows and make comparisions vs. equal-width types.
 	 */
 	size = PAGE_ALIGN(size);
 	while (vma && (loff_t)(vma->vm_end - addr) <= size) {
 		next = vma->vm_next;
 		/* finding a matching vma now does not alter retval */
 		if ((vma->vm_ops == &shm_vm_ops) &&
 			(vma->vm_start - addr)/PAGE_SIZE == vma->vm_pgoff)
 			do_munmap(mm, vma->vm_start, vma->vm_end - vma->vm_start);
 		vma = next;
 	}
+#else /* CONFIG_MMU */
+	/* under NOMMU conditions, the exact address to be destroyed must be
+	 * given */
+	retval = -EINVAL;
+	if (vma->vm_start == addr && vma->vm_ops == &shm_vm_ops) {
+		do_munmap(mm, vma->vm_start, vma->vm_end - vma->vm_start);
+		retval = 0;
+	}
+#endif
 	up_write(&mm->mmap_sem);
 	return retval;
 }
 #ifdef CONFIG_PROC_FS
 static int sysvipc_shm_proc_show(struct seq_file *s, void *it)
 {
 	struct shmid_kernel *shp = it;
 #if BITS_PER_LONG <= 32
 #define SIZE_SPEC "%10lu"
 #else
 #define SIZE_SPEC "%21lu"
 #endif
 	return seq_printf(s,
 			  "%10d %10d  %4o " SIZE_SPEC " %5u %5u  "
 			  "%5lu %5u %5u %5u %5u %10lu %10lu %10lu\n",
 			  shp->shm_perm.key,
 			  shp->shm_perm.id,
 			  shp->shm_perm.mode,
 			  shp->shm_segsz,
 			  shp->shm_cprid,
 			  shp->shm_lprid,
 			  shp->shm_nattch,
 			  shp->shm_perm.uid,
 			  shp->shm_perm.gid,
 			  shp->shm_perm.cuid,
 			  shp->shm_perm.cgid,
 			  shp->shm_atim,
 			  shp->shm_dtim,
 			  shp->shm_ctim);
 }
 #endif

kernel/fork.c

Diff comments View file @ c40f6f8

 /*
  *  linux/kernel/fork.c
  *
  *  Copyright (C) 1991, 1992  Linus Torvalds
  */
 /*
  *  'fork.c' contains the help-routines for the 'fork' system call
  * (see also entry.S and others).
  * Fork is rather simple, once you get the hang of it, but the memory
  * management can be a bitch. See 'mm/memory.c': 'copy_page_range()'
  */
 #include <linux/slab.h>
 #include <linux/init.h>
 #include <linux/unistd.h>
 #include <linux/module.h>
 #include <linux/vmalloc.h>
 #include <linux/completion.h>
 #include <linux/mnt_namespace.h>
 #include <linux/personality.h>
 #include <linux/mempolicy.h>
 #include <linux/sem.h>
 #include <linux/file.h>
 #include <linux/fdtable.h>
 #include <linux/iocontext.h>
 #include <linux/key.h>
 #include <linux/binfmts.h>
 #include <linux/mman.h>
 #include <linux/mmu_notifier.h>
 #include <linux/fs.h>
 #include <linux/nsproxy.h>
 #include <linux/capability.h>
 #include <linux/cpu.h>
 #include <linux/cgroup.h>
 #include <linux/security.h>
 #include <linux/hugetlb.h>
 #include <linux/swap.h>
 #include <linux/syscalls.h>
 #include <linux/jiffies.h>
 #include <linux/tracehook.h>
 #include <linux/futex.h>
 #include <linux/compat.h>
 #include <linux/task_io_accounting_ops.h>
 #include <linux/rcupdate.h>
 #include <linux/ptrace.h>
 #include <linux/mount.h>
 #include <linux/audit.h>
 #include <linux/memcontrol.h>
 #include <linux/ftrace.h>
 #include <linux/profile.h>
 #include <linux/rmap.h>
 #include <linux/acct.h>
 #include <linux/tsacct_kern.h>
 #include <linux/cn_proc.h>
 #include <linux/freezer.h>
 #include <linux/delayacct.h>
 #include <linux/taskstats_kern.h>
 #include <linux/random.h>
 #include <linux/tty.h>
 #include <linux/proc_fs.h>
 #include <linux/blkdev.h>
 #include <trace/sched.h>
 #include <asm/pgtable.h>
 #include <asm/pgalloc.h>
 #include <asm/uaccess.h>
 #include <asm/mmu_context.h>
 #include <asm/cacheflush.h>
 #include <asm/tlbflush.h>
 /*
  * Protected counters by write_lock_irq(&tasklist_lock)
  */
 unsigned long total_forks;	/* Handle normal Linux uptimes. */
 int nr_threads; 		/* The idle threads do not count.. */
 int max_threads;		/* tunable limit on nr_threads */
 DEFINE_PER_CPU(unsigned long, process_counts) = 0;
 __cacheline_aligned DEFINE_RWLOCK(tasklist_lock);  /* outer */
 DEFINE_TRACE(sched_process_fork);
 int nr_processes(void)
 {
 	int cpu;
 	int total = 0;
 	for_each_online_cpu(cpu)
 		total += per_cpu(process_counts, cpu);
 	return total;
 }
 #ifndef __HAVE_ARCH_TASK_STRUCT_ALLOCATOR
 # define alloc_task_struct()	kmem_cache_alloc(task_struct_cachep, GFP_KERNEL)
 # define free_task_struct(tsk)	kmem_cache_free(task_struct_cachep, (tsk))
 static struct kmem_cache *task_struct_cachep;
 #endif
 #ifndef __HAVE_ARCH_THREAD_INFO_ALLOCATOR
 static inline struct thread_info *alloc_thread_info(struct task_struct *tsk)
 {
 #ifdef CONFIG_DEBUG_STACK_USAGE
 	gfp_t mask = GFP_KERNEL | __GFP_ZERO;
 #else
 	gfp_t mask = GFP_KERNEL;
 #endif
 	return (struct thread_info *)__get_free_pages(mask, THREAD_SIZE_ORDER);
 }
 static inline void free_thread_info(struct thread_info *ti)
 {
 	free_pages((unsigned long)ti, THREAD_SIZE_ORDER);
 }
 #endif
 /* SLAB cache for signal_struct structures (tsk->signal) */
 static struct kmem_cache *signal_cachep;
 /* SLAB cache for sighand_struct structures (tsk->sighand) */
 struct kmem_cache *sighand_cachep;
 /* SLAB cache for files_struct structures (tsk->files) */
 struct kmem_cache *files_cachep;
 /* SLAB cache for fs_struct structures (tsk->fs) */
 struct kmem_cache *fs_cachep;
 /* SLAB cache for vm_area_struct structures */
 struct kmem_cache *vm_area_cachep;
 /* SLAB cache for mm_struct structures (tsk->mm) */
 static struct kmem_cache *mm_cachep;
 void free_task(struct task_struct *tsk)
 {
 	prop_local_destroy_single(&tsk->dirties);
 	free_thread_info(tsk->stack);
 	rt_mutex_debug_task_free(tsk);
 	ftrace_graph_exit_task(tsk);
 	free_task_struct(tsk);
 }
 EXPORT_SYMBOL(free_task);
 void __put_task_struct(struct task_struct *tsk)
 {
 	WARN_ON(!tsk->exit_state);
 	WARN_ON(atomic_read(&tsk->usage));
 	WARN_ON(tsk == current);
 	put_cred(tsk->real_cred);
 	put_cred(tsk->cred);
 	delayacct_tsk_free(tsk);
 	if (!profile_handoff_task(tsk))
 		free_task(tsk);
 }
 /*
  * macro override instead of weak attribute alias, to workaround
  * gcc 4.1.0 and 4.1.1 bugs with weak attribute and empty functions.
  */
 #ifndef arch_task_cache_init
 #define arch_task_cache_init()
 #endif
 void __init fork_init(unsigned long mempages)
 {
 #ifndef __HAVE_ARCH_TASK_STRUCT_ALLOCATOR
 #ifndef ARCH_MIN_TASKALIGN
 #define ARCH_MIN_TASKALIGN	L1_CACHE_BYTES
 #endif
 	/* create a slab on which task_structs can be allocated */
 	task_struct_cachep =
 		kmem_cache_create("task_struct", sizeof(struct task_struct),
 			ARCH_MIN_TASKALIGN, SLAB_PANIC, NULL);
 #endif
 	/* do the arch specific task caches init */
 	arch_task_cache_init();
 	/*
 	 * The default maximum number of threads is set to a safe
 	 * value: the thread structures can take up at most half
 	 * of memory.
 	 */
 	max_threads = mempages / (8 * THREAD_SIZE / PAGE_SIZE);
 	/*
 	 * we need to allow at least 20 threads to boot a system
 	 */
 	if(max_threads < 20)
 		max_threads = 20;
 	init_task.signal->rlim[RLIMIT_NPROC].rlim_cur = max_threads/2;
 	init_task.signal->rlim[RLIMIT_NPROC].rlim_max = max_threads/2;
 	init_task.signal->rlim[RLIMIT_SIGPENDING] =
 		init_task.signal->rlim[RLIMIT_NPROC];
 }
 int __attribute__((weak)) arch_dup_task_struct(struct task_struct *dst,
 					       struct task_struct *src)
 {
 	*dst = *src;
 	return 0;
 }
 static struct task_struct *dup_task_struct(struct task_struct *orig)
 {
 	struct task_struct *tsk;
 	struct thread_info *ti;
 	int err;
 	prepare_to_copy(orig);
 	tsk = alloc_task_struct();
 	if (!tsk)
 		return NULL;
 	ti = alloc_thread_info(tsk);
 	if (!ti) {
 		free_task_struct(tsk);
 		return NULL;
 	}
  	err = arch_dup_task_struct(tsk, orig);
 	if (err)
 		goto out;
 	tsk->stack = ti;
 	err = prop_local_init_single(&tsk->dirties);
 	if (err)
 		goto out;
 	setup_thread_stack(tsk, orig);
 #ifdef CONFIG_CC_STACKPROTECTOR
 	tsk->stack_canary = get_random_int();
 #endif
 	/* One for us, one for whoever does the "release_task()" (usually parent) */
 	atomic_set(&tsk->usage,2);
 	atomic_set(&tsk->fs_excl, 0);
 #ifdef CONFIG_BLK_DEV_IO_TRACE
 	tsk->btrace_seq = 0;
 #endif
 	tsk->splice_pipe = NULL;
 	return tsk;
 out:
 	free_thread_info(ti);
 	free_task_struct(tsk);
 	return NULL;
 }
 #ifdef CONFIG_MMU
 static int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm)
 {
 	struct vm_area_struct *mpnt, *tmp, **pprev;
 	struct rb_node **rb_link, *rb_parent;
 	int retval;
 	unsigned long charge;
 	struct mempolicy *pol;
 	down_write(&oldmm->mmap_sem);
 	flush_cache_dup_mm(oldmm);
 	/*
 	 * Not linked in yet - no deadlock potential:
 	 */
 	down_write_nested(&mm->mmap_sem, SINGLE_DEPTH_NESTING);
 	mm->locked_vm = 0;
 	mm->mmap = NULL;
 	mm->mmap_cache = NULL;
 	mm->free_area_cache = oldmm->mmap_base;
 	mm->cached_hole_size = ~0UL;
 	mm->map_count = 0;
 	cpus_clear(mm->cpu_vm_mask);
 	mm->mm_rb = RB_ROOT;
 	rb_link = &mm->mm_rb.rb_node;
 	rb_parent = NULL;
 	pprev = &mm->mmap;
 	for (mpnt = oldmm->mmap; mpnt; mpnt = mpnt->vm_next) {
 		struct file *file;
 		if (mpnt->vm_flags & VM_DONTCOPY) {
 			long pages = vma_pages(mpnt);
 			mm->total_vm -= pages;
 			vm_stat_account(mm, mpnt->vm_flags, mpnt->vm_file,
 								-pages);
 			continue;
 		}
 		charge = 0;
 		if (mpnt->vm_flags & VM_ACCOUNT) {
 			unsigned int len = (mpnt->vm_end - mpnt->vm_start) >> PAGE_SHIFT;
 			if (security_vm_enough_memory(len))
 				goto fail_nomem;
 			charge = len;
 		}
 		tmp = kmem_cache_alloc(vm_area_cachep, GFP_KERNEL);
 		if (!tmp)
 			goto fail_nomem;
 		*tmp = *mpnt;
 		pol = mpol_dup(vma_policy(mpnt));
 		retval = PTR_ERR(pol);
 		if (IS_ERR(pol))
 			goto fail_nomem_policy;
 		vma_set_policy(tmp, pol);
 		tmp->vm_flags &= ~VM_LOCKED;
 		tmp->vm_mm = mm;
 		tmp->vm_next = NULL;
 		anon_vma_link(tmp);
 		file = tmp->vm_file;
 		if (file) {
 			struct inode *inode = file->f_path.dentry->d_inode;
 			struct address_space *mapping = file->f_mapping;
 			get_file(file);
 			if (tmp->vm_flags & VM_DENYWRITE)
 				atomic_dec(&inode->i_writecount);
 			spin_lock(&mapping->i_mmap_lock);
 			if (tmp->vm_flags & VM_SHARED)
 				mapping->i_mmap_writable++;
 			tmp->vm_truncate_count = mpnt->vm_truncate_count;
 			flush_dcache_mmap_lock(mapping);
 			/* insert tmp into the share list, just after mpnt */
 			vma_prio_tree_add(tmp, mpnt);
 			flush_dcache_mmap_unlock(mapping);
 			spin_unlock(&mapping->i_mmap_lock);
 		}
 		/*
 		 * Clear hugetlb-related page reserves for children. This only
 		 * affects MAP_PRIVATE mappings. Faults generated by the child
 		 * are not guaranteed to succeed, even if read-only
 		 */
 		if (is_vm_hugetlb_page(tmp))
 			reset_vma_resv_huge_pages(tmp);
 		/*
 		 * Link in the new vma and copy the page table entries.
 		 */
 		*pprev = tmp;
 		pprev = &tmp->vm_next;
 		__vma_link_rb(mm, tmp, rb_link, rb_parent);
 		rb_link = &tmp->vm_rb.rb_right;
 		rb_parent = &tmp->vm_rb;
 		mm->map_count++;
 		retval = copy_page_range(mm, oldmm, mpnt);
 		if (tmp->vm_ops && tmp->vm_ops->open)
 			tmp->vm_ops->open(tmp);
 		if (retval)
 			goto out;
 	}
 	/* a new mm has just been created */
 	arch_dup_mmap(oldmm, mm);
 	retval = 0;
 out:
 	up_write(&mm->mmap_sem);
 	flush_tlb_mm(oldmm);
 	up_write(&oldmm->mmap_sem);
 	return retval;
 fail_nomem_policy:
 	kmem_cache_free(vm_area_cachep, tmp);
 fail_nomem:
 	retval = -ENOMEM;
 	vm_unacct_memory(charge);
 	goto out;
 }
 static inline int mm_alloc_pgd(struct mm_struct * mm)
 {
 	mm->pgd = pgd_alloc(mm);
 	if (unlikely(!mm->pgd))
 		return -ENOMEM;
 	return 0;
 }
 static inline void mm_free_pgd(struct mm_struct * mm)
 {
 	pgd_free(mm, mm->pgd);
 }
 #else
 #define dup_mmap(mm, oldmm)	(0)
 #define mm_alloc_pgd(mm)	(0)
 #define mm_free_pgd(mm)
 #endif /* CONFIG_MMU */
 __cacheline_aligned_in_smp DEFINE_SPINLOCK(mmlist_lock);
 #define allocate_mm()	(kmem_cache_alloc(mm_cachep, GFP_KERNEL))
 #define free_mm(mm)	(kmem_cache_free(mm_cachep, (mm)))
 static unsigned long default_dump_filter = MMF_DUMP_FILTER_DEFAULT;
 static int __init coredump_filter_setup(char *s)
 {
 	default_dump_filter =
 		(simple_strtoul(s, NULL, 0) << MMF_DUMP_FILTER_SHIFT) &
 		MMF_DUMP_FILTER_MASK;
 	return 1;
 }
 __setup("coredump_filter=", coredump_filter_setup);
 #include <linux/init_task.h>
 static struct mm_struct * mm_init(struct mm_struct * mm, struct task_struct *p)
 {
 	atomic_set(&mm->mm_users, 1);
 	atomic_set(&mm->mm_count, 1);
 	init_rwsem(&mm->mmap_sem);
 	INIT_LIST_HEAD(&mm->mmlist);
 	mm->flags = (current->mm) ? current->mm->flags : default_dump_filter;
 	mm->core_state = NULL;
 	mm->nr_ptes = 0;
 	set_mm_counter(mm, file_rss, 0);
 	set_mm_counter(mm, anon_rss, 0);
 	spin_lock_init(&mm->page_table_lock);
 	spin_lock_init(&mm->ioctx_lock);
 	INIT_HLIST_HEAD(&mm->ioctx_list);
 	mm->free_area_cache = TASK_UNMAPPED_BASE;
 	mm->cached_hole_size = ~0UL;
 	mm_init_owner(mm, p);
 	if (likely(!mm_alloc_pgd(mm))) {
 		mm->def_flags = 0;
 		mmu_notifier_mm_init(mm);
 		return mm;
 	}
 	free_mm(mm);
 	return NULL;
 }
 /*
  * Allocate and initialize an mm_struct.
  */
 struct mm_struct * mm_alloc(void)
 {
 	struct mm_struct * mm;
 	mm = allocate_mm();
 	if (mm) {
 		memset(mm, 0, sizeof(*mm));
 		mm = mm_init(mm, current);
 	}
 	return mm;
 }
 /*
  * Called when the last reference to the mm
  * is dropped: either by a lazy thread or by
  * mmput. Free the page directory and the mm.
  */
 void __mmdrop(struct mm_struct *mm)
 {
 	BUG_ON(mm == &init_mm);
 	mm_free_pgd(mm);
 	destroy_context(mm);
 	mmu_notifier_mm_destroy(mm);
 	free_mm(mm);
 }
 EXPORT_SYMBOL_GPL(__mmdrop);
 /*
  * Decrement the use count and release all resources for an mm.
  */
 void mmput(struct mm_struct *mm)
 {
 	might_sleep();
 	if (atomic_dec_and_test(&mm->mm_users)) {
 		exit_aio(mm);
 		exit_mmap(mm);
 		set_mm_exe_file(mm, NULL);
 		if (!list_empty(&mm->mmlist)) {
 			spin_lock(&mmlist_lock);
 			list_del(&mm->mmlist);
 			spin_unlock(&mmlist_lock);
 		}
 		put_swap_token(mm);
 		mmdrop(mm);
 	}
 }
 EXPORT_SYMBOL_GPL(mmput);
 /**
  * get_task_mm - acquire a reference to the task's mm
  *
  * Returns %NULL if the task has no mm.  Checks PF_KTHREAD (meaning
  * this kernel workthread has transiently adopted a user mm with use_mm,
  * to do its AIO) is not set and if so returns a reference to it, after
  * bumping up the use count.  User must release the mm via mmput()
  * after use.  Typically used by /proc and ptrace.
  */
 struct mm_struct *get_task_mm(struct task_struct *task)
 {
 	struct mm_struct *mm;
 	task_lock(task);
 	mm = task->mm;
 	if (mm) {
 		if (task->flags & PF_KTHREAD)
 			mm = NULL;
 		else
 			atomic_inc(&mm->mm_users);
 	}
 	task_unlock(task);
 	return mm;
 }
 EXPORT_SYMBOL_GPL(get_task_mm);
 /* Please note the differences between mmput and mm_release.
  * mmput is called whenever we stop holding onto a mm_struct,
  * error success whatever.
  *
  * mm_release is called after a mm_struct has been removed
  * from the current process.
  *
  * This difference is important for error handling, when we
  * only half set up a mm_struct for a new process and need to restore
  * the old one.  Because we mmput the new mm_struct before
  * restoring the old one. . .
  * Eric Biederman 10 January 1998
  */
 void mm_release(struct task_struct *tsk, struct mm_struct *mm)
 {
 	struct completion *vfork_done = tsk->vfork_done;
 	/* Get rid of any futexes when releasing the mm */
 #ifdef CONFIG_FUTEX
 	if (unlikely(tsk->robust_list))
 		exit_robust_list(tsk);
 #ifdef CONFIG_COMPAT
 	if (unlikely(tsk->compat_robust_list))
 		compat_exit_robust_list(tsk);
 #endif
 #endif
 	/* Get rid of any cached register state */
 	deactivate_mm(tsk, mm);
 	/* notify parent sleeping on vfork() */
 	if (vfork_done) {
 		tsk->vfork_done = NULL;
 		complete(vfork_done);
 	}
 	/*
 	 * If we're exiting normally, clear a user-space tid field if
 	 * requested.  We leave this alone when dying by signal, to leave
 	 * the value intact in a core dump, and to save the unnecessary
 	 * trouble otherwise.  Userland only wants this done for a sys_exit.
 	 */
 	if (tsk->clear_child_tid
 	    && !(tsk->flags & PF_SIGNALED)
 	    && atomic_read(&mm->mm_users) > 1) {
 		u32 __user * tidptr = tsk->clear_child_tid;
 		tsk->clear_child_tid = NULL;
 		/*
 		 * We don't check the error code - if userspace has
 		 * not set up a proper pointer then tough luck.
 		 */
 		put_user(0, tidptr);
 		sys_futex(tidptr, FUTEX_WAKE, 1, NULL, NULL, 0);
 	}
 }
 /*
  * Allocate a new mm structure and copy contents from the
  * mm structure of the passed in task structure.
  */
 struct mm_struct *dup_mm(struct task_struct *tsk)
 {
 	struct mm_struct *mm, *oldmm = current->mm;
 	int err;
 	if (!oldmm)
 		return NULL;
 	mm = allocate_mm();
 	if (!mm)
 		goto fail_nomem;
 	memcpy(mm, oldmm, sizeof(*mm));
 	/* Initializing for Swap token stuff */
 	mm->token_priority = 0;
 	mm->last_interval = 0;
 	if (!mm_init(mm, tsk))
 		goto fail_nomem;
 	if (init_new_context(tsk, mm))
 		goto fail_nocontext;
 	dup_mm_exe_file(oldmm, mm);
 	err = dup_mmap(mm, oldmm);
 	if (err)
 		goto free_pt;
 	mm->hiwater_rss = get_mm_rss(mm);
 	mm->hiwater_vm = mm->total_vm;
 	return mm;
 free_pt:
 	mmput(mm);
 fail_nomem:
 	return NULL;
 fail_nocontext:
 	/*
 	 * If init_new_context() failed, we cannot use mmput() to free the mm
 	 * because it calls destroy_context()
 	 */
 	mm_free_pgd(mm);
 	free_mm(mm);
 	return NULL;
 }
 static int copy_mm(unsigned long clone_flags, struct task_struct * tsk)
 {
 	struct mm_struct * mm, *oldmm;
 	int retval;
 	tsk->min_flt = tsk->maj_flt = 0;
 	tsk->nvcsw = tsk->nivcsw = 0;
 	tsk->mm = NULL;
 	tsk->active_mm = NULL;
 	/*
 	 * Are we cloning a kernel thread?
 	 *
 	 * We need to steal a active VM for that..
 	 */
 	oldmm = current->mm;
 	if (!oldmm)
 		return 0;
 	if (clone_flags & CLONE_VM) {
 		atomic_inc(&oldmm->mm_users);
 		mm = oldmm;
 		goto good_mm;
 	}
 	retval = -ENOMEM;
 	mm = dup_mm(tsk);
 	if (!mm)
 		goto fail_nomem;
 good_mm:
 	/* Initializing for Swap token stuff */
 	mm->token_priority = 0;
 	mm->last_interval = 0;
 	tsk->mm = mm;
 	tsk->active_mm = mm;
 	return 0;
 fail_nomem:
 	return retval;
 }
 static struct fs_struct *__copy_fs_struct(struct fs_struct *old)
 {
 	struct fs_struct *fs = kmem_cache_alloc(fs_cachep, GFP_KERNEL);
 	/* We don't need to lock fs - think why ;-) */
 	if (fs) {
 		atomic_set(&fs->count, 1);
 		rwlock_init(&fs->lock);
 		fs->umask = old->umask;
 		read_lock(&old->lock);
 		fs->root = old->root;
 		path_get(&old->root);
 		fs->pwd = old->pwd;
 		path_get(&old->pwd);
 		read_unlock(&old->lock);
 	}
 	return fs;
 }
 struct fs_struct *copy_fs_struct(struct fs_struct *old)
 {
 	return __copy_fs_struct(old);
 }
 EXPORT_SYMBOL_GPL(copy_fs_struct);
 static int copy_fs(unsigned long clone_flags, struct task_struct *tsk)
 {
 	if (clone_flags & CLONE_FS) {
 		atomic_inc(&current->fs->count);
 		return 0;
 	}
 	tsk->fs = __copy_fs_struct(current->fs);
 	if (!tsk->fs)
 		return -ENOMEM;
 	return 0;
 }
 static int copy_files(unsigned long clone_flags, struct task_struct * tsk)
 {
 	struct files_struct *oldf, *newf;
 	int error = 0;
 	/*
 	 * A background process may not have any files ...
 	 */
 	oldf = current->files;
 	if (!oldf)
 		goto out;
 	if (clone_flags & CLONE_FILES) {
 		atomic_inc(&oldf->count);
 		goto out;
 	}
 	newf = dup_fd(oldf, &error);
 	if (!newf)
 		goto out;
 	tsk->files = newf;
 	error = 0;
 out:
 	return error;
 }
 static int copy_io(unsigned long clone_flags, struct task_struct *tsk)
 {
 #ifdef CONFIG_BLOCK
 	struct io_context *ioc = current->io_context;
 	if (!ioc)
 		return 0;
 	/*
 	 * Share io context with parent, if CLONE_IO is set
 	 */
 	if (clone_flags & CLONE_IO) {
 		tsk->io_context = ioc_task_link(ioc);
 		if (unlikely(!tsk->io_context))
 			return -ENOMEM;
 	} else if (ioprio_valid(ioc->ioprio)) {
 		tsk->io_context = alloc_io_context(GFP_KERNEL, -1);
 		if (unlikely(!tsk->io_context))
 			return -ENOMEM;
 		tsk->io_context->ioprio = ioc->ioprio;
 	}
 #endif
 	return 0;
 }
 static int copy_sighand(unsigned long clone_flags, struct task_struct *tsk)
 {
 	struct sighand_struct *sig;
 	if (clone_flags & CLONE_SIGHAND) {
 		atomic_inc(&current->sighand->count);
 		return 0;
 	}
 	sig = kmem_cache_alloc(sighand_cachep, GFP_KERNEL);
 	rcu_assign_pointer(tsk->sighand, sig);
 	if (!sig)
 		return -ENOMEM;
 	atomic_set(&sig->count, 1);
 	memcpy(sig->action, current->sighand->action, sizeof(sig->action));
 	return 0;
 }
 void __cleanup_sighand(struct sighand_struct *sighand)
 {
 	if (atomic_dec_and_test(&sighand->count))
 		kmem_cache_free(sighand_cachep, sighand);
 }
 /*
  * Initialize POSIX timer handling for a thread group.
  */
 static void posix_cpu_timers_init_group(struct signal_struct *sig)
 {
 	/* Thread group counters. */
 	thread_group_cputime_init(sig);
 	/* Expiration times and increments. */
 	sig->it_virt_expires = cputime_zero;
 	sig->it_virt_incr = cputime_zero;
 	sig->it_prof_expires = cputime_zero;
 	sig->it_prof_incr = cputime_zero;
 	/* Cached expiration times. */
 	sig->cputime_expires.prof_exp = cputime_zero;
 	sig->cputime_expires.virt_exp = cputime_zero;
 	sig->cputime_expires.sched_exp = 0;
 	/* The timer lists. */
 	INIT_LIST_HEAD(&sig->cpu_timers[0]);
 	INIT_LIST_HEAD(&sig->cpu_timers[1]);
 	INIT_LIST_HEAD(&sig->cpu_timers[2]);
 }
 static int copy_signal(unsigned long clone_flags, struct task_struct *tsk)
 {
 	struct signal_struct *sig;
 	int ret;
 	if (clone_flags & CLONE_THREAD) {
 		ret = thread_group_cputime_clone_thread(current);
 		if (likely(!ret)) {
 			atomic_inc(&current->signal->count);
 			atomic_inc(&current->signal->live);
 		}
 		return ret;
 	}
 	sig = kmem_cache_alloc(signal_cachep, GFP_KERNEL);
 	tsk->signal = sig;
 	if (!sig)
 		return -ENOMEM;
 	atomic_set(&sig->count, 1);
 	atomic_set(&sig->live, 1);
 	init_waitqueue_head(&sig->wait_chldexit);
 	sig->flags = 0;
 	sig->group_exit_code = 0;
 	sig->group_exit_task = NULL;
 	sig->group_stop_count = 0;
 	sig->curr_target = tsk;
 	init_sigpending(&sig->shared_pending);
 	INIT_LIST_HEAD(&sig->posix_timers);
 	hrtimer_init(&sig->real_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
 	sig->it_real_incr.tv64 = 0;
 	sig->real_timer.function = it_real_fn;
 	sig->leader = 0;	/* session leadership doesn't inherit */
 	sig->tty_old_pgrp = NULL;
 	sig->tty = NULL;
 	sig->cutime = sig->cstime = cputime_zero;
 	sig->gtime = cputime_zero;
 	sig->cgtime = cputime_zero;
 	sig->nvcsw = sig->nivcsw = sig->cnvcsw = sig->cnivcsw = 0;
 	sig->min_flt = sig->maj_flt = sig->cmin_flt = sig->cmaj_flt = 0;
 	sig->inblock = sig->oublock = sig->cinblock = sig->coublock = 0;
 	task_io_accounting_init(&sig->ioac);
 	taskstats_tgid_init(sig);
 	task_lock(current->group_leader);
 	memcpy(sig->rlim, current->signal->rlim, sizeof sig->rlim);
 	task_unlock(current->group_leader);
 	posix_cpu_timers_init_group(sig);
 	acct_init_pacct(&sig->pacct);
 	tty_audit_fork(sig);
 	return 0;
 }
 void __cleanup_signal(struct signal_struct *sig)
 {
 	thread_group_cputime_free(sig);
 	tty_kref_put(sig->tty);
 	kmem_cache_free(signal_cachep, sig);
 }
 static void cleanup_signal(struct task_struct *tsk)
 {
 	struct signal_struct *sig = tsk->signal;
 	atomic_dec(&sig->live);
 	if (atomic_dec_and_test(&sig->count))
 		__cleanup_signal(sig);
 }
 static void copy_flags(unsigned long clone_flags, struct task_struct *p)
 {
 	unsigned long new_flags = p->flags;
 	new_flags &= ~PF_SUPERPRIV;
 	new_flags |= PF_FORKNOEXEC;
 	new_flags |= PF_STARTING;
 	p->flags = new_flags;
 	clear_freeze_flag(p);
 }
 asmlinkage long sys_set_tid_address(int __user *tidptr)
 {
 	current->clear_child_tid = tidptr;
 	return task_pid_vnr(current);
 }
 static void rt_mutex_init_task(struct task_struct *p)
 {
 	spin_lock_init(&p->pi_lock);
 #ifdef CONFIG_RT_MUTEXES
 	plist_head_init(&p->pi_waiters, &p->pi_lock);
 	p->pi_blocked_on = NULL;
 #endif
 }
 #ifdef CONFIG_MM_OWNER
 void mm_init_owner(struct mm_struct *mm, struct task_struct *p)
 {
 	mm->owner = p;
 }
 #endif /* CONFIG_MM_OWNER */
 /*
  * Initialize POSIX timer handling for a single task.
  */
 static void posix_cpu_timers_init(struct task_struct *tsk)
 {
 	tsk->cputime_expires.prof_exp = cputime_zero;
 	tsk->cputime_expires.virt_exp = cputime_zero;
 	tsk->cputime_expires.sched_exp = 0;
 	INIT_LIST_HEAD(&tsk->cpu_timers[0]);
 	INIT_LIST_HEAD(&tsk->cpu_timers[1]);
 	INIT_LIST_HEAD(&tsk->cpu_timers[2]);
 }
 /*
  * This creates a new process as a copy of the old one,
  * but does not actually start it yet.
  *
  * It copies the registers, and all the appropriate
  * parts of the process environment (as per the clone
  * flags). The actual kick-off is left to the caller.
  */
 static struct task_struct *copy_process(unsigned long clone_flags,
 					unsigned long stack_start,
 					struct pt_regs *regs,
 					unsigned long stack_size,
 					int __user *child_tidptr,
 					struct pid *pid,
 					int trace)
 {
 	int retval;
 	struct task_struct *p;
 	int cgroup_callbacks_done = 0;
 	if ((clone_flags & (CLONE_NEWNS|CLONE_FS)) == (CLONE_NEWNS|CLONE_FS))
 		return ERR_PTR(-EINVAL);
 	/*
 	 * Thread groups must share signals as well, and detached threads
 	 * can only be started up within the thread group.
 	 */
 	if ((clone_flags & CLONE_THREAD) && !(clone_flags & CLONE_SIGHAND))
 		return ERR_PTR(-EINVAL);
 	/*
 	 * Shared signal handlers imply shared VM. By way of the above,
 	 * thread groups also imply shared VM. Blocking this case allows
 	 * for various simplifications in other code.
 	 */
 	if ((clone_flags & CLONE_SIGHAND) && !(clone_flags & CLONE_VM))
 		return ERR_PTR(-EINVAL);
 	retval = security_task_create(clone_flags);
 	if (retval)
 		goto fork_out;
 	retval = -ENOMEM;
 	p = dup_task_struct(current);
 	if (!p)
 		goto fork_out;
 	rt_mutex_init_task(p);
 #ifdef CONFIG_PROVE_LOCKING
 	DEBUG_LOCKS_WARN_ON(!p->hardirqs_enabled);
 	DEBUG_LOCKS_WARN_ON(!p->softirqs_enabled);
 #endif
 	retval = -EAGAIN;
 	if (atomic_read(&p->real_cred->user->processes) >=
 			p->signal->rlim[RLIMIT_NPROC].rlim_cur) {
 		if (!capable(CAP_SYS_ADMIN) && !capable(CAP_SYS_RESOURCE) &&
 		    p->real_cred->user != INIT_USER)
 			goto bad_fork_free;
 	}
 	retval = copy_creds(p, clone_flags);
 	if (retval < 0)
 		goto bad_fork_free;
 	/*
 	 * If multiple threads are within copy_process(), then this check
 	 * triggers too late. This doesn't hurt, the check is only there
 	 * to stop root fork bombs.
 	 */
 	if (nr_threads >= max_threads)
 		goto bad_fork_cleanup_count;
 	if (!try_module_get(task_thread_info(p)->exec_domain->module))
 		goto bad_fork_cleanup_count;
 	if (p->binfmt && !try_module_get(p->binfmt->module))
 		goto bad_fork_cleanup_put_domain;
 	p->did_exec = 0;
 	delayacct_tsk_init(p);	/* Must remain after dup_task_struct() */
 	copy_flags(clone_flags, p);
 	INIT_LIST_HEAD(&p->children);
 	INIT_LIST_HEAD(&p->sibling);
 #ifdef CONFIG_PREEMPT_RCU
 	p->rcu_read_lock_nesting = 0;
 	p->rcu_flipctr_idx = 0;
 #endif /* #ifdef CONFIG_PREEMPT_RCU */
 	p->vfork_done = NULL;
 	spin_lock_init(&p->alloc_lock);
 	clear_tsk_thread_flag(p, TIF_SIGPENDING);
 	init_sigpending(&p->pending);
 	p->utime = cputime_zero;
 	p->stime = cputime_zero;
 	p->gtime = cputime_zero;
 	p->utimescaled = cputime_zero;
 	p->stimescaled = cputime_zero;
 	p->prev_utime = cputime_zero;
 	p->prev_stime = cputime_zero;
 	p->default_timer_slack_ns = current->timer_slack_ns;
 #ifdef CONFIG_DETECT_SOFTLOCKUP
 	p->last_switch_count = 0;
 	p->last_switch_timestamp = 0;
 #endif
 	task_io_accounting_init(&p->ioac);
 	acct_clear_integrals(p);
 	posix_cpu_timers_init(p);
 	p->lock_depth = -1;		/* -1 = no lock */
 	do_posix_clock_monotonic_gettime(&p->start_time);
 	p->real_start_time = p->start_time;
 	monotonic_to_bootbased(&p->real_start_time);
 	p->io_context = NULL;
 	p->audit_context = NULL;
 	cgroup_fork(p);
 #ifdef CONFIG_NUMA
 	p->mempolicy = mpol_dup(p->mempolicy);
  	if (IS_ERR(p->mempolicy)) {
  		retval = PTR_ERR(p->mempolicy);
  		p->mempolicy = NULL;
  		goto bad_fork_cleanup_cgroup;
  	}
 	mpol_fix_fork_child_flag(p);
 #endif
 #ifdef CONFIG_TRACE_IRQFLAGS
 	p->irq_events = 0;
 #ifdef __ARCH_WANT_INTERRUPTS_ON_CTXSW
 	p->hardirqs_enabled = 1;
 #else
 	p->hardirqs_enabled = 0;
 #endif
 	p->hardirq_enable_ip = 0;
 	p->hardirq_enable_event = 0;
 	p->hardirq_disable_ip = _THIS_IP_;
 	p->hardirq_disable_event = 0;
 	p->softirqs_enabled = 1;
 	p->softirq_enable_ip = _THIS_IP_;
 	p->softirq_enable_event = 0;
 	p->softirq_disable_ip = 0;
 	p->softirq_disable_event = 0;
 	p->hardirq_context = 0;
 	p->softirq_context = 0;
 #endif
 #ifdef CONFIG_LOCKDEP
 	p->lockdep_depth = 0; /* no locks held yet */
 	p->curr_chain_key = 0;
 	p->lockdep_recursion = 0;
 #endif
 #ifdef CONFIG_DEBUG_MUTEXES
 	p->blocked_on = NULL; /* not blocked yet */
 #endif
 	if (unlikely(ptrace_reparented(current)))
 		ptrace_fork(p, clone_flags);
 	/* Perform scheduler related setup. Assign this task to a CPU. */
 	sched_fork(p, clone_flags);
 	if ((retval = audit_alloc(p)))
 		goto bad_fork_cleanup_policy;
 	/* copy all the process information */
 	if ((retval = copy_semundo(clone_flags, p)))
 		goto bad_fork_cleanup_audit;
 	if ((retval = copy_files(clone_flags, p)))
 		goto bad_fork_cleanup_semundo;
 	if ((retval = copy_fs(clone_flags, p)))
 		goto bad_fork_cleanup_files;
 	if ((retval = copy_sighand(clone_flags, p)))
 		goto bad_fork_cleanup_fs;
 	if ((retval = copy_signal(clone_flags, p)))
 		goto bad_fork_cleanup_sighand;
 	if ((retval = copy_mm(clone_flags, p)))
 		goto bad_fork_cleanup_signal;
 	if ((retval = copy_namespaces(clone_flags, p)))
 		goto bad_fork_cleanup_mm;
 	if ((retval = copy_io(clone_flags, p)))
 		goto bad_fork_cleanup_namespaces;
 	retval = copy_thread(0, clone_flags, stack_start, stack_size, p, regs);
 	if (retval)
 		goto bad_fork_cleanup_io;
 	if (pid != &init_struct_pid) {
 		retval = -ENOMEM;
 		pid = alloc_pid(p->nsproxy->pid_ns);
 		if (!pid)
 			goto bad_fork_cleanup_io;
 		if (clone_flags & CLONE_NEWPID) {
 			retval = pid_ns_prepare_proc(p->nsproxy->pid_ns);
 			if (retval < 0)
 				goto bad_fork_free_pid;
 		}
 	}
 	ftrace_graph_init_task(p);
 	p->pid = pid_nr(pid);
 	p->tgid = p->pid;
 	if (clone_flags & CLONE_THREAD)
 		p->tgid = current->tgid;
 	if (current->nsproxy != p->nsproxy) {
 		retval = ns_cgroup_clone(p, pid);
 		if (retval)
 			goto bad_fork_free_graph;
 	}
 	p->set_child_tid = (clone_flags & CLONE_CHILD_SETTID) ? child_tidptr : NULL;
 	/*
 	 * Clear TID on mm_release()?
 	 */
 	p->clear_child_tid = (clone_flags & CLONE_CHILD_CLEARTID) ? child_tidptr: NULL;
 #ifdef CONFIG_FUTEX
 	p->robust_list = NULL;
 #ifdef CONFIG_COMPAT
 	p->compat_robust_list = NULL;
 #endif
 	INIT_LIST_HEAD(&p->pi_state_list);
 	p->pi_state_cache = NULL;
 #endif
 	/*
 	 * sigaltstack should be cleared when sharing the same VM
 	 */
 	if ((clone_flags & (CLONE_VM|CLONE_VFORK)) == CLONE_VM)
 		p->sas_ss_sp = p->sas_ss_size = 0;
 	/*
 	 * Syscall tracing should be turned off in the child regardless
 	 * of CLONE_PTRACE.
 	 */
 	clear_tsk_thread_flag(p, TIF_SYSCALL_TRACE);
 #ifdef TIF_SYSCALL_EMU
 	clear_tsk_thread_flag(p, TIF_SYSCALL_EMU);
 #endif
 	clear_all_latency_tracing(p);
 	/* Our parent execution domain becomes current domain
 	   These must match for thread signalling to apply */
 	p->parent_exec_id = p->self_exec_id;
 	/* ok, now we should be set up.. */
 	p->exit_signal = (clone_flags & CLONE_THREAD) ? -1 : (clone_flags & CSIGNAL);
 	p->pdeath_signal = 0;
 	p->exit_state = 0;
 	/*
 	 * Ok, make it visible to the rest of the system.
 	 * We dont wake it up yet.
 	 */
 	p->group_leader = p;
 	INIT_LIST_HEAD(&p->thread_group);
 	/* Now that the task is set up, run cgroup callbacks if
 	 * necessary. We need to run them before the task is visible
 	 * on the tasklist. */
 	cgroup_fork_callbacks(p);
 	cgroup_callbacks_done = 1;
 	/* Need tasklist lock for parent etc handling! */
 	write_lock_irq(&tasklist_lock);
 	/*
 	 * The task hasn't been attached yet, so its cpus_allowed mask will
 	 * not be changed, nor will its assigned CPU.
 	 *
 	 * The cpus_allowed mask of the parent may have changed after it was
 	 * copied first time - so re-copy it here, then check the child's CPU
 	 * to ensure it is on a valid CPU (and if not, just force it back to
 	 * parent's CPU). This avoids alot of nasty races.
 	 */
 	p->cpus_allowed = current->cpus_allowed;
 	p->rt.nr_cpus_allowed = current->rt.nr_cpus_allowed;
 	if (unlikely(!cpu_isset(task_cpu(p), p->cpus_allowed) ||
 			!cpu_online(task_cpu(p))))
 		set_task_cpu(p, smp_processor_id());
 	/* CLONE_PARENT re-uses the old parent */
 	if (clone_flags & (CLONE_PARENT|CLONE_THREAD))
 		p->real_parent = current->real_parent;
 	else
 		p->real_parent = current;
 	spin_lock(&current->sighand->siglock);
 	/*
 	 * Process group and session signals need to be delivered to just the
 	 * parent before the fork or both the parent and the child after the
 	 * fork. Restart if a signal comes in before we add the new process to
 	 * it's process group.
 	 * A fatal signal pending means that current will exit, so the new
 	 * thread can't slip out of an OOM kill (or normal SIGKILL).
  	 */
 	recalc_sigpending();
 	if (signal_pending(current)) {
 		spin_unlock(&current->sighand->siglock);
 		write_unlock_irq(&tasklist_lock);
 		retval = -ERESTARTNOINTR;
 		goto bad_fork_free_graph;
 	}
 	if (clone_flags & CLONE_THREAD) {
 		p->group_leader = current->group_leader;
 		list_add_tail_rcu(&p->thread_group, &p->group_leader->thread_group);
 	}
 	if (likely(p->pid)) {
 		list_add_tail(&p->sibling, &p->real_parent->children);
 		tracehook_finish_clone(p, clone_flags, trace);
 		if (thread_group_leader(p)) {
 			if (clone_flags & CLONE_NEWPID)
 				p->nsproxy->pid_ns->child_reaper = p;
 			p->signal->leader_pid = pid;
 			tty_kref_put(p->signal->tty);
 			p->signal->tty = tty_kref_get(current->signal->tty);
 			set_task_pgrp(p, task_pgrp_nr(current));
 			set_task_session(p, task_session_nr(current));
 			attach_pid(p, PIDTYPE_PGID, task_pgrp(current));
 			attach_pid(p, PIDTYPE_SID, task_session(current));
 			list_add_tail_rcu(&p->tasks, &init_task.tasks);
 			__get_cpu_var(process_counts)++;
 		}
 		attach_pid(p, PIDTYPE_PID, pid);
 		nr_threads++;
 	}
 	total_forks++;
 	spin_unlock(&current->sighand->siglock);
 	write_unlock_irq(&tasklist_lock);
 	proc_fork_connector(p);
 	cgroup_post_fork(p);
 	return p;
 bad_fork_free_graph:
 	ftrace_graph_exit_task(p);
 bad_fork_free_pid:
 	if (pid != &init_struct_pid)
 		free_pid(pid);
 bad_fork_cleanup_io:
 	put_io_context(p->io_context);
 bad_fork_cleanup_namespaces:
 	exit_task_namespaces(p);
 bad_fork_cleanup_mm:
 	if (p->mm)
 		mmput(p->mm);
 bad_fork_cleanup_signal:
 	cleanup_signal(p);
 bad_fork_cleanup_sighand:
 	__cleanup_sighand(p->sighand);
 bad_fork_cleanup_fs:
 	exit_fs(p); /* blocking */
 bad_fork_cleanup_files:
 	exit_files(p); /* blocking */
 bad_fork_cleanup_semundo:
 	exit_sem(p);
 bad_fork_cleanup_audit:
 	audit_free(p);
 bad_fork_cleanup_policy:
 #ifdef CONFIG_NUMA
 	mpol_put(p->mempolicy);
 bad_fork_cleanup_cgroup:
 #endif
 	cgroup_exit(p, cgroup_callbacks_done);
 	delayacct_tsk_free(p);
 	if (p->binfmt)
 		module_put(p->binfmt->module);
 bad_fork_cleanup_put_domain:
 	module_put(task_thread_info(p)->exec_domain->module);
 bad_fork_cleanup_count:
 	atomic_dec(&p->cred->user->processes);
 	put_cred(p->real_cred);
 	put_cred(p->cred);
 bad_fork_free:
 	free_task(p);
 fork_out:
 	return ERR_PTR(retval);
 }
 noinline struct pt_regs * __cpuinit __attribute__((weak)) idle_regs(struct pt_regs *regs)
 {
 	memset(regs, 0, sizeof(struct pt_regs));
 	return regs;
 }
 struct task_struct * __cpuinit fork_idle(int cpu)
 {
 	struct task_struct *task;
 	struct pt_regs regs;
 	task = copy_process(CLONE_VM, 0, idle_regs(&regs), 0, NULL,
 			    &init_struct_pid, 0);
 	if (!IS_ERR(task))
 		init_idle(task, cpu);
 	return task;
 }
 /*
  *  Ok, this is the main fork-routine.
  *
  * It copies the process, and if successful kick-starts
  * it and waits for it to finish using the VM if required.
  */
 long do_fork(unsigned long clone_flags,
 	      unsigned long stack_start,
 	      struct pt_regs *regs,
 	      unsigned long stack_size,
 	      int __user *parent_tidptr,
 	      int __user *child_tidptr)
 {
 	struct task_struct *p;
 	int trace = 0;
 	long nr;
 	/*
 	 * Do some preliminary argument and permissions checking before we
 	 * actually start allocating stuff
 	 */
 	if (clone_flags & CLONE_NEWUSER) {
 		if (clone_flags & CLONE_THREAD)
 			return -EINVAL;
 		/* hopefully this check will go away when userns support is
 		 * complete
 		 */
 		if (!capable(CAP_SYS_ADMIN) || !capable(CAP_SETUID) ||
 				!capable(CAP_SETGID))
 			return -EPERM;
 	}
 	/*
 	 * We hope to recycle these flags after 2.6.26
 	 */
 	if (unlikely(clone_flags & CLONE_STOPPED)) {
 		static int __read_mostly count = 100;
 		if (count > 0 && printk_ratelimit()) {
 			char comm[TASK_COMM_LEN];
 			count--;
 			printk(KERN_INFO "fork(): process `%s' used deprecated "
 					"clone flags 0x%lx\n",
 				get_task_comm(comm, current),
 				clone_flags & CLONE_STOPPED);
 		}
 	}
 	/*
 	 * When called from kernel_thread, don't do user tracing stuff.
 	 */
 	if (likely(user_mode(regs)))
 		trace = tracehook_prepare_clone(clone_flags);
 	p = copy_process(clone_flags, stack_start, regs, stack_size,
 			 child_tidptr, NULL, trace);
 	/*
 	 * Do this prior waking up the new thread - the thread pointer
 	 * might get invalid after that point, if the thread exits quickly.
 	 */
 	if (!IS_ERR(p)) {
 		struct completion vfork;
 		trace_sched_process_fork(current, p);
 		nr = task_pid_vnr(p);
 		if (clone_flags & CLONE_PARENT_SETTID)
 			put_user(nr, parent_tidptr);
 		if (clone_flags & CLONE_VFORK) {
 			p->vfork_done = &vfork;
 			init_completion(&vfork);
 		}
 		audit_finish_fork(p);
 		tracehook_report_clone(trace, regs, clone_flags, nr, p);
 		/*
 		 * We set PF_STARTING at creation in case tracing wants to
 		 * use this to distinguish a fully live task from one that
 		 * hasn't gotten to tracehook_report_clone() yet.  Now we
 		 * clear it and set the child going.
 		 */
 		p->flags &= ~PF_STARTING;
 		if (unlikely(clone_flags & CLONE_STOPPED)) {
 			/*
 			 * We'll start up with an immediate SIGSTOP.
 			 */
 			sigaddset(&p->pending.signal, SIGSTOP);
 			set_tsk_thread_flag(p, TIF_SIGPENDING);
 			__set_task_state(p, TASK_STOPPED);
 		} else {
 			wake_up_new_task(p, clone_flags);
 		}
 		tracehook_report_clone_complete(trace, regs,
 						clone_flags, nr, p);
 		if (clone_flags & CLONE_VFORK) {
 			freezer_do_not_count();
 			wait_for_completion(&vfork);
 			freezer_count();
 			tracehook_report_vfork_done(p, nr);
 		}
 	} else {
 		nr = PTR_ERR(p);
 	}
 	return nr;
 }
 #ifndef ARCH_MIN_MMSTRUCT_ALIGN
 #define ARCH_MIN_MMSTRUCT_ALIGN 0
 #endif
 static void sighand_ctor(void *data)
 {
 	struct sighand_struct *sighand = data;
 	spin_lock_init(&sighand->siglock);
 	init_waitqueue_head(&sighand->signalfd_wqh);
 }
 void __init proc_caches_init(void)
 {
 	sighand_cachep = kmem_cache_create("sighand_cache",
 			sizeof(struct sighand_struct), 0,
 			SLAB_HWCACHE_ALIGN|SLAB_PANIC|SLAB_DESTROY_BY_RCU,
 			sighand_ctor);
 	signal_cachep = kmem_cache_create("signal_cache",
 			sizeof(struct signal_struct), 0,
 			SLAB_HWCACHE_ALIGN|SLAB_PANIC, NULL);
 	files_cachep = kmem_cache_create("files_cache",
 			sizeof(struct files_struct), 0,
 			SLAB_HWCACHE_ALIGN|SLAB_PANIC, NULL);
 	fs_cachep = kmem_cache_create("fs_cache",
 			sizeof(struct fs_struct), 0,
 			SLAB_HWCACHE_ALIGN|SLAB_PANIC, NULL);
-	vm_area_cachep = kmem_cache_create("vm_area_struct",
-			sizeof(struct vm_area_struct), 0,
-			SLAB_PANIC, NULL);
 	mm_cachep = kmem_cache_create("mm_struct",
 			sizeof(struct mm_struct), ARCH_MIN_MMSTRUCT_ALIGN,
 			SLAB_HWCACHE_ALIGN|SLAB_PANIC, NULL);
+	mmap_init();
 }
 /*
  * Check constraints on flags passed to the unshare system call and
  * force unsharing of additional process context as appropriate.
  */
 static void check_unshare_flags(unsigned long *flags_ptr)
 {
 	/*
 	 * If unsharing a thread from a thread group, must also
 	 * unshare vm.
 	 */
 	if (*flags_ptr & CLONE_THREAD)
 		*flags_ptr |= CLONE_VM;
 	/*
 	 * If unsharing vm, must also unshare signal handlers.
 	 */
 	if (*flags_ptr & CLONE_VM)
 		*flags_ptr |= CLONE_SIGHAND;
 	/*
 	 * If unsharing signal handlers and the task was created
 	 * using CLONE_THREAD, then must unshare the thread
 	 */
 	if ((*flags_ptr & CLONE_SIGHAND) &&
 	    (atomic_read(&current->signal->count) > 1))
 		*flags_ptr |= CLONE_THREAD;
 	/*
 	 * If unsharing namespace, must also unshare filesystem information.
 	 */
 	if (*flags_ptr & CLONE_NEWNS)
 		*flags_ptr |= CLONE_FS;
 }
 /*
  * Unsharing of tasks created with CLONE_THREAD is not supported yet
  */
 static int unshare_thread(unsigned long unshare_flags)
 {
 	if (unshare_flags & CLONE_THREAD)
 		return -EINVAL;
 	return 0;
 }
 /*
  * Unshare the filesystem structure if it is being shared
  */
 static int unshare_fs(unsigned long unshare_flags, struct fs_struct **new_fsp)
 {
 	struct fs_struct *fs = current->fs;
 	if ((unshare_flags & CLONE_FS) &&
 	    (fs && atomic_read(&fs->count) > 1)) {
 		*new_fsp = __copy_fs_struct(current->fs);
 		if (!*new_fsp)
 			return -ENOMEM;
 	}
 	return 0;
 }
 /*
  * Unsharing of sighand is not supported yet
  */
 static int unshare_sighand(unsigned long unshare_flags, struct sighand_struct **new_sighp)
 {
 	struct sighand_struct *sigh = current->sighand;
 	if ((unshare_flags & CLONE_SIGHAND) && atomic_read(&sigh->count) > 1)
 		return -EINVAL;
 	else
 		return 0;
 }
 /*
  * Unshare vm if it is being shared
  */
 static int unshare_vm(unsigned long unshare_flags, struct mm_struct **new_mmp)
 {
 	struct mm_struct *mm = current->mm;
 	if ((unshare_flags & CLONE_VM) &&
 	    (mm && atomic_read(&mm->mm_users) > 1)) {
 		return -EINVAL;
 	}
 	return 0;
 }
 /*
  * Unshare file descriptor table if it is being shared
  */
 static int unshare_fd(unsigned long unshare_flags, struct files_struct **new_fdp)
 {
 	struct files_struct *fd = current->files;
 	int error = 0;
 	if ((unshare_flags & CLONE_FILES) &&
 	    (fd && atomic_read(&fd->count) > 1)) {
 		*new_fdp = dup_fd(fd, &error);
 		if (!*new_fdp)
 			return error;
 	}
 	return 0;
 }
 /*
  * unshare allows a process to 'unshare' part of the process
  * context which was originally shared using clone.  copy_*
  * functions used by do_fork() cannot be used here directly
  * because they modify an inactive task_struct that is being
  * constructed. Here we are modifying the current, active,
  * task_struct.
  */
 asmlinkage long sys_unshare(unsigned long unshare_flags)
 {
 	int err = 0;
 	struct fs_struct *fs, *new_fs = NULL;
 	struct sighand_struct *new_sigh = NULL;
 	struct mm_struct *mm, *new_mm = NULL, *active_mm = NULL;
 	struct files_struct *fd, *new_fd = NULL;
 	struct nsproxy *new_nsproxy = NULL;
 	int do_sysvsem = 0;
 	check_unshare_flags(&unshare_flags);
 	/* Return -EINVAL for all unsupported flags */
 	err = -EINVAL;
 	if (unshare_flags & ~(CLONE_THREAD|CLONE_FS|CLONE_NEWNS|CLONE_SIGHAND|
 				CLONE_VM|CLONE_FILES|CLONE_SYSVSEM|
 				CLONE_NEWUTS|CLONE_NEWIPC|CLONE_NEWNET))
 		goto bad_unshare_out;
 	/*
 	 * CLONE_NEWIPC must also detach from the undolist: after switching
 	 * to a new ipc namespace, the semaphore arrays from the old
 	 * namespace are unreachable.
 	 */
 	if (unshare_flags & (CLONE_NEWIPC|CLONE_SYSVSEM))
 		do_sysvsem = 1;
 	if ((err = unshare_thread(unshare_flags)))
 		goto bad_unshare_out;
 	if ((err = unshare_fs(unshare_flags, &new_fs)))
 		goto bad_unshare_cleanup_thread;
 	if ((err = unshare_sighand(unshare_flags, &new_sigh)))
 		goto bad_unshare_cleanup_fs;
 	if ((err = unshare_vm(unshare_flags, &new_mm)))
 		goto bad_unshare_cleanup_sigh;
 	if ((err = unshare_fd(unshare_flags, &new_fd)))
 		goto bad_unshare_cleanup_vm;
 	if ((err = unshare_nsproxy_namespaces(unshare_flags, &new_nsproxy,
 			new_fs)))
 		goto bad_unshare_cleanup_fd;
 	if (new_fs ||  new_mm || new_fd || do_sysvsem || new_nsproxy) {
 		if (do_sysvsem) {
 			/*
 			 * CLONE_SYSVSEM is equivalent to sys_exit().
 			 */
 			exit_sem(current);
 		}
 		if (new_nsproxy) {
 			switch_task_namespaces(current, new_nsproxy);
 			new_nsproxy = NULL;
 		}
 		task_lock(current);
 		if (new_fs) {
 			fs = current->fs;
 			current->fs = new_fs;
 			new_fs = fs;
 		}
 		if (new_mm) {
 			mm = current->mm;
 			active_mm = current->active_mm;
 			current->mm = new_mm;
 			current->active_mm = new_mm;
 			activate_mm(active_mm, new_mm);
 			new_mm = mm;
 		}
 		if (new_fd) {
 			fd = current->files;
 			current->files = new_fd;
 			new_fd = fd;
 		}
 		task_unlock(current);
 	}
 	if (new_nsproxy)
 		put_nsproxy(new_nsproxy);
 bad_unshare_cleanup_fd:
 	if (new_fd)
 		put_files_struct(new_fd);
 bad_unshare_cleanup_vm:
 	if (new_mm)
 		mmput(new_mm);
 bad_unshare_cleanup_sigh:
 	if (new_sigh)
 		if (atomic_dec_and_test(&new_sigh->count))
 			kmem_cache_free(sighand_cachep, new_sigh);
 bad_unshare_cleanup_fs:
 	if (new_fs)
 		put_fs_struct(new_fs);
 bad_unshare_cleanup_thread:
 bad_unshare_out:
 	return err;
 }
 /*
  *	Helper to unshare the files of the current task.
  *	We don't want to expose copy_files internals to
  *	the exec layer of the kernel.
  */
 int unshare_files(struct files_struct **displaced)
 {
 	struct task_struct *task = current;
 	struct files_struct *copy = NULL;
 	int error;
 	error = unshare_fd(CLONE_FILES, &copy);
 	if (error || !copy) {
 		*displaced = NULL;
 		return error;
 	}
 	*displaced = task->files;
 	task_lock(task);
 	task->files = copy;
 	task_unlock(task);
 	return 0;
 }

kernel/sysctl.c

Diff comments View file @ c40f6f8

 /*
  * sysctl.c: General linux system control interface
  *
  * Begun 24 March 1995, Stephen Tweedie
  * Added /proc support, Dec 1995
  * Added bdflush entry and intvec min/max checking, 2/23/96, Tom Dyas.
  * Added hooks for /proc/sys/net (minor, minor patch), 96/4/1, Mike Shaver.
  * Added kernel/java-{interpreter,appletviewer}, 96/5/10, Mike Shaver.
  * Dynamic registration fixes, Stephen Tweedie.
  * Added kswapd-interval, ctrl-alt-del, printk stuff, 1/8/97, Chris Horn.
  * Made sysctl support optional via CONFIG_SYSCTL, 1/10/97, Chris
  *  Horn.
  * Added proc_doulongvec_ms_jiffies_minmax, 09/08/99, Carlos H. Bauer.
  * Added proc_doulongvec_minmax, 09/08/99, Carlos H. Bauer.
  * Changed linked lists to use list.h instead of lists.h, 02/24/00, Bill
  *  Wendling.
  * The list_for_each() macro wasn't appropriate for the sysctl loop.
  *  Removed it and replaced it with older style, 03/23/00, Bill Wendling
  */
 #include <linux/module.h>
 #include <linux/mm.h>
 #include <linux/swap.h>
 #include <linux/slab.h>
 #include <linux/sysctl.h>
 #include <linux/proc_fs.h>
 #include <linux/security.h>
 #include <linux/ctype.h>
 #include <linux/utsname.h>
 #include <linux/smp_lock.h>
 #include <linux/fs.h>
 #include <linux/init.h>
 #include <linux/kernel.h>
 #include <linux/kobject.h>
 #include <linux/net.h>
 #include <linux/sysrq.h>
 #include <linux/highuid.h>
 #include <linux/writeback.h>
 #include <linux/hugetlb.h>
 #include <linux/initrd.h>
 #include <linux/key.h>
 #include <linux/times.h>
 #include <linux/limits.h>
 #include <linux/dcache.h>
 #include <linux/syscalls.h>
 #include <linux/vmstat.h>
 #include <linux/nfs_fs.h>
 #include <linux/acpi.h>
 #include <linux/reboot.h>
 #include <linux/ftrace.h>
 #include <asm/uaccess.h>
 #include <asm/processor.h>
 #ifdef CONFIG_X86
 #include <asm/nmi.h>
 #include <asm/stacktrace.h>
 #include <asm/io.h>
 #endif
 static int deprecated_sysctl_warning(struct __sysctl_args *args);
 #if defined(CONFIG_SYSCTL)
 /* External variables not in a header file. */
 extern int C_A_D;
 extern int print_fatal_signals;
 extern int sysctl_overcommit_memory;
 extern int sysctl_overcommit_ratio;
 extern int sysctl_panic_on_oom;
 extern int sysctl_oom_kill_allocating_task;
 extern int sysctl_oom_dump_tasks;
 extern int max_threads;
 extern int core_uses_pid;
 extern int suid_dumpable;
 extern char core_pattern[];
 extern int pid_max;
 extern int min_free_kbytes;
 extern int pid_max_min, pid_max_max;
 extern int sysctl_drop_caches;
 extern int percpu_pagelist_fraction;
 extern int compat_log;
 extern int latencytop_enabled;
 extern int sysctl_nr_open_min, sysctl_nr_open_max;
+#ifndef CONFIG_MMU
+extern int sysctl_nr_trim_pages;
+#endif
 #ifdef CONFIG_RCU_TORTURE_TEST
 extern int rcutorture_runnable;
 #endif /* #ifdef CONFIG_RCU_TORTURE_TEST */
 /* Constants used for minimum and  maximum */
 #ifdef CONFIG_DETECT_SOFTLOCKUP
 static int sixty = 60;
 static int neg_one = -1;
 #endif
 #if defined(CONFIG_MMU) && defined(CONFIG_FILE_LOCKING)
 static int two = 2;
 #endif
 static int zero;
 static int one = 1;
 static int one_hundred = 100;
 /* this is needed for the proc_dointvec_minmax for [fs_]overflow UID and GID */
 static int maxolduid = 65535;
 static int minolduid;
 static int min_percpu_pagelist_fract = 8;
 static int ngroups_max = NGROUPS_MAX;
 #ifdef CONFIG_MODULES
 extern char modprobe_path[];
 #endif
 #ifdef CONFIG_CHR_DEV_SG
 extern int sg_big_buff;
 #endif
 #ifdef CONFIG_SPARC
 #include <asm/system.h>
 #endif
 #ifdef CONFIG_SPARC64
 extern int sysctl_tsb_ratio;
 #endif
 #ifdef __hppa__
 extern int pwrsw_enabled;
 extern int unaligned_enabled;
 #endif
 #ifdef CONFIG_S390
 #ifdef CONFIG_MATHEMU
 extern int sysctl_ieee_emulation_warnings;
 #endif
 extern int sysctl_userprocess_debug;
 extern int spin_retry;
 #endif
 #ifdef CONFIG_BSD_PROCESS_ACCT
 extern int acct_parm[];
 #endif
 #ifdef CONFIG_IA64
 extern int no_unaligned_warning;
 #endif
 #ifdef CONFIG_RT_MUTEXES
 extern int max_lock_depth;
 #endif
 #ifdef CONFIG_PROC_SYSCTL
 static int proc_do_cad_pid(struct ctl_table *table, int write, struct file *filp,
 		  void __user *buffer, size_t *lenp, loff_t *ppos);
 static int proc_taint(struct ctl_table *table, int write, struct file *filp,
 			       void __user *buffer, size_t *lenp, loff_t *ppos);
 #endif
 static struct ctl_table root_table[];
 static struct ctl_table_root sysctl_table_root;
 static struct ctl_table_header root_table_header = {
 	.count = 1,
 	.ctl_table = root_table,
 	.ctl_entry = LIST_HEAD_INIT(sysctl_table_root.default_set.list),
 	.root = &sysctl_table_root,
 	.set = &sysctl_table_root.default_set,
 };
 static struct ctl_table_root sysctl_table_root = {
 	.root_list = LIST_HEAD_INIT(sysctl_table_root.root_list),
 	.default_set.list = LIST_HEAD_INIT(root_table_header.ctl_entry),
 };
 static struct ctl_table kern_table[];
 static struct ctl_table vm_table[];
 static struct ctl_table fs_table[];
 static struct ctl_table debug_table[];
 static struct ctl_table dev_table[];
 extern struct ctl_table random_table[];
 #ifdef CONFIG_INOTIFY_USER
 extern struct ctl_table inotify_table[];
 #endif
 #ifdef CONFIG_EPOLL
 extern struct ctl_table epoll_table[];
 #endif
 #ifdef HAVE_ARCH_PICK_MMAP_LAYOUT
 int sysctl_legacy_va_layout;
 #endif
 extern int prove_locking;
 extern int lock_stat;
 /* The default sysctl tables: */
 static struct ctl_table root_table[] = {
 	{
 		.ctl_name	= CTL_KERN,
 		.procname	= "kernel",
 		.mode		= 0555,
 		.child		= kern_table,
 	},
 	{
 		.ctl_name	= CTL_VM,
 		.procname	= "vm",
 		.mode		= 0555,
 		.child		= vm_table,
 	},
 	{
 		.ctl_name	= CTL_FS,
 		.procname	= "fs",
 		.mode		= 0555,
 		.child		= fs_table,
 	},
 	{
 		.ctl_name	= CTL_DEBUG,
 		.procname	= "debug",
 		.mode		= 0555,
 		.child		= debug_table,
 	},
 	{
 		.ctl_name	= CTL_DEV,
 		.procname	= "dev",
 		.mode		= 0555,
 		.child		= dev_table,
 	},
 /*
  * NOTE: do not add new entries to this table unless you have read
  * Documentation/sysctl/ctl_unnumbered.txt
  */
 	{ .ctl_name = 0 }
 };
 #ifdef CONFIG_SCHED_DEBUG
 static int min_sched_granularity_ns = 100000;		/* 100 usecs */
 static int max_sched_granularity_ns = NSEC_PER_SEC;	/* 1 second */
 static int min_wakeup_granularity_ns;			/* 0 usecs */
 static int max_wakeup_granularity_ns = NSEC_PER_SEC;	/* 1 second */
 #endif
 static struct ctl_table kern_table[] = {
 #ifdef CONFIG_SCHED_DEBUG
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "sched_min_granularity_ns",
 		.data		= &sysctl_sched_min_granularity,
 		.maxlen		= sizeof(unsigned int),
 		.mode		= 0644,
 		.proc_handler	= &sched_nr_latency_handler,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &min_sched_granularity_ns,
 		.extra2		= &max_sched_granularity_ns,
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "sched_latency_ns",
 		.data		= &sysctl_sched_latency,
 		.maxlen		= sizeof(unsigned int),
 		.mode		= 0644,
 		.proc_handler	= &sched_nr_latency_handler,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &min_sched_granularity_ns,
 		.extra2		= &max_sched_granularity_ns,
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "sched_wakeup_granularity_ns",
 		.data		= &sysctl_sched_wakeup_granularity,
 		.maxlen		= sizeof(unsigned int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec_minmax,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &min_wakeup_granularity_ns,
 		.extra2		= &max_wakeup_granularity_ns,
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "sched_shares_ratelimit",
 		.data		= &sysctl_sched_shares_ratelimit,
 		.maxlen		= sizeof(unsigned int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "sched_shares_thresh",
 		.data		= &sysctl_sched_shares_thresh,
 		.maxlen		= sizeof(unsigned int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec_minmax,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &zero,
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "sched_child_runs_first",
 		.data		= &sysctl_sched_child_runs_first,
 		.maxlen		= sizeof(unsigned int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "sched_features",
 		.data		= &sysctl_sched_features,
 		.maxlen		= sizeof(unsigned int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "sched_migration_cost",
 		.data		= &sysctl_sched_migration_cost,
 		.maxlen		= sizeof(unsigned int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "sched_nr_migrate",
 		.data		= &sysctl_sched_nr_migrate,
 		.maxlen		= sizeof(unsigned int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "sched_rt_period_us",
 		.data		= &sysctl_sched_rt_period,
 		.maxlen		= sizeof(unsigned int),
 		.mode		= 0644,
 		.proc_handler	= &sched_rt_handler,
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "sched_rt_runtime_us",
 		.data		= &sysctl_sched_rt_runtime,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &sched_rt_handler,
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "sched_compat_yield",
 		.data		= &sysctl_sched_compat_yield,
 		.maxlen		= sizeof(unsigned int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #ifdef CONFIG_PROVE_LOCKING
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "prove_locking",
 		.data		= &prove_locking,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 #ifdef CONFIG_LOCK_STAT
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "lock_stat",
 		.data		= &lock_stat,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 	{
 		.ctl_name	= KERN_PANIC,
 		.procname	= "panic",
 		.data		= &panic_timeout,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= KERN_CORE_USES_PID,
 		.procname	= "core_uses_pid",
 		.data		= &core_uses_pid,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= KERN_CORE_PATTERN,
 		.procname	= "core_pattern",
 		.data		= core_pattern,
 		.maxlen		= CORENAME_MAX_SIZE,
 		.mode		= 0644,
 		.proc_handler	= &proc_dostring,
 		.strategy	= &sysctl_string,
 	},
 #ifdef CONFIG_PROC_SYSCTL
 	{
 		.procname	= "tainted",
 		.maxlen 	= sizeof(long),
 		.mode		= 0644,
 		.proc_handler	= &proc_taint,
 	},
 #endif
 #ifdef CONFIG_LATENCYTOP
 	{
 		.procname	= "latencytop",
 		.data		= &latencytop_enabled,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 #ifdef CONFIG_BLK_DEV_INITRD
 	{
 		.ctl_name	= KERN_REALROOTDEV,
 		.procname	= "real-root-dev",
 		.data		= &real_root_dev,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "print-fatal-signals",
 		.data		= &print_fatal_signals,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #ifdef CONFIG_SPARC
 	{
 		.ctl_name	= KERN_SPARC_REBOOT,
 		.procname	= "reboot-cmd",
 		.data		= reboot_command,
 		.maxlen		= 256,
 		.mode		= 0644,
 		.proc_handler	= &proc_dostring,
 		.strategy	= &sysctl_string,
 	},
 	{
 		.ctl_name	= KERN_SPARC_STOP_A,
 		.procname	= "stop-a",
 		.data		= &stop_a_enabled,
 		.maxlen		= sizeof (int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= KERN_SPARC_SCONS_PWROFF,
 		.procname	= "scons-poweroff",
 		.data		= &scons_pwroff,
 		.maxlen		= sizeof (int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 #ifdef CONFIG_SPARC64
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "tsb-ratio",
 		.data		= &sysctl_tsb_ratio,
 		.maxlen		= sizeof (int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 #ifdef __hppa__
 	{
 		.ctl_name	= KERN_HPPA_PWRSW,
 		.procname	= "soft-power",
 		.data		= &pwrsw_enabled,
 		.maxlen		= sizeof (int),
 	 	.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= KERN_HPPA_UNALIGNED,
 		.procname	= "unaligned-trap",
 		.data		= &unaligned_enabled,
 		.maxlen		= sizeof (int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 	{
 		.ctl_name	= KERN_CTLALTDEL,
 		.procname	= "ctrl-alt-del",
 		.data		= &C_A_D,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #ifdef CONFIG_FUNCTION_TRACER
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "ftrace_enabled",
 		.data		= &ftrace_enabled,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &ftrace_enable_sysctl,
 	},
 #endif
 #ifdef CONFIG_STACK_TRACER
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "stack_tracer_enabled",
 		.data		= &stack_tracer_enabled,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &stack_trace_sysctl,
 	},
 #endif
 #ifdef CONFIG_TRACING
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "ftrace_dump_on_oops",
 		.data		= &ftrace_dump_on_oops,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 #ifdef CONFIG_MODULES
 	{
 		.ctl_name	= KERN_MODPROBE,
 		.procname	= "modprobe",
 		.data		= &modprobe_path,
 		.maxlen		= KMOD_PATH_LEN,
 		.mode		= 0644,
 		.proc_handler	= &proc_dostring,
 		.strategy	= &sysctl_string,
 	},
 #endif
 #if defined(CONFIG_HOTPLUG) && defined(CONFIG_NET)
 	{
 		.ctl_name	= KERN_HOTPLUG,
 		.procname	= "hotplug",
 		.data		= &uevent_helper,
 		.maxlen		= UEVENT_HELPER_PATH_LEN,
 		.mode		= 0644,
 		.proc_handler	= &proc_dostring,
 		.strategy	= &sysctl_string,
 	},
 #endif
 #ifdef CONFIG_CHR_DEV_SG
 	{
 		.ctl_name	= KERN_SG_BIG_BUFF,
 		.procname	= "sg-big-buff",
 		.data		= &sg_big_buff,
 		.maxlen		= sizeof (int),
 		.mode		= 0444,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 #ifdef CONFIG_BSD_PROCESS_ACCT
 	{
 		.ctl_name	= KERN_ACCT,
 		.procname	= "acct",
 		.data		= &acct_parm,
 		.maxlen		= 3*sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 #ifdef CONFIG_MAGIC_SYSRQ
 	{
 		.ctl_name	= KERN_SYSRQ,
 		.procname	= "sysrq",
 		.data		= &__sysrq_enabled,
 		.maxlen		= sizeof (int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 #ifdef CONFIG_PROC_SYSCTL
 	{
 		.procname	= "cad_pid",
 		.data		= NULL,
 		.maxlen		= sizeof (int),
 		.mode		= 0600,
 		.proc_handler	= &proc_do_cad_pid,
 	},
 #endif
 	{
 		.ctl_name	= KERN_MAX_THREADS,
 		.procname	= "threads-max",
 		.data		= &max_threads,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= KERN_RANDOM,
 		.procname	= "random",
 		.mode		= 0555,
 		.child		= random_table,
 	},
 	{
 		.ctl_name	= KERN_OVERFLOWUID,
 		.procname	= "overflowuid",
 		.data		= &overflowuid,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec_minmax,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &minolduid,
 		.extra2		= &maxolduid,
 	},
 	{
 		.ctl_name	= KERN_OVERFLOWGID,
 		.procname	= "overflowgid",
 		.data		= &overflowgid,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec_minmax,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &minolduid,
 		.extra2		= &maxolduid,
 	},
 #ifdef CONFIG_S390
 #ifdef CONFIG_MATHEMU
 	{
 		.ctl_name	= KERN_IEEE_EMULATION_WARNINGS,
 		.procname	= "ieee_emulation_warnings",
 		.data		= &sysctl_ieee_emulation_warnings,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 	{
 		.ctl_name	= KERN_S390_USER_DEBUG_LOGGING,
 		.procname	= "userprocess_debug",
 		.data		= &sysctl_userprocess_debug,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 	{
 		.ctl_name	= KERN_PIDMAX,
 		.procname	= "pid_max",
 		.data		= &pid_max,
 		.maxlen		= sizeof (int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec_minmax,
 		.strategy	= sysctl_intvec,
 		.extra1		= &pid_max_min,
 		.extra2		= &pid_max_max,
 	},
 	{
 		.ctl_name	= KERN_PANIC_ON_OOPS,
 		.procname	= "panic_on_oops",
 		.data		= &panic_on_oops,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #if defined CONFIG_PRINTK
 	{
 		.ctl_name	= KERN_PRINTK,
 		.procname	= "printk",
 		.data		= &console_loglevel,
 		.maxlen		= 4*sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= KERN_PRINTK_RATELIMIT,
 		.procname	= "printk_ratelimit",
 		.data		= &printk_ratelimit_state.interval,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec_jiffies,
 		.strategy	= &sysctl_jiffies,
 	},
 	{
 		.ctl_name	= KERN_PRINTK_RATELIMIT_BURST,
 		.procname	= "printk_ratelimit_burst",
 		.data		= &printk_ratelimit_state.burst,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 	{
 		.ctl_name	= KERN_NGROUPS_MAX,
 		.procname	= "ngroups_max",
 		.data		= &ngroups_max,
 		.maxlen		= sizeof (int),
 		.mode		= 0444,
 		.proc_handler	= &proc_dointvec,
 	},
 #if defined(CONFIG_X86_LOCAL_APIC) && defined(CONFIG_X86)
 	{
 		.ctl_name       = KERN_UNKNOWN_NMI_PANIC,
 		.procname       = "unknown_nmi_panic",
 		.data           = &unknown_nmi_panic,
 		.maxlen         = sizeof (int),
 		.mode           = 0644,
 		.proc_handler   = &proc_dointvec,
 	},
 	{
 		.procname       = "nmi_watchdog",
 		.data           = &nmi_watchdog_enabled,
 		.maxlen         = sizeof (int),
 		.mode           = 0644,
 		.proc_handler   = &proc_nmi_enabled,
 	},
 #endif
 #if defined(CONFIG_X86)
 	{
 		.ctl_name	= KERN_PANIC_ON_NMI,
 		.procname	= "panic_on_unrecovered_nmi",
 		.data		= &panic_on_unrecovered_nmi,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= KERN_BOOTLOADER_TYPE,
 		.procname	= "bootloader_type",
 		.data		= &bootloader_type,
 		.maxlen		= sizeof (int),
 		.mode		= 0444,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "kstack_depth_to_print",
 		.data		= &kstack_depth_to_print,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "io_delay_type",
 		.data		= &io_delay_type,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 #if defined(CONFIG_MMU)
 	{
 		.ctl_name	= KERN_RANDOMIZE,
 		.procname	= "randomize_va_space",
 		.data		= &randomize_va_space,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 #if defined(CONFIG_S390) && defined(CONFIG_SMP)
 	{
 		.ctl_name	= KERN_SPIN_RETRY,
 		.procname	= "spin_retry",
 		.data		= &spin_retry,
 		.maxlen		= sizeof (int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 #if	defined(CONFIG_ACPI_SLEEP) && defined(CONFIG_X86)
 	{
 		.procname	= "acpi_video_flags",
 		.data		= &acpi_realmode_flags,
 		.maxlen		= sizeof (unsigned long),
 		.mode		= 0644,
 		.proc_handler	= &proc_doulongvec_minmax,
 	},
 #endif
 #ifdef CONFIG_IA64
 	{
 		.ctl_name	= KERN_IA64_UNALIGNED,
 		.procname	= "ignore-unaligned-usertrap",
 		.data		= &no_unaligned_warning,
 		.maxlen		= sizeof (int),
 	 	.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 #ifdef CONFIG_DETECT_SOFTLOCKUP
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "softlockup_panic",
 		.data		= &softlockup_panic,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec_minmax,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &zero,
 		.extra2		= &one,
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "softlockup_thresh",
 		.data		= &softlockup_thresh,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec_minmax,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &neg_one,
 		.extra2		= &sixty,
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "hung_task_check_count",
 		.data		= &sysctl_hung_task_check_count,
 		.maxlen		= sizeof(unsigned long),
 		.mode		= 0644,
 		.proc_handler	= &proc_doulongvec_minmax,
 		.strategy	= &sysctl_intvec,
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "hung_task_timeout_secs",
 		.data		= &sysctl_hung_task_timeout_secs,
 		.maxlen		= sizeof(unsigned long),
 		.mode		= 0644,
 		.proc_handler	= &proc_doulongvec_minmax,
 		.strategy	= &sysctl_intvec,
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "hung_task_warnings",
 		.data		= &sysctl_hung_task_warnings,
 		.maxlen		= sizeof(unsigned long),
 		.mode		= 0644,
 		.proc_handler	= &proc_doulongvec_minmax,
 		.strategy	= &sysctl_intvec,
 	},
 #endif
 #ifdef CONFIG_COMPAT
 	{
 		.ctl_name	= KERN_COMPAT_LOG,
 		.procname	= "compat-log",
 		.data		= &compat_log,
 		.maxlen		= sizeof (int),
 	 	.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 #ifdef CONFIG_RT_MUTEXES
 	{
 		.ctl_name	= KERN_MAX_LOCK_DEPTH,
 		.procname	= "max_lock_depth",
 		.data		= &max_lock_depth,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "poweroff_cmd",
 		.data		= &poweroff_cmd,
 		.maxlen		= POWEROFF_CMD_PATH_LEN,
 		.mode		= 0644,
 		.proc_handler	= &proc_dostring,
 		.strategy	= &sysctl_string,
 	},
 #ifdef CONFIG_KEYS
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "keys",
 		.mode		= 0555,
 		.child		= key_sysctls,
 	},
 #endif
 #ifdef CONFIG_RCU_TORTURE_TEST
 	{
 		.ctl_name       = CTL_UNNUMBERED,
 		.procname       = "rcutorture_runnable",
 		.data           = &rcutorture_runnable,
 		.maxlen         = sizeof(int),
 		.mode           = 0644,
 		.proc_handler   = &proc_dointvec,
 	},
 #endif
 #ifdef CONFIG_UNEVICTABLE_LRU
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "scan_unevictable_pages",
 		.data		= &scan_unevictable_pages,
 		.maxlen		= sizeof(scan_unevictable_pages),
 		.mode		= 0644,
 		.proc_handler	= &scan_unevictable_handler,
 	},
 #endif
 /*
  * NOTE: do not add new entries to this table unless you have read
  * Documentation/sysctl/ctl_unnumbered.txt
  */
 	{ .ctl_name = 0 }
 };
 static struct ctl_table vm_table[] = {
 	{
 		.ctl_name	= VM_OVERCOMMIT_MEMORY,
 		.procname	= "overcommit_memory",
 		.data		= &sysctl_overcommit_memory,
 		.maxlen		= sizeof(sysctl_overcommit_memory),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= VM_PANIC_ON_OOM,
 		.procname	= "panic_on_oom",
 		.data		= &sysctl_panic_on_oom,
 		.maxlen		= sizeof(sysctl_panic_on_oom),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "oom_kill_allocating_task",
 		.data		= &sysctl_oom_kill_allocating_task,
 		.maxlen		= sizeof(sysctl_oom_kill_allocating_task),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "oom_dump_tasks",
 		.data		= &sysctl_oom_dump_tasks,
 		.maxlen		= sizeof(sysctl_oom_dump_tasks),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= VM_OVERCOMMIT_RATIO,
 		.procname	= "overcommit_ratio",
 		.data		= &sysctl_overcommit_ratio,
 		.maxlen		= sizeof(sysctl_overcommit_ratio),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= VM_PAGE_CLUSTER,
 		.procname	= "page-cluster",
 		.data		= &page_cluster,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= VM_DIRTY_BACKGROUND,
 		.procname	= "dirty_background_ratio",
 		.data		= &dirty_background_ratio,
 		.maxlen		= sizeof(dirty_background_ratio),
 		.mode		= 0644,
 		.proc_handler	= &dirty_background_ratio_handler,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &zero,
 		.extra2		= &one_hundred,
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "dirty_background_bytes",
 		.data		= &dirty_background_bytes,
 		.maxlen		= sizeof(dirty_background_bytes),
 		.mode		= 0644,
 		.proc_handler	= &dirty_background_bytes_handler,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &one,
 	},
 	{
 		.ctl_name	= VM_DIRTY_RATIO,
 		.procname	= "dirty_ratio",
 		.data		= &vm_dirty_ratio,
 		.maxlen		= sizeof(vm_dirty_ratio),
 		.mode		= 0644,
 		.proc_handler	= &dirty_ratio_handler,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &zero,
 		.extra2		= &one_hundred,
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "dirty_bytes",
 		.data		= &vm_dirty_bytes,
 		.maxlen		= sizeof(vm_dirty_bytes),
 		.mode		= 0644,
 		.proc_handler	= &dirty_bytes_handler,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &one,
 	},
 	{
 		.procname	= "dirty_writeback_centisecs",
 		.data		= &dirty_writeback_interval,
 		.maxlen		= sizeof(dirty_writeback_interval),
 		.mode		= 0644,
 		.proc_handler	= &dirty_writeback_centisecs_handler,
 	},
 	{
 		.procname	= "dirty_expire_centisecs",
 		.data		= &dirty_expire_interval,
 		.maxlen		= sizeof(dirty_expire_interval),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec_userhz_jiffies,
 	},
 	{
 		.ctl_name	= VM_NR_PDFLUSH_THREADS,
 		.procname	= "nr_pdflush_threads",
 		.data		= &nr_pdflush_threads,
 		.maxlen		= sizeof nr_pdflush_threads,
 		.mode		= 0444 /* read-only*/,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= VM_SWAPPINESS,
 		.procname	= "swappiness",
 		.data		= &vm_swappiness,
 		.maxlen		= sizeof(vm_swappiness),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec_minmax,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &zero,
 		.extra2		= &one_hundred,
 	},
 #ifdef CONFIG_HUGETLB_PAGE
 	 {
 		.procname	= "nr_hugepages",
 		.data		= NULL,
 		.maxlen		= sizeof(unsigned long),
 		.mode		= 0644,
 		.proc_handler	= &hugetlb_sysctl_handler,
 		.extra1		= (void *)&hugetlb_zero,
 		.extra2		= (void *)&hugetlb_infinity,
 	 },
 	 {
 		.ctl_name	= VM_HUGETLB_GROUP,
 		.procname	= "hugetlb_shm_group",
 		.data		= &sysctl_hugetlb_shm_group,
 		.maxlen		= sizeof(gid_t),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	 },
 	 {
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "hugepages_treat_as_movable",
 		.data		= &hugepages_treat_as_movable,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &hugetlb_treat_movable_handler,
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "nr_overcommit_hugepages",
 		.data		= NULL,
 		.maxlen		= sizeof(unsigned long),
 		.mode		= 0644,
 		.proc_handler	= &hugetlb_overcommit_handler,
 		.extra1		= (void *)&hugetlb_zero,
 		.extra2		= (void *)&hugetlb_infinity,
 	},
 #endif
 	{
 		.ctl_name	= VM_LOWMEM_RESERVE_RATIO,
 		.procname	= "lowmem_reserve_ratio",
 		.data		= &sysctl_lowmem_reserve_ratio,
 		.maxlen		= sizeof(sysctl_lowmem_reserve_ratio),
 		.mode		= 0644,
 		.proc_handler	= &lowmem_reserve_ratio_sysctl_handler,
 		.strategy	= &sysctl_intvec,
 	},
 	{
 		.ctl_name	= VM_DROP_PAGECACHE,
 		.procname	= "drop_caches",
 		.data		= &sysctl_drop_caches,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= drop_caches_sysctl_handler,
 		.strategy	= &sysctl_intvec,
 	},
 	{
 		.ctl_name	= VM_MIN_FREE_KBYTES,
 		.procname	= "min_free_kbytes",
 		.data		= &min_free_kbytes,
 		.maxlen		= sizeof(min_free_kbytes),
 		.mode		= 0644,
 		.proc_handler	= &min_free_kbytes_sysctl_handler,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &zero,
 	},
 	{
 		.ctl_name	= VM_PERCPU_PAGELIST_FRACTION,
 		.procname	= "percpu_pagelist_fraction",
 		.data		= &percpu_pagelist_fraction,
 		.maxlen		= sizeof(percpu_pagelist_fraction),
 		.mode		= 0644,
 		.proc_handler	= &percpu_pagelist_fraction_sysctl_handler,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &min_percpu_pagelist_fract,
 	},
 #ifdef CONFIG_MMU
 	{
 		.ctl_name	= VM_MAX_MAP_COUNT,
 		.procname	= "max_map_count",
 		.data		= &sysctl_max_map_count,
 		.maxlen		= sizeof(sysctl_max_map_count),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec
+	},
+#else
+	{
+		.ctl_name	= CTL_UNNUMBERED,
+		.procname	= "nr_trim_pages",
+		.data		= &sysctl_nr_trim_pages,
+		.maxlen		= sizeof(sysctl_nr_trim_pages),
+		.mode		= 0644,
+		.proc_handler	= &proc_dointvec_minmax,
+		.strategy	= &sysctl_intvec,
+		.extra1		= &zero,
 	},
 #endif
 	{
 		.ctl_name	= VM_LAPTOP_MODE,
 		.procname	= "laptop_mode",
 		.data		= &laptop_mode,
 		.maxlen		= sizeof(laptop_mode),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec_jiffies,
 		.strategy	= &sysctl_jiffies,
 	},
 	{
 		.ctl_name	= VM_BLOCK_DUMP,
 		.procname	= "block_dump",
 		.data		= &block_dump,
 		.maxlen		= sizeof(block_dump),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &zero,
 	},
 	{
 		.ctl_name	= VM_VFS_CACHE_PRESSURE,
 		.procname	= "vfs_cache_pressure",
 		.data		= &sysctl_vfs_cache_pressure,
 		.maxlen		= sizeof(sysctl_vfs_cache_pressure),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &zero,
 	},
 #ifdef HAVE_ARCH_PICK_MMAP_LAYOUT
 	{
 		.ctl_name	= VM_LEGACY_VA_LAYOUT,
 		.procname	= "legacy_va_layout",
 		.data		= &sysctl_legacy_va_layout,
 		.maxlen		= sizeof(sysctl_legacy_va_layout),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &zero,
 	},
 #endif
 #ifdef CONFIG_NUMA
 	{
 		.ctl_name	= VM_ZONE_RECLAIM_MODE,
 		.procname	= "zone_reclaim_mode",
 		.data		= &zone_reclaim_mode,
 		.maxlen		= sizeof(zone_reclaim_mode),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &zero,
 	},
 	{
 		.ctl_name	= VM_MIN_UNMAPPED,
 		.procname	= "min_unmapped_ratio",
 		.data		= &sysctl_min_unmapped_ratio,
 		.maxlen		= sizeof(sysctl_min_unmapped_ratio),
 		.mode		= 0644,
 		.proc_handler	= &sysctl_min_unmapped_ratio_sysctl_handler,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &zero,
 		.extra2		= &one_hundred,
 	},
 	{
 		.ctl_name	= VM_MIN_SLAB,
 		.procname	= "min_slab_ratio",
 		.data		= &sysctl_min_slab_ratio,
 		.maxlen		= sizeof(sysctl_min_slab_ratio),
 		.mode		= 0644,
 		.proc_handler	= &sysctl_min_slab_ratio_sysctl_handler,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &zero,
 		.extra2		= &one_hundred,
 	},
 #endif
 #ifdef CONFIG_SMP
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "stat_interval",
 		.data		= &sysctl_stat_interval,
 		.maxlen		= sizeof(sysctl_stat_interval),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec_jiffies,
 		.strategy	= &sysctl_jiffies,
 	},
 #endif
 #ifdef CONFIG_SECURITY
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "mmap_min_addr",
 		.data		= &mmap_min_addr,
 		.maxlen         = sizeof(unsigned long),
 		.mode		= 0644,
 		.proc_handler	= &proc_doulongvec_minmax,
 	},
 #endif
 #ifdef CONFIG_NUMA
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "numa_zonelist_order",
 		.data		= &numa_zonelist_order,
 		.maxlen		= NUMA_ZONELIST_ORDER_LEN,
 		.mode		= 0644,
 		.proc_handler	= &numa_zonelist_order_handler,
 		.strategy	= &sysctl_string,
 	},
 #endif
 #if (defined(CONFIG_X86_32) && !defined(CONFIG_UML))|| \
    (defined(CONFIG_SUPERH) && defined(CONFIG_VSYSCALL))
 	{
 		.ctl_name	= VM_VDSO_ENABLED,
 		.procname	= "vdso_enabled",
 		.data		= &vdso_enabled,
 		.maxlen		= sizeof(vdso_enabled),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &zero,
 	},
 #endif
 #ifdef CONFIG_HIGHMEM
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "highmem_is_dirtyable",
 		.data		= &vm_highmem_is_dirtyable,
 		.maxlen		= sizeof(vm_highmem_is_dirtyable),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec_minmax,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &zero,
 		.extra2		= &one,
 	},
 #endif
 /*
  * NOTE: do not add new entries to this table unless you have read
  * Documentation/sysctl/ctl_unnumbered.txt
  */
 	{ .ctl_name = 0 }
 };
 #if defined(CONFIG_BINFMT_MISC) || defined(CONFIG_BINFMT_MISC_MODULE)
 static struct ctl_table binfmt_misc_table[] = {
 	{ .ctl_name = 0 }
 };
 #endif
 static struct ctl_table fs_table[] = {
 	{
 		.ctl_name	= FS_NRINODE,
 		.procname	= "inode-nr",
 		.data		= &inodes_stat,
 		.maxlen		= 2*sizeof(int),
 		.mode		= 0444,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= FS_STATINODE,
 		.procname	= "inode-state",
 		.data		= &inodes_stat,
 		.maxlen		= 7*sizeof(int),
 		.mode		= 0444,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.procname	= "file-nr",
 		.data		= &files_stat,
 		.maxlen		= 3*sizeof(int),
 		.mode		= 0444,
 		.proc_handler	= &proc_nr_files,
 	},
 	{
 		.ctl_name	= FS_MAXFILE,
 		.procname	= "file-max",
 		.data		= &files_stat.max_files,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "nr_open",
 		.data		= &sysctl_nr_open,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec_minmax,
 		.extra1		= &sysctl_nr_open_min,
 		.extra2		= &sysctl_nr_open_max,
 	},
 	{
 		.ctl_name	= FS_DENTRY,
 		.procname	= "dentry-state",
 		.data		= &dentry_stat,
 		.maxlen		= 6*sizeof(int),
 		.mode		= 0444,
 		.proc_handler	= &proc_dointvec,
 	},
 	{
 		.ctl_name	= FS_OVERFLOWUID,
 		.procname	= "overflowuid",
 		.data		= &fs_overflowuid,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec_minmax,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &minolduid,
 		.extra2		= &maxolduid,
 	},
 	{
 		.ctl_name	= FS_OVERFLOWGID,
 		.procname	= "overflowgid",
 		.data		= &fs_overflowgid,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec_minmax,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &minolduid,
 		.extra2		= &maxolduid,
 	},
 #ifdef CONFIG_FILE_LOCKING
 	{
 		.ctl_name	= FS_LEASES,
 		.procname	= "leases-enable",
 		.data		= &leases_enable,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 #ifdef CONFIG_DNOTIFY
 	{
 		.ctl_name	= FS_DIR_NOTIFY,
 		.procname	= "dir-notify-enable",
 		.data		= &dir_notify_enable,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #endif
 #ifdef CONFIG_MMU
 #ifdef CONFIG_FILE_LOCKING
 	{
 		.ctl_name	= FS_LEASE_TIME,
 		.procname	= "lease-break-time",
 		.data		= &lease_break_time,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec_minmax,
 		.strategy	= &sysctl_intvec,
 		.extra1		= &zero,
 		.extra2		= &two,
 	},
 #endif
 #ifdef CONFIG_AIO
 	{
 		.procname	= "aio-nr",
 		.data		= &aio_nr,
 		.maxlen		= sizeof(aio_nr),
 		.mode		= 0444,
 		.proc_handler	= &proc_doulongvec_minmax,
 	},
 	{
 		.procname	= "aio-max-nr",
 		.data		= &aio_max_nr,
 		.maxlen		= sizeof(aio_max_nr),
 		.mode		= 0644,
 		.proc_handler	= &proc_doulongvec_minmax,
 	},
 #endif /* CONFIG_AIO */
 #ifdef CONFIG_INOTIFY_USER
 	{
 		.ctl_name	= FS_INOTIFY,
 		.procname	= "inotify",
 		.mode		= 0555,
 		.child		= inotify_table,
 	},
 #endif
 #ifdef CONFIG_EPOLL
 	{
 		.procname	= "epoll",
 		.mode		= 0555,
 		.child		= epoll_table,
 	},
 #endif
 #endif
 	{
 		.ctl_name	= KERN_SETUID_DUMPABLE,
 		.procname	= "suid_dumpable",
 		.data		= &suid_dumpable,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= &proc_dointvec,
 	},
 #if defined(CONFIG_BINFMT_MISC) || defined(CONFIG_BINFMT_MISC_MODULE)
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "binfmt_misc",
 		.mode		= 0555,
 		.child		= binfmt_misc_table,
 	},
 #endif
 /*
  * NOTE: do not add new entries to this table unless you have read
  * Documentation/sysctl/ctl_unnumbered.txt
  */
 	{ .ctl_name = 0 }
 };
 static struct ctl_table debug_table[] = {
 #if defined(CONFIG_X86) || defined(CONFIG_PPC)
 	{
 		.ctl_name	= CTL_UNNUMBERED,
 		.procname	= "exception-trace",
 		.data		= &show_unhandled_signals,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
 		.proc_handler	= proc_dointvec
 	},
 #endif
 	{ .ctl_name = 0 }
 };
 static struct ctl_table dev_table[] = {
 	{ .ctl_name = 0 }
 };
 static DEFINE_SPINLOCK(sysctl_lock);
 /* called under sysctl_lock */
 static int use_table(struct ctl_table_header *p)
 {
 	if (unlikely(p->unregistering))
 		return 0;
 	p->used++;
 	return 1;
 }
 /* called under sysctl_lock */
 static void unuse_table(struct ctl_table_header *p)
 {
 	if (!--p->used)
 		if (unlikely(p->unregistering))
 			complete(p->unregistering);
 }
 /* called under sysctl_lock, will reacquire if has to wait */
 static void start_unregistering(struct ctl_table_header *p)
 {
 	/*
 	 * if p->used is 0, nobody will ever touch that entry again;
 	 * we'll eliminate all paths to it before dropping sysctl_lock
 	 */
 	if (unlikely(p->used)) {
 		struct completion wait;
 		init_completion(&wait);
 		p->unregistering = &wait;
 		spin_unlock(&sysctl_lock);
 		wait_for_completion(&wait);
 		spin_lock(&sysctl_lock);
 	} else {
 		/* anything non-NULL; we'll never dereference it */
 		p->unregistering = ERR_PTR(-EINVAL);
 	}
 	/*
 	 * do not remove from the list until nobody holds it; walking the
 	 * list in do_sysctl() relies on that.
 	 */
 	list_del_init(&p->ctl_entry);
 }
 void sysctl_head_get(struct ctl_table_header *head)
 {
 	spin_lock(&sysctl_lock);
 	head->count++;
 	spin_unlock(&sysctl_lock);
 }
 void sysctl_head_put(struct ctl_table_header *head)
 {
 	spin_lock(&sysctl_lock);
 	if (!--head->count)
 		kfree(head);
 	spin_unlock(&sysctl_lock);
 }
 struct ctl_table_header *sysctl_head_grab(struct ctl_table_header *head)
 {
 	if (!head)
 		BUG();
 	spin_lock(&sysctl_lock);
 	if (!use_table(head))
 		head = ERR_PTR(-ENOENT);
 	spin_unlock(&sysctl_lock);
 	return head;
 }
 void sysctl_head_finish(struct ctl_table_header *head)
 {
 	if (!head)
 		return;
 	spin_lock(&sysctl_lock);
 	unuse_table(head);
 	spin_unlock(&sysctl_lock);
 }
 static struct ctl_table_set *
 lookup_header_set(struct ctl_table_root *root, struct nsproxy *namespaces)
 {
 	struct ctl_table_set *set = &root->default_set;
 	if (root->lookup)
 		set = root->lookup(root, namespaces);
 	return set;
 }
 static struct list_head *
 lookup_header_list(struct ctl_table_root *root, struct nsproxy *namespaces)
 {
 	struct ctl_table_set *set = lookup_header_set(root, namespaces);
 	return &set->list;
 }
 struct ctl_table_header *__sysctl_head_next(struct nsproxy *namespaces,
 					    struct ctl_table_header *prev)
 {
 	struct ctl_table_root *root;
 	struct list_head *header_list;
 	struct ctl_table_header *head;
 	struct list_head *tmp;
 	spin_lock(&sysctl_lock);
 	if (prev) {
 		head = prev;
 		tmp = &prev->ctl_entry;
 		unuse_table(prev);
 		goto next;
 	}
 	tmp = &root_table_header.ctl_entry;
 	for (;;) {
 		head = list_entry(tmp, struct ctl_table_header, ctl_entry);
 		if (!use_table(head))
 			goto next;
 		spin_unlock(&sysctl_lock);
 		return head;
 	next:
 		root = head->root;
 		tmp = tmp->next;
 		header_list = lookup_header_list(root, namespaces);
 		if (tmp != header_list)
 			continue;
 		do {
 			root = list_entry(root->root_list.next,
 					struct ctl_table_root, root_list);
 			if (root == &sysctl_table_root)
 				goto out;
 			header_list = lookup_header_list(root, namespaces);
 		} while (list_empty(header_list));
 		tmp = header_list->next;
 	}
 out:
 	spin_unlock(&sysctl_lock);
 	return NULL;
 }
 struct ctl_table_header *sysctl_head_next(struct ctl_table_header *prev)
 {
 	return __sysctl_head_next(current->nsproxy, prev);
 }
 void register_sysctl_root(struct ctl_table_root *root)
 {
 	spin_lock(&sysctl_lock);
 	list_add_tail(&root->root_list, &sysctl_table_root.root_list);
 	spin_unlock(&sysctl_lock);
 }
 #ifdef CONFIG_SYSCTL_SYSCALL
 /* Perform the actual read/write of a sysctl table entry. */
 static int do_sysctl_strategy(struct ctl_table_root *root,
 			struct ctl_table *table,
 			void __user *oldval, size_t __user *oldlenp,
 			void __user *newval, size_t newlen)
 {
 	int op = 0, rc;
 	if (oldval)
 		op |= MAY_READ;
 	if (newval)
 		op |= MAY_WRITE;
 	if (sysctl_perm(root, table, op))
 		return -EPERM;
 	if (table->strategy) {
 		rc = table->strategy(table, oldval, oldlenp, newval, newlen);
 		if (rc < 0)
 			return rc;
 		if (rc > 0)
 			return 0;
 	}
 	/* If there is no strategy routine, or if the strategy returns
 	 * zero, proceed with automatic r/w */
 	if (table->data && table->maxlen) {
 		rc = sysctl_data(table, oldval, oldlenp, newval, newlen);
 		if (rc < 0)
 			return rc;
 	}
 	return 0;
 }
 static int parse_table(int __user *name, int nlen,
 		       void __user *oldval, size_t __user *oldlenp,
 		       void __user *newval, size_t newlen,
 		       struct ctl_table_root *root,
 		       struct ctl_table *table)
 {
 	int n;
 repeat:
 	if (!nlen)
 		return -ENOTDIR;
 	if (get_user(n, name))
 		return -EFAULT;
 	for ( ; table->ctl_name || table->procname; table++) {
 		if (!table->ctl_name)
 			continue;
 		if (n == table->ctl_name) {
 			int error;
 			if (table->child) {
 				if (sysctl_perm(root, table, MAY_EXEC))
 					return -EPERM;
 				name++;
 				nlen--;
 				table = table->child;
 				goto repeat;
 			}
 			error = do_sysctl_strategy(root, table,
 						   oldval, oldlenp,
 						   newval, newlen);
 			return error;
 		}
 	}
 	return -ENOTDIR;
 }
 int do_sysctl(int __user *name, int nlen, void __user *oldval, size_t __user *oldlenp,
 	       void __user *newval, size_t newlen)
 {
 	struct ctl_table_header *head;
 	int error = -ENOTDIR;
 	if (nlen <= 0 || nlen >= CTL_MAXNAME)
 		return -ENOTDIR;
 	if (oldval) {
 		int old_len;
 		if (!oldlenp || get_user(old_len, oldlenp))
 			return -EFAULT;
 	}
 	for (head = sysctl_head_next(NULL); head;
 			head = sysctl_head_next(head)) {
 		error = parse_table(name, nlen, oldval, oldlenp,
 					newval, newlen,
 					head->root, head->ctl_table);
 		if (error != -ENOTDIR) {
 			sysctl_head_finish(head);
 			break;
 		}
 	}
 	return error;
 }
 asmlinkage long sys_sysctl(struct __sysctl_args __user *args)
 {
 	struct __sysctl_args tmp;
 	int error;
 	if (copy_from_user(&tmp, args, sizeof(tmp)))
 		return -EFAULT;
 	error = deprecated_sysctl_warning(&tmp);
 	if (error)
 		goto out;
 	lock_kernel();
 	error = do_sysctl(tmp.name, tmp.nlen, tmp.oldval, tmp.oldlenp,
 			  tmp.newval, tmp.newlen);
 	unlock_kernel();
 out:
 	return error;
 }
 #endif /* CONFIG_SYSCTL_SYSCALL */
 /*
  * sysctl_perm does NOT grant the superuser all rights automatically, because
  * some sysctl variables are readonly even to root.
  */
 static int test_perm(int mode, int op)
 {
 	if (!current_euid())
 		mode >>= 6;
 	else if (in_egroup_p(0))
 		mode >>= 3;
 	if ((op & ~mode & (MAY_READ|MAY_WRITE|MAY_EXEC)) == 0)
 		return 0;
 	return -EACCES;
 }
 int sysctl_perm(struct ctl_table_root *root, struct ctl_table *table, int op)
 {
 	int error;
 	int mode;
 	error = security_sysctl(table, op & (MAY_READ | MAY_WRITE | MAY_EXEC));
 	if (error)
 		return error;
 	if (root->permissions)
 		mode = root->permissions(root, current->nsproxy, table);
 	else
 		mode = table->mode;
 	return test_perm(mode, op);
 }
 static void sysctl_set_parent(struct ctl_table *parent, struct ctl_table *table)
 {
 	for (; table->ctl_name || table->procname; table++) {
 		table->parent = parent;
 		if (table->child)
 			sysctl_set_parent(table, table->child);
 	}
 }
 static __init int sysctl_init(void)
 {
 	sysctl_set_parent(NULL, root_table);
 #ifdef CONFIG_SYSCTL_SYSCALL_CHECK
 	{
 		int err;
 		err = sysctl_check_table(current->nsproxy, root_table);
 	}
 #endif
 	return 0;
 }
 core_initcall(sysctl_init);
 static struct ctl_table *is_branch_in(struct ctl_table *branch,
 				      struct ctl_table *table)
 {
 	struct ctl_table *p;
 	const char *s = branch->procname;
 	/* branch should have named subdirectory as its first element */
 	if (!s || !branch->child)
 		return NULL;
 	/* ... and nothing else */
 	if (branch[1].procname || branch[1].ctl_name)
 		return NULL;
 	/* table should contain subdirectory with the same name */
 	for (p = table; p->procname || p->ctl_name; p++) {
 		if (!p->child)
 			continue;
 		if (p->procname && strcmp(p->procname, s) == 0)
 			return p;
 	}
 	return NULL;
 }
 /* see if attaching q to p would be an improvement */
 static void try_attach(struct ctl_table_header *p, struct ctl_table_header *q)
 {
 	struct ctl_table *to = p->ctl_table, *by = q->ctl_table;
 	struct ctl_table *next;
 	int is_better = 0;
 	int not_in_parent = !p->attached_by;
 	while ((next = is_branch_in(by, to)) != NULL) {
 		if (by == q->attached_by)
 			is_better = 1;
 		if (to == p->attached_by)
 			not_in_parent = 1;
 		by = by->child;
 		to = next->child;
 	}
 	if (is_better && not_in_parent) {
 		q->attached_by = by;
 		q->attached_to = to;
 		q->parent = p;
 	}
 }
 /**
  * __register_sysctl_paths - register a sysctl hierarchy
  * @root: List of sysctl headers to register on
  * @namespaces: Data to compute which lists of sysctl entries are visible
  * @path: The path to the directory the sysctl table is in.
  * @table: the top-level table structure
  *
  * Register a sysctl table hierarchy. @table should be a filled in ctl_table
  * array. A completely 0 filled entry terminates the table.
  *
  * The members of the &struct ctl_table structure are used as follows:
  *
  * ctl_name - This is the numeric sysctl value used by sysctl(2). The number
  *            must be unique within that level of sysctl
  *
  * procname - the name of the sysctl file under /proc/sys. Set to %NULL to not
  *            enter a sysctl file
  *
  * data - a pointer to data for use by proc_handler
  *
  * maxlen - the maximum size in bytes of the data
  *
  * mode - the file permissions for the /proc/sys file, and for sysctl(2)
  *
  * child - a pointer to the child sysctl table if this entry is a directory, or
  *         %NULL.
  *
  * proc_handler - the text handler routine (described below)
  *
  * strategy - the strategy routine (described below)
  *
  * de - for internal use by the sysctl routines
  *
  * extra1, extra2 - extra pointers usable by the proc handler routines
  *
  * Leaf nodes in the sysctl tree will be represented by a single file
  * under /proc; non-leaf nodes will be represented by directories.
  *
  * sysctl(2) can automatically manage read and write requests through
  * the sysctl table.  The data and maxlen fields of the ctl_table
  * struct enable minimal validation of the values being written to be
  * performed, and the mode field allows minimal authentication.
  *
  * More sophisticated management can be enabled by the provision of a
  * strategy routine with the table entry.  This will be called before
  * any automatic read or write of the data is performed.
  *
  * The strategy routine may return
  *
  * < 0 - Error occurred (error is passed to user process)
  *
  * 0   - OK - proceed with automatic read or write.
  *
  * > 0 - OK - read or write has been done by the strategy routine, so
  *       return immediately.
  *
  * There must be a proc_handler routine for any terminal nodes
  * mirrored under /proc/sys (non-terminals are handled by a built-in
  * directory handler).  Several default handlers are available to
  * cover common cases -
  *
  * proc_dostring(), proc_dointvec(), proc_dointvec_jiffies(),
  * proc_dointvec_userhz_jiffies(), proc_dointvec_minmax(),
  * proc_doulongvec_ms_jiffies_minmax(), proc_doulongvec_minmax()
  *
  * It is the handler's job to read the input buffer from user memory
  * and process it. The handler should return 0 on success.
  *
  * This routine returns %NULL on a failure to register, and a pointer
  * to the table header on success.
  */
 struct ctl_table_header *__register_sysctl_paths(
 	struct ctl_table_root *root,
 	struct nsproxy *namespaces,
 	const struct ctl_path *path, struct ctl_table *table)
 {
 	struct ctl_table_header *header;
 	struct ctl_table *new, **prevp;
 	unsigned int n, npath;
 	struct ctl_table_set *set;
 	/* Count the path components */
 	for (npath = 0; path[npath].ctl_name || path[npath].procname; ++npath)
 		;
 	/*
 	 * For each path component, allocate a 2-element ctl_table array.
 	 * The first array element will be filled with the sysctl entry
 	 * for this, the second will be the sentinel (ctl_name == 0).
 	 *
 	 * We allocate everything in one go so that we don't have to
 	 * worry about freeing additional memory in unregister_sysctl_table.
 	 */
 	header = kzalloc(sizeof(struct ctl_table_header) +
 			 (2 * npath * sizeof(struct ctl_table)), GFP_KERNEL);
 	if (!header)
 		return NULL;
 	new = (struct ctl_table *) (header + 1);
 	/* Now connect the dots */
 	prevp = &header->ctl_table;
 	for (n = 0; n < npath; ++n, ++path) {
 		/* Copy the procname */
 		new->procname = path->procname;
 		new->ctl_name = path->ctl_name;
 		new->mode     = 0555;
 		*prevp = new;
 		prevp = &new->child;
 		new += 2;
 	}
 	*prevp = table;
 	header->ctl_table_arg = table;
 	INIT_LIST_HEAD(&header->ctl_entry);
 	header->used = 0;
 	header->unregistering = NULL;
 	header->root = root;
 	sysctl_set_parent(NULL, header->ctl_table);
 	header->count = 1;
 #ifdef CONFIG_SYSCTL_SYSCALL_CHECK
 	if (sysctl_check_table(namespaces, header->ctl_table)) {
 		kfree(header);
 		return NULL;
 	}
 #endif
 	spin_lock(&sysctl_lock);
 	header->set = lookup_header_set(root, namespaces);
 	header->attached_by = header->ctl_table;
 	header->attached_to = root_table;
 	header->parent = &root_table_header;
 	for (set = header->set; set; set = set->parent) {
 		struct ctl_table_header *p;
 		list_for_each_entry(p, &set->list, ctl_entry) {
 			if (p->unregistering)
 				continue;
 			try_attach(p, header);
 		}
 	}
 	header->parent->count++;
 	list_add_tail(&header->ctl_entry, &header->set->list);
 	spin_unlock(&sysctl_lock);
 	return header;
 }
 /**
  * register_sysctl_table_path - register a sysctl table hierarchy
  * @path: The path to the directory the sysctl table is in.
  * @table: the top-level table structure
  *
  * Register a sysctl table hierarchy. @table should be a filled in ctl_table
  * array. A completely 0 filled entry terminates the table.
  *
  * See __register_sysctl_paths for more details.
  */
 struct ctl_table_header *register_sysctl_paths(const struct ctl_path *path,
 						struct ctl_table *table)
 {
 	return __register_sysctl_paths(&sysctl_table_root, current->nsproxy,
 					path, table);
 }
 /**
  * register_sysctl_table - register a sysctl table hierarchy
  * @table: the top-level table structure
  *
  * Register a sysctl table hierarchy. @table should be a filled in ctl_table
  * array. A completely 0 filled entry terminates the table.
  *
  * See register_sysctl_paths for more details.
  */
 struct ctl_table_header *register_sysctl_table(struct ctl_table *table)
 {
 	static const struct ctl_path null_path[] = { {} };
 	return register_sysctl_paths(null_path, table);
 }
 /**
  * unregister_sysctl_table - unregister a sysctl table hierarchy
  * @header: the header returned from register_sysctl_table
  *
  * Unregisters the sysctl table and all children. proc entries may not
  * actually be removed until they are no longer used by anyone.
  */
 void unregister_sysctl_table(struct ctl_table_header * header)
 {
 	might_sleep();
 	if (header == NULL)
 		return;
 	spin_lock(&sysctl_lock);
 	start_unregistering(header);
 	if (!--header->parent->count) {
 		WARN_ON(1);
 		kfree(header->parent);
 	}
 	if (!--header->count)
 		kfree(header);
 	spin_unlock(&sysctl_lock);
 }
 int sysctl_is_seen(struct ctl_table_header *p)
 {
 	struct ctl_table_set *set = p->set;
 	int res;
 	spin_lock(&sysctl_lock);
 	if (p->unregistering)
 		res = 0;
 	else if (!set->is_seen)
 		res = 1;
 	else
 		res = set->is_seen(set);
 	spin_unlock(&sysctl_lock);
 	return res;
 }
 void setup_sysctl_set(struct ctl_table_set *p,
 	struct ctl_table_set *parent,
 	int (*is_seen)(struct ctl_table_set *))
 {
 	INIT_LIST_HEAD(&p->list);
 	p->parent = parent ? parent : &sysctl_table_root.default_set;
 	p->is_seen = is_seen;
 }
 #else /* !CONFIG_SYSCTL */
 struct ctl_table_header *register_sysctl_table(struct ctl_table * table)
 {
 	return NULL;
 }
 struct ctl_table_header *register_sysctl_paths(const struct ctl_path *path,
 						    struct ctl_table *table)
 {
 	return NULL;
 }
 void unregister_sysctl_table(struct ctl_table_header * table)
 {
 }
 void setup_sysctl_set(struct ctl_table_set *p,
 	struct ctl_table_set *parent,
 	int (*is_seen)(struct ctl_table_set *))
 {
 }
 void sysctl_head_put(struct ctl_table_header *head)
 {
 }
 #endif /* CONFIG_SYSCTL */
 /*
  * /proc/sys support
  */
 #ifdef CONFIG_PROC_SYSCTL
 static int _proc_do_string(void* data, int maxlen, int write,
 			   struct file *filp, void __user *buffer,
 			   size_t *lenp, loff_t *ppos)
 {
 	size_t len;
 	char __user *p;
 	char c;
 	if (!data || !maxlen || !*lenp) {
 		*lenp = 0;
 		return 0;
 	}
 	if (write) {
 		len = 0;
 		p = buffer;
 		while (len < *lenp) {
 			if (get_user(c, p++))
 				return -EFAULT;
 			if (c == 0 || c == '\n')
 				break;
 			len++;
 		}
 		if (len >= maxlen)
 			len = maxlen-1;
 		if(copy_from_user(data, buffer, len))
 			return -EFAULT;
 		((char *) data)[len] = 0;
 		*ppos += *lenp;
 	} else {
 		len = strlen(data);
 		if (len > maxlen)
 			len = maxlen;
 		if (*ppos > len) {
 			*lenp = 0;
 			return 0;
 		}
 		data += *ppos;
 		len  -= *ppos;
 		if (len > *lenp)
 			len = *lenp;
 		if (len)
 			if(copy_to_user(buffer, data, len))
 				return -EFAULT;
 		if (len < *lenp) {
 			if(put_user('\n', ((char __user *) buffer) + len))
 				return -EFAULT;
 			len++;
 		}
 		*lenp = len;
 		*ppos += len;
 	}
 	return 0;
 }
 /**
  * proc_dostring - read a string sysctl
  * @table: the sysctl table
  * @write: %TRUE if this is a write to the sysctl file
  * @filp: the file structure
  * @buffer: the user buffer
  * @lenp: the size of the user buffer
  * @ppos: file position
  *
  * Reads/writes a string from/to the user buffer. If the kernel
  * buffer provided is not large enough to hold the string, the
  * string is truncated. The copied string is %NULL-terminated.
  * If the string is being read by the user process, it is copied
  * and a newline '\n' is added. It is truncated if the buffer is
  * not large enough.
  *
  * Returns 0 on success.
  */
 int proc_dostring(struct ctl_table *table, int write, struct file *filp,
 		  void __user *buffer, size_t *lenp, loff_t *ppos)
 {
 	return _proc_do_string(table->data, table->maxlen, write, filp,
 			       buffer, lenp, ppos);
 }
 static int do_proc_dointvec_conv(int *negp, unsigned long *lvalp,
 				 int *valp,
 				 int write, void *data)
 {
 	if (write) {
 		*valp = *negp ? -*lvalp : *lvalp;
 	} else {
 		int val = *valp;
 		if (val < 0) {
 			*negp = -1;
 			*lvalp = (unsigned long)-val;
 		} else {
 			*negp = 0;
 			*lvalp = (unsigned long)val;
 		}
 	}
 	return 0;
 }
 static int __do_proc_dointvec(void *tbl_data, struct ctl_table *table,
 		  int write, struct file *filp, void __user *buffer,
 		  size_t *lenp, loff_t *ppos,
 		  int (*conv)(int *negp, unsigned long *lvalp, int *valp,
 			      int write, void *data),
 		  void *data)
 {
 #define TMPBUFLEN 21
 	int *i, vleft, first=1, neg, val;
 	unsigned long lval;
 	size_t left, len;
 	char buf[TMPBUFLEN], *p;
 	char __user *s = buffer;
 	if (!tbl_data || !table->maxlen || !*lenp ||
 	    (*ppos && !write)) {
 		*lenp = 0;
 		return 0;
 	}
 	i = (int *) tbl_data;
 	vleft = table->maxlen / sizeof(*i);
 	left = *lenp;
 	if (!conv)
 		conv = do_proc_dointvec_conv;
 	for (; left && vleft--; i++, first=0) {
 		if (write) {
 			while (left) {
 				char c;
 				if (get_user(c, s))
 					return -EFAULT;
 				if (!isspace(c))
 					break;
 				left--;
 				s++;
 			}
 			if (!left)
 				break;
 			neg = 0;
 			len = left;
 			if (len > sizeof(buf) - 1)
 				len = sizeof(buf) - 1;
 			if (copy_from_user(buf, s, len))
 				return -EFAULT;
 			buf[len] = 0;
 			p = buf;
 			if (*p == '-' && left > 1) {
 				neg = 1;
 				p++;
 			}
 			if (*p < '0' || *p > '9')
 				break;
 			lval = simple_strtoul(p, &p, 0);
 			len = p-buf;
 			if ((len < left) && *p && !isspace(*p))
 				break;
 			if (neg)
 				val = -val;
 			s += len;
 			left -= len;
 			if (conv(&neg, &lval, i, 1, data))
 				break;
 		} else {
 			p = buf;
 			if (!first)
 				*p++ = '\t';
 			if (conv(&neg, &lval, i, 0, data))
 				break;
 			sprintf(p, "%s%lu", neg ? "-" : "", lval);
 			len = strlen(buf);
 			if (len > left)
 				len = left;
 			if(copy_to_user(s, buf, len))
 				return -EFAULT;
 			left -= len;
 			s += len;
 		}
 	}
 	if (!write && !first && left) {
 		if(put_user('\n', s))
 			return -EFAULT;
 		left--, s++;
 	}
 	if (write) {
 		while (left) {
 			char c;
 			if (get_user(c, s++))
 				return -EFAULT;
 			if (!isspace(c))
 				break;
 			left--;
 		}
 	}
 	if (write && first)
 		return -EINVAL;
 	*lenp -= left;
 	*ppos += *lenp;
 	return 0;
 #undef TMPBUFLEN
 }
 static int do_proc_dointvec(struct ctl_table *table, int write, struct file *filp,
 		  void __user *buffer, size_t *lenp, loff_t *ppos,
 		  int (*conv)(int *negp, unsigned long *lvalp, int *valp,
 			      int write, void *data),
 		  void *data)
 {
 	return __do_proc_dointvec(table->data, table, write, filp,
 			buffer, lenp, ppos, conv, data);
 }
 /**
  * proc_dointvec - read a vector of integers
  * @table: the sysctl table
  * @write: %TRUE if this is a write to the sysctl file
  * @filp: the file structure
  * @buffer: the user buffer
  * @lenp: the size of the user buffer
  * @ppos: file position
  *
  * Reads/writes up to table->maxlen/sizeof(unsigned int) integer
  * values from/to the user buffer, treated as an ASCII string.
  *
  * Returns 0 on success.
  */
 int proc_dointvec(struct ctl_table *table, int write, struct file *filp,
 		     void __user *buffer, size_t *lenp, loff_t *ppos)
 {
     return do_proc_dointvec(table,write,filp,buffer,lenp,ppos,
 		    	    NULL,NULL);
 }
 /*
  * Taint values can only be increased
  * This means we can safely use a temporary.
  */
 static int proc_taint(struct ctl_table *table, int write, struct file *filp,
 			       void __user *buffer, size_t *lenp, loff_t *ppos)
 {
 	struct ctl_table t;
 	unsigned long tmptaint = get_taint();
 	int err;
 	if (write && !capable(CAP_SYS_ADMIN))
 		return -EPERM;
 	t = *table;
 	t.data = &tmptaint;
 	err = proc_doulongvec_minmax(&t, write, filp, buffer, lenp, ppos);
 	if (err < 0)
 		return err;
 	if (write) {
 		/*
 		 * Poor man's atomic or. Not worth adding a primitive
 		 * to everyone's atomic.h for this
 		 */
 		int i;
 		for (i = 0; i < BITS_PER_LONG && tmptaint >> i; i++) {
 			if ((tmptaint >> i) & 1)
 				add_taint(i);
 		}
 	}
 	return err;
 }
 struct do_proc_dointvec_minmax_conv_param {
 	int *min;
 	int *max;
 };
 static int do_proc_dointvec_minmax_conv(int *negp, unsigned long *lvalp,
 					int *valp,
 					int write, void *data)
 {
 	struct do_proc_dointvec_minmax_conv_param *param = data;
 	if (write) {
 		int val = *negp ? -*lvalp : *lvalp;
 		if ((param->min && *param->min > val) ||
 		    (param->max && *param->max < val))
 			return -EINVAL;
 		*valp = val;
 	} else {
 		int val = *valp;
 		if (val < 0) {
 			*negp = -1;
 			*lvalp = (unsigned long)-val;
 		} else {
 			*negp = 0;
 			*lvalp = (unsigned long)val;
 		}
 	}
 	return 0;
 }
 /**
  * proc_dointvec_minmax - read a vector of integers with min/max values
  * @table: the sysctl table
  * @write: %TRUE if this is a write to the sysctl file
  * @filp: the file structure
  * @buffer: the user buffer
  * @lenp: the size of the user buffer
  * @ppos: file position
  *
  * Reads/writes up to table->maxlen/sizeof(unsigned int) integer
  * values from/to the user buffer, treated as an ASCII string.
  *
  * This routine will ensure the values are within the range specified by
  * table->extra1 (min) and table->extra2 (max).
  *
  * Returns 0 on success.
  */
 int proc_dointvec_minmax(struct ctl_table *table, int write, struct file *filp,
 		  void __user *buffer, size_t *lenp, loff_t *ppos)
 {
 	struct do_proc_dointvec_minmax_conv_param param = {
 		.min = (int *) table->extra1,
 		.max = (int *) table->extra2,
 	};
 	return do_proc_dointvec(table, write, filp, buffer, lenp, ppos,
 				do_proc_dointvec_minmax_conv, &param);
 }
 static int __do_proc_doulongvec_minmax(void *data, struct ctl_table *table, int write,
 				     struct file *filp,
 				     void __user *buffer,
 				     size_t *lenp, loff_t *ppos,
 				     unsigned long convmul,
 				     unsigned long convdiv)
 {
 #define TMPBUFLEN 21
 	unsigned long *i, *min, *max, val;
 	int vleft, first=1, neg;
 	size_t len, left;
 	char buf[TMPBUFLEN], *p;
 	char __user *s = buffer;
 	if (!data || !table->maxlen || !*lenp ||
 	    (*ppos && !write)) {
 		*lenp = 0;
 		return 0;
 	}
 	i = (unsigned long *) data;
 	min = (unsigned long *) table->extra1;
 	max = (unsigned long *) table->extra2;
 	vleft = table->maxlen / sizeof(unsigned long);
 	left = *lenp;
 	for (; left && vleft--; i++, min++, max++, first=0) {
 		if (write) {
 			while (left) {
 				char c;
 				if (get_user(c, s))
 					return -EFAULT;
 				if (!isspace(c))
 					break;
 				left--;
 				s++;
 			}
 			if (!left)
 				break;
 			neg = 0;
 			len = left;
 			if (len > TMPBUFLEN-1)
 				len = TMPBUFLEN-1;
 			if (copy_from_user(buf, s, len))
 				return -EFAULT;
 			buf[len] = 0;
 			p = buf;
 			if (*p == '-' && left > 1) {
 				neg = 1;
 				p++;
 			}
 			if (*p < '0' || *p > '9')
 				break;
 			val = simple_strtoul(p, &p, 0) * convmul / convdiv ;
 			len = p-buf;
 			if ((len < left) && *p && !isspace(*p))
 				break;
 			if (neg)
 				val = -val;
 			s += len;
 			left -= len;
 			if(neg)
 				continue;
 			if ((min && val < *min) || (max && val > *max))
 				continue;
 			*i = val;
 		} else {
 			p = buf;
 			if (!first)
 				*p++ = '\t';
 			sprintf(p, "%lu", convdiv * (*i) / convmul);
 			len = strlen(buf);
 			if (len > left)
 				len = left;
 			if(copy_to_user(s, buf, len))
 				return -EFAULT;
 			left -= len;
 			s += len;
 		}
 	}
 	if (!write && !first && left) {
 		if(put_user('\n', s))
 			return -EFAULT;
 		left--, s++;
 	}
 	if (write) {
 		while (left) {
 			char c;
 			if (get_user(c, s++))
 				return -EFAULT;
 			if (!isspace(c))
 				break;
 			left--;
 		}
 	}
 	if (write && first)
 		return -EINVAL;
 	*lenp -= left;
 	*ppos += *lenp;
 	return 0;
 #undef TMPBUFLEN
 }
 static int do_proc_doulongvec_minmax(struct ctl_table *table, int write,
 				     struct file *filp,
 				     void __user *buffer,
 				     size_t *lenp, loff_t *ppos,
 				     unsigned long convmul,
 				     unsigned long convdiv)
 {
 	return __do_proc_doulongvec_minmax(table->data, table, write,
 			filp, buffer, lenp, ppos, convmul, convdiv);
 }
 /**
  * proc_doulongvec_minmax - read a vector of long integers with min/max values
  * @table: the sysctl table
  * @write: %TRUE if this is a write to the sysctl file
  * @filp: the file structure
  * @buffer: the user buffer
  * @lenp: the size of the user buffer
  * @ppos: file position
  *
  * Reads/writes up to table->maxlen/sizeof(unsigned long) unsigned long
  * values from/to the user buffer, treated as an ASCII string.
  *
  * This routine will ensure the values are within the range specified by
  * table->extra1 (min) and table->extra2 (max).
  *
  * Returns 0 on success.
  */
 int proc_doulongvec_minmax(struct ctl_table *table, int write, struct file *filp,
 			   void __user *buffer, size_t *lenp, loff_t *ppos)
 {
     return do_proc_doulongvec_minmax(table, write, filp, buffer, lenp, ppos, 1l, 1l);
 }
 /**
  * proc_doulongvec_ms_jiffies_minmax - read a vector of millisecond values with min/max values
  * @table: the sysctl table
  * @write: %TRUE if this is a write to the sysctl file
  * @filp: the file structure
  * @buffer: the user buffer
  * @lenp: the size of the user buffer
  * @ppos: file position
  *
  * Reads/writes up to table->maxlen/sizeof(unsigned long) unsigned long
  * values from/to the user buffer, treated as an ASCII string. The values
  * are treated as milliseconds, and converted to jiffies when they are stored.
  *
  * This routine will ensure the values are within the range specified by
  * table->extra1 (min) and table->extra2 (max).
  *
  * Returns 0 on success.
  */
 int proc_doulongvec_ms_jiffies_minmax(struct ctl_table *table, int write,
 				      struct file *filp,
 				      void __user *buffer,
 				      size_t *lenp, loff_t *ppos)
 {
     return do_proc_doulongvec_minmax(table, write, filp, buffer,
 				     lenp, ppos, HZ, 1000l);
 }
 static int do_proc_dointvec_jiffies_conv(int *negp, unsigned long *lvalp,
 					 int *valp,
 					 int write, void *data)
 {
 	if (write) {
 		if (*lvalp > LONG_MAX / HZ)
 			return 1;
 		*valp = *negp ? -(*lvalp*HZ) : (*lvalp*HZ);
 	} else {
 		int val = *valp;
 		unsigned long lval;
 		if (val < 0) {
 			*negp = -1;
 			lval = (unsigned long)-val;
 		} else {
 			*negp = 0;
 			lval = (unsigned long)val;
 		}
 		*lvalp = lval / HZ;
 	}
 	return 0;
 }
 static int do_proc_dointvec_userhz_jiffies_conv(int *negp, unsigned long *lvalp,
 						int *valp,
 						int write, void *data)
 {
 	if (write) {
 		if (USER_HZ < HZ && *lvalp > (LONG_MAX / HZ) * USER_HZ)
 			return 1;
 		*valp = clock_t_to_jiffies(*negp ? -*lvalp : *lvalp);
 	} else {
 		int val = *valp;
 		unsigned long lval;
 		if (val < 0) {
 			*negp = -1;
 			lval = (unsigned long)-val;
 		} else {
 			*negp = 0;
 			lval = (unsigned long)val;
 		}
 		*lvalp = jiffies_to_clock_t(lval);
 	}
 	return 0;
 }
 static int do_proc_dointvec_ms_jiffies_conv(int *negp, unsigned long *lvalp,
 					    int *valp,
 					    int write, void *data)
 {
 	if (write) {
 		*valp = msecs_to_jiffies(*negp ? -*lvalp : *lvalp);
 	} else {
 		int val = *valp;
 		unsigned long lval;
 		if (val < 0) {
 			*negp = -1;
 			lval = (unsigned long)-val;
 		} else {
 			*negp = 0;
 			lval = (unsigned long)val;
 		}
 		*lvalp = jiffies_to_msecs(lval);
 	}
 	return 0;
 }
 /**
  * proc_dointvec_jiffies - read a vector of integers as seconds
  * @table: the sysctl table
  * @write: %TRUE if this is a write to the sysctl file
  * @filp: the file structure
  * @buffer: the user buffer
  * @lenp: the size of the user buffer
  * @ppos: file position
  *
  * Reads/writes up to table->maxlen/sizeof(unsigned int) integer
  * values from/to the user buffer, treated as an ASCII string.
  * The values read are assumed to be in seconds, and are converted into
  * jiffies.
  *
  * Returns 0 on success.
  */
 int proc_dointvec_jiffies(struct ctl_table *table, int write, struct file *filp,
 			  void __user *buffer, size_t *lenp, loff_t *ppos)
 {
     return do_proc_dointvec(table,write,filp,buffer,lenp,ppos,
 		    	    do_proc_dointvec_jiffies_conv,NULL);
 }
 /**
  * proc_dointvec_userhz_jiffies - read a vector of integers as 1/USER_HZ seconds
  * @table: the sysctl table
  * @write: %TRUE if this is a write to the sysctl file
  * @filp: the file structure
  * @buffer: the user buffer
  * @lenp: the size of the user buffer
  * @ppos: pointer to the file position
  *
  * Reads/writes up to table->maxlen/sizeof(unsigned int) integer
  * values from/to the user buffer, treated as an ASCII string.
  * The values read are assumed to be in 1/USER_HZ seconds, and
  * are converted into jiffies.
  *
  * Returns 0 on success.
  */
 int proc_dointvec_userhz_jiffies(struct ctl_table *table, int write, struct file *filp,
 				 void __user *buffer, size_t *lenp, loff_t *ppos)
 {
     return do_proc_dointvec(table,write,filp,buffer,lenp,ppos,
 		    	    do_proc_dointvec_userhz_jiffies_conv,NULL);
 }
 /**
  * proc_dointvec_ms_jiffies - read a vector of integers as 1 milliseconds
  * @table: the sysctl table
  * @write: %TRUE if this is a write to the sysctl file
  * @filp: the file structure
  * @buffer: the user buffer
  * @lenp: the size of the user buffer
  * @ppos: file position
  * @ppos: the current position in the file
  *
  * Reads/writes up to table->maxlen/sizeof(unsigned int) integer
  * values from/to the user buffer, treated as an ASCII string.
  * The values read are assumed to be in 1/1000 seconds, and
  * are converted into jiffies.
  *
  * Returns 0 on success.
  */
 int proc_dointvec_ms_jiffies(struct ctl_table *table, int write, struct file *filp,
 			     void __user *buffer, size_t *lenp, loff_t *ppos)
 {
 	return do_proc_dointvec(table, write, filp, buffer, lenp, ppos,
 				do_proc_dointvec_ms_jiffies_conv, NULL);
 }
 static int proc_do_cad_pid(struct ctl_table *table, int write, struct file *filp,
 			   void __user *buffer, size_t *lenp, loff_t *ppos)
 {
 	struct pid *new_pid;
 	pid_t tmp;
 	int r;
 	tmp = pid_vnr(cad_pid);
 	r = __do_proc_dointvec(&tmp, table, write, filp, buffer,
 			       lenp, ppos, NULL, NULL);
 	if (r || !write)
 		return r;
 	new_pid = find_get_pid(tmp);
 	if (!new_pid)
 		return -ESRCH;
 	put_pid(xchg(&cad_pid, new_pid));
 	return 0;
 }
 #else /* CONFIG_PROC_FS */
 int proc_dostring(struct ctl_table *table, int write, struct file *filp,
 		  void __user *buffer, size_t *lenp, loff_t *ppos)
 {
 	return -ENOSYS;
 }
 int proc_dointvec(struct ctl_table *table, int write, struct file *filp,
 		  void __user *buffer, size_t *lenp, loff_t *ppos)
 {
 	return -ENOSYS;
 }
 int proc_dointvec_minmax(struct ctl_table *table, int write, struct file *filp,
 		    void __user *buffer, size_t *lenp, loff_t *ppos)
 {
 	return -ENOSYS;
 }
 int proc_dointvec_jiffies(struct ctl_table *table, int write, struct file *filp,
 		    void __user *buffer, size_t *lenp, loff_t *ppos)
 {
 	return -ENOSYS;
 }
 int proc_dointvec_userhz_jiffies(struct ctl_table *table, int write, struct file *filp,
 		    void __user *buffer, size_t *lenp, loff_t *ppos)
 {
 	return -ENOSYS;
 }
 int proc_dointvec_ms_jiffies(struct ctl_table *table, int write, struct file *filp,
 			     void __user *buffer, size_t *lenp, loff_t *ppos)
 {
 	return -ENOSYS;
 }
 int proc_doulongvec_minmax(struct ctl_table *table, int write, struct file *filp,
 		    void __user *buffer, size_t *lenp, loff_t *ppos)
 {
 	return -ENOSYS;
 }
 int proc_doulongvec_ms_jiffies_minmax(struct ctl_table *table, int write,
 				      struct file *filp,
 				      void __user *buffer,
 				      size_t *lenp, loff_t *ppos)
 {
     return -ENOSYS;
 }
 #endif /* CONFIG_PROC_FS */
 #ifdef CONFIG_SYSCTL_SYSCALL
 /*
  * General sysctl support routines
  */
 /* The generic sysctl data routine (used if no strategy routine supplied) */
 int sysctl_data(struct ctl_table *table,
 		void __user *oldval, size_t __user *oldlenp,
 		void __user *newval, size_t newlen)
 {
 	size_t len;
 	/* Get out of I don't have a variable */
 	if (!table->data || !table->maxlen)
 		return -ENOTDIR;
 	if (oldval && oldlenp) {
 		if (get_user(len, oldlenp))
 			return -EFAULT;
 		if (len) {
 			if (len > table->maxlen)
 				len = table->maxlen;
 			if (copy_to_user(oldval, table->data, len))
 				return -EFAULT;
 			if (put_user(len, oldlenp))
 				return -EFAULT;
 		}
 	}
 	if (newval && newlen) {
 		if (newlen > table->maxlen)
 			newlen = table->maxlen;
 		if (copy_from_user(table->data, newval, newlen))
 			return -EFAULT;
 	}
 	return 1;
 }
 /* The generic string strategy routine: */
 int sysctl_string(struct ctl_table *table,
 		  void __user *oldval, size_t __user *oldlenp,
 		  void __user *newval, size_t newlen)
 {
 	if (!table->data || !table->maxlen)
 		return -ENOTDIR;
 	if (oldval && oldlenp) {
 		size_t bufsize;
 		if (get_user(bufsize, oldlenp))
 			return -EFAULT;
 		if (bufsize) {
 			size_t len = strlen(table->data), copied;
 			/* This shouldn't trigger for a well-formed sysctl */
 			if (len > table->maxlen)
 				len = table->maxlen;
 			/* Copy up to a max of bufsize-1 bytes of the string */
 			copied = (len >= bufsize) ? bufsize - 1 : len;
 			if (copy_to_user(oldval, table->data, copied) ||
 			    put_user(0, (char __user *)(oldval + copied)))
 				return -EFAULT;
 			if (put_user(len, oldlenp))
 				return -EFAULT;
 		}
 	}
 	if (newval && newlen) {
 		size_t len = newlen;
 		if (len > table->maxlen)
 			len = table->maxlen;
 		if(copy_from_user(table->data, newval, len))
 			return -EFAULT;
 		if (len == table->maxlen)
 			len--;
 		((char *) table->data)[len] = 0;
 	}
 	return 1;
 }
 /*
  * This function makes sure that all of the integers in the vector
  * are between the minimum and maximum values given in the arrays
  * table->extra1 and table->extra2, respectively.
  */
 int sysctl_intvec(struct ctl_table *table,
 		void __user *oldval, size_t __user *oldlenp,
 		void __user *newval, size_t newlen)
 {
 	if (newval && newlen) {
 		int __user *vec = (int __user *) newval;
 		int *min = (int *) table->extra1;
 		int *max = (int *) table->extra2;
 		size_t length;
 		int i;
 		if (newlen % sizeof(int) != 0)
 			return -EINVAL;
 		if (!table->extra1 && !table->extra2)
 			return 0;
 		if (newlen > table->maxlen)
 			newlen = table->maxlen;
 		length = newlen / sizeof(int);
 		for (i = 0; i < length; i++) {
 			int value;
 			if (get_user(value, vec + i))
 				return -EFAULT;
 			if (min && value < min[i])
 				return -EINVAL;
 			if (max && value > max[i])
 				return -EINVAL;
 		}
 	}
 	return 0;
 }
 /* Strategy function to convert jiffies to seconds */
 int sysctl_jiffies(struct ctl_table *table,
 		void __user *oldval, size_t __user *oldlenp,
 		void __user *newval, size_t newlen)
 {
 	if (oldval && oldlenp) {
 		size_t olen;
 		if (get_user(olen, oldlenp))
 			return -EFAULT;
 		if (olen) {
 			int val;
 			if (olen < sizeof(int))
 				return -EINVAL;
 			val = *(int *)(table->data) / HZ;
 			if (put_user(val, (int __user *)oldval))
 				return -EFAULT;
 			if (put_user(sizeof(int), oldlenp))
 				return -EFAULT;
 		}
 	}
 	if (newval && newlen) {
 		int new;
 		if (newlen != sizeof(int))
 			return -EINVAL;
 		if (get_user(new, (int __user *)newval))
 			return -EFAULT;
 		*(int *)(table->data) = new*HZ;
 	}
 	return 1;
 }
 /* Strategy function to convert jiffies to seconds */
 int sysctl_ms_jiffies(struct ctl_table *table,
 		void __user *oldval, size_t __user *oldlenp,
 		void __user *newval, size_t newlen)
 {
 	if (oldval && oldlenp) {
 		size_t olen;
 		if (get_user(olen, oldlenp))
 			return -EFAULT;
 		if (olen) {
 			int val;
 			if (olen < sizeof(int))
 				return -EINVAL;
 			val = jiffies_to_msecs(*(int *)(table->data));
 			if (put_user(val, (int __user *)oldval))
 				return -EFAULT;
 			if (put_user(sizeof(int), oldlenp))
 				return -EFAULT;
 		}
 	}
 	if (newval && newlen) {
 		int new;
 		if (newlen != sizeof(int))
 			return -EINVAL;
 		if (get_user(new, (int __user *)newval))
 			return -EFAULT;
 		*(int *)(table->data) = msecs_to_jiffies(new);
 	}
 	return 1;
 }
 #else /* CONFIG_SYSCTL_SYSCALL */
 asmlinkage long sys_sysctl(struct __sysctl_args __user *args)
 {
 	struct __sysctl_args tmp;
 	int error;
 	if (copy_from_user(&tmp, args, sizeof(tmp)))
 		return -EFAULT;
 	error = deprecated_sysctl_warning(&tmp);
 	/* If no error reading the parameters then just -ENOSYS ... */
 	if (!error)
 		error = -ENOSYS;
 	return error;
 }
 int sysctl_data(struct ctl_table *table,
 		  void __user *oldval, size_t __user *oldlenp,
 		  void __user *newval, size_t newlen)
 {
 	return -ENOSYS;
 }
 int sysctl_string(struct ctl_table *table,
 		  void __user *oldval, size_t __user *oldlenp,
 		  void __user *newval, size_t newlen)
 {
 	return -ENOSYS;
 }
 int sysctl_intvec(struct ctl_table *table,
 		void __user *oldval, size_t __user *oldlenp,
 		void __user *newval, size_t newlen)
 {
 	return -ENOSYS;
 }
 int sysctl_jiffies(struct ctl_table *table,
 		void __user *oldval, size_t __user *oldlenp,
 		void __user *newval, size_t newlen)
 {
 	return -ENOSYS;
 }
 int sysctl_ms_jiffies(struct ctl_table *table,
 		void __user *oldval, size_t __user *oldlenp,
 		void __user *newval, size_t newlen)
 {
 	return -ENOSYS;
 }
 #endif /* CONFIG_SYSCTL_SYSCALL */
 static int deprecated_sysctl_warning(struct __sysctl_args *args)
 {
 	static int msg_count;
 	int name[CTL_MAXNAME];
 	int i;
 	/* Check args->nlen. */
 	if (args->nlen < 0 || args->nlen > CTL_MAXNAME)
 		return -ENOTDIR;
 	/* Read in the sysctl name for better debug message logging */
 	for (i = 0; i < args->nlen; i++)
 		if (get_user(name[i], args->name + i))
 			return -EFAULT;
 	/* Ignore accesses to kernel.version */
 	if ((args->nlen == 2) && (name[0] == CTL_KERN) && (name[1] == KERN_VERSION))
 		return 0;
 	if (msg_count < 5) {
 		msg_count++;
 		printk(KERN_INFO
 			"warning: process `%s' used the deprecated sysctl "
 			"system call with ", current->comm);
 		for (i = 0; i < args->nlen; i++)
 			printk("%d.", name[i]);
 		printk("\n");
 	}
 	return 0;
 }
 /*
  * No sense putting this after each symbol definition, twice,
  * exception granted :-)
  */
 EXPORT_SYMBOL(proc_dointvec);
 EXPORT_SYMBOL(proc_dointvec_jiffies);
 EXPORT_SYMBOL(proc_dointvec_minmax);
 EXPORT_SYMBOL(proc_dointvec_userhz_jiffies);
 EXPORT_SYMBOL(proc_dointvec_ms_jiffies);
 EXPORT_SYMBOL(proc_dostring);
 EXPORT_SYMBOL(proc_doulongvec_minmax);
 EXPORT_SYMBOL(proc_doulongvec_ms_jiffies_minmax);
 EXPORT_SYMBOL(register_sysctl_table);
 EXPORT_SYMBOL(register_sysctl_paths);
 EXPORT_SYMBOL(sysctl_intvec);
 EXPORT_SYMBOL(sysctl_jiffies);
 EXPORT_SYMBOL(sysctl_ms_jiffies);
 EXPORT_SYMBOL(sysctl_string);
 EXPORT_SYMBOL(sysctl_data);
 EXPORT_SYMBOL(unregister_sysctl_table);

lib/Kconfig.debug

Diff comments View file @ c40f6f8

 config PRINTK_TIME
 	bool "Show timing information on printks"
 	depends on PRINTK
 	help
 	  Selecting this option causes timing information to be
 	  included in printk output.  This allows you to measure
 	  the interval between kernel operations, including bootup
 	  operations.  This is useful for identifying long delays
 	  in kernel startup.
 config ENABLE_WARN_DEPRECATED
 	bool "Enable __deprecated logic"
 	default y
 	help
 	  Enable the __deprecated logic in the kernel build.
 	  Disable this to suppress the "warning: 'foo' is deprecated
 	  (declared at kernel/power/somefile.c:1234)" messages.
 config ENABLE_MUST_CHECK
 	bool "Enable __must_check logic"
 	default y
 	help
 	  Enable the __must_check logic in the kernel build.  Disable this to
 	  suppress the "warning: ignoring return value of 'foo', declared with
 	  attribute warn_unused_result" messages.
 config FRAME_WARN
 	int "Warn for stack frames larger than (needs gcc 4.4)"
 	range 0 8192
 	default 1024 if !64BIT
 	default 2048 if 64BIT
 	help
 	  Tell gcc to warn at build time for stack frames larger than this.
 	  Setting this too low will cause a lot of warnings.
 	  Setting it to 0 disables the warning.
 	  Requires gcc 4.4
 config MAGIC_SYSRQ
 	bool "Magic SysRq key"
 	depends on !UML
 	help
 	  If you say Y here, you will have some control over the system even
 	  if the system crashes for example during kernel debugging (e.g., you
 	  will be able to flush the buffer cache to disk, reboot the system
 	  immediately or dump some status information). This is accomplished
 	  by pressing various keys while holding SysRq (Alt+PrintScreen). It
 	  also works on a serial console (on PC hardware at least), if you
 	  send a BREAK and then within 5 seconds a command keypress. The
 	  keys are documented in <file:Documentation/sysrq.txt>. Don't say Y
 	  unless you really know what this hack does.
 config UNUSED_SYMBOLS
 	bool "Enable unused/obsolete exported symbols"
 	default y if X86
 	help
 	  Unused but exported symbols make the kernel needlessly bigger.  For
 	  that reason most of these unused exports will soon be removed.  This
 	  option is provided temporarily to provide a transition period in case
 	  some external kernel module needs one of these symbols anyway. If you
 	  encounter such a case in your module, consider if you are actually
 	  using the right API.  (rationale: since nobody in the kernel is using
 	  this in a module, there is a pretty good chance it's actually the
 	  wrong interface to use).  If you really need the symbol, please send a
 	  mail to the linux kernel mailing list mentioning the symbol and why
 	  you really need it, and what the merge plan to the mainline kernel for
 	  your module is.
 config DEBUG_FS
 	bool "Debug Filesystem"
 	depends on SYSFS
 	help
 	  debugfs is a virtual file system that kernel developers use to put
 	  debugging files into.  Enable this option to be able to read and
 	  write to these files.
 	  For detailed documentation on the debugfs API, see
 	  Documentation/DocBook/filesystems.
 	  If unsure, say N.
 config HEADERS_CHECK
 	bool "Run 'make headers_check' when building vmlinux"
 	depends on !UML
 	help
 	  This option will extract the user-visible kernel headers whenever
 	  building the kernel, and will run basic sanity checks on them to
 	  ensure that exported files do not attempt to include files which
 	  were not exported, etc.
 	  If you're making modifications to header files which are
 	  relevant for userspace, say 'Y', and check the headers
 	  exported to $(INSTALL_HDR_PATH) (usually 'usr/include' in
 	  your build tree), to make sure they're suitable.
 config DEBUG_SECTION_MISMATCH
 	bool "Enable full Section mismatch analysis"
 	depends on UNDEFINED
 	# This option is on purpose disabled for now.
 	# It will be enabled when we are down to a resonable number
 	# of section mismatch warnings (< 10 for an allyesconfig build)
 	help
 	  The section mismatch analysis checks if there are illegal
 	  references from one section to another section.
 	  Linux will during link or during runtime drop some sections
 	  and any use of code/data previously in these sections will
 	  most likely result in an oops.
 	  In the code functions and variables are annotated with
 	  __init, __devinit etc. (see full list in include/linux/init.h)
 	  which results in the code/data being placed in specific sections.
 	  The section mismatch analysis is always done after a full
 	  kernel build but enabling this option will in addition
 	  do the following:
 	  - Add the option -fno-inline-functions-called-once to gcc
 	    When inlining a function annotated __init in a non-init
 	    function we would lose the section information and thus
 	    the analysis would not catch the illegal reference.
 	    This option tells gcc to inline less but will also
 	    result in a larger kernel.
 	  - Run the section mismatch analysis for each module/built-in.o
 	    When we run the section mismatch analysis on vmlinux.o we
 	    lose valueble information about where the mismatch was
 	    introduced.
 	    Running the analysis for each module/built-in.o file
 	    will tell where the mismatch happens much closer to the
 	    source. The drawback is that we will report the same
 	    mismatch at least twice.
 	  - Enable verbose reporting from modpost to help solving
 	    the section mismatches reported.
 config DEBUG_KERNEL
 	bool "Kernel debugging"
 	help
 	  Say Y here if you are developing drivers or trying to debug and
 	  identify kernel problems.
 config DEBUG_SHIRQ
 	bool "Debug shared IRQ handlers"
 	depends on DEBUG_KERNEL && GENERIC_HARDIRQS
 	help
 	  Enable this to generate a spurious interrupt as soon as a shared
 	  interrupt handler is registered, and just before one is deregistered.
 	  Drivers ought to be able to handle interrupts coming in at those
 	  points; some don't and need to be caught.
 config DETECT_SOFTLOCKUP
 	bool "Detect Soft Lockups"
 	depends on DEBUG_KERNEL && !S390
 	default y
 	help
 	  Say Y here to enable the kernel to detect "soft lockups",
 	  which are bugs that cause the kernel to loop in kernel
 	  mode for more than 60 seconds, without giving other tasks a
 	  chance to run.
 	  When a soft-lockup is detected, the kernel will print the
 	  current stack trace (which you should report), but the
 	  system will stay locked up. This feature has negligible
 	  overhead.
 	  (Note that "hard lockups" are separate type of bugs that
 	   can be detected via the NMI-watchdog, on platforms that
 	   support it.)
 config BOOTPARAM_SOFTLOCKUP_PANIC
 	bool "Panic (Reboot) On Soft Lockups"
 	depends on DETECT_SOFTLOCKUP
 	help
 	  Say Y here to enable the kernel to panic on "soft lockups",
 	  which are bugs that cause the kernel to loop in kernel
 	  mode for more than 60 seconds, without giving other tasks a
 	  chance to run.
 	  The panic can be used in combination with panic_timeout,
 	  to cause the system to reboot automatically after a
 	  lockup has been detected. This feature is useful for
 	  high-availability systems that have uptime guarantees and
 	  where a lockup must be resolved ASAP.
 	  Say N if unsure.
 config BOOTPARAM_SOFTLOCKUP_PANIC_VALUE
 	int
 	depends on DETECT_SOFTLOCKUP
 	range 0 1
 	default 0 if !BOOTPARAM_SOFTLOCKUP_PANIC
 	default 1 if BOOTPARAM_SOFTLOCKUP_PANIC
 config SCHED_DEBUG
 	bool "Collect scheduler debugging info"
 	depends on DEBUG_KERNEL && PROC_FS
 	default y
 	help
 	  If you say Y here, the /proc/sched_debug file will be provided
 	  that can help debug the scheduler. The runtime overhead of this
 	  option is minimal.
 config SCHEDSTATS
 	bool "Collect scheduler statistics"
 	depends on DEBUG_KERNEL && PROC_FS
 	help
 	  If you say Y here, additional code will be inserted into the
 	  scheduler and related routines to collect statistics about
 	  scheduler behavior and provide them in /proc/schedstat.  These
 	  stats may be useful for both tuning and debugging the scheduler
 	  If you aren't debugging the scheduler or trying to tune a specific
 	  application, you can say N to avoid the very slight overhead
 	  this adds.
 config TIMER_STATS
 	bool "Collect kernel timers statistics"
 	depends on DEBUG_KERNEL && PROC_FS
 	help
 	  If you say Y here, additional code will be inserted into the
 	  timer routines to collect statistics about kernel timers being
 	  reprogrammed. The statistics can be read from /proc/timer_stats.
 	  The statistics collection is started by writing 1 to /proc/timer_stats,
 	  writing 0 stops it. This feature is useful to collect information
 	  about timer usage patterns in kernel and userspace. This feature
 	  is lightweight if enabled in the kernel config but not activated
 	  (it defaults to deactivated on bootup and will only be activated
 	  if some application like powertop activates it explicitly).
 config DEBUG_OBJECTS
 	bool "Debug object operations"
 	depends on DEBUG_KERNEL
 	help
 	  If you say Y here, additional code will be inserted into the
 	  kernel to track the life time of various objects and validate
 	  the operations on those objects.
 config DEBUG_OBJECTS_SELFTEST
 	bool "Debug objects selftest"
 	depends on DEBUG_OBJECTS
 	help
 	  This enables the selftest of the object debug code.
 config DEBUG_OBJECTS_FREE
 	bool "Debug objects in freed memory"
 	depends on DEBUG_OBJECTS
 	help
 	  This enables checks whether a k/v free operation frees an area
 	  which contains an object which has not been deactivated
 	  properly. This can make kmalloc/kfree-intensive workloads
 	  much slower.
 config DEBUG_OBJECTS_TIMERS
 	bool "Debug timer objects"
 	depends on DEBUG_OBJECTS
 	help
 	  If you say Y here, additional code will be inserted into the
 	  timer routines to track the life time of timer objects and
 	  validate the timer operations.
 config DEBUG_OBJECTS_ENABLE_DEFAULT
 	int "debug_objects bootup default value (0-1)"
         range 0 1
         default "1"
         depends on DEBUG_OBJECTS
         help
           Debug objects boot parameter default value
 config DEBUG_SLAB
 	bool "Debug slab memory allocations"
 	depends on DEBUG_KERNEL && SLAB
 	help
 	  Say Y here to have the kernel do limited verification on memory
 	  allocation as well as poisoning memory on free to catch use of freed
 	  memory. This can make kmalloc/kfree-intensive workloads much slower.
 config DEBUG_SLAB_LEAK
 	bool "Memory leak debugging"
 	depends on DEBUG_SLAB
 config SLUB_DEBUG_ON
 	bool "SLUB debugging on by default"
 	depends on SLUB && SLUB_DEBUG
 	default n
 	help
 	  Boot with debugging on by default. SLUB boots by default with
 	  the runtime debug capabilities switched off. Enabling this is
 	  equivalent to specifying the "slub_debug" parameter on boot.
 	  There is no support for more fine grained debug control like
 	  possible with slub_debug=xxx. SLUB debugging may be switched
 	  off in a kernel built with CONFIG_SLUB_DEBUG_ON by specifying
 	  "slub_debug=-".
 config SLUB_STATS
 	default n
 	bool "Enable SLUB performance statistics"
 	depends on SLUB && SLUB_DEBUG && SYSFS
 	help
 	  SLUB statistics are useful to debug SLUBs allocation behavior in
 	  order find ways to optimize the allocator. This should never be
 	  enabled for production use since keeping statistics slows down
 	  the allocator by a few percentage points. The slabinfo command
 	  supports the determination of the most active slabs to figure
 	  out which slabs are relevant to a particular load.
 	  Try running: slabinfo -DA
 config DEBUG_PREEMPT
 	bool "Debug preemptible kernel"
 	depends on DEBUG_KERNEL && PREEMPT && (TRACE_IRQFLAGS_SUPPORT || PPC64)
 	default y
 	help
 	  If you say Y here then the kernel will use a debug variant of the
 	  commonly used smp_processor_id() function and will print warnings
 	  if kernel code uses it in a preemption-unsafe way. Also, the kernel
 	  will detect preemption count underflows.
 config DEBUG_RT_MUTEXES
 	bool "RT Mutex debugging, deadlock detection"
 	depends on DEBUG_KERNEL && RT_MUTEXES
 	help
 	 This allows rt mutex semantics violations and rt mutex related
 	 deadlocks (lockups) to be detected and reported automatically.
 config DEBUG_PI_LIST
 	bool
 	default y
 	depends on DEBUG_RT_MUTEXES
 config RT_MUTEX_TESTER
 	bool "Built-in scriptable tester for rt-mutexes"
 	depends on DEBUG_KERNEL && RT_MUTEXES
 	help
 	  This option enables a rt-mutex tester.
 config DEBUG_SPINLOCK
 	bool "Spinlock and rw-lock debugging: basic checks"
 	depends on DEBUG_KERNEL
 	help
 	  Say Y here and build SMP to catch missing spinlock initialization
 	  and certain other kinds of spinlock errors commonly made.  This is
 	  best used in conjunction with the NMI watchdog so that spinlock
 	  deadlocks are also debuggable.
 config DEBUG_MUTEXES
 	bool "Mutex debugging: basic checks"
 	depends on DEBUG_KERNEL
 	help
 	 This feature allows mutex semantics violations to be detected and
 	 reported.
 config DEBUG_LOCK_ALLOC
 	bool "Lock debugging: detect incorrect freeing of live locks"
 	depends on DEBUG_KERNEL && TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT
 	select DEBUG_SPINLOCK
 	select DEBUG_MUTEXES
 	select LOCKDEP
 	help
 	 This feature will check whether any held lock (spinlock, rwlock,
 	 mutex or rwsem) is incorrectly freed by the kernel, via any of the
 	 memory-freeing routines (kfree(), kmem_cache_free(), free_pages(),
 	 vfree(), etc.), whether a live lock is incorrectly reinitialized via
 	 spin_lock_init()/mutex_init()/etc., or whether there is any lock
 	 held during task exit.
 config PROVE_LOCKING
 	bool "Lock debugging: prove locking correctness"
 	depends on DEBUG_KERNEL && TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT
 	select LOCKDEP
 	select DEBUG_SPINLOCK
 	select DEBUG_MUTEXES
 	select DEBUG_LOCK_ALLOC
 	default n
 	help
 	 This feature enables the kernel to prove that all locking
 	 that occurs in the kernel runtime is mathematically
 	 correct: that under no circumstance could an arbitrary (and
 	 not yet triggered) combination of observed locking
 	 sequences (on an arbitrary number of CPUs, running an
 	 arbitrary number of tasks and interrupt contexts) cause a
 	 deadlock.
 	 In short, this feature enables the kernel to report locking
 	 related deadlocks before they actually occur.
 	 The proof does not depend on how hard and complex a
 	 deadlock scenario would be to trigger: how many
 	 participant CPUs, tasks and irq-contexts would be needed
 	 for it to trigger. The proof also does not depend on
 	 timing: if a race and a resulting deadlock is possible
 	 theoretically (no matter how unlikely the race scenario
 	 is), it will be proven so and will immediately be
 	 reported by the kernel (once the event is observed that
 	 makes the deadlock theoretically possible).
 	 If a deadlock is impossible (i.e. the locking rules, as
 	 observed by the kernel, are mathematically correct), the
 	 kernel reports nothing.
 	 NOTE: this feature can also be enabled for rwlocks, mutexes
 	 and rwsems - in which case all dependencies between these
 	 different locking variants are observed and mapped too, and
 	 the proof of observed correctness is also maintained for an
 	 arbitrary combination of these separate locking variants.
 	 For more details, see Documentation/lockdep-design.txt.
 config LOCKDEP
 	bool
 	depends on DEBUG_KERNEL && TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT
 	select STACKTRACE
 	select FRAME_POINTER if !X86 && !MIPS && !PPC
 	select KALLSYMS
 	select KALLSYMS_ALL
 config LOCK_STAT
 	bool "Lock usage statistics"
 	depends on DEBUG_KERNEL && TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT
 	select LOCKDEP
 	select DEBUG_SPINLOCK
 	select DEBUG_MUTEXES
 	select DEBUG_LOCK_ALLOC
 	default n
 	help
 	 This feature enables tracking lock contention points
 	 For more details, see Documentation/lockstat.txt
 config DEBUG_LOCKDEP
 	bool "Lock dependency engine debugging"
 	depends on DEBUG_KERNEL && LOCKDEP
 	help
 	  If you say Y here, the lock dependency engine will do
 	  additional runtime checks to debug itself, at the price
 	  of more runtime overhead.
 config TRACE_IRQFLAGS
 	depends on DEBUG_KERNEL
 	bool
 	default y
 	depends on TRACE_IRQFLAGS_SUPPORT
 	depends on PROVE_LOCKING
 config DEBUG_SPINLOCK_SLEEP
 	bool "Spinlock debugging: sleep-inside-spinlock checking"
 	depends on DEBUG_KERNEL
 	help
 	  If you say Y here, various routines which may sleep will become very
 	  noisy if they are called with a spinlock held.
 config DEBUG_LOCKING_API_SELFTESTS
 	bool "Locking API boot-time self-tests"
 	depends on DEBUG_KERNEL
 	help
 	  Say Y here if you want the kernel to run a short self-test during
 	  bootup. The self-test checks whether common types of locking bugs
 	  are detected by debugging mechanisms or not. (if you disable
 	  lock debugging then those bugs wont be detected of course.)
 	  The following locking APIs are covered: spinlocks, rwlocks,
 	  mutexes and rwsems.
 config STACKTRACE
 	bool
 	depends on STACKTRACE_SUPPORT
 config DEBUG_KOBJECT
 	bool "kobject debugging"
 	depends on DEBUG_KERNEL
 	help
 	  If you say Y here, some extra kobject debugging messages will be sent
 	  to the syslog.
 config DEBUG_HIGHMEM
 	bool "Highmem debugging"
 	depends on DEBUG_KERNEL && HIGHMEM
 	help
 	  This options enables addition error checking for high memory systems.
 	  Disable for production systems.
 config DEBUG_BUGVERBOSE
 	bool "Verbose BUG() reporting (adds 70K)" if DEBUG_KERNEL && EMBEDDED
 	depends on BUG
 	depends on ARM || AVR32 || M32R || M68K || SPARC32 || SPARC64 || \
 		   FRV || SUPERH || GENERIC_BUG || BLACKFIN || MN10300
 	default !EMBEDDED
 	help
 	  Say Y here to make BUG() panics output the file name and line number
 	  of the BUG call as well as the EIP and oops trace.  This aids
 	  debugging but costs about 70-100K of memory.
 config DEBUG_INFO
 	bool "Compile the kernel with debug info"
 	depends on DEBUG_KERNEL
 	help
           If you say Y here the resulting kernel image will include
 	  debugging info resulting in a larger kernel image.
 	  This adds debug symbols to the kernel and modules (gcc -g), and
 	  is needed if you intend to use kernel crashdump or binary object
 	  tools like crash, kgdb, LKCD, gdb, etc on the kernel.
 	  Say Y here only if you plan to debug the kernel.
 	  If unsure, say N.
 config DEBUG_VM
 	bool "Debug VM"
 	depends on DEBUG_KERNEL
 	help
 	  Enable this to turn on extended checks in the virtual-memory system
           that may impact performance.
 	  If unsure, say N.
 config DEBUG_VIRTUAL
 	bool "Debug VM translations"
 	depends on DEBUG_KERNEL && X86
 	help
 	  Enable some costly sanity checks in virtual to page code. This can
 	  catch mistakes with virt_to_page() and friends.
 	  If unsure, say N.
+config DEBUG_NOMMU_REGIONS
+	bool "Debug the global anon/private NOMMU mapping region tree"
+	depends on DEBUG_KERNEL && !MMU
+	help
+	  This option causes the global tree of anonymous and private mapping
+	  regions to be regularly checked for invalid topology.
 config DEBUG_WRITECOUNT
 	bool "Debug filesystem writers count"
 	depends on DEBUG_KERNEL
 	help
 	  Enable this to catch wrong use of the writers count in struct
 	  vfsmount.  This will increase the size of each file struct by
 	  32 bits.
 	  If unsure, say N.
 config DEBUG_MEMORY_INIT
 	bool "Debug memory initialisation" if EMBEDDED
 	default !EMBEDDED
 	help
 	  Enable this for additional checks during memory initialisation.
 	  The sanity checks verify aspects of the VM such as the memory model
 	  and other information provided by the architecture. Verbose
 	  information will be printed at KERN_DEBUG loglevel depending
 	  on the mminit_loglevel= command-line option.
 	  If unsure, say Y
 config DEBUG_LIST
 	bool "Debug linked list manipulation"
 	depends on DEBUG_KERNEL
 	help
 	  Enable this to turn on extended checks in the linked-list
 	  walking routines.
 	  If unsure, say N.
 config DEBUG_SG
 	bool "Debug SG table operations"
 	depends on DEBUG_KERNEL
 	help
 	  Enable this to turn on checks on scatter-gather tables. This can
 	  help find problems with drivers that do not properly initialize
 	  their sg tables.
 	  If unsure, say N.
 config DEBUG_NOTIFIERS
 	bool "Debug notifier call chains"
 	depends on DEBUG_KERNEL
 	help
 	  Enable this to turn on sanity checking for notifier call chains.
 	  This is most useful for kernel developers to make sure that
 	  modules properly unregister themselves from notifier chains.
 	  This is a relatively cheap check but if you care about maximum
 	  performance, say N.
 config FRAME_POINTER
 	bool "Compile the kernel with frame pointers"
 	depends on DEBUG_KERNEL && \
 		(X86 || CRIS || M68K || M68KNOMMU || FRV || UML || S390 || \
 		 AVR32 || SUPERH || BLACKFIN || MN10300)
 	default y if DEBUG_INFO && UML
 	help
 	  If you say Y here the resulting kernel image will be slightly larger
 	  and slower, but it might give very useful debugging information on
 	  some architectures or if you use external debuggers.
 	  If you don't debug the kernel, you can say N.
 config BOOT_PRINTK_DELAY
 	bool "Delay each boot printk message by N milliseconds"
 	depends on DEBUG_KERNEL && PRINTK && GENERIC_CALIBRATE_DELAY
 	help
 	  This build option allows you to read kernel boot messages
 	  by inserting a short delay after each one.  The delay is
 	  specified in milliseconds on the kernel command line,
 	  using "boot_delay=N".
 	  It is likely that you would also need to use "lpj=M" to preset
 	  the "loops per jiffie" value.
 	  See a previous boot log for the "lpj" value to use for your
 	  system, and then set "lpj=M" before setting "boot_delay=N".
 	  NOTE:  Using this option may adversely affect SMP systems.
 	  I.e., processors other than the first one may not boot up.
 	  BOOT_PRINTK_DELAY also may cause DETECT_SOFTLOCKUP to detect
 	  what it believes to be lockup conditions.
 config RCU_TORTURE_TEST
 	tristate "torture tests for RCU"
 	depends on DEBUG_KERNEL
 	default n
 	help
 	  This option provides a kernel module that runs torture tests
 	  on the RCU infrastructure.  The kernel module may be built
 	  after the fact on the running kernel to be tested, if desired.
 	  Say Y here if you want RCU torture tests to be built into
 	  the kernel.
 	  Say M if you want the RCU torture tests to build as a module.
 	  Say N if you are unsure.
 config RCU_TORTURE_TEST_RUNNABLE
 	bool "torture tests for RCU runnable by default"
 	depends on RCU_TORTURE_TEST = y
 	default n
 	help
 	  This option provides a way to build the RCU torture tests
 	  directly into the kernel without them starting up at boot
 	  time.  You can use /proc/sys/kernel/rcutorture_runnable
 	  to manually override this setting.  This /proc file is
 	  available only when the RCU torture tests have been built
 	  into the kernel.
 	  Say Y here if you want the RCU torture tests to start during
 	  boot (you probably don't).
 	  Say N here if you want the RCU torture tests to start only
 	  after being manually enabled via /proc.
 config RCU_CPU_STALL_DETECTOR
 	bool "Check for stalled CPUs delaying RCU grace periods"
 	depends on CLASSIC_RCU
 	default n
 	help
 	  This option causes RCU to printk information on which
 	  CPUs are delaying the current grace period, but only when
 	  the grace period extends for excessive time periods.
 	  Say Y if you want RCU to perform such checks.
 	  Say N if you are unsure.
 config RCU_CPU_STALL_DETECTOR
 	bool "Check for stalled CPUs delaying RCU grace periods"
 	depends on CLASSIC_RCU || TREE_RCU
 	default n
 	help
 	  This option causes RCU to printk information on which
 	  CPUs are delaying the current grace period, but only when
 	  the grace period extends for excessive time periods.
 	  Say Y if you want RCU to perform such checks.
 	  Say N if you are unsure.
 config KPROBES_SANITY_TEST
 	bool "Kprobes sanity tests"
 	depends on DEBUG_KERNEL
 	depends on KPROBES
 	default n
 	help
 	  This option provides for testing basic kprobes functionality on
 	  boot. A sample kprobe, jprobe and kretprobe are inserted and
 	  verified for functionality.
 	  Say N if you are unsure.
 config BACKTRACE_SELF_TEST
 	tristate "Self test for the backtrace code"
 	depends on DEBUG_KERNEL
 	default n
 	help
 	  This option provides a kernel module that can be used to test
 	  the kernel stack backtrace code. This option is not useful
 	  for distributions or general kernels, but only for kernel
 	  developers working on architecture code.
 	  Note that if you want to also test saved backtraces, you will
 	  have to enable STACKTRACE as well.
 	  Say N if you are unsure.
 config DEBUG_BLOCK_EXT_DEVT
         bool "Force extended block device numbers and spread them"
 	depends on DEBUG_KERNEL
 	depends on BLOCK
 	default n
 	help
 	  BIG FAT WARNING: ENABLING THIS OPTION MIGHT BREAK BOOTING ON
 	  SOME DISTRIBUTIONS.  DO NOT ENABLE THIS UNLESS YOU KNOW WHAT
 	  YOU ARE DOING.  Distros, please enable this and fix whatever
 	  is broken.
 	  Conventionally, block device numbers are allocated from
 	  predetermined contiguous area.  However, extended block area
 	  may introduce non-contiguous block device numbers.  This
 	  option forces most block device numbers to be allocated from
 	  the extended space and spreads them to discover kernel or
 	  userland code paths which assume predetermined contiguous
 	  device number allocation.
 	  Note that turning on this debug option shuffles all the
 	  device numbers for all IDE and SCSI devices including libata
 	  ones, so root partition specified using device number
 	  directly (via rdev or root=MAJ:MIN) won't work anymore.
 	  Textual device names (root=/dev/sdXn) will continue to work.
 	  Say N if you are unsure.
 config LKDTM
 	tristate "Linux Kernel Dump Test Tool Module"
 	depends on DEBUG_KERNEL
 	depends on KPROBES
 	depends on BLOCK
 	default n
 	help
 	This module enables testing of the different dumping mechanisms by
 	inducing system failures at predefined crash points.
 	If you don't need it: say N
 	Choose M here to compile this code as a module. The module will be
 	called lkdtm.
 	Documentation on how to use the module can be found in
 	drivers/misc/lkdtm.c
 config FAULT_INJECTION
 	bool "Fault-injection framework"
 	depends on DEBUG_KERNEL
 	help
 	  Provide fault-injection framework.
 	  For more details, see Documentation/fault-injection/.
 config FAILSLAB
 	bool "Fault-injection capability for kmalloc"
 	depends on FAULT_INJECTION
 	depends on SLAB || SLUB
 	help
 	  Provide fault-injection capability for kmalloc.
 config FAIL_PAGE_ALLOC
 	bool "Fault-injection capabilitiy for alloc_pages()"
 	depends on FAULT_INJECTION
 	help
 	  Provide fault-injection capability for alloc_pages().
 config FAIL_MAKE_REQUEST
 	bool "Fault-injection capability for disk IO"
 	depends on FAULT_INJECTION && BLOCK
 	help
 	  Provide fault-injection capability for disk IO.
 config FAIL_IO_TIMEOUT
 	bool "Faul-injection capability for faking disk interrupts"
 	depends on FAULT_INJECTION && BLOCK
 	help
 	  Provide fault-injection capability on end IO handling. This
 	  will make the block layer "forget" an interrupt as configured,
 	  thus exercising the error handling.
 	  Only works with drivers that use the generic timeout handling,
 	  for others it wont do anything.
 config FAULT_INJECTION_DEBUG_FS
 	bool "Debugfs entries for fault-injection capabilities"
 	depends on FAULT_INJECTION && SYSFS && DEBUG_FS
 	help
 	  Enable configuration of fault-injection capabilities via debugfs.
 config FAULT_INJECTION_STACKTRACE_FILTER
 	bool "stacktrace filter for fault-injection capabilities"
 	depends on FAULT_INJECTION_DEBUG_FS && STACKTRACE_SUPPORT
 	depends on !X86_64
 	select STACKTRACE
 	select FRAME_POINTER if !PPC
 	help
 	  Provide stacktrace filter for fault-injection capabilities
 config LATENCYTOP
 	bool "Latency measuring infrastructure"
 	select FRAME_POINTER if !MIPS && !PPC
 	select KALLSYMS
 	select KALLSYMS_ALL
 	select STACKTRACE
 	select SCHEDSTATS
 	select SCHED_DEBUG
 	depends on HAVE_LATENCYTOP_SUPPORT
 	help
 	  Enable this option if you want to use the LatencyTOP tool
 	  to find out which userspace is blocking on what kernel operations.
 config SYSCTL_SYSCALL_CHECK
 	bool "Sysctl checks"
 	depends on SYSCTL_SYSCALL
 	---help---
 	  sys_sysctl uses binary paths that have been found challenging
 	  to properly maintain and use. This enables checks that help
 	  you to keep things correct.
 source kernel/trace/Kconfig
 config PROVIDE_OHCI1394_DMA_INIT
 	bool "Remote debugging over FireWire early on boot"
 	depends on PCI && X86
 	help
 	  If you want to debug problems which hang or crash the kernel early
 	  on boot and the crashing machine has a FireWire port, you can use
 	  this feature to remotely access the memory of the crashed machine
 	  over FireWire. This employs remote DMA as part of the OHCI1394
 	  specification which is now the standard for FireWire controllers.
 	  With remote DMA, you can monitor the printk buffer remotely using
 	  firescope and access all memory below 4GB using fireproxy from gdb.
 	  Even controlling a kernel debugger is possible using remote DMA.
 	  Usage:
 	  If ohci1394_dma=early is used as boot parameter, it will initialize
 	  all OHCI1394 controllers which are found in the PCI config space.
 	  As all changes to the FireWire bus such as enabling and disabling
 	  devices cause a bus reset and thereby disable remote DMA for all
 	  devices, be sure to have the cable plugged and FireWire enabled on
 	  the debugging host before booting the debug target for debugging.
 	  This code (~1k) is freed after boot. By then, the firewire stack
 	  in charge of the OHCI-1394 controllers should be used instead.
 	  See Documentation/debugging-via-ohci1394.txt for more information.
 config FIREWIRE_OHCI_REMOTE_DMA
 	bool "Remote debugging over FireWire with firewire-ohci"
 	depends on FIREWIRE_OHCI
 	help
 	  This option lets you use the FireWire bus for remote debugging
 	  with help of the firewire-ohci driver. It enables unfiltered
 	  remote DMA in firewire-ohci.
 	  See Documentation/debugging-via-ohci1394.txt for more information.
 	  If unsure, say N.
 menuconfig BUILD_DOCSRC
 	bool "Build targets in Documentation/ tree"
 	depends on HEADERS_CHECK
 	help
 	  This option attempts to build objects from the source files in the
 	  kernel Documentation/ tree.
 	  Say N if you are unsure.
 config DYNAMIC_PRINTK_DEBUG
 	bool "Enable dynamic printk() call support"
 	default n
 	depends on PRINTK
 	select PRINTK_DEBUG
 	help
 	  Compiles debug level messages into the kernel, which would not
 	  otherwise be available at runtime. These messages can then be
 	  enabled/disabled on a per module basis. This mechanism implicitly
 	  enables all pr_debug() and dev_dbg() calls. The impact of this
 	  compile option is a larger kernel text size of about 2%.
 	  Usage:
 	  Dynamic debugging is controlled by the debugfs file,
 	  dynamic_printk/modules. This file contains a list of the modules that
 	  can be enabled. The format of the file is the module name, followed
 	  by a set of flags that can be enabled. The first flag is always the
 	  'enabled' flag. For example:
 		<module_name> <enabled=0/1>
 				.
 				.
 				.
 	  <module_name> : Name of the module in which the debug call resides
 	  <enabled=0/1> : whether the messages are enabled or not
 	  From a live system:
 		snd_hda_intel enabled=0
 		fixup enabled=0
 		driver enabled=0
 	  Enable a module:
 	  	$echo "set enabled=1 <module_name>" > dynamic_printk/modules
 	  Disable a module:
 	  	$echo "set enabled=0 <module_name>" > dynamic_printk/modules
 	  Enable all modules:
 		$echo "set enabled=1 all" > dynamic_printk/modules
 	  Disable all modules:
 		$echo "set enabled=0 all" > dynamic_printk/modules
 	  Finally, passing "dynamic_printk" at the command line enables
 	  debugging for all modules. This mode can be turned off via the above
 	  disable command.
 source "samples/Kconfig"
 source "lib/Kconfig.kgdb"

mm/mmap.c

Diff comments View file @ c40f6f8

 /*
  * mm/mmap.c
  *
  * Written by obz.
  *
  * Address space accounting code	<alan@lxorguk.ukuu.org.uk>
  */
 #include <linux/slab.h>
 #include <linux/backing-dev.h>
 #include <linux/mm.h>
 #include <linux/shm.h>
 #include <linux/mman.h>
 #include <linux/pagemap.h>
 #include <linux/swap.h>
 #include <linux/syscalls.h>
 #include <linux/capability.h>
 #include <linux/init.h>
 #include <linux/file.h>
 #include <linux/fs.h>
 #include <linux/personality.h>
 #include <linux/security.h>
 #include <linux/hugetlb.h>
 #include <linux/profile.h>
 #include <linux/module.h>
 #include <linux/mount.h>
 #include <linux/mempolicy.h>
 #include <linux/rmap.h>
 #include <linux/mmu_notifier.h>
 #include <asm/uaccess.h>
 #include <asm/cacheflush.h>
 #include <asm/tlb.h>
 #include <asm/mmu_context.h>
 #include "internal.h"
 #ifndef arch_mmap_check
 #define arch_mmap_check(addr, len, flags)	(0)
 #endif
 #ifndef arch_rebalance_pgtables
 #define arch_rebalance_pgtables(addr, len)		(addr)
 #endif
 static void unmap_region(struct mm_struct *mm,
 		struct vm_area_struct *vma, struct vm_area_struct *prev,
 		unsigned long start, unsigned long end);
 /*
  * WARNING: the debugging will use recursive algorithms so never enable this
  * unless you know what you are doing.
  */
 #undef DEBUG_MM_RB
 /* description of effects of mapping type and prot in current implementation.
  * this is due to the limited x86 page protection hardware.  The expected
  * behavior is in parens:
  *
  * map_type	prot
  *		PROT_NONE	PROT_READ	PROT_WRITE	PROT_EXEC
  * MAP_SHARED	r: (no) no	r: (yes) yes	r: (no) yes	r: (no) yes
  *		w: (no) no	w: (no) no	w: (yes) yes	w: (no) no
  *		x: (no) no	x: (no) yes	x: (no) yes	x: (yes) yes
  *
  * MAP_PRIVATE	r: (no) no	r: (yes) yes	r: (no) yes	r: (no) yes
  *		w: (no) no	w: (no) no	w: (copy) copy	w: (no) no
  *		x: (no) no	x: (no) yes	x: (no) yes	x: (yes) yes
  *
  */
 pgprot_t protection_map[16] = {
 	__P000, __P001, __P010, __P011, __P100, __P101, __P110, __P111,
 	__S000, __S001, __S010, __S011, __S100, __S101, __S110, __S111
 };
 pgprot_t vm_get_page_prot(unsigned long vm_flags)
 {
 	return __pgprot(pgprot_val(protection_map[vm_flags &
 				(VM_READ|VM_WRITE|VM_EXEC|VM_SHARED)]) |
 			pgprot_val(arch_vm_get_page_prot(vm_flags)));
 }
 EXPORT_SYMBOL(vm_get_page_prot);
 int sysctl_overcommit_memory = OVERCOMMIT_GUESS;  /* heuristic overcommit */
 int sysctl_overcommit_ratio = 50;	/* default is 50% */
 int sysctl_max_map_count __read_mostly = DEFAULT_MAX_MAP_COUNT;
 atomic_long_t vm_committed_space = ATOMIC_LONG_INIT(0);
 /*
  * Check that a process has enough memory to allocate a new virtual
  * mapping. 0 means there is enough memory for the allocation to
  * succeed and -ENOMEM implies there is not.
  *
  * We currently support three overcommit policies, which are set via the
  * vm.overcommit_memory sysctl.  See Documentation/vm/overcommit-accounting
  *
  * Strict overcommit modes added 2002 Feb 26 by Alan Cox.
  * Additional code 2002 Jul 20 by Robert Love.
  *
  * cap_sys_admin is 1 if the process has admin privileges, 0 otherwise.
  *
  * Note this is a helper function intended to be used by LSMs which
  * wish to use this logic.
  */
 int __vm_enough_memory(struct mm_struct *mm, long pages, int cap_sys_admin)
 {
 	unsigned long free, allowed;
 	vm_acct_memory(pages);
 	/*
 	 * Sometimes we want to use more memory than we have
 	 */
 	if (sysctl_overcommit_memory == OVERCOMMIT_ALWAYS)
 		return 0;
 	if (sysctl_overcommit_memory == OVERCOMMIT_GUESS) {
 		unsigned long n;
 		free = global_page_state(NR_FILE_PAGES);
 		free += nr_swap_pages;
 		/*
 		 * Any slabs which are created with the
 		 * SLAB_RECLAIM_ACCOUNT flag claim to have contents
 		 * which are reclaimable, under pressure.  The dentry
 		 * cache and most inode caches should fall into this
 		 */
 		free += global_page_state(NR_SLAB_RECLAIMABLE);
 		/*
 		 * Leave the last 3% for root
 		 */
 		if (!cap_sys_admin)
 			free -= free / 32;
 		if (free > pages)
 			return 0;
 		/*
 		 * nr_free_pages() is very expensive on large systems,
 		 * only call if we're about to fail.
 		 */
 		n = nr_free_pages();
 		/*
 		 * Leave reserved pages. The pages are not for anonymous pages.
 		 */
 		if (n <= totalreserve_pages)
 			goto error;
 		else
 			n -= totalreserve_pages;
 		/*
 		 * Leave the last 3% for root
 		 */
 		if (!cap_sys_admin)
 			n -= n / 32;
 		free += n;
 		if (free > pages)
 			return 0;
 		goto error;
 	}
 	allowed = (totalram_pages - hugetlb_total_pages())
 	       	* sysctl_overcommit_ratio / 100;
 	/*
 	 * Leave the last 3% for root
 	 */
 	if (!cap_sys_admin)
 		allowed -= allowed / 32;
 	allowed += total_swap_pages;
 	/* Don't let a single process grow too big:
 	   leave 3% of the size of this process for other processes */
 	if (mm)
 		allowed -= mm->total_vm / 32;
 	/*
 	 * cast `allowed' as a signed long because vm_committed_space
 	 * sometimes has a negative value
 	 */
 	if (atomic_long_read(&vm_committed_space) < (long)allowed)
 		return 0;
 error:
 	vm_unacct_memory(pages);
 	return -ENOMEM;
 }
 /*
  * Requires inode->i_mapping->i_mmap_lock
  */
 static void __remove_shared_vm_struct(struct vm_area_struct *vma,
 		struct file *file, struct address_space *mapping)
 {
 	if (vma->vm_flags & VM_DENYWRITE)
 		atomic_inc(&file->f_path.dentry->d_inode->i_writecount);
 	if (vma->vm_flags & VM_SHARED)
 		mapping->i_mmap_writable--;
 	flush_dcache_mmap_lock(mapping);
 	if (unlikely(vma->vm_flags & VM_NONLINEAR))
 		list_del_init(&vma->shared.vm_set.list);
 	else
 		vma_prio_tree_remove(vma, &mapping->i_mmap);
 	flush_dcache_mmap_unlock(mapping);
 }
 /*
  * Unlink a file-based vm structure from its prio_tree, to hide
  * vma from rmap and vmtruncate before freeing its page tables.
  */
 void unlink_file_vma(struct vm_area_struct *vma)
 {
 	struct file *file = vma->vm_file;
 	if (file) {
 		struct address_space *mapping = file->f_mapping;
 		spin_lock(&mapping->i_mmap_lock);
 		__remove_shared_vm_struct(vma, file, mapping);
 		spin_unlock(&mapping->i_mmap_lock);
 	}
 }
 /*
  * Close a vm structure and free it, returning the next.
  */
 static struct vm_area_struct *remove_vma(struct vm_area_struct *vma)
 {
 	struct vm_area_struct *next = vma->vm_next;
 	might_sleep();
 	if (vma->vm_ops && vma->vm_ops->close)
 		vma->vm_ops->close(vma);
 	if (vma->vm_file) {
 		fput(vma->vm_file);
 		if (vma->vm_flags & VM_EXECUTABLE)
 			removed_exe_file_vma(vma->vm_mm);
 	}
 	mpol_put(vma_policy(vma));
 	kmem_cache_free(vm_area_cachep, vma);
 	return next;
 }
 asmlinkage unsigned long sys_brk(unsigned long brk)
 {
 	unsigned long rlim, retval;
 	unsigned long newbrk, oldbrk;
 	struct mm_struct *mm = current->mm;
 	unsigned long min_brk;
 	down_write(&mm->mmap_sem);
 #ifdef CONFIG_COMPAT_BRK
 	min_brk = mm->end_code;
 #else
 	min_brk = mm->start_brk;
 #endif
 	if (brk < min_brk)
 		goto out;
 	/*
 	 * Check against rlimit here. If this check is done later after the test
 	 * of oldbrk with newbrk then it can escape the test and let the data
 	 * segment grow beyond its set limit the in case where the limit is
 	 * not page aligned -Ram Gupta
 	 */
 	rlim = current->signal->rlim[RLIMIT_DATA].rlim_cur;
 	if (rlim < RLIM_INFINITY && (brk - mm->start_brk) +
 			(mm->end_data - mm->start_data) > rlim)
 		goto out;
 	newbrk = PAGE_ALIGN(brk);
 	oldbrk = PAGE_ALIGN(mm->brk);
 	if (oldbrk == newbrk)
 		goto set_brk;
 	/* Always allow shrinking brk. */
 	if (brk <= mm->brk) {
 		if (!do_munmap(mm, newbrk, oldbrk-newbrk))
 			goto set_brk;
 		goto out;
 	}
 	/* Check against existing mmap mappings. */
 	if (find_vma_intersection(mm, oldbrk, newbrk+PAGE_SIZE))
 		goto out;
 	/* Ok, looks good - let it rip. */
 	if (do_brk(oldbrk, newbrk-oldbrk) != oldbrk)
 		goto out;
 set_brk:
 	mm->brk = brk;
 out:
 	retval = mm->brk;
 	up_write(&mm->mmap_sem);
 	return retval;
 }
 #ifdef DEBUG_MM_RB
 static int browse_rb(struct rb_root *root)
 {
 	int i = 0, j;
 	struct rb_node *nd, *pn = NULL;
 	unsigned long prev = 0, pend = 0;
 	for (nd = rb_first(root); nd; nd = rb_next(nd)) {
 		struct vm_area_struct *vma;
 		vma = rb_entry(nd, struct vm_area_struct, vm_rb);
 		if (vma->vm_start < prev)
 			printk("vm_start %lx prev %lx\n", vma->vm_start, prev), i = -1;
 		if (vma->vm_start < pend)
 			printk("vm_start %lx pend %lx\n", vma->vm_start, pend);
 		if (vma->vm_start > vma->vm_end)
 			printk("vm_end %lx < vm_start %lx\n", vma->vm_end, vma->vm_start);
 		i++;
 		pn = nd;
 		prev = vma->vm_start;
 		pend = vma->vm_end;
 	}
 	j = 0;
 	for (nd = pn; nd; nd = rb_prev(nd)) {
 		j++;
 	}
 	if (i != j)
 		printk("backwards %d, forwards %d\n", j, i), i = 0;
 	return i;
 }
 void validate_mm(struct mm_struct *mm)
 {
 	int bug = 0;
 	int i = 0;
 	struct vm_area_struct *tmp = mm->mmap;
 	while (tmp) {
 		tmp = tmp->vm_next;
 		i++;
 	}
 	if (i != mm->map_count)
 		printk("map_count %d vm_next %d\n", mm->map_count, i), bug = 1;
 	i = browse_rb(&mm->mm_rb);
 	if (i != mm->map_count)
 		printk("map_count %d rb %d\n", mm->map_count, i), bug = 1;
 	BUG_ON(bug);
 }
 #else
 #define validate_mm(mm) do { } while (0)
 #endif
 static struct vm_area_struct *
 find_vma_prepare(struct mm_struct *mm, unsigned long addr,
 		struct vm_area_struct **pprev, struct rb_node ***rb_link,
 		struct rb_node ** rb_parent)
 {
 	struct vm_area_struct * vma;
 	struct rb_node ** __rb_link, * __rb_parent, * rb_prev;
 	__rb_link = &mm->mm_rb.rb_node;
 	rb_prev = __rb_parent = NULL;
 	vma = NULL;
 	while (*__rb_link) {
 		struct vm_area_struct *vma_tmp;
 		__rb_parent = *__rb_link;
 		vma_tmp = rb_entry(__rb_parent, struct vm_area_struct, vm_rb);
 		if (vma_tmp->vm_end > addr) {
 			vma = vma_tmp;
 			if (vma_tmp->vm_start <= addr)
 				break;
 			__rb_link = &__rb_parent->rb_left;
 		} else {
 			rb_prev = __rb_parent;
 			__rb_link = &__rb_parent->rb_right;
 		}
 	}
 	*pprev = NULL;
 	if (rb_prev)
 		*pprev = rb_entry(rb_prev, struct vm_area_struct, vm_rb);
 	*rb_link = __rb_link;
 	*rb_parent = __rb_parent;
 	return vma;
 }
 static inline void
 __vma_link_list(struct mm_struct *mm, struct vm_area_struct *vma,
 		struct vm_area_struct *prev, struct rb_node *rb_parent)
 {
 	if (prev) {
 		vma->vm_next = prev->vm_next;
 		prev->vm_next = vma;
 	} else {
 		mm->mmap = vma;
 		if (rb_parent)
 			vma->vm_next = rb_entry(rb_parent,
 					struct vm_area_struct, vm_rb);
 		else
 			vma->vm_next = NULL;
 	}
 }
 void __vma_link_rb(struct mm_struct *mm, struct vm_area_struct *vma,
 		struct rb_node **rb_link, struct rb_node *rb_parent)
 {
 	rb_link_node(&vma->vm_rb, rb_parent, rb_link);
 	rb_insert_color(&vma->vm_rb, &mm->mm_rb);
 }
 static void __vma_link_file(struct vm_area_struct *vma)
 {
 	struct file *file;
 	file = vma->vm_file;
 	if (file) {
 		struct address_space *mapping = file->f_mapping;
 		if (vma->vm_flags & VM_DENYWRITE)
 			atomic_dec(&file->f_path.dentry->d_inode->i_writecount);
 		if (vma->vm_flags & VM_SHARED)
 			mapping->i_mmap_writable++;
 		flush_dcache_mmap_lock(mapping);
 		if (unlikely(vma->vm_flags & VM_NONLINEAR))
 			vma_nonlinear_insert(vma, &mapping->i_mmap_nonlinear);
 		else
 			vma_prio_tree_insert(vma, &mapping->i_mmap);
 		flush_dcache_mmap_unlock(mapping);
 	}
 }
 static void
 __vma_link(struct mm_struct *mm, struct vm_area_struct *vma,
 	struct vm_area_struct *prev, struct rb_node **rb_link,
 	struct rb_node *rb_parent)
 {
 	__vma_link_list(mm, vma, prev, rb_parent);
 	__vma_link_rb(mm, vma, rb_link, rb_parent);
 	__anon_vma_link(vma);
 }
 static void vma_link(struct mm_struct *mm, struct vm_area_struct *vma,
 			struct vm_area_struct *prev, struct rb_node **rb_link,
 			struct rb_node *rb_parent)
 {
 	struct address_space *mapping = NULL;
 	if (vma->vm_file)
 		mapping = vma->vm_file->f_mapping;
 	if (mapping) {
 		spin_lock(&mapping->i_mmap_lock);
 		vma->vm_truncate_count = mapping->truncate_count;
 	}
 	anon_vma_lock(vma);
 	__vma_link(mm, vma, prev, rb_link, rb_parent);
 	__vma_link_file(vma);
 	anon_vma_unlock(vma);
 	if (mapping)
 		spin_unlock(&mapping->i_mmap_lock);
 	mm->map_count++;
 	validate_mm(mm);
 }
 /*
  * Helper for vma_adjust in the split_vma insert case:
  * insert vm structure into list and rbtree and anon_vma,
  * but it has already been inserted into prio_tree earlier.
  */
 static void __insert_vm_struct(struct mm_struct *mm, struct vm_area_struct *vma)
 {
 	struct vm_area_struct *__vma, *prev;
 	struct rb_node **rb_link, *rb_parent;
 	__vma = find_vma_prepare(mm, vma->vm_start,&prev, &rb_link, &rb_parent);
 	BUG_ON(__vma && __vma->vm_start < vma->vm_end);
 	__vma_link(mm, vma, prev, rb_link, rb_parent);
 	mm->map_count++;
 }
 static inline void
 __vma_unlink(struct mm_struct *mm, struct vm_area_struct *vma,
 		struct vm_area_struct *prev)
 {
 	prev->vm_next = vma->vm_next;
 	rb_erase(&vma->vm_rb, &mm->mm_rb);
 	if (mm->mmap_cache == vma)
 		mm->mmap_cache = prev;
 }
 /*
  * We cannot adjust vm_start, vm_end, vm_pgoff fields of a vma that
  * is already present in an i_mmap tree without adjusting the tree.
  * The following helper function should be used when such adjustments
  * are necessary.  The "insert" vma (if any) is to be inserted
  * before we drop the necessary locks.
  */
 void vma_adjust(struct vm_area_struct *vma, unsigned long start,
 	unsigned long end, pgoff_t pgoff, struct vm_area_struct *insert)
 {
 	struct mm_struct *mm = vma->vm_mm;
 	struct vm_area_struct *next = vma->vm_next;
 	struct vm_area_struct *importer = NULL;
 	struct address_space *mapping = NULL;
 	struct prio_tree_root *root = NULL;
 	struct file *file = vma->vm_file;
 	struct anon_vma *anon_vma = NULL;
 	long adjust_next = 0;
 	int remove_next = 0;
 	if (next && !insert) {
 		if (end >= next->vm_end) {
 			/*
 			 * vma expands, overlapping all the next, and
 			 * perhaps the one after too (mprotect case 6).
 			 */
 again:			remove_next = 1 + (end > next->vm_end);
 			end = next->vm_end;
 			anon_vma = next->anon_vma;
 			importer = vma;
 		} else if (end > next->vm_start) {
 			/*
 			 * vma expands, overlapping part of the next:
 			 * mprotect case 5 shifting the boundary up.
 			 */
 			adjust_next = (end - next->vm_start) >> PAGE_SHIFT;
 			anon_vma = next->anon_vma;
 			importer = vma;
 		} else if (end < vma->vm_end) {
 			/*
 			 * vma shrinks, and !insert tells it's not
 			 * split_vma inserting another: so it must be
 			 * mprotect case 4 shifting the boundary down.
 			 */
 			adjust_next = - ((vma->vm_end - end) >> PAGE_SHIFT);
 			anon_vma = next->anon_vma;
 			importer = next;
 		}
 	}
 	if (file) {
 		mapping = file->f_mapping;
 		if (!(vma->vm_flags & VM_NONLINEAR))
 			root = &mapping->i_mmap;
 		spin_lock(&mapping->i_mmap_lock);
 		if (importer &&
 		    vma->vm_truncate_count != next->vm_truncate_count) {
 			/*
 			 * unmap_mapping_range might be in progress:
 			 * ensure that the expanding vma is rescanned.
 			 */
 			importer->vm_truncate_count = 0;
 		}
 		if (insert) {
 			insert->vm_truncate_count = vma->vm_truncate_count;
 			/*
 			 * Put into prio_tree now, so instantiated pages
 			 * are visible to arm/parisc __flush_dcache_page
 			 * throughout; but we cannot insert into address
 			 * space until vma start or end is updated.
 			 */
 			__vma_link_file(insert);
 		}
 	}
 	/*
 	 * When changing only vma->vm_end, we don't really need
 	 * anon_vma lock: but is that case worth optimizing out?
 	 */
 	if (vma->anon_vma)
 		anon_vma = vma->anon_vma;
 	if (anon_vma) {
 		spin_lock(&anon_vma->lock);
 		/*
 		 * Easily overlooked: when mprotect shifts the boundary,
 		 * make sure the expanding vma has anon_vma set if the
 		 * shrinking vma had, to cover any anon pages imported.
 		 */
 		if (importer && !importer->anon_vma) {
 			importer->anon_vma = anon_vma;
 			__anon_vma_link(importer);
 		}
 	}
 	if (root) {
 		flush_dcache_mmap_lock(mapping);
 		vma_prio_tree_remove(vma, root);
 		if (adjust_next)
 			vma_prio_tree_remove(next, root);
 	}
 	vma->vm_start = start;
 	vma->vm_end = end;
 	vma->vm_pgoff = pgoff;
 	if (adjust_next) {
 		next->vm_start += adjust_next << PAGE_SHIFT;
 		next->vm_pgoff += adjust_next;
 	}
 	if (root) {
 		if (adjust_next)
 			vma_prio_tree_insert(next, root);
 		vma_prio_tree_insert(vma, root);
 		flush_dcache_mmap_unlock(mapping);
 	}
 	if (remove_next) {
 		/*
 		 * vma_merge has merged next into vma, and needs
 		 * us to remove next before dropping the locks.
 		 */
 		__vma_unlink(mm, next, vma);
 		if (file)
 			__remove_shared_vm_struct(next, file, mapping);
 		if (next->anon_vma)
 			__anon_vma_merge(vma, next);
 	} else if (insert) {
 		/*
 		 * split_vma has split insert from vma, and needs
 		 * us to insert it before dropping the locks
 		 * (it may either follow vma or precede it).
 		 */
 		__insert_vm_struct(mm, insert);
 	}
 	if (anon_vma)
 		spin_unlock(&anon_vma->lock);
 	if (mapping)
 		spin_unlock(&mapping->i_mmap_lock);
 	if (remove_next) {
 		if (file) {
 			fput(file);
 			if (next->vm_flags & VM_EXECUTABLE)
 				removed_exe_file_vma(mm);
 		}
 		mm->map_count--;
 		mpol_put(vma_policy(next));
 		kmem_cache_free(vm_area_cachep, next);
 		/*
 		 * In mprotect's case 6 (see comments on vma_merge),
 		 * we must remove another next too. It would clutter
 		 * up the code too much to do both in one go.
 		 */
 		if (remove_next == 2) {
 			next = vma->vm_next;
 			goto again;
 		}
 	}
 	validate_mm(mm);
 }
 /*
  * If the vma has a ->close operation then the driver probably needs to release
  * per-vma resources, so we don't attempt to merge those.
  */
 static inline int is_mergeable_vma(struct vm_area_struct *vma,
 			struct file *file, unsigned long vm_flags)
 {
 	if (vma->vm_flags != vm_flags)
 		return 0;
 	if (vma->vm_file != file)
 		return 0;
 	if (vma->vm_ops && vma->vm_ops->close)
 		return 0;
 	return 1;
 }
 static inline int is_mergeable_anon_vma(struct anon_vma *anon_vma1,
 					struct anon_vma *anon_vma2)
 {
 	return !anon_vma1 || !anon_vma2 || (anon_vma1 == anon_vma2);
 }
 /*
  * Return true if we can merge this (vm_flags,anon_vma,file,vm_pgoff)
  * in front of (at a lower virtual address and file offset than) the vma.
  *
  * We cannot merge two vmas if they have differently assigned (non-NULL)
  * anon_vmas, nor if same anon_vma is assigned but offsets incompatible.
  *
  * We don't check here for the merged mmap wrapping around the end of pagecache
  * indices (16TB on ia32) because do_mmap_pgoff() does not permit mmap's which
  * wrap, nor mmaps which cover the final page at index -1UL.
  */
 static int
 can_vma_merge_before(struct vm_area_struct *vma, unsigned long vm_flags,
 	struct anon_vma *anon_vma, struct file *file, pgoff_t vm_pgoff)
 {
 	if (is_mergeable_vma(vma, file, vm_flags) &&
 	    is_mergeable_anon_vma(anon_vma, vma->anon_vma)) {
 		if (vma->vm_pgoff == vm_pgoff)
 			return 1;
 	}
 	return 0;
 }
 /*
  * Return true if we can merge this (vm_flags,anon_vma,file,vm_pgoff)
  * beyond (at a higher virtual address and file offset than) the vma.
  *
  * We cannot merge two vmas if they have differently assigned (non-NULL)
  * anon_vmas, nor if same anon_vma is assigned but offsets incompatible.
  */
 static int
 can_vma_merge_after(struct vm_area_struct *vma, unsigned long vm_flags,
 	struct anon_vma *anon_vma, struct file *file, pgoff_t vm_pgoff)
 {
 	if (is_mergeable_vma(vma, file, vm_flags) &&
 	    is_mergeable_anon_vma(anon_vma, vma->anon_vma)) {
 		pgoff_t vm_pglen;
 		vm_pglen = (vma->vm_end - vma->vm_start) >> PAGE_SHIFT;
 		if (vma->vm_pgoff + vm_pglen == vm_pgoff)
 			return 1;
 	}
 	return 0;
 }
 /*
  * Given a mapping request (addr,end,vm_flags,file,pgoff), figure out
  * whether that can be merged with its predecessor or its successor.
  * Or both (it neatly fills a hole).
  *
  * In most cases - when called for mmap, brk or mremap - [addr,end) is
  * certain not to be mapped by the time vma_merge is called; but when
  * called for mprotect, it is certain to be already mapped (either at
  * an offset within prev, or at the start of next), and the flags of
  * this area are about to be changed to vm_flags - and the no-change
  * case has already been eliminated.
  *
  * The following mprotect cases have to be considered, where AAAA is
  * the area passed down from mprotect_fixup, never extending beyond one
  * vma, PPPPPP is the prev vma specified, and NNNNNN the next vma after:
  *
  *     AAAA             AAAA                AAAA          AAAA
  *    PPPPPPNNNNNN    PPPPPPNNNNNN    PPPPPPNNNNNN    PPPPNNNNXXXX
  *    cannot merge    might become    might become    might become
  *                    PPNNNNNNNNNN    PPPPPPPPPPNN    PPPPPPPPPPPP 6 or
  *    mmap, brk or    case 4 below    case 5 below    PPPPPPPPXXXX 7 or
  *    mremap move:                                    PPPPNNNNNNNN 8
  *        AAAA
  *    PPPP    NNNN    PPPPPPPPPPPP    PPPPPPPPNNNN    PPPPNNNNNNNN
  *    might become    case 1 below    case 2 below    case 3 below
  *
  * Odd one out? Case 8, because it extends NNNN but needs flags of XXXX:
  * mprotect_fixup updates vm_flags & vm_page_prot on successful return.
  */
 struct vm_area_struct *vma_merge(struct mm_struct *mm,
 			struct vm_area_struct *prev, unsigned long addr,
 			unsigned long end, unsigned long vm_flags,
 		     	struct anon_vma *anon_vma, struct file *file,
 			pgoff_t pgoff, struct mempolicy *policy)
 {
 	pgoff_t pglen = (end - addr) >> PAGE_SHIFT;
 	struct vm_area_struct *area, *next;
 	/*
 	 * We later require that vma->vm_flags == vm_flags,
 	 * so this tests vma->vm_flags & VM_SPECIAL, too.
 	 */
 	if (vm_flags & VM_SPECIAL)
 		return NULL;
 	if (prev)
 		next = prev->vm_next;
 	else
 		next = mm->mmap;
 	area = next;
 	if (next && next->vm_end == end)		/* cases 6, 7, 8 */
 		next = next->vm_next;
 	/*
 	 * Can it merge with the predecessor?
 	 */
 	if (prev && prev->vm_end == addr &&
   			mpol_equal(vma_policy(prev), policy) &&
 			can_vma_merge_after(prev, vm_flags,
 						anon_vma, file, pgoff)) {
 		/*
 		 * OK, it can.  Can we now merge in the successor as well?
 		 */
 		if (next && end == next->vm_start &&
 				mpol_equal(policy, vma_policy(next)) &&
 				can_vma_merge_before(next, vm_flags,
 					anon_vma, file, pgoff+pglen) &&
 				is_mergeable_anon_vma(prev->anon_vma,
 						      next->anon_vma)) {
 							/* cases 1, 6 */
 			vma_adjust(prev, prev->vm_start,
 				next->vm_end, prev->vm_pgoff, NULL);
 		} else					/* cases 2, 5, 7 */
 			vma_adjust(prev, prev->vm_start,
 				end, prev->vm_pgoff, NULL);
 		return prev;
 	}
 	/*
 	 * Can this new request be merged in front of next?
 	 */
 	if (next && end == next->vm_start &&
  			mpol_equal(policy, vma_policy(next)) &&
 			can_vma_merge_before(next, vm_flags,
 					anon_vma, file, pgoff+pglen)) {
 		if (prev && addr < prev->vm_end)	/* case 4 */
 			vma_adjust(prev, prev->vm_start,
 				addr, prev->vm_pgoff, NULL);
 		else					/* cases 3, 8 */
 			vma_adjust(area, addr, next->vm_end,
 				next->vm_pgoff - pglen, NULL);
 		return area;
 	}
 	return NULL;
 }
 /*
  * find_mergeable_anon_vma is used by anon_vma_prepare, to check
  * neighbouring vmas for a suitable anon_vma, before it goes off
  * to allocate a new anon_vma.  It checks because a repetitive
  * sequence of mprotects and faults may otherwise lead to distinct
  * anon_vmas being allocated, preventing vma merge in subsequent
  * mprotect.
  */
 struct anon_vma *find_mergeable_anon_vma(struct vm_area_struct *vma)
 {
 	struct vm_area_struct *near;
 	unsigned long vm_flags;
 	near = vma->vm_next;
 	if (!near)
 		goto try_prev;
 	/*
 	 * Since only mprotect tries to remerge vmas, match flags
 	 * which might be mprotected into each other later on.
 	 * Neither mlock nor madvise tries to remerge at present,
 	 * so leave their flags as obstructing a merge.
 	 */
 	vm_flags = vma->vm_flags & ~(VM_READ|VM_WRITE|VM_EXEC);
 	vm_flags |= near->vm_flags & (VM_READ|VM_WRITE|VM_EXEC);
 	if (near->anon_vma && vma->vm_end == near->vm_start &&
  			mpol_equal(vma_policy(vma), vma_policy(near)) &&
 			can_vma_merge_before(near, vm_flags,
 				NULL, vma->vm_file, vma->vm_pgoff +
 				((vma->vm_end - vma->vm_start) >> PAGE_SHIFT)))
 		return near->anon_vma;
 try_prev:
 	/*
 	 * It is potentially slow to have to call find_vma_prev here.
 	 * But it's only on the first write fault on the vma, not
 	 * every time, and we could devise a way to avoid it later
 	 * (e.g. stash info in next's anon_vma_node when assigning
 	 * an anon_vma, or when trying vma_merge).  Another time.
 	 */
 	BUG_ON(find_vma_prev(vma->vm_mm, vma->vm_start, &near) != vma);
 	if (!near)
 		goto none;
 	vm_flags = vma->vm_flags & ~(VM_READ|VM_WRITE|VM_EXEC);
 	vm_flags |= near->vm_flags & (VM_READ|VM_WRITE|VM_EXEC);
 	if (near->anon_vma && near->vm_end == vma->vm_start &&
   			mpol_equal(vma_policy(near), vma_policy(vma)) &&
 			can_vma_merge_after(near, vm_flags,
 				NULL, vma->vm_file, vma->vm_pgoff))
 		return near->anon_vma;
 none:
 	/*
 	 * There's no absolute need to look only at touching neighbours:
 	 * we could search further afield for "compatible" anon_vmas.
 	 * But it would probably just be a waste of time searching,
 	 * or lead to too many vmas hanging off the same anon_vma.
 	 * We're trying to allow mprotect remerging later on,
 	 * not trying to minimize memory used for anon_vmas.
 	 */
 	return NULL;
 }
 #ifdef CONFIG_PROC_FS
 void vm_stat_account(struct mm_struct *mm, unsigned long flags,
 						struct file *file, long pages)
 {
 	const unsigned long stack_flags
 		= VM_STACK_FLAGS & (VM_GROWSUP|VM_GROWSDOWN);
 	if (file) {
 		mm->shared_vm += pages;
 		if ((flags & (VM_EXEC|VM_WRITE)) == VM_EXEC)
 			mm->exec_vm += pages;
 	} else if (flags & stack_flags)
 		mm->stack_vm += pages;
 	if (flags & (VM_RESERVED|VM_IO))
 		mm->reserved_vm += pages;
 }
 #endif /* CONFIG_PROC_FS */
 /*
  * The caller must hold down_write(current->mm->mmap_sem).
  */
 unsigned long do_mmap_pgoff(struct file *file, unsigned long addr,
 			unsigned long len, unsigned long prot,
 			unsigned long flags, unsigned long pgoff)
 {
 	struct mm_struct * mm = current->mm;
 	struct inode *inode;
 	unsigned int vm_flags;
 	int error;
 	int accountable = 1;
 	unsigned long reqprot = prot;
 	/*
 	 * Does the application expect PROT_READ to imply PROT_EXEC?
 	 *
 	 * (the exception is when the underlying filesystem is noexec
 	 *  mounted, in which case we dont add PROT_EXEC.)
 	 */
 	if ((prot & PROT_READ) && (current->personality & READ_IMPLIES_EXEC))
 		if (!(file && (file->f_path.mnt->mnt_flags & MNT_NOEXEC)))
 			prot |= PROT_EXEC;
 	if (!len)
 		return -EINVAL;
 	if (!(flags & MAP_FIXED))
 		addr = round_hint_to_min(addr);
 	error = arch_mmap_check(addr, len, flags);
 	if (error)
 		return error;
 	/* Careful about overflows.. */
 	len = PAGE_ALIGN(len);
 	if (!len || len > TASK_SIZE)
 		return -ENOMEM;
 	/* offset overflow? */
 	if ((pgoff + (len >> PAGE_SHIFT)) < pgoff)
                return -EOVERFLOW;
 	/* Too many mappings? */
 	if (mm->map_count > sysctl_max_map_count)
 		return -ENOMEM;
 	/* Obtain the address to map to. we verify (or select) it and ensure
 	 * that it represents a valid section of the address space.
 	 */
 	addr = get_unmapped_area(file, addr, len, pgoff, flags);
 	if (addr & ~PAGE_MASK)
 		return addr;
 	/* Do simple checking here so the lower-level routines won't have
 	 * to. we assume access permissions have been handled by the open
 	 * of the memory object, so we don't do any here.
 	 */
 	vm_flags = calc_vm_prot_bits(prot) | calc_vm_flag_bits(flags) |
 			mm->def_flags | VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC;
 	if (flags & MAP_LOCKED) {
 		if (!can_do_mlock())
 			return -EPERM;
 		vm_flags |= VM_LOCKED;
 	}
 	/* mlock MCL_FUTURE? */
 	if (vm_flags & VM_LOCKED) {
 		unsigned long locked, lock_limit;
 		locked = len >> PAGE_SHIFT;
 		locked += mm->locked_vm;
 		lock_limit = current->signal->rlim[RLIMIT_MEMLOCK].rlim_cur;
 		lock_limit >>= PAGE_SHIFT;
 		if (locked > lock_limit && !capable(CAP_IPC_LOCK))
 			return -EAGAIN;
 	}
 	inode = file ? file->f_path.dentry->d_inode : NULL;
 	if (file) {
 		switch (flags & MAP_TYPE) {
 		case MAP_SHARED:
 			if ((prot&PROT_WRITE) && !(file->f_mode&FMODE_WRITE))
 				return -EACCES;
 			/*
 			 * Make sure we don't allow writing to an append-only
 			 * file..
 			 */
 			if (IS_APPEND(inode) && (file->f_mode & FMODE_WRITE))
 				return -EACCES;
 			/*
 			 * Make sure there are no mandatory locks on the file.
 			 */
 			if (locks_verify_locked(inode))
 				return -EAGAIN;
 			vm_flags |= VM_SHARED | VM_MAYSHARE;
 			if (!(file->f_mode & FMODE_WRITE))
 				vm_flags &= ~(VM_MAYWRITE | VM_SHARED);
 			/* fall through */
 		case MAP_PRIVATE:
 			if (!(file->f_mode & FMODE_READ))
 				return -EACCES;
 			if (file->f_path.mnt->mnt_flags & MNT_NOEXEC) {
 				if (vm_flags & VM_EXEC)
 					return -EPERM;
 				vm_flags &= ~VM_MAYEXEC;
 			}
 			if (is_file_hugepages(file))
 				accountable = 0;
 			if (!file->f_op || !file->f_op->mmap)
 				return -ENODEV;
 			break;
 		default:
 			return -EINVAL;
 		}
 	} else {
 		switch (flags & MAP_TYPE) {
 		case MAP_SHARED:
 			/*
 			 * Ignore pgoff.
 			 */
 			pgoff = 0;
 			vm_flags |= VM_SHARED | VM_MAYSHARE;
 			break;
 		case MAP_PRIVATE:
 			/*
 			 * Set pgoff according to addr for anon_vma.
 			 */
 			pgoff = addr >> PAGE_SHIFT;
 			break;
 		default:
 			return -EINVAL;
 		}
 	}
 	error = security_file_mmap(file, reqprot, prot, flags, addr, 0);
 	if (error)
 		return error;
 	return mmap_region(file, addr, len, flags, vm_flags, pgoff,
 			   accountable);
 }
 EXPORT_SYMBOL(do_mmap_pgoff);
 /*
  * Some shared mappigns will want the pages marked read-only
  * to track write events. If so, we'll downgrade vm_page_prot
  * to the private version (using protection_map[] without the
  * VM_SHARED bit).
  */
 int vma_wants_writenotify(struct vm_area_struct *vma)
 {
 	unsigned int vm_flags = vma->vm_flags;
 	/* If it was private or non-writable, the write bit is already clear */
 	if ((vm_flags & (VM_WRITE|VM_SHARED)) != ((VM_WRITE|VM_SHARED)))
 		return 0;
 	/* The backer wishes to know when pages are first written to? */
 	if (vma->vm_ops && vma->vm_ops->page_mkwrite)
 		return 1;
 	/* The open routine did something to the protections already? */
 	if (pgprot_val(vma->vm_page_prot) !=
 	    pgprot_val(vm_get_page_prot(vm_flags)))
 		return 0;
 	/* Specialty mapping? */
 	if (vm_flags & (VM_PFNMAP|VM_INSERTPAGE))
 		return 0;
 	/* Can the mapping track the dirty pages? */
 	return vma->vm_file && vma->vm_file->f_mapping &&
 		mapping_cap_account_dirty(vma->vm_file->f_mapping);
 }
 unsigned long mmap_region(struct file *file, unsigned long addr,
 			  unsigned long len, unsigned long flags,
 			  unsigned int vm_flags, unsigned long pgoff,
 			  int accountable)
 {
 	struct mm_struct *mm = current->mm;
 	struct vm_area_struct *vma, *prev;
 	int correct_wcount = 0;
 	int error;
 	struct rb_node **rb_link, *rb_parent;
 	unsigned long charged = 0;
 	struct inode *inode =  file ? file->f_path.dentry->d_inode : NULL;
 	/* Clear old maps */
 	error = -ENOMEM;
 munmap_back:
 	vma = find_vma_prepare(mm, addr, &prev, &rb_link, &rb_parent);
 	if (vma && vma->vm_start < addr + len) {
 		if (do_munmap(mm, addr, len))
 			return -ENOMEM;
 		goto munmap_back;
 	}
 	/* Check against address space limit. */
 	if (!may_expand_vm(mm, len >> PAGE_SHIFT))
 		return -ENOMEM;
 	if (flags & MAP_NORESERVE)
 		vm_flags |= VM_NORESERVE;
 	if (accountable && (!(flags & MAP_NORESERVE) ||
 			    sysctl_overcommit_memory == OVERCOMMIT_NEVER)) {
 		if (vm_flags & VM_SHARED) {
 			/* Check memory availability in shmem_file_setup? */
 			vm_flags |= VM_ACCOUNT;
 		} else if (vm_flags & VM_WRITE) {
 			/*
 			 * Private writable mapping: check memory availability
 			 */
 			charged = len >> PAGE_SHIFT;
 			if (security_vm_enough_memory(charged))
 				return -ENOMEM;
 			vm_flags |= VM_ACCOUNT;
 		}
 	}
 	/*
 	 * Can we just expand an old private anonymous mapping?
 	 * The VM_SHARED test is necessary because shmem_zero_setup
 	 * will create the file object for a shared anonymous map below.
 	 */
 	if (!file && !(vm_flags & VM_SHARED)) {
 		vma = vma_merge(mm, prev, addr, addr + len, vm_flags,
 					NULL, NULL, pgoff, NULL);
 		if (vma)
 			goto out;
 	}
 	/*
 	 * Determine the object being mapped and call the appropriate
 	 * specific mapper. the address has already been validated, but
 	 * not unmapped, but the maps are removed from the list.
 	 */
 	vma = kmem_cache_zalloc(vm_area_cachep, GFP_KERNEL);
 	if (!vma) {
 		error = -ENOMEM;
 		goto unacct_error;
 	}
 	vma->vm_mm = mm;
 	vma->vm_start = addr;
 	vma->vm_end = addr + len;
 	vma->vm_flags = vm_flags;
 	vma->vm_page_prot = vm_get_page_prot(vm_flags);
 	vma->vm_pgoff = pgoff;
 	if (file) {
 		error = -EINVAL;
 		if (vm_flags & (VM_GROWSDOWN|VM_GROWSUP))
 			goto free_vma;
 		if (vm_flags & VM_DENYWRITE) {
 			error = deny_write_access(file);
 			if (error)
 				goto free_vma;
 			correct_wcount = 1;
 		}
 		vma->vm_file = file;
 		get_file(file);
 		error = file->f_op->mmap(file, vma);
 		if (error)
 			goto unmap_and_free_vma;
 		if (vm_flags & VM_EXECUTABLE)
 			added_exe_file_vma(mm);
 	} else if (vm_flags & VM_SHARED) {
 		error = shmem_zero_setup(vma);
 		if (error)
 			goto free_vma;
 	}
 	/* We set VM_ACCOUNT in a shared mapping's vm_flags, to inform
 	 * shmem_zero_setup (perhaps called through /dev/zero's ->mmap)
 	 * that memory reservation must be checked; but that reservation
 	 * belongs to shared memory object, not to vma: so now clear it.
 	 */
 	if ((vm_flags & (VM_SHARED|VM_ACCOUNT)) == (VM_SHARED|VM_ACCOUNT))
 		vma->vm_flags &= ~VM_ACCOUNT;
 	/* Can addr have changed??
 	 *
 	 * Answer: Yes, several device drivers can do it in their
 	 *         f_op->mmap method. -DaveM
 	 */
 	addr = vma->vm_start;
 	pgoff = vma->vm_pgoff;
 	vm_flags = vma->vm_flags;
 	if (vma_wants_writenotify(vma))
 		vma->vm_page_prot = vm_get_page_prot(vm_flags & ~VM_SHARED);
 	if (file && vma_merge(mm, prev, addr, vma->vm_end,
 			vma->vm_flags, NULL, file, pgoff, vma_policy(vma))) {
 		mpol_put(vma_policy(vma));
 		kmem_cache_free(vm_area_cachep, vma);
 		fput(file);
 		if (vm_flags & VM_EXECUTABLE)
 			removed_exe_file_vma(mm);
 	} else {
 		vma_link(mm, vma, prev, rb_link, rb_parent);
 		file = vma->vm_file;
 	}
 	/* Once vma denies write, undo our temporary denial count */
 	if (correct_wcount)
 		atomic_inc(&inode->i_writecount);
 out:
 	mm->total_vm += len >> PAGE_SHIFT;
 	vm_stat_account(mm, vm_flags, file, len >> PAGE_SHIFT);
 	if (vm_flags & VM_LOCKED) {
 		/*
 		 * makes pages present; downgrades, drops, reacquires mmap_sem
 		 */
 		long nr_pages = mlock_vma_pages_range(vma, addr, addr + len);
 		if (nr_pages < 0)
 			return nr_pages;	/* vma gone! */
 		mm->locked_vm += (len >> PAGE_SHIFT) - nr_pages;
 	} else if ((flags & MAP_POPULATE) && !(flags & MAP_NONBLOCK))
 		make_pages_present(addr, addr + len);
 	return addr;
 unmap_and_free_vma:
 	if (correct_wcount)
 		atomic_inc(&inode->i_writecount);
 	vma->vm_file = NULL;
 	fput(file);
 	/* Undo any partial mapping done by a device driver. */
 	unmap_region(mm, vma, prev, vma->vm_start, vma->vm_end);
 	charged = 0;
 free_vma:
 	kmem_cache_free(vm_area_cachep, vma);
 unacct_error:
 	if (charged)
 		vm_unacct_memory(charged);
 	return error;
 }
 /* Get an address range which is currently unmapped.
  * For shmat() with addr=0.
  *
  * Ugly calling convention alert:
  * Return value with the low bits set means error value,
  * ie
  *	if (ret & ~PAGE_MASK)
  *		error = ret;
  *
  * This function "knows" that -ENOMEM has the bits set.
  */
 #ifndef HAVE_ARCH_UNMAPPED_AREA
 unsigned long
 arch_get_unmapped_area(struct file *filp, unsigned long addr,
 		unsigned long len, unsigned long pgoff, unsigned long flags)
 {
 	struct mm_struct *mm = current->mm;
 	struct vm_area_struct *vma;
 	unsigned long start_addr;
 	if (len > TASK_SIZE)
 		return -ENOMEM;
 	if (flags & MAP_FIXED)
 		return addr;
 	if (addr) {
 		addr = PAGE_ALIGN(addr);
 		vma = find_vma(mm, addr);
 		if (TASK_SIZE - len >= addr &&
 		    (!vma || addr + len <= vma->vm_start))
 			return addr;
 	}
 	if (len > mm->cached_hole_size) {
 	        start_addr = addr = mm->free_area_cache;
 	} else {
 	        start_addr = addr = TASK_UNMAPPED_BASE;
 	        mm->cached_hole_size = 0;
 	}
 full_search:
 	for (vma = find_vma(mm, addr); ; vma = vma->vm_next) {
 		/* At this point:  (!vma || addr < vma->vm_end). */
 		if (TASK_SIZE - len < addr) {
 			/*
 			 * Start a new search - just in case we missed
 			 * some holes.
 			 */
 			if (start_addr != TASK_UNMAPPED_BASE) {
 				addr = TASK_UNMAPPED_BASE;
 			        start_addr = addr;
 				mm->cached_hole_size = 0;
 				goto full_search;
 			}
 			return -ENOMEM;
 		}
 		if (!vma || addr + len <= vma->vm_start) {
 			/*
 			 * Remember the place where we stopped the search:
 			 */
 			mm->free_area_cache = addr + len;
 			return addr;
 		}
 		if (addr + mm->cached_hole_size < vma->vm_start)
 		        mm->cached_hole_size = vma->vm_start - addr;
 		addr = vma->vm_end;
 	}
 }
 #endif
 void arch_unmap_area(struct mm_struct *mm, unsigned long addr)
 {
 	/*
 	 * Is this a new hole at the lowest possible address?
 	 */
 	if (addr >= TASK_UNMAPPED_BASE && addr < mm->free_area_cache) {
 		mm->free_area_cache = addr;
 		mm->cached_hole_size = ~0UL;
 	}
 }
 /*
  * This mmap-allocator allocates new areas top-down from below the
  * stack's low limit (the base):
  */
 #ifndef HAVE_ARCH_UNMAPPED_AREA_TOPDOWN
 unsigned long
 arch_get_unmapped_area_topdown(struct file *filp, const unsigned long addr0,
 			  const unsigned long len, const unsigned long pgoff,
 			  const unsigned long flags)
 {
 	struct vm_area_struct *vma;
 	struct mm_struct *mm = current->mm;
 	unsigned long addr = addr0;
 	/* requested length too big for entire address space */
 	if (len > TASK_SIZE)
 		return -ENOMEM;
 	if (flags & MAP_FIXED)
 		return addr;
 	/* requesting a specific address */
 	if (addr) {
 		addr = PAGE_ALIGN(addr);
 		vma = find_vma(mm, addr);
 		if (TASK_SIZE - len >= addr &&
 				(!vma || addr + len <= vma->vm_start))
 			return addr;
 	}
 	/* check if free_area_cache is useful for us */
 	if (len <= mm->cached_hole_size) {
  	        mm->cached_hole_size = 0;
  		mm->free_area_cache = mm->mmap_base;
  	}
 	/* either no address requested or can't fit in requested address hole */
 	addr = mm->free_area_cache;
 	/* make sure it can fit in the remaining address space */
 	if (addr > len) {
 		vma = find_vma(mm, addr-len);
 		if (!vma || addr <= vma->vm_start)
 			/* remember the address as a hint for next time */
 			return (mm->free_area_cache = addr-len);
 	}
 	if (mm->mmap_base < len)
 		goto bottomup;
 	addr = mm->mmap_base-len;
 	do {
 		/*
 		 * Lookup failure means no vma is above this address,
 		 * else if new region fits below vma->vm_start,
 		 * return with success:
 		 */
 		vma = find_vma(mm, addr);
 		if (!vma || addr+len <= vma->vm_start)
 			/* remember the address as a hint for next time */
 			return (mm->free_area_cache = addr);
  		/* remember the largest hole we saw so far */
  		if (addr + mm->cached_hole_size < vma->vm_start)
  		        mm->cached_hole_size = vma->vm_start - addr;
 		/* try just below the current vma->vm_start */
 		addr = vma->vm_start-len;
 	} while (len < vma->vm_start);
 bottomup:
 	/*
 	 * A failed mmap() very likely causes application failure,
 	 * so fall back to the bottom-up function here. This scenario
 	 * can happen with large stack limits and large mmap()
 	 * allocations.
 	 */
 	mm->cached_hole_size = ~0UL;
   	mm->free_area_cache = TASK_UNMAPPED_BASE;
 	addr = arch_get_unmapped_area(filp, addr0, len, pgoff, flags);
 	/*
 	 * Restore the topdown base:
 	 */
 	mm->free_area_cache = mm->mmap_base;
 	mm->cached_hole_size = ~0UL;
 	return addr;
 }
 #endif
 void arch_unmap_area_topdown(struct mm_struct *mm, unsigned long addr)
 {
 	/*
 	 * Is this a new hole at the highest possible address?
 	 */
 	if (addr > mm->free_area_cache)
 		mm->free_area_cache = addr;
 	/* dont allow allocations above current base */
 	if (mm->free_area_cache > mm->mmap_base)
 		mm->free_area_cache = mm->mmap_base;
 }
 unsigned long
 get_unmapped_area(struct file *file, unsigned long addr, unsigned long len,
 		unsigned long pgoff, unsigned long flags)
 {
 	unsigned long (*get_area)(struct file *, unsigned long,
 				  unsigned long, unsigned long, unsigned long);
 	get_area = current->mm->get_unmapped_area;
 	if (file && file->f_op && file->f_op->get_unmapped_area)
 		get_area = file->f_op->get_unmapped_area;
 	addr = get_area(file, addr, len, pgoff, flags);
 	if (IS_ERR_VALUE(addr))
 		return addr;
 	if (addr > TASK_SIZE - len)
 		return -ENOMEM;
 	if (addr & ~PAGE_MASK)
 		return -EINVAL;
 	return arch_rebalance_pgtables(addr, len);
 }
 EXPORT_SYMBOL(get_unmapped_area);
 /* Look up the first VMA which satisfies  addr < vm_end,  NULL if none. */
 struct vm_area_struct *find_vma(struct mm_struct *mm, unsigned long addr)
 {
 	struct vm_area_struct *vma = NULL;
 	if (mm) {
 		/* Check the cache first. */
 		/* (Cache hit rate is typically around 35%.) */
 		vma = mm->mmap_cache;
 		if (!(vma && vma->vm_end > addr && vma->vm_start <= addr)) {
 			struct rb_node * rb_node;
 			rb_node = mm->mm_rb.rb_node;
 			vma = NULL;
 			while (rb_node) {
 				struct vm_area_struct * vma_tmp;
 				vma_tmp = rb_entry(rb_node,
 						struct vm_area_struct, vm_rb);
 				if (vma_tmp->vm_end > addr) {
 					vma = vma_tmp;
 					if (vma_tmp->vm_start <= addr)
 						break;
 					rb_node = rb_node->rb_left;
 				} else
 					rb_node = rb_node->rb_right;
 			}
 			if (vma)
 				mm->mmap_cache = vma;
 		}
 	}
 	return vma;
 }
 EXPORT_SYMBOL(find_vma);
 /* Same as find_vma, but also return a pointer to the previous VMA in *pprev. */
 struct vm_area_struct *
 find_vma_prev(struct mm_struct *mm, unsigned long addr,
 			struct vm_area_struct **pprev)
 {
 	struct vm_area_struct *vma = NULL, *prev = NULL;
 	struct rb_node *rb_node;
 	if (!mm)
 		goto out;
 	/* Guard against addr being lower than the first VMA */
 	vma = mm->mmap;
 	/* Go through the RB tree quickly. */
 	rb_node = mm->mm_rb.rb_node;
 	while (rb_node) {
 		struct vm_area_struct *vma_tmp;
 		vma_tmp = rb_entry(rb_node, struct vm_area_struct, vm_rb);
 		if (addr < vma_tmp->vm_end) {
 			rb_node = rb_node->rb_left;
 		} else {
 			prev = vma_tmp;
 			if (!prev->vm_next || (addr < prev->vm_next->vm_end))
 				break;
 			rb_node = rb_node->rb_right;
 		}
 	}
 out:
 	*pprev = prev;
 	return prev ? prev->vm_next : vma;
 }
 /*
  * Verify that the stack growth is acceptable and
  * update accounting. This is shared with both the
  * grow-up and grow-down cases.
  */
 static int acct_stack_growth(struct vm_area_struct *vma, unsigned long size, unsigned long grow)
 {
 	struct mm_struct *mm = vma->vm_mm;
 	struct rlimit *rlim = current->signal->rlim;
 	unsigned long new_start;
 	/* address space limit tests */
 	if (!may_expand_vm(mm, grow))
 		return -ENOMEM;
 	/* Stack limit test */
 	if (size > rlim[RLIMIT_STACK].rlim_cur)
 		return -ENOMEM;
 	/* mlock limit tests */
 	if (vma->vm_flags & VM_LOCKED) {
 		unsigned long locked;
 		unsigned long limit;
 		locked = mm->locked_vm + grow;
 		limit = rlim[RLIMIT_MEMLOCK].rlim_cur >> PAGE_SHIFT;
 		if (locked > limit && !capable(CAP_IPC_LOCK))
 			return -ENOMEM;
 	}
 	/* Check to ensure the stack will not grow into a hugetlb-only region */
 	new_start = (vma->vm_flags & VM_GROWSUP) ? vma->vm_start :
 			vma->vm_end - size;
 	if (is_hugepage_only_range(vma->vm_mm, new_start, size))
 		return -EFAULT;
 	/*
 	 * Overcommit..  This must be the final test, as it will
 	 * update security statistics.
 	 */
 	if (security_vm_enough_memory(grow))
 		return -ENOMEM;
 	/* Ok, everything looks good - let it rip */
 	mm->total_vm += grow;
 	if (vma->vm_flags & VM_LOCKED)
 		mm->locked_vm += grow;
 	vm_stat_account(mm, vma->vm_flags, vma->vm_file, grow);
 	return 0;
 }
 #if defined(CONFIG_STACK_GROWSUP) || defined(CONFIG_IA64)
 /*
  * PA-RISC uses this for its stack; IA64 for its Register Backing Store.
  * vma is the last one with address > vma->vm_end.  Have to extend vma.
  */
 #ifndef CONFIG_IA64
 static
 #endif
 int expand_upwards(struct vm_area_struct *vma, unsigned long address)
 {
 	int error;
 	if (!(vma->vm_flags & VM_GROWSUP))
 		return -EFAULT;
 	/*
 	 * We must make sure the anon_vma is allocated
 	 * so that the anon_vma locking is not a noop.
 	 */
 	if (unlikely(anon_vma_prepare(vma)))
 		return -ENOMEM;
 	anon_vma_lock(vma);
 	/*
 	 * vma->vm_start/vm_end cannot change under us because the caller
 	 * is required to hold the mmap_sem in read mode.  We need the
 	 * anon_vma lock to serialize against concurrent expand_stacks.
 	 * Also guard against wrapping around to address 0.
 	 */
 	if (address < PAGE_ALIGN(address+4))
 		address = PAGE_ALIGN(address+4);
 	else {
 		anon_vma_unlock(vma);
 		return -ENOMEM;
 	}
 	error = 0;
 	/* Somebody else might have raced and expanded it already */
 	if (address > vma->vm_end) {
 		unsigned long size, grow;
 		size = address - vma->vm_start;
 		grow = (address - vma->vm_end) >> PAGE_SHIFT;
 		error = acct_stack_growth(vma, size, grow);
 		if (!error)
 			vma->vm_end = address;
 	}
 	anon_vma_unlock(vma);
 	return error;
 }
 #endif /* CONFIG_STACK_GROWSUP || CONFIG_IA64 */
 /*
  * vma is the first one with address < vma->vm_start.  Have to extend vma.
  */
 static int expand_downwards(struct vm_area_struct *vma,
 				   unsigned long address)
 {
 	int error;
 	/*
 	 * We must make sure the anon_vma is allocated
 	 * so that the anon_vma locking is not a noop.
 	 */
 	if (unlikely(anon_vma_prepare(vma)))
 		return -ENOMEM;
 	address &= PAGE_MASK;
 	error = security_file_mmap(NULL, 0, 0, 0, address, 1);
 	if (error)
 		return error;
 	anon_vma_lock(vma);
 	/*
 	 * vma->vm_start/vm_end cannot change under us because the caller
 	 * is required to hold the mmap_sem in read mode.  We need the
 	 * anon_vma lock to serialize against concurrent expand_stacks.
 	 */
 	/* Somebody else might have raced and expanded it already */
 	if (address < vma->vm_start) {
 		unsigned long size, grow;
 		size = vma->vm_end - address;
 		grow = (vma->vm_start - address) >> PAGE_SHIFT;
 		error = acct_stack_growth(vma, size, grow);
 		if (!error) {
 			vma->vm_start = address;
 			vma->vm_pgoff -= grow;
 		}
 	}
 	anon_vma_unlock(vma);
 	return error;
 }
 int expand_stack_downwards(struct vm_area_struct *vma, unsigned long address)
 {
 	return expand_downwards(vma, address);
 }
 #ifdef CONFIG_STACK_GROWSUP
 int expand_stack(struct vm_area_struct *vma, unsigned long address)
 {
 	return expand_upwards(vma, address);
 }
 struct vm_area_struct *
 find_extend_vma(struct mm_struct *mm, unsigned long addr)
 {
 	struct vm_area_struct *vma, *prev;
 	addr &= PAGE_MASK;
 	vma = find_vma_prev(mm, addr, &prev);
 	if (vma && (vma->vm_start <= addr))
 		return vma;
 	if (!prev || expand_stack(prev, addr))
 		return NULL;
 	if (prev->vm_flags & VM_LOCKED) {
 		if (mlock_vma_pages_range(prev, addr, prev->vm_end) < 0)
 			return NULL;	/* vma gone! */
 	}
 	return prev;
 }
 #else
 int expand_stack(struct vm_area_struct *vma, unsigned long address)
 {
 	return expand_downwards(vma, address);
 }
 struct vm_area_struct *
 find_extend_vma(struct mm_struct * mm, unsigned long addr)
 {
 	struct vm_area_struct * vma;
 	unsigned long start;
 	addr &= PAGE_MASK;
 	vma = find_vma(mm,addr);
 	if (!vma)
 		return NULL;
 	if (vma->vm_start <= addr)
 		return vma;
 	if (!(vma->vm_flags & VM_GROWSDOWN))
 		return NULL;
 	start = vma->vm_start;
 	if (expand_stack(vma, addr))
 		return NULL;
 	if (vma->vm_flags & VM_LOCKED) {
 		if (mlock_vma_pages_range(vma, addr, start) < 0)
 			return NULL;	/* vma gone! */
 	}
 	return vma;
 }
 #endif
 /*
  * Ok - we have the memory areas we should free on the vma list,
  * so release them, and do the vma updates.
  *
  * Called with the mm semaphore held.
  */
 static void remove_vma_list(struct mm_struct *mm, struct vm_area_struct *vma)
 {
 	/* Update high watermark before we lower total_vm */
 	update_hiwater_vm(mm);
 	do {
 		long nrpages = vma_pages(vma);
 		mm->total_vm -= nrpages;
 		vm_stat_account(mm, vma->vm_flags, vma->vm_file, -nrpages);
 		vma = remove_vma(vma);
 	} while (vma);
 	validate_mm(mm);
 }
 /*
  * Get rid of page table information in the indicated region.
  *
  * Called with the mm semaphore held.
  */
 static void unmap_region(struct mm_struct *mm,
 		struct vm_area_struct *vma, struct vm_area_struct *prev,
 		unsigned long start, unsigned long end)
 {
 	struct vm_area_struct *next = prev? prev->vm_next: mm->mmap;
 	struct mmu_gather *tlb;
 	unsigned long nr_accounted = 0;
 	lru_add_drain();
 	tlb = tlb_gather_mmu(mm, 0);
 	update_hiwater_rss(mm);
 	unmap_vmas(&tlb, vma, start, end, &nr_accounted, NULL);
 	vm_unacct_memory(nr_accounted);
 	free_pgtables(tlb, vma, prev? prev->vm_end: FIRST_USER_ADDRESS,
 				 next? next->vm_start: 0);
 	tlb_finish_mmu(tlb, start, end);
 }
 /*
  * Create a list of vma's touched by the unmap, removing them from the mm's
  * vma list as we go..
  */
 static void
 detach_vmas_to_be_unmapped(struct mm_struct *mm, struct vm_area_struct *vma,
 	struct vm_area_struct *prev, unsigned long end)
 {
 	struct vm_area_struct **insertion_point;
 	struct vm_area_struct *tail_vma = NULL;
 	unsigned long addr;
 	insertion_point = (prev ? &prev->vm_next : &mm->mmap);
 	do {
 		rb_erase(&vma->vm_rb, &mm->mm_rb);
 		mm->map_count--;
 		tail_vma = vma;
 		vma = vma->vm_next;
 	} while (vma && vma->vm_start < end);
 	*insertion_point = vma;
 	tail_vma->vm_next = NULL;
 	if (mm->unmap_area == arch_unmap_area)
 		addr = prev ? prev->vm_end : mm->mmap_base;
 	else
 		addr = vma ?  vma->vm_start : mm->mmap_base;
 	mm->unmap_area(mm, addr);
 	mm->mmap_cache = NULL;		/* Kill the cache. */
 }
 /*
  * Split a vma into two pieces at address 'addr', a new vma is allocated
  * either for the first part or the tail.
  */
 int split_vma(struct mm_struct * mm, struct vm_area_struct * vma,
 	      unsigned long addr, int new_below)
 {
 	struct mempolicy *pol;
 	struct vm_area_struct *new;
 	if (is_vm_hugetlb_page(vma) && (addr &
 					~(huge_page_mask(hstate_vma(vma)))))
 		return -EINVAL;
 	if (mm->map_count >= sysctl_max_map_count)
 		return -ENOMEM;
 	new = kmem_cache_alloc(vm_area_cachep, GFP_KERNEL);
 	if (!new)
 		return -ENOMEM;
 	/* most fields are the same, copy all, and then fixup */
 	*new = *vma;
 	if (new_below)
 		new->vm_end = addr;
 	else {
 		new->vm_start = addr;
 		new->vm_pgoff += ((addr - vma->vm_start) >> PAGE_SHIFT);
 	}
 	pol = mpol_dup(vma_policy(vma));
 	if (IS_ERR(pol)) {
 		kmem_cache_free(vm_area_cachep, new);
 		return PTR_ERR(pol);
 	}
 	vma_set_policy(new, pol);
 	if (new->vm_file) {
 		get_file(new->vm_file);
 		if (vma->vm_flags & VM_EXECUTABLE)
 			added_exe_file_vma(mm);
 	}
 	if (new->vm_ops && new->vm_ops->open)
 		new->vm_ops->open(new);
 	if (new_below)
 		vma_adjust(vma, addr, vma->vm_end, vma->vm_pgoff +
 			((addr - new->vm_start) >> PAGE_SHIFT), new);
 	else
 		vma_adjust(vma, vma->vm_start, addr, vma->vm_pgoff, new);
 	return 0;
 }
 /* Munmap is split into 2 main parts -- this part which finds
  * what needs doing, and the areas themselves, which do the
  * work.  This now handles partial unmappings.
  * Jeremy Fitzhardinge <jeremy@goop.org>
  */
 int do_munmap(struct mm_struct *mm, unsigned long start, size_t len)
 {
 	unsigned long end;
 	struct vm_area_struct *vma, *prev, *last;
 	if ((start & ~PAGE_MASK) || start > TASK_SIZE || len > TASK_SIZE-start)
 		return -EINVAL;
 	if ((len = PAGE_ALIGN(len)) == 0)
 		return -EINVAL;
 	/* Find the first overlapping VMA */
 	vma = find_vma_prev(mm, start, &prev);
 	if (!vma)
 		return 0;
 	/* we have  start < vma->vm_end  */
 	/* if it doesn't overlap, we have nothing.. */
 	end = start + len;
 	if (vma->vm_start >= end)
 		return 0;
 	/*
 	 * If we need to split any vma, do it now to save pain later.
 	 *
 	 * Note: mremap's move_vma VM_ACCOUNT handling assumes a partially
 	 * unmapped vm_area_struct will remain in use: so lower split_vma
 	 * places tmp vma above, and higher split_vma places tmp vma below.
 	 */
 	if (start > vma->vm_start) {
 		int error = split_vma(mm, vma, start, 0);
 		if (error)
 			return error;
 		prev = vma;
 	}
 	/* Does it split the last one? */
 	last = find_vma(mm, end);
 	if (last && end > last->vm_start) {
 		int error = split_vma(mm, last, end, 1);
 		if (error)
 			return error;
 	}
 	vma = prev? prev->vm_next: mm->mmap;
 	/*
 	 * unlock any mlock()ed ranges before detaching vmas
 	 */
 	if (mm->locked_vm) {
 		struct vm_area_struct *tmp = vma;
 		while (tmp && tmp->vm_start < end) {
 			if (tmp->vm_flags & VM_LOCKED) {
 				mm->locked_vm -= vma_pages(tmp);
 				munlock_vma_pages_all(tmp);
 			}
 			tmp = tmp->vm_next;
 		}
 	}
 	/*
 	 * Remove the vma's, and unmap the actual pages
 	 */
 	detach_vmas_to_be_unmapped(mm, vma, prev, end);
 	unmap_region(mm, vma, prev, start, end);
 	/* Fix up all other VM information */
 	remove_vma_list(mm, vma);
 	return 0;
 }
 EXPORT_SYMBOL(do_munmap);
 asmlinkage long sys_munmap(unsigned long addr, size_t len)
 {
 	int ret;
 	struct mm_struct *mm = current->mm;
 	profile_munmap(addr);
 	down_write(&mm->mmap_sem);
 	ret = do_munmap(mm, addr, len);
 	up_write(&mm->mmap_sem);
 	return ret;
 }
 static inline void verify_mm_writelocked(struct mm_struct *mm)
 {
 #ifdef CONFIG_DEBUG_VM
 	if (unlikely(down_read_trylock(&mm->mmap_sem))) {
 		WARN_ON(1);
 		up_read(&mm->mmap_sem);
 	}
 #endif
 }
 /*
  *  this is really a simplified "do_mmap".  it only handles
  *  anonymous maps.  eventually we may be able to do some
  *  brk-specific accounting here.
  */
 unsigned long do_brk(unsigned long addr, unsigned long len)
 {
 	struct mm_struct * mm = current->mm;
 	struct vm_area_struct * vma, * prev;
 	unsigned long flags;
 	struct rb_node ** rb_link, * rb_parent;
 	pgoff_t pgoff = addr >> PAGE_SHIFT;
 	int error;
 	len = PAGE_ALIGN(len);
 	if (!len)
 		return addr;
 	if ((addr + len) > TASK_SIZE || (addr + len) < addr)
 		return -EINVAL;
 	if (is_hugepage_only_range(mm, addr, len))
 		return -EINVAL;
 	error = security_file_mmap(NULL, 0, 0, 0, addr, 1);
 	if (error)
 		return error;
 	flags = VM_DATA_DEFAULT_FLAGS | VM_ACCOUNT | mm->def_flags;
 	error = arch_mmap_check(addr, len, flags);
 	if (error)
 		return error;
 	/*
 	 * mlock MCL_FUTURE?
 	 */
 	if (mm->def_flags & VM_LOCKED) {
 		unsigned long locked, lock_limit;
 		locked = len >> PAGE_SHIFT;
 		locked += mm->locked_vm;
 		lock_limit = current->signal->rlim[RLIMIT_MEMLOCK].rlim_cur;
 		lock_limit >>= PAGE_SHIFT;
 		if (locked > lock_limit && !capable(CAP_IPC_LOCK))
 			return -EAGAIN;
 	}
 	/*
 	 * mm->mmap_sem is required to protect against another thread
 	 * changing the mappings in case we sleep.
 	 */
 	verify_mm_writelocked(mm);
 	/*
 	 * Clear old maps.  this also does some error checking for us
 	 */
  munmap_back:
 	vma = find_vma_prepare(mm, addr, &prev, &rb_link, &rb_parent);
 	if (vma && vma->vm_start < addr + len) {
 		if (do_munmap(mm, addr, len))
 			return -ENOMEM;
 		goto munmap_back;
 	}
 	/* Check against address space limits *after* clearing old maps... */
 	if (!may_expand_vm(mm, len >> PAGE_SHIFT))
 		return -ENOMEM;
 	if (mm->map_count > sysctl_max_map_count)
 		return -ENOMEM;
 	if (security_vm_enough_memory(len >> PAGE_SHIFT))
 		return -ENOMEM;
 	/* Can we just expand an old private anonymous mapping? */
 	vma = vma_merge(mm, prev, addr, addr + len, flags,
 					NULL, NULL, pgoff, NULL);
 	if (vma)
 		goto out;
 	/*
 	 * create a vma struct for an anonymous mapping
 	 */
 	vma = kmem_cache_zalloc(vm_area_cachep, GFP_KERNEL);
 	if (!vma) {
 		vm_unacct_memory(len >> PAGE_SHIFT);
 		return -ENOMEM;
 	}
 	vma->vm_mm = mm;
 	vma->vm_start = addr;
 	vma->vm_end = addr + len;
 	vma->vm_pgoff = pgoff;
 	vma->vm_flags = flags;
 	vma->vm_page_prot = vm_get_page_prot(flags);
 	vma_link(mm, vma, prev, rb_link, rb_parent);
 out:
 	mm->total_vm += len >> PAGE_SHIFT;
 	if (flags & VM_LOCKED) {
 		if (!mlock_vma_pages_range(vma, addr, addr + len))
 			mm->locked_vm += (len >> PAGE_SHIFT);
 	}
 	return addr;
 }
 EXPORT_SYMBOL(do_brk);
 /* Release all mmaps. */
 void exit_mmap(struct mm_struct *mm)
 {
 	struct mmu_gather *tlb;
 	struct vm_area_struct *vma;
 	unsigned long nr_accounted = 0;
 	unsigned long end;
 	/* mm's last user has gone, and its about to be pulled down */
 	arch_exit_mmap(mm);
 	mmu_notifier_release(mm);
 	if (!mm->mmap)	/* Can happen if dup_mmap() received an OOM */
 		return;
 	if (mm->locked_vm) {
 		vma = mm->mmap;
 		while (vma) {
 			if (vma->vm_flags & VM_LOCKED)
 				munlock_vma_pages_all(vma);
 			vma = vma->vm_next;
 		}
 	}
 	vma = mm->mmap;
 	lru_add_drain();
 	flush_cache_mm(mm);
 	tlb = tlb_gather_mmu(mm, 1);
 	/* update_hiwater_rss(mm) here? but nobody should be looking */
 	/* Use -1 here to ensure all VMAs in the mm are unmapped */
 	end = unmap_vmas(&tlb, vma, 0, -1, &nr_accounted, NULL);
 	vm_unacct_memory(nr_accounted);
 	free_pgtables(tlb, vma, FIRST_USER_ADDRESS, 0);
 	tlb_finish_mmu(tlb, 0, end);
 	/*
 	 * Walk the list again, actually closing and freeing it,
 	 * with preemption enabled, without holding any MM locks.
 	 */
 	while (vma)
 		vma = remove_vma(vma);
 	BUG_ON(mm->nr_ptes > (FIRST_USER_ADDRESS+PMD_SIZE-1)>>PMD_SHIFT);
 }
 /* Insert vm structure into process list sorted by address
  * and into the inode's i_mmap tree.  If vm_file is non-NULL
  * then i_mmap_lock is taken here.
  */
 int insert_vm_struct(struct mm_struct * mm, struct vm_area_struct * vma)
 {
 	struct vm_area_struct * __vma, * prev;
 	struct rb_node ** rb_link, * rb_parent;
 	/*
 	 * The vm_pgoff of a purely anonymous vma should be irrelevant
 	 * until its first write fault, when page's anon_vma and index
 	 * are set.  But now set the vm_pgoff it will almost certainly
 	 * end up with (unless mremap moves it elsewhere before that
 	 * first wfault), so /proc/pid/maps tells a consistent story.
 	 *
 	 * By setting it to reflect the virtual start address of the
 	 * vma, merges and splits can happen in a seamless way, just
 	 * using the existing file pgoff checks and manipulations.
 	 * Similarly in do_mmap_pgoff and in do_brk.
 	 */
 	if (!vma->vm_file) {
 		BUG_ON(vma->anon_vma);
 		vma->vm_pgoff = vma->vm_start >> PAGE_SHIFT;
 	}
 	__vma = find_vma_prepare(mm,vma->vm_start,&prev,&rb_link,&rb_parent);
 	if (__vma && __vma->vm_start < vma->vm_end)
 		return -ENOMEM;
 	if ((vma->vm_flags & VM_ACCOUNT) &&
 	     security_vm_enough_memory_mm(mm, vma_pages(vma)))
 		return -ENOMEM;
 	vma_link(mm, vma, prev, rb_link, rb_parent);
 	return 0;
 }
 /*
  * Copy the vma structure to a new location in the same mm,
  * prior to moving page table entries, to effect an mremap move.
  */
 struct vm_area_struct *copy_vma(struct vm_area_struct **vmap,
 	unsigned long addr, unsigned long len, pgoff_t pgoff)
 {
 	struct vm_area_struct *vma = *vmap;
 	unsigned long vma_start = vma->vm_start;
 	struct mm_struct *mm = vma->vm_mm;
 	struct vm_area_struct *new_vma, *prev;
 	struct rb_node **rb_link, *rb_parent;
 	struct mempolicy *pol;
 	/*
 	 * If anonymous vma has not yet been faulted, update new pgoff
 	 * to match new location, to increase its chance of merging.
 	 */
 	if (!vma->vm_file && !vma->anon_vma)
 		pgoff = addr >> PAGE_SHIFT;
 	find_vma_prepare(mm, addr, &prev, &rb_link, &rb_parent);
 	new_vma = vma_merge(mm, prev, addr, addr + len, vma->vm_flags,
 			vma->anon_vma, vma->vm_file, pgoff, vma_policy(vma));
 	if (new_vma) {
 		/*
 		 * Source vma may have been merged into new_vma
 		 */
 		if (vma_start >= new_vma->vm_start &&
 		    vma_start < new_vma->vm_end)
 			*vmap = new_vma;
 	} else {
 		new_vma = kmem_cache_alloc(vm_area_cachep, GFP_KERNEL);
 		if (new_vma) {
 			*new_vma = *vma;
 			pol = mpol_dup(vma_policy(vma));
 			if (IS_ERR(pol)) {
 				kmem_cache_free(vm_area_cachep, new_vma);
 				return NULL;
 			}
 			vma_set_policy(new_vma, pol);
 			new_vma->vm_start = addr;
 			new_vma->vm_end = addr + len;
 			new_vma->vm_pgoff = pgoff;
 			if (new_vma->vm_file) {
 				get_file(new_vma->vm_file);
 				if (vma->vm_flags & VM_EXECUTABLE)
 					added_exe_file_vma(mm);
 			}
 			if (new_vma->vm_ops && new_vma->vm_ops->open)
 				new_vma->vm_ops->open(new_vma);
 			vma_link(mm, new_vma, prev, rb_link, rb_parent);
 		}
 	}
 	return new_vma;
 }
 /*
  * Return true if the calling process may expand its vm space by the passed
  * number of pages
  */
 int may_expand_vm(struct mm_struct *mm, unsigned long npages)
 {
 	unsigned long cur = mm->total_vm;	/* pages */
 	unsigned long lim;
 	lim = current->signal->rlim[RLIMIT_AS].rlim_cur >> PAGE_SHIFT;
 	if (cur + npages > lim)
 		return 0;
 	return 1;
 }
 static int special_mapping_fault(struct vm_area_struct *vma,
 				struct vm_fault *vmf)
 {
 	pgoff_t pgoff;
 	struct page **pages;
 	/*
 	 * special mappings have no vm_file, and in that case, the mm
 	 * uses vm_pgoff internally. So we have to subtract it from here.
 	 * We are allowed to do this because we are the mm; do not copy
 	 * this code into drivers!
 	 */
 	pgoff = vmf->pgoff - vma->vm_pgoff;
 	for (pages = vma->vm_private_data; pgoff && *pages; ++pages)
 		pgoff--;
 	if (*pages) {
 		struct page *page = *pages;
 		get_page(page);
 		vmf->page = page;
 		return 0;
 	}
 	return VM_FAULT_SIGBUS;
 }
 /*
  * Having a close hook prevents vma merging regardless of flags.
  */
 static void special_mapping_close(struct vm_area_struct *vma)
 {
 }
 static struct vm_operations_struct special_mapping_vmops = {
 	.close = special_mapping_close,
 	.fault = special_mapping_fault,
 };
 /*
  * Called with mm->mmap_sem held for writing.
  * Insert a new vma covering the given region, with the given flags.
  * Its pages are supplied by the given array of struct page *.
  * The array can be shorter than len >> PAGE_SHIFT if it's null-terminated.
  * The region past the last page supplied will always produce SIGBUS.
  * The array pointer and the pages it points to are assumed to stay alive
  * for as long as this mapping might exist.
  */
 int install_special_mapping(struct mm_struct *mm,
 			    unsigned long addr, unsigned long len,
 			    unsigned long vm_flags, struct page **pages)
 {
 	struct vm_area_struct *vma;
 	vma = kmem_cache_zalloc(vm_area_cachep, GFP_KERNEL);
 	if (unlikely(vma == NULL))
 		return -ENOMEM;
 	vma->vm_mm = mm;
 	vma->vm_start = addr;
 	vma->vm_end = addr + len;
 	vma->vm_flags = vm_flags | mm->def_flags | VM_DONTEXPAND;
 	vma->vm_page_prot = vm_get_page_prot(vma->vm_flags);
 	vma->vm_ops = &special_mapping_vmops;
 	vma->vm_private_data = pages;
 	if (unlikely(insert_vm_struct(mm, vma))) {
 		kmem_cache_free(vm_area_cachep, vma);
 		return -ENOMEM;
 	}
 	mm->total_vm += len >> PAGE_SHIFT;
 	return 0;
 }
 static DEFINE_MUTEX(mm_all_locks_mutex);
 static void vm_lock_anon_vma(struct mm_struct *mm, struct anon_vma *anon_vma)
 {
 	if (!test_bit(0, (unsigned long *) &anon_vma->head.next)) {
 		/*
 		 * The LSB of head.next can't change from under us
 		 * because we hold the mm_all_locks_mutex.
 		 */
 		spin_lock_nest_lock(&anon_vma->lock, &mm->mmap_sem);
 		/*
 		 * We can safely modify head.next after taking the
 		 * anon_vma->lock. If some other vma in this mm shares
 		 * the same anon_vma we won't take it again.
 		 *
 		 * No need of atomic instructions here, head.next
 		 * can't change from under us thanks to the
 		 * anon_vma->lock.
 		 */
 		if (__test_and_set_bit(0, (unsigned long *)
 				       &anon_vma->head.next))
 			BUG();
 	}
 }
 static void vm_lock_mapping(struct mm_struct *mm, struct address_space *mapping)
 {
 	if (!test_bit(AS_MM_ALL_LOCKS, &mapping->flags)) {
 		/*
 		 * AS_MM_ALL_LOCKS can't change from under us because
 		 * we hold the mm_all_locks_mutex.
 		 *
 		 * Operations on ->flags have to be atomic because
 		 * even if AS_MM_ALL_LOCKS is stable thanks to the
 		 * mm_all_locks_mutex, there may be other cpus
 		 * changing other bitflags in parallel to us.
 		 */
 		if (test_and_set_bit(AS_MM_ALL_LOCKS, &mapping->flags))
 			BUG();
 		spin_lock_nest_lock(&mapping->i_mmap_lock, &mm->mmap_sem);
 	}
 }
 /*
  * This operation locks against the VM for all pte/vma/mm related
  * operations that could ever happen on a certain mm. This includes
  * vmtruncate, try_to_unmap, and all page faults.
  *
  * The caller must take the mmap_sem in write mode before calling
  * mm_take_all_locks(). The caller isn't allowed to release the
  * mmap_sem until mm_drop_all_locks() returns.
  *
  * mmap_sem in write mode is required in order to block all operations
  * that could modify pagetables and free pages without need of
  * altering the vma layout (for example populate_range() with
  * nonlinear vmas). It's also needed in write mode to avoid new
  * anon_vmas to be associated with existing vmas.
  *
  * A single task can't take more than one mm_take_all_locks() in a row
  * or it would deadlock.
  *
  * The LSB in anon_vma->head.next and the AS_MM_ALL_LOCKS bitflag in
  * mapping->flags avoid to take the same lock twice, if more than one
  * vma in this mm is backed by the same anon_vma or address_space.
  *
  * We can take all the locks in random order because the VM code
  * taking i_mmap_lock or anon_vma->lock outside the mmap_sem never
  * takes more than one of them in a row. Secondly we're protected
  * against a concurrent mm_take_all_locks() by the mm_all_locks_mutex.
  *
  * mm_take_all_locks() and mm_drop_all_locks are expensive operations
  * that may have to take thousand of locks.
  *
  * mm_take_all_locks() can fail if it's interrupted by signals.
  */
 int mm_take_all_locks(struct mm_struct *mm)
 {
 	struct vm_area_struct *vma;
 	int ret = -EINTR;
 	BUG_ON(down_read_trylock(&mm->mmap_sem));
 	mutex_lock(&mm_all_locks_mutex);
 	for (vma = mm->mmap; vma; vma = vma->vm_next) {
 		if (signal_pending(current))
 			goto out_unlock;
 		if (vma->vm_file && vma->vm_file->f_mapping)
 			vm_lock_mapping(mm, vma->vm_file->f_mapping);
 	}
 	for (vma = mm->mmap; vma; vma = vma->vm_next) {
 		if (signal_pending(current))
 			goto out_unlock;
 		if (vma->anon_vma)
 			vm_lock_anon_vma(mm, vma->anon_vma);
 	}
 	ret = 0;
 out_unlock:
 	if (ret)
 		mm_drop_all_locks(mm);
 	return ret;
 }
 static void vm_unlock_anon_vma(struct anon_vma *anon_vma)
 {
 	if (test_bit(0, (unsigned long *) &anon_vma->head.next)) {
 		/*
 		 * The LSB of head.next can't change to 0 from under
 		 * us because we hold the mm_all_locks_mutex.
 		 *
 		 * We must however clear the bitflag before unlocking
 		 * the vma so the users using the anon_vma->head will
 		 * never see our bitflag.
 		 *
 		 * No need of atomic instructions here, head.next
 		 * can't change from under us until we release the
 		 * anon_vma->lock.
 		 */
 		if (!__test_and_clear_bit(0, (unsigned long *)
 					  &anon_vma->head.next))
 			BUG();
 		spin_unlock(&anon_vma->lock);
 	}
 }
 static void vm_unlock_mapping(struct address_space *mapping)
 {
 	if (test_bit(AS_MM_ALL_LOCKS, &mapping->flags)) {
 		/*
 		 * AS_MM_ALL_LOCKS can't change to 0 from under us
 		 * because we hold the mm_all_locks_mutex.
 		 */
 		spin_unlock(&mapping->i_mmap_lock);
 		if (!test_and_clear_bit(AS_MM_ALL_LOCKS,
 					&mapping->flags))
 			BUG();
 	}
 }
 /*
  * The mmap_sem cannot be released by the caller until
  * mm_drop_all_locks() returns.
  */
 void mm_drop_all_locks(struct mm_struct *mm)
 {
 	struct vm_area_struct *vma;
 	BUG_ON(down_read_trylock(&mm->mmap_sem));
 	BUG_ON(!mutex_is_locked(&mm_all_locks_mutex));
 	for (vma = mm->mmap; vma; vma = vma->vm_next) {
 		if (vma->anon_vma)
 			vm_unlock_anon_vma(vma->anon_vma);
 		if (vma->vm_file && vma->vm_file->f_mapping)
 			vm_unlock_mapping(vma->vm_file->f_mapping);
 	}
 	mutex_unlock(&mm_all_locks_mutex);
 }
+/*
+ * initialise the VMA slab
+ */
+void __init mmap_init(void)
+{
+	vm_area_cachep = kmem_cache_create("vm_area_struct",
+			sizeof(struct vm_area_struct), 0,
+			SLAB_PANIC, NULL);
+}

mm/nommu.c

Diff comments View file @ c40f6f8

1	/*	1	/*
2	* linux/mm/nommu.c	2	* linux/mm/nommu.c
3	*	3	*
4	* Replacement code for mm functions to support CPU's that don't	4	* Replacement code for mm functions to support CPU's that don't
5	* have any form of memory management unit (thus no virtual memory).	5	* have any form of memory management unit (thus no virtual memory).
6	*	6	*
7	* See Documentation/nommu-mmap.txt	7	* See Documentation/nommu-mmap.txt
8	*	8	*
9	* Copyright (c) 2004-2005 David Howells <dhowells@redhat.com>	9	* Copyright (c) 2004-2008 David Howells <dhowells@redhat.com>
10	* Copyright (c) 2000-2003 David McCullough <davidm@snapgear.com>	10	* Copyright (c) 2000-2003 David McCullough <davidm@snapgear.com>
11	* Copyright (c) 2000-2001 D Jeff Dionne <jeff@uClinux.org>	11	* Copyright (c) 2000-2001 D Jeff Dionne <jeff@uClinux.org>
12	* Copyright (c) 2002 Greg Ungerer <gerg@snapgear.com>	12	* Copyright (c) 2002 Greg Ungerer <gerg@snapgear.com>
13	* Copyright (c) 2007 Paul Mundt <lethal@linux-sh.org>	13	* Copyright (c) 2007-2008 Paul Mundt <lethal@linux-sh.org>
14	*/	14	*/
15		15
16	#include <linux/module.h>	16	#include <linux/module.h>
17	#include <linux/mm.h>	17	#include <linux/mm.h>
18	#include <linux/mman.h>	18	#include <linux/mman.h>
19	#include <linux/swap.h>	19	#include <linux/swap.h>
20	#include <linux/file.h>	20	#include <linux/file.h>
21	#include <linux/highmem.h>	21	#include <linux/highmem.h>
22	#include <linux/pagemap.h>	22	#include <linux/pagemap.h>
23	#include <linux/slab.h>	23	#include <linux/slab.h>
24	#include <linux/vmalloc.h>	24	#include <linux/vmalloc.h>
25	#include <linux/tracehook.h>	25	#include <linux/tracehook.h>
26	#include <linux/blkdev.h>	26	#include <linux/blkdev.h>
27	#include <linux/backing-dev.h>	27	#include <linux/backing-dev.h>
28	#include <linux/mount.h>	28	#include <linux/mount.h>
29	#include <linux/personality.h>	29	#include <linux/personality.h>
30	#include <linux/security.h>	30	#include <linux/security.h>
31	#include <linux/syscalls.h>	31	#include <linux/syscalls.h>
32		32
33	#include <asm/uaccess.h>	33	#include <asm/uaccess.h>
34	#include <asm/tlb.h>	34	#include <asm/tlb.h>
35	#include <asm/tlbflush.h>	35	#include <asm/tlbflush.h>
		36	#include "internal.h"
36		37
		38	static inline __attribute__((format(printf, 1, 2)))
		39	void no_printk(const char *fmt, ...)
		40	{
		41	}
		42
		43	#if 0
		44	#define kenter(FMT, ...) \
		45	printk(KERN_DEBUG "==> %s("FMT")\n", __func__, ##__VA_ARGS__)
		46	#define kleave(FMT, ...) \
		47	printk(KERN_DEBUG "<== %s()"FMT"\n", __func__, ##__VA_ARGS__)
		48	#define kdebug(FMT, ...) \
		49	printk(KERN_DEBUG "xxx" FMT"yyy\n", ##__VA_ARGS__)
		50	#else
		51	#define kenter(FMT, ...) \
		52	no_printk(KERN_DEBUG "==> %s("FMT")\n", __func__, ##__VA_ARGS__)
		53	#define kleave(FMT, ...) \
		54	no_printk(KERN_DEBUG "<== %s()"FMT"\n", __func__, ##__VA_ARGS__)
		55	#define kdebug(FMT, ...) \
		56	no_printk(KERN_DEBUG FMT"\n", ##__VA_ARGS__)
		57	#endif
		58
37	#include "internal.h"	59	#include "internal.h"
38		60
39	void *high_memory;	61	void *high_memory;
40	struct page *mem_map;	62	struct page *mem_map;
41	unsigned long max_mapnr;	63	unsigned long max_mapnr;
42	unsigned long num_physpages;	64	unsigned long num_physpages;
43	unsigned long askedalloc, realalloc;
44	atomic_long_t vm_committed_space = ATOMIC_LONG_INIT(0);	65	atomic_long_t vm_committed_space = ATOMIC_LONG_INIT(0);
45	int sysctl_overcommit_memory = OVERCOMMIT_GUESS; /* heuristic overcommit */	66	int sysctl_overcommit_memory = OVERCOMMIT_GUESS; /* heuristic overcommit */
46	int sysctl_overcommit_ratio = 50; /* default is 50% */	67	int sysctl_overcommit_ratio = 50; /* default is 50% */
47	int sysctl_max_map_count = DEFAULT_MAX_MAP_COUNT;	68	int sysctl_max_map_count = DEFAULT_MAX_MAP_COUNT;
		69	int sysctl_nr_trim_pages = 1; /* page trimming behaviour */
48	int heap_stack_gap = 0;	70	int heap_stack_gap = 0;
49		71
		72	atomic_t mmap_pages_allocated;
		73
50	EXPORT_SYMBOL(mem_map);	74	EXPORT_SYMBOL(mem_map);
51	EXPORT_SYMBOL(num_physpages);	75	EXPORT_SYMBOL(num_physpages);
52		76
53	/* list of shareable VMAs */	77	/* list of mapped, potentially shareable regions */
54	struct rb_root nommu_vma_tree = RB_ROOT;	78	static struct kmem_cache *vm_region_jar;
55	DECLARE_RWSEM(nommu_vma_sem);	79	struct rb_root nommu_region_tree = RB_ROOT;
		80	DECLARE_RWSEM(nommu_region_sem);
56		81
57	struct vm_operations_struct generic_file_vm_ops = {	82	struct vm_operations_struct generic_file_vm_ops = {
58	};	83	};
59		84
60	/*	85	/*
61	* Handle all mappings that got truncated by a "truncate()"	86	* Handle all mappings that got truncated by a "truncate()"
62	* system call.	87	* system call.
63	*	88	*
64	* NOTE! We have to be ready to update the memory sharing	89	* NOTE! We have to be ready to update the memory sharing
65	* between the file and the memory map for a potential last	90	* between the file and the memory map for a potential last
66	* incomplete page. Ugly, but necessary.	91	* incomplete page. Ugly, but necessary.
67	*/	92	*/
68	int vmtruncate(struct inode *inode, loff_t offset)	93	int vmtruncate(struct inode *inode, loff_t offset)
69	{	94	{
70	struct address_space *mapping = inode->i_mapping;	95	struct address_space *mapping = inode->i_mapping;
71	unsigned long limit;	96	unsigned long limit;
72		97
73	if (inode->i_size < offset)	98	if (inode->i_size < offset)
74	goto do_expand;	99	goto do_expand;
75	i_size_write(inode, offset);	100	i_size_write(inode, offset);
76		101
77	truncate_inode_pages(mapping, offset);	102	truncate_inode_pages(mapping, offset);
78	goto out_truncate;	103	goto out_truncate;
79		104
80	do_expand:	105	do_expand:
81	limit = current->signal->rlim[RLIMIT_FSIZE].rlim_cur;	106	limit = current->signal->rlim[RLIMIT_FSIZE].rlim_cur;
82	if (limit != RLIM_INFINITY && offset > limit)	107	if (limit != RLIM_INFINITY && offset > limit)
83	goto out_sig;	108	goto out_sig;
84	if (offset > inode->i_sb->s_maxbytes)	109	if (offset > inode->i_sb->s_maxbytes)
85	goto out;	110	goto out;
86	i_size_write(inode, offset);	111	i_size_write(inode, offset);
87		112
88	out_truncate:	113	out_truncate:
89	if (inode->i_op->truncate)	114	if (inode->i_op->truncate)
90	inode->i_op->truncate(inode);	115	inode->i_op->truncate(inode);
91	return 0;	116	return 0;
92	out_sig:	117	out_sig:
93	send_sig(SIGXFSZ, current, 0);	118	send_sig(SIGXFSZ, current, 0);
94	out:	119	out:
95	return -EFBIG;	120	return -EFBIG;
96	}	121	}
97		122
98	EXPORT_SYMBOL(vmtruncate);	123	EXPORT_SYMBOL(vmtruncate);
99		124
100	/*	125	/*
101	* Return the total memory allocated for this pointer, not	126	* Return the total memory allocated for this pointer, not
102	* just what the caller asked for.	127	* just what the caller asked for.
103	*	128	*
104	* Doesn't have to be accurate, i.e. may have races.	129	* Doesn't have to be accurate, i.e. may have races.
105	*/	130	*/
106	unsigned int kobjsize(const void *objp)	131	unsigned int kobjsize(const void *objp)
107	{	132	{
108	struct page *page;	133	struct page *page;
109		134
110	/*	135	/*
111	* If the object we have should not have ksize performed on it,	136	* If the object we have should not have ksize performed on it,
112	* return size of 0	137	* return size of 0
113	*/	138	*/
114	if (!objp \|\| !virt_addr_valid(objp))	139	if (!objp \|\| !virt_addr_valid(objp))
115	return 0;	140	return 0;
116		141
117	page = virt_to_head_page(objp);	142	page = virt_to_head_page(objp);
118		143
119	/*	144	/*
120	* If the allocator sets PageSlab, we know the pointer came from	145	* If the allocator sets PageSlab, we know the pointer came from
121	* kmalloc().	146	* kmalloc().
122	*/	147	*/
123	if (PageSlab(page))	148	if (PageSlab(page))
124	return ksize(objp);	149	return ksize(objp);
125		150
126	/*	151	/*
		152	* If it's not a compound page, see if we have a matching VMA
		153	* region. This test is intentionally done in reverse order,
		154	* so if there's no VMA, we still fall through and hand back
		155	* PAGE_SIZE for 0-order pages.
		156	*/
		157	if (!PageCompound(page)) {
		158	struct vm_area_struct *vma;
		159
		160	vma = find_vma(current->mm, (unsigned long)objp);
		161	if (vma)
		162	return vma->vm_end - vma->vm_start;
		163	}
		164
		165	/*
127	* The ksize() function is only guaranteed to work for pointers	166	* The ksize() function is only guaranteed to work for pointers
128	* returned by kmalloc(). So handle arbitrary pointers here.	167	* returned by kmalloc(). So handle arbitrary pointers here.
129	*/	168	*/
130	return PAGE_SIZE << compound_order(page);	169	return PAGE_SIZE << compound_order(page);
131	}	170	}
132		171
133	int __get_user_pages(struct task_struct tsk, struct mm_struct mm,	172	int __get_user_pages(struct task_struct tsk, struct mm_struct mm,
134	unsigned long start, int len, int flags,	173	unsigned long start, int len, int flags,
135	struct page pages, struct vm_area_struct vmas)	174	struct page pages, struct vm_area_struct vmas)
136	{	175	{
137	struct vm_area_struct *vma;	176	struct vm_area_struct *vma;
138	unsigned long vm_flags;	177	unsigned long vm_flags;
139	int i;	178	int i;
140	int write = !!(flags & GUP_FLAGS_WRITE);	179	int write = !!(flags & GUP_FLAGS_WRITE);
141	int force = !!(flags & GUP_FLAGS_FORCE);	180	int force = !!(flags & GUP_FLAGS_FORCE);
142	int ignore = !!(flags & GUP_FLAGS_IGNORE_VMA_PERMISSIONS);	181	int ignore = !!(flags & GUP_FLAGS_IGNORE_VMA_PERMISSIONS);
143		182
144	/* calculate required read or write permissions.	183	/* calculate required read or write permissions.
145	* - if 'force' is set, we only require the "MAY" flags.	184	* - if 'force' is set, we only require the "MAY" flags.
146	*/	185	*/
147	vm_flags = write ? (VM_WRITE \| VM_MAYWRITE) : (VM_READ \| VM_MAYREAD);	186	vm_flags = write ? (VM_WRITE \| VM_MAYWRITE) : (VM_READ \| VM_MAYREAD);
148	vm_flags &= force ? (VM_MAYREAD \| VM_MAYWRITE) : (VM_READ \| VM_WRITE);	187	vm_flags &= force ? (VM_MAYREAD \| VM_MAYWRITE) : (VM_READ \| VM_WRITE);
149		188
150	for (i = 0; i < len; i++) {	189	for (i = 0; i < len; i++) {
151	vma = find_vma(mm, start);	190	vma = find_vma(mm, start);
152	if (!vma)	191	if (!vma)
153	goto finish_or_fault;	192	goto finish_or_fault;
154		193
155	/* protect what we can, including chardevs */	194	/* protect what we can, including chardevs */
156	if (vma->vm_flags & (VM_IO \| VM_PFNMAP) \|\|	195	if (vma->vm_flags & (VM_IO \| VM_PFNMAP) \|\|
157	(!ignore && !(vm_flags & vma->vm_flags)))	196	(!ignore && !(vm_flags & vma->vm_flags)))
158	goto finish_or_fault;	197	goto finish_or_fault;
159		198
160	if (pages) {	199	if (pages) {
161	pages[i] = virt_to_page(start);	200	pages[i] = virt_to_page(start);
162	if (pages[i])	201	if (pages[i])
163	page_cache_get(pages[i]);	202	page_cache_get(pages[i]);
164	}	203	}
165	if (vmas)	204	if (vmas)
166	vmas[i] = vma;	205	vmas[i] = vma;
167	start += PAGE_SIZE;	206	start += PAGE_SIZE;
168	}	207	}
169		208
170	return i;	209	return i;
171		210
172	finish_or_fault:	211	finish_or_fault:
173	return i ? : -EFAULT;	212	return i ? : -EFAULT;
174	}	213	}
175		214
176		215
177	/*	216	/*
178	* get a list of pages in an address range belonging to the specified process	217	* get a list of pages in an address range belonging to the specified process
179	* and indicate the VMA that covers each page	218	* and indicate the VMA that covers each page
180	* - this is potentially dodgy as we may end incrementing the page count of a	219	* - this is potentially dodgy as we may end incrementing the page count of a
181	* slab page or a secondary page from a compound page	220	* slab page or a secondary page from a compound page
182	* - don't permit access to VMAs that don't support it, such as I/O mappings	221	* - don't permit access to VMAs that don't support it, such as I/O mappings
183	*/	222	*/
184	int get_user_pages(struct task_struct tsk, struct mm_struct mm,	223	int get_user_pages(struct task_struct tsk, struct mm_struct mm,
185	unsigned long start, int len, int write, int force,	224	unsigned long start, int len, int write, int force,
186	struct page pages, struct vm_area_struct vmas)	225	struct page pages, struct vm_area_struct vmas)
187	{	226	{
188	int flags = 0;	227	int flags = 0;
189		228
190	if (write)	229	if (write)
191	flags \|= GUP_FLAGS_WRITE;	230	flags \|= GUP_FLAGS_WRITE;
192	if (force)	231	if (force)
193	flags \|= GUP_FLAGS_FORCE;	232	flags \|= GUP_FLAGS_FORCE;
194		233
195	return __get_user_pages(tsk, mm,	234	return __get_user_pages(tsk, mm,
196	start, len, flags,	235	start, len, flags,
197	pages, vmas);	236	pages, vmas);
198	}	237	}
199	EXPORT_SYMBOL(get_user_pages);	238	EXPORT_SYMBOL(get_user_pages);
200		239
201	DEFINE_RWLOCK(vmlist_lock);	240	DEFINE_RWLOCK(vmlist_lock);
202	struct vm_struct *vmlist;	241	struct vm_struct *vmlist;
203		242
204	void vfree(const void *addr)	243	void vfree(const void *addr)
205	{	244	{
206	kfree(addr);	245	kfree(addr);
207	}	246	}
208	EXPORT_SYMBOL(vfree);	247	EXPORT_SYMBOL(vfree);
209		248
210	void *__vmalloc(unsigned long size, gfp_t gfp_mask, pgprot_t prot)	249	void *__vmalloc(unsigned long size, gfp_t gfp_mask, pgprot_t prot)
211	{	250	{
212	/*	251	/*
213	* You can't specify __GFP_HIGHMEM with kmalloc() since kmalloc()	252	* You can't specify __GFP_HIGHMEM with kmalloc() since kmalloc()
214	* returns only a logical address.	253	* returns only a logical address.
215	*/	254	*/
216	return kmalloc(size, (gfp_mask \| __GFP_COMP) & ~__GFP_HIGHMEM);	255	return kmalloc(size, (gfp_mask \| __GFP_COMP) & ~__GFP_HIGHMEM);
217	}	256	}
218	EXPORT_SYMBOL(__vmalloc);	257	EXPORT_SYMBOL(__vmalloc);
219		258
220	void *vmalloc_user(unsigned long size)	259	void *vmalloc_user(unsigned long size)
221	{	260	{
222	void *ret;	261	void *ret;
223		262
224	ret = __vmalloc(size, GFP_KERNEL \| __GFP_HIGHMEM \| __GFP_ZERO,	263	ret = __vmalloc(size, GFP_KERNEL \| __GFP_HIGHMEM \| __GFP_ZERO,
225	PAGE_KERNEL);	264	PAGE_KERNEL);
226	if (ret) {	265	if (ret) {
227	struct vm_area_struct *vma;	266	struct vm_area_struct *vma;
228		267
229	down_write(&current->mm->mmap_sem);	268	down_write(&current->mm->mmap_sem);
230	vma = find_vma(current->mm, (unsigned long)ret);	269	vma = find_vma(current->mm, (unsigned long)ret);
231	if (vma)	270	if (vma)
232	vma->vm_flags \|= VM_USERMAP;	271	vma->vm_flags \|= VM_USERMAP;
233	up_write(&current->mm->mmap_sem);	272	up_write(&current->mm->mmap_sem);
234	}	273	}
235		274
236	return ret;	275	return ret;
237	}	276	}
238	EXPORT_SYMBOL(vmalloc_user);	277	EXPORT_SYMBOL(vmalloc_user);
239		278
240	struct page vmalloc_to_page(const void addr)	279	struct page vmalloc_to_page(const void addr)
241	{	280	{
242	return virt_to_page(addr);	281	return virt_to_page(addr);
243	}	282	}
244	EXPORT_SYMBOL(vmalloc_to_page);	283	EXPORT_SYMBOL(vmalloc_to_page);
245		284
246	unsigned long vmalloc_to_pfn(const void *addr)	285	unsigned long vmalloc_to_pfn(const void *addr)
247	{	286	{
248	return page_to_pfn(virt_to_page(addr));	287	return page_to_pfn(virt_to_page(addr));
249	}	288	}
250	EXPORT_SYMBOL(vmalloc_to_pfn);	289	EXPORT_SYMBOL(vmalloc_to_pfn);
251		290
252	long vread(char buf, char addr, unsigned long count)	291	long vread(char buf, char addr, unsigned long count)
253	{	292	{
254	memcpy(buf, addr, count);	293	memcpy(buf, addr, count);
255	return count;	294	return count;
256	}	295	}
257		296
258	long vwrite(char buf, char addr, unsigned long count)	297	long vwrite(char buf, char addr, unsigned long count)
259	{	298	{
260	/* Don't allow overflow */	299	/* Don't allow overflow */
261	if ((unsigned long) addr + count < count)	300	if ((unsigned long) addr + count < count)
262	count = -(unsigned long) addr;	301	count = -(unsigned long) addr;
263		302
264	memcpy(addr, buf, count);	303	memcpy(addr, buf, count);
265	return(count);	304	return(count);
266	}	305	}
267		306
268	/*	307	/*
269	* vmalloc - allocate virtually continguos memory	308	* vmalloc - allocate virtually continguos memory
270	*	309	*
271	* @size: allocation size	310	* @size: allocation size
272	*	311	*
273	* Allocate enough pages to cover @size from the page level	312	* Allocate enough pages to cover @size from the page level
274	* allocator and map them into continguos kernel virtual space.	313	* allocator and map them into continguos kernel virtual space.
275	*	314	*
276	* For tight control over page level allocator and protection flags	315	* For tight control over page level allocator and protection flags
277	* use __vmalloc() instead.	316	* use __vmalloc() instead.
278	*/	317	*/
279	void *vmalloc(unsigned long size)	318	void *vmalloc(unsigned long size)
280	{	319	{
281	return __vmalloc(size, GFP_KERNEL \| __GFP_HIGHMEM, PAGE_KERNEL);	320	return __vmalloc(size, GFP_KERNEL \| __GFP_HIGHMEM, PAGE_KERNEL);
282	}	321	}
283	EXPORT_SYMBOL(vmalloc);	322	EXPORT_SYMBOL(vmalloc);
284		323
285	void *vmalloc_node(unsigned long size, int node)	324	void *vmalloc_node(unsigned long size, int node)
286	{	325	{
287	return vmalloc(size);	326	return vmalloc(size);
288	}	327	}
289	EXPORT_SYMBOL(vmalloc_node);	328	EXPORT_SYMBOL(vmalloc_node);
290		329
291	#ifndef PAGE_KERNEL_EXEC	330	#ifndef PAGE_KERNEL_EXEC
292	# define PAGE_KERNEL_EXEC PAGE_KERNEL	331	# define PAGE_KERNEL_EXEC PAGE_KERNEL
293	#endif	332	#endif
294		333
295	/**	334	/**
296	* vmalloc_exec - allocate virtually contiguous, executable memory	335	* vmalloc_exec - allocate virtually contiguous, executable memory
297	* @size: allocation size	336	* @size: allocation size
298	*	337	*
299	* Kernel-internal function to allocate enough pages to cover @size	338	* Kernel-internal function to allocate enough pages to cover @size
300	* the page level allocator and map them into contiguous and	339	* the page level allocator and map them into contiguous and
301	* executable kernel virtual space.	340	* executable kernel virtual space.
302	*	341	*
303	* For tight control over page level allocator and protection flags	342	* For tight control over page level allocator and protection flags
304	* use __vmalloc() instead.	343	* use __vmalloc() instead.
305	*/	344	*/
306		345
307	void *vmalloc_exec(unsigned long size)	346	void *vmalloc_exec(unsigned long size)
308	{	347	{
309	return __vmalloc(size, GFP_KERNEL \| __GFP_HIGHMEM, PAGE_KERNEL_EXEC);	348	return __vmalloc(size, GFP_KERNEL \| __GFP_HIGHMEM, PAGE_KERNEL_EXEC);
310	}	349	}
311		350
312	/**	351	/**
313	* vmalloc_32 - allocate virtually contiguous memory (32bit addressable)	352	* vmalloc_32 - allocate virtually contiguous memory (32bit addressable)
314	* @size: allocation size	353	* @size: allocation size
315	*	354	*
316	* Allocate enough 32bit PA addressable pages to cover @size from the	355	* Allocate enough 32bit PA addressable pages to cover @size from the
317	* page level allocator and map them into continguos kernel virtual space.	356	* page level allocator and map them into continguos kernel virtual space.
318	*/	357	*/
319	void *vmalloc_32(unsigned long size)	358	void *vmalloc_32(unsigned long size)
320	{	359	{
321	return __vmalloc(size, GFP_KERNEL, PAGE_KERNEL);	360	return __vmalloc(size, GFP_KERNEL, PAGE_KERNEL);
322	}	361	}
323	EXPORT_SYMBOL(vmalloc_32);	362	EXPORT_SYMBOL(vmalloc_32);
324		363
325	/**	364	/**
326	* vmalloc_32_user - allocate zeroed virtually contiguous 32bit memory	365	* vmalloc_32_user - allocate zeroed virtually contiguous 32bit memory
327	* @size: allocation size	366	* @size: allocation size
328	*	367	*
329	* The resulting memory area is 32bit addressable and zeroed so it can be	368	* The resulting memory area is 32bit addressable and zeroed so it can be
330	* mapped to userspace without leaking data.	369	* mapped to userspace without leaking data.
331	*	370	*
332	* VM_USERMAP is set on the corresponding VMA so that subsequent calls to	371	* VM_USERMAP is set on the corresponding VMA so that subsequent calls to
333	* remap_vmalloc_range() are permissible.	372	* remap_vmalloc_range() are permissible.
334	*/	373	*/
335	void *vmalloc_32_user(unsigned long size)	374	void *vmalloc_32_user(unsigned long size)
336	{	375	{
337	/*	376	/*
338	* We'll have to sort out the ZONE_DMA bits for 64-bit,	377	* We'll have to sort out the ZONE_DMA bits for 64-bit,
339	* but for now this can simply use vmalloc_user() directly.	378	* but for now this can simply use vmalloc_user() directly.
340	*/	379	*/
341	return vmalloc_user(size);	380	return vmalloc_user(size);
342	}	381	}
343	EXPORT_SYMBOL(vmalloc_32_user);	382	EXPORT_SYMBOL(vmalloc_32_user);
344		383
345	void vmap(struct page *pages, unsigned int count, unsigned long flags, pgprot_t prot)	384	void vmap(struct page *pages, unsigned int count, unsigned long flags, pgprot_t prot)
346	{	385	{
347	BUG();	386	BUG();
348	return NULL;	387	return NULL;
349	}	388	}
350	EXPORT_SYMBOL(vmap);	389	EXPORT_SYMBOL(vmap);
351		390
352	void vunmap(const void *addr)	391	void vunmap(const void *addr)
353	{	392	{
354	BUG();	393	BUG();
355	}	394	}
356	EXPORT_SYMBOL(vunmap);	395	EXPORT_SYMBOL(vunmap);
357		396
358	/*	397	/*
359	* Implement a stub for vmalloc_sync_all() if the architecture chose not to	398	* Implement a stub for vmalloc_sync_all() if the architecture chose not to
360	* have one.	399	* have one.
361	*/	400	*/
362	void __attribute__((weak)) vmalloc_sync_all(void)	401	void __attribute__((weak)) vmalloc_sync_all(void)
363	{	402	{
364	}	403	}
365		404
366	int vm_insert_page(struct vm_area_struct *vma, unsigned long addr,	405	int vm_insert_page(struct vm_area_struct *vma, unsigned long addr,
367	struct page *page)	406	struct page *page)
368	{	407	{
369	return -EINVAL;	408	return -EINVAL;
370	}	409	}
371	EXPORT_SYMBOL(vm_insert_page);	410	EXPORT_SYMBOL(vm_insert_page);
372		411
373	/*	412	/*
374	* sys_brk() for the most part doesn't need the global kernel	413	* sys_brk() for the most part doesn't need the global kernel
375	* lock, except when an application is doing something nasty	414	* lock, except when an application is doing something nasty
376	* like trying to un-brk an area that has already been mapped	415	* like trying to un-brk an area that has already been mapped
377	* to a regular file. in this case, the unmapping will need	416	* to a regular file. in this case, the unmapping will need
378	* to invoke file system routines that need the global lock.	417	* to invoke file system routines that need the global lock.
379	*/	418	*/
380	asmlinkage unsigned long sys_brk(unsigned long brk)	419	asmlinkage unsigned long sys_brk(unsigned long brk)
381	{	420	{
382	struct mm_struct *mm = current->mm;	421	struct mm_struct *mm = current->mm;
383		422
384	if (brk < mm->start_brk \|\| brk > mm->context.end_brk)	423	if (brk < mm->start_brk \|\| brk > mm->context.end_brk)
385	return mm->brk;	424	return mm->brk;
386		425
387	if (mm->brk == brk)	426	if (mm->brk == brk)
388	return mm->brk;	427	return mm->brk;
389		428
390	/*	429	/*
391	* Always allow shrinking brk	430	* Always allow shrinking brk
392	*/	431	*/
393	if (brk <= mm->brk) {	432	if (brk <= mm->brk) {
394	mm->brk = brk;	433	mm->brk = brk;
395	return brk;	434	return brk;
396	}	435	}
397		436
398	/*	437	/*
399	* Ok, looks good - let it rip.	438	* Ok, looks good - let it rip.
400	*/	439	*/
401	return mm->brk = brk;	440	return mm->brk = brk;
402	}	441	}
403		442
404	#ifdef DEBUG	443	/*
405	static void show_process_blocks(void)	444	* initialise the VMA and region record slabs
		445	*/
		446	void __init mmap_init(void)
406	{	447	{
407	struct vm_list_struct *vml;	448	vm_region_jar = kmem_cache_create("vm_region_jar",
408		449	sizeof(struct vm_region), 0,
409	printk("Process blocks %d:", current->pid);	450	SLAB_PANIC, NULL);
410		451	vm_area_cachep = kmem_cache_create("vm_area_struct",
411	for (vml = &current->mm->context.vmlist; vml; vml = vml->next) {	452	sizeof(struct vm_area_struct), 0,
412	printk(" %p: %p", vml, vml->vma);	453	SLAB_PANIC, NULL);
413	if (vml->vma)
414	printk(" (%d @%lx #%d)",
415	kobjsize((void *) vml->vma->vm_start),
416	vml->vma->vm_start,
417	atomic_read(&vml->vma->vm_usage));
418	printk(vml->next ? " ->" : ".\n");
419	}
420	}	454	}
421	#endif /* DEBUG */
422		455
423	/*	456	/*
424	* add a VMA into a process's mm_struct in the appropriate place in the list	457	* validate the region tree
425	* - should be called with mm->mmap_sem held writelocked	458	* - the caller must hold the region lock
426	*/	459	*/
427	static void add_vma_to_mm(struct mm_struct mm, struct vm_list_struct vml)	460	#ifdef CONFIG_DEBUG_NOMMU_REGIONS
		461	static noinline void validate_nommu_regions(void)
428	{	462	{
429	struct vm_list_struct **ppv;	463	struct vm_region region, last;
		464	struct rb_node p, lastp;
430		465
431	for (ppv = &current->mm->context.vmlist; ppv; ppv = &(ppv)->next)	466	lastp = rb_first(&nommu_region_tree);
432	if ((*ppv)->vma->vm_start > vml->vma->vm_start)	467	if (!lastp)
433	break;	468	return;
434		469
435	vml->next = *ppv;	470	last = rb_entry(lastp, struct vm_region, vm_rb);
436	*ppv = vml;	471	if (unlikely(last->vm_end <= last->vm_start))
		472	BUG();
		473	if (unlikely(last->vm_top < last->vm_end))
		474	BUG();
		475
		476	while ((p = rb_next(lastp))) {
		477	region = rb_entry(p, struct vm_region, vm_rb);
		478	last = rb_entry(lastp, struct vm_region, vm_rb);
		479
		480	if (unlikely(region->vm_end <= region->vm_start))
		481	BUG();
		482	if (unlikely(region->vm_top < region->vm_end))
		483	BUG();
		484	if (unlikely(region->vm_start < last->vm_top))
		485	BUG();
		486
		487	lastp = p;
		488	}
437	}	489	}
		490	#else
		491	#define validate_nommu_regions() do {} while(0)
		492	#endif
438		493
439	/*	494	/*
440	* look up the first VMA in which addr resides, NULL if none	495	* add a region into the global tree
441	* - should be called with mm->mmap_sem at least held readlocked
442	*/	496	*/
443	struct vm_area_struct find_vma(struct mm_struct mm, unsigned long addr)	497	static void add_nommu_region(struct vm_region *region)
444	{	498	{
445	struct vm_list_struct loop, vml;	499	struct vm_region *pregion;
		500	struct rb_node *p, parent;
446		501
447	/* search the vm_start ordered list */	502	validate_nommu_regions();
448	vml = NULL;	503
449	for (loop = mm->context.vmlist; loop; loop = loop->next) {	504	BUG_ON(region->vm_start & ~PAGE_MASK);
450	if (loop->vma->vm_start > addr)	505
451	break;	506	parent = NULL;
452	vml = loop;	507	p = &nommu_region_tree.rb_node;
		508	while (*p) {
		509	parent = *p;
		510	pregion = rb_entry(parent, struct vm_region, vm_rb);
		511	if (region->vm_start < pregion->vm_start)
		512	p = &(*p)->rb_left;
		513	else if (region->vm_start > pregion->vm_start)
		514	p = &(*p)->rb_right;
		515	else if (pregion == region)
		516	return;
		517	else
		518	BUG();
453	}	519	}
454		520
455	if (vml && vml->vma->vm_end > addr)	521	rb_link_node(&region->vm_rb, parent, p);
456	return vml->vma;	522	rb_insert_color(&region->vm_rb, &nommu_region_tree);
457		523
458	return NULL;	524	validate_nommu_regions();
459	}	525	}
460	EXPORT_SYMBOL(find_vma);
461		526
462	/*	527	/*
463	* find a VMA	528	* delete a region from the global tree
464	* - we don't extend stack VMAs under NOMMU conditions
465	*/	529	*/
466	struct vm_area_struct find_extend_vma(struct mm_struct mm, unsigned long addr)	530	static void delete_nommu_region(struct vm_region *region)
467	{	531	{
468	return find_vma(mm, addr);	532	BUG_ON(!nommu_region_tree.rb_node);
469	}
470		533
471	int expand_stack(struct vm_area_struct *vma, unsigned long address)	534	validate_nommu_regions();
472	{	535	rb_erase(&region->vm_rb, &nommu_region_tree);
473	return -ENOMEM;	536	validate_nommu_regions();
474	}	537	}
475		538
476	/*	539	/*
477	* look up the first VMA exactly that exactly matches addr	540	* free a contiguous series of pages
478	* - should be called with mm->mmap_sem at least held readlocked
479	*/	541	*/
480	static inline struct vm_area_struct find_vma_exact(struct mm_struct mm,	542	static void free_page_series(unsigned long from, unsigned long to)
481	unsigned long addr)
482	{	543	{
483	struct vm_list_struct *vml;	544	for (; from < to; from += PAGE_SIZE) {
		545	struct page *page = virt_to_page(from);
484		546
485	/* search the vm_start ordered list */	547	kdebug("- free %lx", from);
486	for (vml = mm->context.vmlist; vml; vml = vml->next) {	548	atomic_dec(&mmap_pages_allocated);
487	if (vml->vma->vm_start == addr)	549	if (page_count(page) != 1)
488	return vml->vma;	550	kdebug("free page %p [%d]", page, page_count(page));
489	if (vml->vma->vm_start > addr)	551	put_page(page);
490	break;
491	}	552	}
492
493	return NULL;
494	}	553	}
495		554
496	/*	555	/*
497	* find a VMA in the global tree	556	* release a reference to a region
		557	* - the caller must hold the region semaphore, which this releases
		558	* - the region may not have been added to the tree yet, in which case vm_top
		559	* will equal vm_start
498	*/	560	*/
499	static inline struct vm_area_struct *find_nommu_vma(unsigned long start)	561	static void __put_nommu_region(struct vm_region *region)
		562	__releases(nommu_region_sem)
500	{	563	{
501	struct vm_area_struct *vma;	564	kenter("%p{%d}", region, atomic_read(&region->vm_usage));
502	struct rb_node *n = nommu_vma_tree.rb_node;
503		565
504	while (n) {	566	BUG_ON(!nommu_region_tree.rb_node);
505	vma = rb_entry(n, struct vm_area_struct, vm_rb);
506		567
507	if (start < vma->vm_start)	568	if (atomic_dec_and_test(&region->vm_usage)) {
508	n = n->rb_left;	569	if (region->vm_top > region->vm_start)
509	else if (start > vma->vm_start)	570	delete_nommu_region(region);
510	n = n->rb_right;	571	up_write(&nommu_region_sem);
511	else	572
512	return vma;	573	if (region->vm_file)
		574	fput(region->vm_file);
		575
		576	/* IO memory and memory shared directly out of the pagecache
		577	* from ramfs/tmpfs mustn't be released here */
		578	if (region->vm_flags & VM_MAPPED_COPY) {
		579	kdebug("free series");
		580	free_page_series(region->vm_start, region->vm_top);
		581	}
		582	kmem_cache_free(vm_region_jar, region);
		583	} else {
		584	up_write(&nommu_region_sem);
513	}	585	}
		586	}
514		587
515	return NULL;	588	/*
		589	* release a reference to a region
		590	*/
		591	static void put_nommu_region(struct vm_region *region)
		592	{
		593	down_write(&nommu_region_sem);
		594	__put_nommu_region(region);
516	}	595	}
517		596
518	/*	597	/*
519	* add a VMA in the global tree	598	* add a VMA into a process's mm_struct in the appropriate place in the list
		599	* and tree and add to the address space's page tree also if not an anonymous
		600	* page
		601	* - should be called with mm->mmap_sem held writelocked
520	*/	602	*/
521	static void add_nommu_vma(struct vm_area_struct *vma)	603	static void add_vma_to_mm(struct mm_struct mm, struct vm_area_struct vma)
522	{	604	{
523	struct vm_area_struct *pvma;	605	struct vm_area_struct pvma, *pp;
524	struct address_space *mapping;	606	struct address_space *mapping;
525	struct rb_node **p = &nommu_vma_tree.rb_node;	607	struct rb_node *p, parent;
526	struct rb_node *parent = NULL;
527		608
		609	kenter(",%p", vma);
		610
		611	BUG_ON(!vma->vm_region);
		612
		613	mm->map_count++;
		614	vma->vm_mm = mm;
		615
528	/* add the VMA to the mapping */	616	/* add the VMA to the mapping */
529	if (vma->vm_file) {	617	if (vma->vm_file) {
530	mapping = vma->vm_file->f_mapping;	618	mapping = vma->vm_file->f_mapping;
531		619
532	flush_dcache_mmap_lock(mapping);	620	flush_dcache_mmap_lock(mapping);
533	vma_prio_tree_insert(vma, &mapping->i_mmap);	621	vma_prio_tree_insert(vma, &mapping->i_mmap);
534	flush_dcache_mmap_unlock(mapping);	622	flush_dcache_mmap_unlock(mapping);
535	}	623	}
536		624
537	/* add the VMA to the master list */	625	/* add the VMA to the tree */
		626	parent = NULL;
		627	p = &mm->mm_rb.rb_node;
538	while (*p) {	628	while (*p) {
539	parent = *p;	629	parent = *p;
540	pvma = rb_entry(parent, struct vm_area_struct, vm_rb);	630	pvma = rb_entry(parent, struct vm_area_struct, vm_rb);
541		631
542	if (vma->vm_start < pvma->vm_start) {	632	/* sort by: start addr, end addr, VMA struct addr in that order
		633	* (the latter is necessary as we may get identical VMAs) */
		634	if (vma->vm_start < pvma->vm_start)
543	p = &(*p)->rb_left;	635	p = &(*p)->rb_left;
544	}	636	else if (vma->vm_start > pvma->vm_start)
545	else if (vma->vm_start > pvma->vm_start) {
546	p = &(*p)->rb_right;	637	p = &(*p)->rb_right;
547	}	638	else if (vma->vm_end < pvma->vm_end)
548	else {	639	p = &(*p)->rb_left;
549	/* mappings are at the same address - this can only	640	else if (vma->vm_end > pvma->vm_end)
550	* happen for shared-mem chardevs and shared file	641	p = &(*p)->rb_right;
551	* mappings backed by ramfs/tmpfs */	642	else if (vma < pvma)
552	BUG_ON(!(pvma->vm_flags & VM_SHARED));	643	p = &(*p)->rb_left;
553		644	else if (vma > pvma)
554	if (vma < pvma)	645	p = &(*p)->rb_right;
555	p = &(*p)->rb_left;	646	else
556	else if (vma > pvma)	647	BUG();
557	p = &(*p)->rb_right;
558	else
559	BUG();
560	}
561	}	648	}
562		649
563	rb_link_node(&vma->vm_rb, parent, p);	650	rb_link_node(&vma->vm_rb, parent, p);
564	rb_insert_color(&vma->vm_rb, &nommu_vma_tree);	651	rb_insert_color(&vma->vm_rb, &mm->mm_rb);
		652
		653	/* add VMA to the VMA list also */
		654	for (pp = &mm->mmap; (pvma = pp); pp = &(pp)->vm_next) {
		655	if (pvma->vm_start > vma->vm_start)
		656	break;
		657	if (pvma->vm_start < vma->vm_start)
		658	continue;
		659	if (pvma->vm_end < vma->vm_end)
		660	break;
		661	}
		662
		663	vma->vm_next = *pp;
		664	*pp = vma;
565	}	665	}
566		666
567	/*	667	/*
568	* delete a VMA from the global list	668	* delete a VMA from its owning mm_struct and address space
569	*/	669	*/
570	static void delete_nommu_vma(struct vm_area_struct *vma)	670	static void delete_vma_from_mm(struct vm_area_struct *vma)
571	{	671	{
		672	struct vm_area_struct **pp;
572	struct address_space *mapping;	673	struct address_space *mapping;
		674	struct mm_struct *mm = vma->vm_mm;
573		675
		676	kenter("%p", vma);
		677
		678	mm->map_count--;
		679	if (mm->mmap_cache == vma)
		680	mm->mmap_cache = NULL;
		681
574	/* remove the VMA from the mapping */	682	/* remove the VMA from the mapping */
575	if (vma->vm_file) {	683	if (vma->vm_file) {
576	mapping = vma->vm_file->f_mapping;	684	mapping = vma->vm_file->f_mapping;
577		685
578	flush_dcache_mmap_lock(mapping);	686	flush_dcache_mmap_lock(mapping);
579	vma_prio_tree_remove(vma, &mapping->i_mmap);	687	vma_prio_tree_remove(vma, &mapping->i_mmap);
580	flush_dcache_mmap_unlock(mapping);	688	flush_dcache_mmap_unlock(mapping);
581	}	689	}
582		690
583	/* remove from the master list */	691	/* remove from the MM's tree and list */
584	rb_erase(&vma->vm_rb, &nommu_vma_tree);	692	rb_erase(&vma->vm_rb, &mm->mm_rb);
		693	for (pp = &mm->mmap; pp; pp = &(pp)->vm_next) {
		694	if (*pp == vma) {
		695	*pp = vma->vm_next;
		696	break;
		697	}
		698	}
		699
		700	vma->vm_mm = NULL;
585	}	701	}
586		702
587	/*	703	/*
		704	* destroy a VMA record
		705	*/
		706	static void delete_vma(struct mm_struct mm, struct vm_area_struct vma)
		707	{
		708	kenter("%p", vma);
		709	if (vma->vm_ops && vma->vm_ops->close)
		710	vma->vm_ops->close(vma);
		711	if (vma->vm_file) {
		712	fput(vma->vm_file);
		713	if (vma->vm_flags & VM_EXECUTABLE)
		714	removed_exe_file_vma(mm);
		715	}
		716	put_nommu_region(vma->vm_region);
		717	kmem_cache_free(vm_area_cachep, vma);
		718	}
		719
		720	/*
		721	* look up the first VMA in which addr resides, NULL if none
		722	* - should be called with mm->mmap_sem at least held readlocked
		723	*/
		724	struct vm_area_struct find_vma(struct mm_struct mm, unsigned long addr)
		725	{
		726	struct vm_area_struct *vma;
		727	struct rb_node *n = mm->mm_rb.rb_node;
		728
		729	/* check the cache first */
		730	vma = mm->mmap_cache;
		731	if (vma && vma->vm_start <= addr && vma->vm_end > addr)
		732	return vma;
		733
		734	/* trawl the tree (there may be multiple mappings in which addr
		735	* resides) */
		736	for (n = rb_first(&mm->mm_rb); n; n = rb_next(n)) {
		737	vma = rb_entry(n, struct vm_area_struct, vm_rb);
		738	if (vma->vm_start > addr)
		739	return NULL;
		740	if (vma->vm_end > addr) {
		741	mm->mmap_cache = vma;
		742	return vma;
		743	}
		744	}
		745
		746	return NULL;
		747	}
		748	EXPORT_SYMBOL(find_vma);
		749
		750	/*
		751	* find a VMA
		752	* - we don't extend stack VMAs under NOMMU conditions
		753	*/
		754	struct vm_area_struct find_extend_vma(struct mm_struct mm, unsigned long addr)
		755	{
		756	return find_vma(mm, addr);
		757	}
		758
		759	/*
		760	* expand a stack to a given address
		761	* - not supported under NOMMU conditions
		762	*/
		763	int expand_stack(struct vm_area_struct *vma, unsigned long address)
		764	{
		765	return -ENOMEM;
		766	}
		767
		768	/*
		769	* look up the first VMA exactly that exactly matches addr
		770	* - should be called with mm->mmap_sem at least held readlocked
		771	*/
		772	static struct vm_area_struct find_vma_exact(struct mm_struct mm,
		773	unsigned long addr,
		774	unsigned long len)
		775	{
		776	struct vm_area_struct *vma;
		777	struct rb_node *n = mm->mm_rb.rb_node;
		778	unsigned long end = addr + len;
		779
		780	/* check the cache first */
		781	vma = mm->mmap_cache;
		782	if (vma && vma->vm_start == addr && vma->vm_end == end)
		783	return vma;
		784
		785	/* trawl the tree (there may be multiple mappings in which addr
		786	* resides) */
		787	for (n = rb_first(&mm->mm_rb); n; n = rb_next(n)) {
		788	vma = rb_entry(n, struct vm_area_struct, vm_rb);
		789	if (vma->vm_start < addr)
		790	continue;
		791	if (vma->vm_start > addr)
		792	return NULL;
		793	if (vma->vm_end == end) {
		794	mm->mmap_cache = vma;
		795	return vma;
		796	}
		797	}
		798
		799	return NULL;
		800	}
		801
		802	/*
588	* determine whether a mapping should be permitted and, if so, what sort of	803	* determine whether a mapping should be permitted and, if so, what sort of
589	* mapping we're capable of supporting	804	* mapping we're capable of supporting
590	*/	805	*/
591	static int validate_mmap_request(struct file *file,	806	static int validate_mmap_request(struct file *file,
592	unsigned long addr,	807	unsigned long addr,
593	unsigned long len,	808	unsigned long len,
594	unsigned long prot,	809	unsigned long prot,
595	unsigned long flags,	810	unsigned long flags,
596	unsigned long pgoff,	811	unsigned long pgoff,
597	unsigned long *_capabilities)	812	unsigned long *_capabilities)
598	{	813	{
599	unsigned long capabilities;	814	unsigned long capabilities, rlen;
600	unsigned long reqprot = prot;	815	unsigned long reqprot = prot;
601	int ret;	816	int ret;
602		817
603	/* do the simple checks first */	818	/* do the simple checks first */
604	if (flags & MAP_FIXED \|\| addr) {	819	if (flags & MAP_FIXED \|\| addr) {
605	printk(KERN_DEBUG	820	printk(KERN_DEBUG
606	"%d: Can't do fixed-address/overlay mmap of RAM\n",	821	"%d: Can't do fixed-address/overlay mmap of RAM\n",
607	current->pid);	822	current->pid);
608	return -EINVAL;	823	return -EINVAL;
609	}	824	}
610		825
611	if ((flags & MAP_TYPE) != MAP_PRIVATE &&	826	if ((flags & MAP_TYPE) != MAP_PRIVATE &&
612	(flags & MAP_TYPE) != MAP_SHARED)	827	(flags & MAP_TYPE) != MAP_SHARED)
613	return -EINVAL;	828	return -EINVAL;
614		829
615	if (!len)	830	if (!len)
616	return -EINVAL;	831	return -EINVAL;
617		832
618	/* Careful about overflows.. */	833	/* Careful about overflows.. */
619	len = PAGE_ALIGN(len);	834	rlen = PAGE_ALIGN(len);
620	if (!len \|\| len > TASK_SIZE)	835	if (!rlen \|\| rlen > TASK_SIZE)
621	return -ENOMEM;	836	return -ENOMEM;
622		837
623	/* offset overflow? */	838	/* offset overflow? */
624	if ((pgoff + (len >> PAGE_SHIFT)) < pgoff)	839	if ((pgoff + (rlen >> PAGE_SHIFT)) < pgoff)
625	return -EOVERFLOW;	840	return -EOVERFLOW;
626		841
627	if (file) {	842	if (file) {
628	/* validate file mapping requests */	843	/* validate file mapping requests */
629	struct address_space *mapping;	844	struct address_space *mapping;
630		845
631	/* files must support mmap */	846	/* files must support mmap */
632	if (!file->f_op \|\| !file->f_op->mmap)	847	if (!file->f_op \|\| !file->f_op->mmap)
633	return -ENODEV;	848	return -ENODEV;
634		849
635	/* work out if what we've got could possibly be shared	850	/* work out if what we've got could possibly be shared
636	* - we support chardevs that provide their own "memory"	851	* - we support chardevs that provide their own "memory"
637	* - we support files/blockdevs that are memory backed	852	* - we support files/blockdevs that are memory backed
638	*/	853	*/
639	mapping = file->f_mapping;	854	mapping = file->f_mapping;
640	if (!mapping)	855	if (!mapping)
641	mapping = file->f_path.dentry->d_inode->i_mapping;	856	mapping = file->f_path.dentry->d_inode->i_mapping;
642		857
643	capabilities = 0;	858	capabilities = 0;
644	if (mapping && mapping->backing_dev_info)	859	if (mapping && mapping->backing_dev_info)
645	capabilities = mapping->backing_dev_info->capabilities;	860	capabilities = mapping->backing_dev_info->capabilities;
646		861
647	if (!capabilities) {	862	if (!capabilities) {
648	/* no explicit capabilities set, so assume some	863	/* no explicit capabilities set, so assume some
649	* defaults */	864	* defaults */
650	switch (file->f_path.dentry->d_inode->i_mode & S_IFMT) {	865	switch (file->f_path.dentry->d_inode->i_mode & S_IFMT) {
651	case S_IFREG:	866	case S_IFREG:
652	case S_IFBLK:	867	case S_IFBLK:
653	capabilities = BDI_CAP_MAP_COPY;	868	capabilities = BDI_CAP_MAP_COPY;
654	break;	869	break;
655		870
656	case S_IFCHR:	871	case S_IFCHR:
657	capabilities =	872	capabilities =
658	BDI_CAP_MAP_DIRECT \|	873	BDI_CAP_MAP_DIRECT \|
659	BDI_CAP_READ_MAP \|	874	BDI_CAP_READ_MAP \|
660	BDI_CAP_WRITE_MAP;	875	BDI_CAP_WRITE_MAP;
661	break;	876	break;
662		877
663	default:	878	default:
664	return -EINVAL;	879	return -EINVAL;
665	}	880	}
666	}	881	}
667		882
668	/* eliminate any capabilities that we can't support on this	883	/* eliminate any capabilities that we can't support on this
669	* device */	884	* device */
670	if (!file->f_op->get_unmapped_area)	885	if (!file->f_op->get_unmapped_area)
671	capabilities &= ~BDI_CAP_MAP_DIRECT;	886	capabilities &= ~BDI_CAP_MAP_DIRECT;
672	if (!file->f_op->read)	887	if (!file->f_op->read)
673	capabilities &= ~BDI_CAP_MAP_COPY;	888	capabilities &= ~BDI_CAP_MAP_COPY;
674		889
675	if (flags & MAP_SHARED) {	890	if (flags & MAP_SHARED) {
676	/* do checks for writing, appending and locking */	891	/* do checks for writing, appending and locking */
677	if ((prot & PROT_WRITE) &&	892	if ((prot & PROT_WRITE) &&
678	!(file->f_mode & FMODE_WRITE))	893	!(file->f_mode & FMODE_WRITE))
679	return -EACCES;	894	return -EACCES;
680		895
681	if (IS_APPEND(file->f_path.dentry->d_inode) &&	896	if (IS_APPEND(file->f_path.dentry->d_inode) &&
682	(file->f_mode & FMODE_WRITE))	897	(file->f_mode & FMODE_WRITE))
683	return -EACCES;	898	return -EACCES;
684		899
685	if (locks_verify_locked(file->f_path.dentry->d_inode))	900	if (locks_verify_locked(file->f_path.dentry->d_inode))
686	return -EAGAIN;	901	return -EAGAIN;
687		902
688	if (!(capabilities & BDI_CAP_MAP_DIRECT))	903	if (!(capabilities & BDI_CAP_MAP_DIRECT))
689	return -ENODEV;	904	return -ENODEV;
690		905
691	if (((prot & PROT_READ) && !(capabilities & BDI_CAP_READ_MAP)) \|\|	906	if (((prot & PROT_READ) && !(capabilities & BDI_CAP_READ_MAP)) \|\|
692	((prot & PROT_WRITE) && !(capabilities & BDI_CAP_WRITE_MAP)) \|\|	907	((prot & PROT_WRITE) && !(capabilities & BDI_CAP_WRITE_MAP)) \|\|
693	((prot & PROT_EXEC) && !(capabilities & BDI_CAP_EXEC_MAP))	908	((prot & PROT_EXEC) && !(capabilities & BDI_CAP_EXEC_MAP))
694	) {	909	) {
695	printk("MAP_SHARED not completely supported on !MMU\n");	910	printk("MAP_SHARED not completely supported on !MMU\n");
696	return -EINVAL;	911	return -EINVAL;
697	}	912	}
698		913
699	/* we mustn't privatise shared mappings */	914	/* we mustn't privatise shared mappings */
700	capabilities &= ~BDI_CAP_MAP_COPY;	915	capabilities &= ~BDI_CAP_MAP_COPY;
701	}	916	}
702	else {	917	else {
703	/* we're going to read the file into private memory we	918	/* we're going to read the file into private memory we
704	* allocate */	919	* allocate */
705	if (!(capabilities & BDI_CAP_MAP_COPY))	920	if (!(capabilities & BDI_CAP_MAP_COPY))
706	return -ENODEV;	921	return -ENODEV;
707		922
708	/* we don't permit a private writable mapping to be	923	/* we don't permit a private writable mapping to be
709	* shared with the backing device */	924	* shared with the backing device */
710	if (prot & PROT_WRITE)	925	if (prot & PROT_WRITE)
711	capabilities &= ~BDI_CAP_MAP_DIRECT;	926	capabilities &= ~BDI_CAP_MAP_DIRECT;
712	}	927	}
713		928
714	/* handle executable mappings and implied executable	929	/* handle executable mappings and implied executable
715	* mappings */	930	* mappings */
716	if (file->f_path.mnt->mnt_flags & MNT_NOEXEC) {	931	if (file->f_path.mnt->mnt_flags & MNT_NOEXEC) {
717	if (prot & PROT_EXEC)	932	if (prot & PROT_EXEC)
718	return -EPERM;	933	return -EPERM;
719	}	934	}
720	else if ((prot & PROT_READ) && !(prot & PROT_EXEC)) {	935	else if ((prot & PROT_READ) && !(prot & PROT_EXEC)) {
721	/* handle implication of PROT_EXEC by PROT_READ */	936	/* handle implication of PROT_EXEC by PROT_READ */
722	if (current->personality & READ_IMPLIES_EXEC) {	937	if (current->personality & READ_IMPLIES_EXEC) {
723	if (capabilities & BDI_CAP_EXEC_MAP)	938	if (capabilities & BDI_CAP_EXEC_MAP)
724	prot \|= PROT_EXEC;	939	prot \|= PROT_EXEC;
725	}	940	}
726	}	941	}
727	else if ((prot & PROT_READ) &&	942	else if ((prot & PROT_READ) &&
728	(prot & PROT_EXEC) &&	943	(prot & PROT_EXEC) &&
729	!(capabilities & BDI_CAP_EXEC_MAP)	944	!(capabilities & BDI_CAP_EXEC_MAP)
730	) {	945	) {
731	/* backing file is not executable, try to copy */	946	/* backing file is not executable, try to copy */
732	capabilities &= ~BDI_CAP_MAP_DIRECT;	947	capabilities &= ~BDI_CAP_MAP_DIRECT;
733	}	948	}
734	}	949	}
735	else {	950	else {
736	/* anonymous mappings are always memory backed and can be	951	/* anonymous mappings are always memory backed and can be
737	* privately mapped	952	* privately mapped
738	*/	953	*/
739	capabilities = BDI_CAP_MAP_COPY;	954	capabilities = BDI_CAP_MAP_COPY;
740		955
741	/* handle PROT_EXEC implication by PROT_READ */	956	/* handle PROT_EXEC implication by PROT_READ */
742	if ((prot & PROT_READ) &&	957	if ((prot & PROT_READ) &&
743	(current->personality & READ_IMPLIES_EXEC))	958	(current->personality & READ_IMPLIES_EXEC))
744	prot \|= PROT_EXEC;	959	prot \|= PROT_EXEC;
745	}	960	}
746		961
747	/* allow the security API to have its say */	962	/* allow the security API to have its say */
748	ret = security_file_mmap(file, reqprot, prot, flags, addr, 0);	963	ret = security_file_mmap(file, reqprot, prot, flags, addr, 0);
749	if (ret < 0)	964	if (ret < 0)
750	return ret;	965	return ret;
751		966
752	/* looks okay */	967	/* looks okay */
753	*_capabilities = capabilities;	968	*_capabilities = capabilities;
754	return 0;	969	return 0;
755	}	970	}
756		971
757	/*	972	/*
758	* we've determined that we can make the mapping, now translate what we	973	* we've determined that we can make the mapping, now translate what we
759	* now know into VMA flags	974	* now know into VMA flags
760	*/	975	*/
761	static unsigned long determine_vm_flags(struct file *file,	976	static unsigned long determine_vm_flags(struct file *file,
762	unsigned long prot,	977	unsigned long prot,
763	unsigned long flags,	978	unsigned long flags,
764	unsigned long capabilities)	979	unsigned long capabilities)
765	{	980	{
766	unsigned long vm_flags;	981	unsigned long vm_flags;
767		982
768	vm_flags = calc_vm_prot_bits(prot) \| calc_vm_flag_bits(flags);	983	vm_flags = calc_vm_prot_bits(prot) \| calc_vm_flag_bits(flags);
769	vm_flags \|= VM_MAYREAD \| VM_MAYWRITE \| VM_MAYEXEC;	984	vm_flags \|= VM_MAYREAD \| VM_MAYWRITE \| VM_MAYEXEC;
770	/* vm_flags \|= mm->def_flags; */	985	/* vm_flags \|= mm->def_flags; */
771		986
772	if (!(capabilities & BDI_CAP_MAP_DIRECT)) {	987	if (!(capabilities & BDI_CAP_MAP_DIRECT)) {
773	/* attempt to share read-only copies of mapped file chunks */	988	/* attempt to share read-only copies of mapped file chunks */
774	if (file && !(prot & PROT_WRITE))	989	if (file && !(prot & PROT_WRITE))
775	vm_flags \|= VM_MAYSHARE;	990	vm_flags \|= VM_MAYSHARE;
776	}	991	}
777	else {	992	else {
778	/* overlay a shareable mapping on the backing device or inode	993	/* overlay a shareable mapping on the backing device or inode
779	* if possible - used for chardevs, ramfs/tmpfs/shmfs and	994	* if possible - used for chardevs, ramfs/tmpfs/shmfs and
780	* romfs/cramfs */	995	* romfs/cramfs */
781	if (flags & MAP_SHARED)	996	if (flags & MAP_SHARED)
782	vm_flags \|= VM_MAYSHARE \| VM_SHARED;	997	vm_flags \|= VM_MAYSHARE \| VM_SHARED;
783	else if ((((vm_flags & capabilities) ^ vm_flags) & BDI_CAP_VMFLAGS) == 0)	998	else if ((((vm_flags & capabilities) ^ vm_flags) & BDI_CAP_VMFLAGS) == 0)
784	vm_flags \|= VM_MAYSHARE;	999	vm_flags \|= VM_MAYSHARE;
785	}	1000	}
786		1001
787	/* refuse to let anyone share private mappings with this process if	1002	/* refuse to let anyone share private mappings with this process if
788	* it's being traced - otherwise breakpoints set in it may interfere	1003	* it's being traced - otherwise breakpoints set in it may interfere
789	* with another untraced process	1004	* with another untraced process
790	*/	1005	*/
791	if ((flags & MAP_PRIVATE) && tracehook_expect_breakpoints(current))	1006	if ((flags & MAP_PRIVATE) && tracehook_expect_breakpoints(current))
792	vm_flags &= ~VM_MAYSHARE;	1007	vm_flags &= ~VM_MAYSHARE;
793		1008
794	return vm_flags;	1009	return vm_flags;
795	}	1010	}
796		1011
797	/*	1012	/*
798	* set up a shared mapping on a file	1013	* set up a shared mapping on a file (the driver or filesystem provides and
		1014	* pins the storage)
799	*/	1015	*/
800	static int do_mmap_shared_file(struct vm_area_struct *vma, unsigned long len)	1016	static int do_mmap_shared_file(struct vm_area_struct *vma)
801	{	1017	{
802	int ret;	1018	int ret;
803		1019
804	ret = vma->vm_file->f_op->mmap(vma->vm_file, vma);	1020	ret = vma->vm_file->f_op->mmap(vma->vm_file, vma);
		1021	if (ret == 0) {
		1022	vma->vm_region->vm_top = vma->vm_region->vm_end;
		1023	return ret;
		1024	}
805	if (ret != -ENOSYS)	1025	if (ret != -ENOSYS)
806	return ret;	1026	return ret;
807		1027
808	/* getting an ENOSYS error indicates that direct mmap isn't	1028	/* getting an ENOSYS error indicates that direct mmap isn't
809	* possible (as opposed to tried but failed) so we'll fall	1029	* possible (as opposed to tried but failed) so we'll fall
810	* through to making a private copy of the data and mapping	1030	* through to making a private copy of the data and mapping
811	* that if we can */	1031	* that if we can */
812	return -ENODEV;	1032	return -ENODEV;
813	}	1033	}
814		1034
815	/*	1035	/*
816	* set up a private mapping or an anonymous shared mapping	1036	* set up a private mapping or an anonymous shared mapping
817	*/	1037	*/
818	static int do_mmap_private(struct vm_area_struct *vma, unsigned long len)	1038	static int do_mmap_private(struct vm_area_struct *vma,
		1039	struct vm_region *region,
		1040	unsigned long len)
819	{	1041	{
		1042	struct page *pages;
		1043	unsigned long total, point, n, rlen;
820	void *base;	1044	void *base;
821	int ret;	1045	int ret, order;
822		1046
823	/* invoke the file's mapping function so that it can keep track of	1047	/* invoke the file's mapping function so that it can keep track of
824	* shared mappings on devices or memory	1048	* shared mappings on devices or memory
825	* - VM_MAYSHARE will be set if it may attempt to share	1049	* - VM_MAYSHARE will be set if it may attempt to share
826	*/	1050	*/
827	if (vma->vm_file) {	1051	if (vma->vm_file) {
828	ret = vma->vm_file->f_op->mmap(vma->vm_file, vma);	1052	ret = vma->vm_file->f_op->mmap(vma->vm_file, vma);
829	if (ret != -ENOSYS) {	1053	if (ret == 0) {
830	/* shouldn't return success if we're not sharing */	1054	/* shouldn't return success if we're not sharing */
831	BUG_ON(ret == 0 && !(vma->vm_flags & VM_MAYSHARE));	1055	BUG_ON(!(vma->vm_flags & VM_MAYSHARE));
832	return ret; /* success or a real error */	1056	vma->vm_region->vm_top = vma->vm_region->vm_end;
		1057	return ret;
833	}	1058	}
		1059	if (ret != -ENOSYS)
		1060	return ret;
834		1061
835	/* getting an ENOSYS error indicates that direct mmap isn't	1062	/* getting an ENOSYS error indicates that direct mmap isn't
836	* possible (as opposed to tried but failed) so we'll try to	1063	* possible (as opposed to tried but failed) so we'll try to
837	* make a private copy of the data and map that instead */	1064	* make a private copy of the data and map that instead */
838	}	1065	}
839		1066
		1067	rlen = PAGE_ALIGN(len);
		1068
840	/* allocate some memory to hold the mapping	1069	/* allocate some memory to hold the mapping
841	* - note that this may not return a page-aligned address if the object	1070	* - note that this may not return a page-aligned address if the object
842	* we're allocating is smaller than a page	1071	* we're allocating is smaller than a page
843	*/	1072	*/
844	base = kmalloc(len, GFP_KERNEL\|__GFP_COMP);	1073	order = get_order(rlen);
845	if (!base)	1074	kdebug("alloc order %d for %lx", order, len);
		1075
		1076	pages = alloc_pages(GFP_KERNEL, order);
		1077	if (!pages)
846	goto enomem;	1078	goto enomem;
847		1079
848	vma->vm_start = (unsigned long) base;	1080	total = 1 << order;
849	vma->vm_end = vma->vm_start + len;	1081	atomic_add(total, &mmap_pages_allocated);
850	vma->vm_flags \|= VM_MAPPED_COPY;
851		1082
852	#ifdef WARN_ON_SLACK	1083	point = rlen >> PAGE_SHIFT;
853	if (len + WARN_ON_SLACK <= kobjsize(result))
854	printk("Allocation of %lu bytes from process %d has %lu bytes of slack\n",
855	len, current->pid, kobjsize(result) - len);
856	#endif
857		1084
		1085	/* we allocated a power-of-2 sized page set, so we may want to trim off
		1086	* the excess */
		1087	if (sysctl_nr_trim_pages && total - point >= sysctl_nr_trim_pages) {
		1088	while (total > point) {
		1089	order = ilog2(total - point);
		1090	n = 1 << order;
		1091	kdebug("shave %lu/%lu @%lu", n, total - point, total);
		1092	atomic_sub(n, &mmap_pages_allocated);
		1093	total -= n;
		1094	set_page_refcounted(pages + total);
		1095	__free_pages(pages + total, order);
		1096	}
		1097	}
		1098
		1099	for (point = 1; point < total; point++)
		1100	set_page_refcounted(&pages[point]);
		1101
		1102	base = page_address(pages);
		1103	region->vm_flags = vma->vm_flags \|= VM_MAPPED_COPY;
		1104	region->vm_start = (unsigned long) base;
		1105	region->vm_end = region->vm_start + rlen;
		1106	region->vm_top = region->vm_start + (total << PAGE_SHIFT);
		1107
		1108	vma->vm_start = region->vm_start;
		1109	vma->vm_end = region->vm_start + len;
		1110
858	if (vma->vm_file) {	1111	if (vma->vm_file) {
859	/* read the contents of a file into the copy */	1112	/* read the contents of a file into the copy */
860	mm_segment_t old_fs;	1113	mm_segment_t old_fs;
861	loff_t fpos;	1114	loff_t fpos;
862		1115
863	fpos = vma->vm_pgoff;	1116	fpos = vma->vm_pgoff;
864	fpos <<= PAGE_SHIFT;	1117	fpos <<= PAGE_SHIFT;
865		1118
866	old_fs = get_fs();	1119	old_fs = get_fs();
867	set_fs(KERNEL_DS);	1120	set_fs(KERNEL_DS);
868	ret = vma->vm_file->f_op->read(vma->vm_file, base, len, &fpos);	1121	ret = vma->vm_file->f_op->read(vma->vm_file, base, rlen, &fpos);
869	set_fs(old_fs);	1122	set_fs(old_fs);
870		1123
871	if (ret < 0)	1124	if (ret < 0)
872	goto error_free;	1125	goto error_free;
873		1126
874	/* clear the last little bit */	1127	/* clear the last little bit */
875	if (ret < len)	1128	if (ret < rlen)
876	memset(base + ret, 0, len - ret);	1129	memset(base + ret, 0, rlen - ret);
877		1130
878	} else {	1131	} else {
879	/* if it's an anonymous mapping, then just clear it */	1132	/* if it's an anonymous mapping, then just clear it */
880	memset(base, 0, len);	1133	memset(base, 0, rlen);
881	}	1134	}
882		1135
883	return 0;	1136	return 0;
884		1137
885	error_free:	1138	error_free:
886	kfree(base);	1139	free_page_series(region->vm_start, region->vm_end);
887	vma->vm_start = 0;	1140	region->vm_start = vma->vm_start = 0;
		1141	region->vm_end = vma->vm_end = 0;
		1142	region->vm_top = 0;
888	return ret;	1143	return ret;
889		1144
890	enomem:	1145	enomem:
891	printk("Allocation of length %lu from process %d failed\n",	1146	printk("Allocation of length %lu from process %d failed\n",
892	len, current->pid);	1147	len, current->pid);
893	show_free_areas();	1148	show_free_areas();
894	return -ENOMEM;	1149	return -ENOMEM;
895	}	1150	}
896		1151
897	/*	1152	/*
898	* handle mapping creation for uClinux	1153	* handle mapping creation for uClinux
899	*/	1154	*/
900	unsigned long do_mmap_pgoff(struct file *file,	1155	unsigned long do_mmap_pgoff(struct file *file,
901	unsigned long addr,	1156	unsigned long addr,
902	unsigned long len,	1157	unsigned long len,
903	unsigned long prot,	1158	unsigned long prot,
904	unsigned long flags,	1159	unsigned long flags,
905	unsigned long pgoff)	1160	unsigned long pgoff)
906	{	1161	{
907	struct vm_list_struct *vml = NULL;	1162	struct vm_area_struct *vma;
908	struct vm_area_struct *vma = NULL;	1163	struct vm_region *region;
909	struct rb_node *rb;	1164	struct rb_node *rb;
910	unsigned long capabilities, vm_flags;	1165	unsigned long capabilities, vm_flags, result;
911	void *result;
912	int ret;	1166	int ret;
913		1167
		1168	kenter(",%lx,%lx,%lx,%lx,%lx", addr, len, prot, flags, pgoff);
		1169
914	if (!(flags & MAP_FIXED))	1170	if (!(flags & MAP_FIXED))
915	addr = round_hint_to_min(addr);	1171	addr = round_hint_to_min(addr);
916		1172
917	/* decide whether we should attempt the mapping, and if so what sort of	1173	/* decide whether we should attempt the mapping, and if so what sort of
918	* mapping */	1174	* mapping */
919	ret = validate_mmap_request(file, addr, len, prot, flags, pgoff,	1175	ret = validate_mmap_request(file, addr, len, prot, flags, pgoff,
920	&capabilities);	1176	&capabilities);
921	if (ret < 0)	1177	if (ret < 0) {
		1178	kleave(" = %d [val]", ret);
922	return ret;	1179	return ret;
		1180	}
923		1181
924	/* we've determined that we can make the mapping, now translate what we	1182	/* we've determined that we can make the mapping, now translate what we
925	* now know into VMA flags */	1183	* now know into VMA flags */
926	vm_flags = determine_vm_flags(file, prot, flags, capabilities);	1184	vm_flags = determine_vm_flags(file, prot, flags, capabilities);
927		1185
928	/* we're going to need to record the mapping if it works */	1186	/* we're going to need to record the mapping */
929	vml = kzalloc(sizeof(struct vm_list_struct), GFP_KERNEL);	1187	region = kmem_cache_zalloc(vm_region_jar, GFP_KERNEL);
930	if (!vml)	1188	if (!region)
931	goto error_getting_vml;	1189	goto error_getting_region;
932		1190
933	down_write(&nommu_vma_sem);	1191	vma = kmem_cache_zalloc(vm_area_cachep, GFP_KERNEL);
		1192	if (!vma)
		1193	goto error_getting_vma;
934		1194
935	/* if we want to share, we need to check for VMAs created by other	1195	atomic_set(&region->vm_usage, 1);
		1196	region->vm_flags = vm_flags;
		1197	region->vm_pgoff = pgoff;
		1198
		1199	INIT_LIST_HEAD(&vma->anon_vma_node);
		1200	vma->vm_flags = vm_flags;
		1201	vma->vm_pgoff = pgoff;
		1202
		1203	if (file) {
		1204	region->vm_file = file;
		1205	get_file(file);
		1206	vma->vm_file = file;
		1207	get_file(file);
		1208	if (vm_flags & VM_EXECUTABLE) {
		1209	added_exe_file_vma(current->mm);
		1210	vma->vm_mm = current->mm;
		1211	}
		1212	}
		1213
		1214	down_write(&nommu_region_sem);
		1215
		1216	/* if we want to share, we need to check for regions created by other
936	* mmap() calls that overlap with our proposed mapping	1217	* mmap() calls that overlap with our proposed mapping
937	* - we can only share with an exact match on most regular files	1218	* - we can only share with a superset match on most regular files
938	* - shared mappings on character devices and memory backed files are	1219	* - shared mappings on character devices and memory backed files are
939	* permitted to overlap inexactly as far as we are concerned for in	1220	* permitted to overlap inexactly as far as we are concerned for in
940	* these cases, sharing is handled in the driver or filesystem rather	1221	* these cases, sharing is handled in the driver or filesystem rather
941	* than here	1222	* than here
942	*/	1223	*/
943	if (vm_flags & VM_MAYSHARE) {	1224	if (vm_flags & VM_MAYSHARE) {
944	unsigned long pglen = (len + PAGE_SIZE - 1) >> PAGE_SHIFT;	1225	struct vm_region *pregion;
945	unsigned long vmpglen;	1226	unsigned long pglen, rpglen, pgend, rpgend, start;
946		1227
947	/* suppress VMA sharing for shared regions */	1228	pglen = (len + PAGE_SIZE - 1) >> PAGE_SHIFT;
948	if (vm_flags & VM_SHARED &&	1229	pgend = pgoff + pglen;
949	capabilities & BDI_CAP_MAP_DIRECT)
950	goto dont_share_VMAs;
951		1230
952	for (rb = rb_first(&nommu_vma_tree); rb; rb = rb_next(rb)) {	1231	for (rb = rb_first(&nommu_region_tree); rb; rb = rb_next(rb)) {
953	vma = rb_entry(rb, struct vm_area_struct, vm_rb);	1232	pregion = rb_entry(rb, struct vm_region, vm_rb);
954		1233
955	if (!(vma->vm_flags & VM_MAYSHARE))	1234	if (!(pregion->vm_flags & VM_MAYSHARE))
956	continue;	1235	continue;
957		1236
958	/* search for overlapping mappings on the same file */	1237	/* search for overlapping mappings on the same file */
959	if (vma->vm_file->f_path.dentry->d_inode != file->f_path.dentry->d_inode)	1238	if (pregion->vm_file->f_path.dentry->d_inode !=
		1239	file->f_path.dentry->d_inode)
960	continue;	1240	continue;
961		1241
962	if (vma->vm_pgoff >= pgoff + pglen)	1242	if (pregion->vm_pgoff >= pgend)
963	continue;	1243	continue;
964		1244
965	vmpglen = vma->vm_end - vma->vm_start + PAGE_SIZE - 1;	1245	rpglen = pregion->vm_end - pregion->vm_start;
966	vmpglen >>= PAGE_SHIFT;	1246	rpglen = (rpglen + PAGE_SIZE - 1) >> PAGE_SHIFT;
967	if (pgoff >= vma->vm_pgoff + vmpglen)	1247	rpgend = pregion->vm_pgoff + rpglen;
		1248	if (pgoff >= rpgend)
968	continue;	1249	continue;
969		1250
970	/* handle inexactly overlapping matches between mappings */	1251	/* handle inexactly overlapping matches between
971	if (vma->vm_pgoff != pgoff \|\| vmpglen != pglen) {	1252	* mappings */
		1253	if ((pregion->vm_pgoff != pgoff \|\| rpglen != pglen) &&
		1254	!(pgoff >= pregion->vm_pgoff && pgend <= rpgend)) {
		1255	/* new mapping is not a subset of the region */
972	if (!(capabilities & BDI_CAP_MAP_DIRECT))	1256	if (!(capabilities & BDI_CAP_MAP_DIRECT))
973	goto sharing_violation;	1257	goto sharing_violation;
974	continue;	1258	continue;
975	}	1259	}
976		1260
977	/* we've found a VMA we can share */	1261	/* we've found a region we can share */
978	atomic_inc(&vma->vm_usage);	1262	atomic_inc(&pregion->vm_usage);
		1263	vma->vm_region = pregion;
		1264	start = pregion->vm_start;
		1265	start += (pgoff - pregion->vm_pgoff) << PAGE_SHIFT;
		1266	vma->vm_start = start;
		1267	vma->vm_end = start + len;
979		1268
980	vml->vma = vma;	1269	if (pregion->vm_flags & VM_MAPPED_COPY) {
981	result = (void *) vma->vm_start;	1270	kdebug("share copy");
982	goto shared;	1271	vma->vm_flags \|= VM_MAPPED_COPY;
		1272	} else {
		1273	kdebug("share mmap");
		1274	ret = do_mmap_shared_file(vma);
		1275	if (ret < 0) {
		1276	vma->vm_region = NULL;
		1277	vma->vm_start = 0;
		1278	vma->vm_end = 0;
		1279	atomic_dec(&pregion->vm_usage);
		1280	pregion = NULL;
		1281	goto error_just_free;
		1282	}
		1283	}
		1284	fput(region->vm_file);
		1285	kmem_cache_free(vm_region_jar, region);
		1286	region = pregion;
		1287	result = start;
		1288	goto share;
983	}	1289	}
984		1290
985	dont_share_VMAs:
986	vma = NULL;
987
988	/* obtain the address at which to make a shared mapping	1291	/* obtain the address at which to make a shared mapping
989	* - this is the hook for quasi-memory character devices to	1292	* - this is the hook for quasi-memory character devices to
990	* tell us the location of a shared mapping	1293	* tell us the location of a shared mapping
991	*/	1294	*/
992	if (file && file->f_op->get_unmapped_area) {	1295	if (file && file->f_op->get_unmapped_area) {
993	addr = file->f_op->get_unmapped_area(file, addr, len,	1296	addr = file->f_op->get_unmapped_area(file, addr, len,
994	pgoff, flags);	1297	pgoff, flags);
995	if (IS_ERR((void *) addr)) {	1298	if (IS_ERR((void *) addr)) {
996	ret = addr;	1299	ret = addr;
997	if (ret != (unsigned long) -ENOSYS)	1300	if (ret != (unsigned long) -ENOSYS)
998	goto error;	1301	goto error_just_free;
999		1302
1000	/* the driver refused to tell us where to site	1303	/* the driver refused to tell us where to site
1001	* the mapping so we'll have to attempt to copy	1304	* the mapping so we'll have to attempt to copy
1002	* it */	1305	* it */
1003	ret = (unsigned long) -ENODEV;	1306	ret = (unsigned long) -ENODEV;
1004	if (!(capabilities & BDI_CAP_MAP_COPY))	1307	if (!(capabilities & BDI_CAP_MAP_COPY))
1005	goto error;	1308	goto error_just_free;
1006		1309
1007	capabilities &= ~BDI_CAP_MAP_DIRECT;	1310	capabilities &= ~BDI_CAP_MAP_DIRECT;
		1311	} else {
		1312	vma->vm_start = region->vm_start = addr;
		1313	vma->vm_end = region->vm_end = addr + len;
1008	}	1314	}
1009	}	1315	}
1010	}	1316	}
1011		1317
1012	/* we're going to need a VMA struct as well */	1318	vma->vm_region = region;
1013	vma = kzalloc(sizeof(struct vm_area_struct), GFP_KERNEL);
1014	if (!vma)
1015	goto error_getting_vma;
1016		1319
1017	INIT_LIST_HEAD(&vma->anon_vma_node);
1018	atomic_set(&vma->vm_usage, 1);
1019	if (file) {
1020	get_file(file);
1021	if (vm_flags & VM_EXECUTABLE) {
1022	added_exe_file_vma(current->mm);
1023	vma->vm_mm = current->mm;
1024	}
1025	}
1026	vma->vm_file = file;
1027	vma->vm_flags = vm_flags;
1028	vma->vm_start = addr;
1029	vma->vm_end = addr + len;
1030	vma->vm_pgoff = pgoff;
1031
1032	vml->vma = vma;
1033
1034	/* set up the mapping */	1320	/* set up the mapping */
1035	if (file && vma->vm_flags & VM_SHARED)	1321	if (file && vma->vm_flags & VM_SHARED)
1036	ret = do_mmap_shared_file(vma, len);	1322	ret = do_mmap_shared_file(vma);
1037	else	1323	else
1038	ret = do_mmap_private(vma, len);	1324	ret = do_mmap_private(vma, region, len);
1039	if (ret < 0)	1325	if (ret < 0)
1040	goto error;	1326	goto error_put_region;
1041		1327
		1328	add_nommu_region(region);
		1329
1042	/* okay... we have a mapping; now we have to register it */	1330	/* okay... we have a mapping; now we have to register it */
1043	result = (void *) vma->vm_start;	1331	result = vma->vm_start;
1044		1332
1045	if (vma->vm_flags & VM_MAPPED_COPY) {
1046	realalloc += kobjsize(result);
1047	askedalloc += len;
1048	}
1049
1050	realalloc += kobjsize(vma);
1051	askedalloc += sizeof(*vma);
1052
1053	current->mm->total_vm += len >> PAGE_SHIFT;	1333	current->mm->total_vm += len >> PAGE_SHIFT;
1054		1334
1055	add_nommu_vma(vma);	1335	share:
		1336	add_vma_to_mm(current->mm, vma);
1056		1337
1057	shared:	1338	up_write(&nommu_region_sem);
1058	realalloc += kobjsize(vml);
1059	askedalloc += sizeof(*vml);
1060		1339
1061	add_vma_to_mm(current->mm, vml);
1062
1063	up_write(&nommu_vma_sem);
1064
1065	if (prot & PROT_EXEC)	1340	if (prot & PROT_EXEC)
1066	flush_icache_range((unsigned long) result,	1341	flush_icache_range(result, result + len);
1067	(unsigned long) result + len);
1068		1342
1069	#ifdef DEBUG	1343	kleave(" = %lx", result);
1070	printk("do_mmap:\n");	1344	return result;
1071	show_process_blocks();
1072	#endif
1073		1345
1074	return (unsigned long) result;	1346	error_put_region:
1075		1347	__put_nommu_region(region);
1076	error:
1077	up_write(&nommu_vma_sem);
1078	kfree(vml);
1079	if (vma) {	1348	if (vma) {
1080	if (vma->vm_file) {	1349	if (vma->vm_file) {
1081	fput(vma->vm_file);	1350	fput(vma->vm_file);
1082	if (vma->vm_flags & VM_EXECUTABLE)	1351	if (vma->vm_flags & VM_EXECUTABLE)
1083	removed_exe_file_vma(vma->vm_mm);	1352	removed_exe_file_vma(vma->vm_mm);
1084	}	1353	}
1085	kfree(vma);	1354	kmem_cache_free(vm_area_cachep, vma);
1086	}	1355	}
		1356	kleave(" = %d [pr]", ret);
1087	return ret;	1357	return ret;
1088		1358
1089	sharing_violation:	1359	error_just_free:
1090	up_write(&nommu_vma_sem);	1360	up_write(&nommu_region_sem);
1091	printk("Attempt to share mismatched mappings\n");	1361	error:
1092	kfree(vml);	1362	fput(region->vm_file);
1093	return -EINVAL;	1363	kmem_cache_free(vm_region_jar, region);
		1364	fput(vma->vm_file);
		1365	if (vma->vm_flags & VM_EXECUTABLE)
		1366	removed_exe_file_vma(vma->vm_mm);
		1367	kmem_cache_free(vm_area_cachep, vma);
		1368	kleave(" = %d", ret);
		1369	return ret;
1094		1370
1095	error_getting_vma:	1371	sharing_violation:
1096	up_write(&nommu_vma_sem);	1372	up_write(&nommu_region_sem);
1097	kfree(vml);	1373	printk(KERN_WARNING "Attempt to share mismatched mappings\n");
1098	printk("Allocation of vma for %lu byte allocation from process %d failed\n",	1374	ret = -EINVAL;
		1375	goto error;
		1376
		1377	error_getting_vma:
		1378	kmem_cache_free(vm_region_jar, region);
		1379	printk(KERN_WARNING "Allocation of vma for %lu byte allocation"
		1380	" from process %d failed\n",
1099	len, current->pid);	1381	len, current->pid);
1100	show_free_areas();	1382	show_free_areas();
1101	return -ENOMEM;	1383	return -ENOMEM;
1102		1384
1103	error_getting_vml:	1385	error_getting_region:
1104	printk("Allocation of vml for %lu byte allocation from process %d failed\n",	1386	printk(KERN_WARNING "Allocation of vm region for %lu byte allocation"
		1387	" from process %d failed\n",
1105	len, current->pid);	1388	len, current->pid);
1106	show_free_areas();	1389	show_free_areas();
1107	return -ENOMEM;	1390	return -ENOMEM;
1108	}	1391	}
1109	EXPORT_SYMBOL(do_mmap_pgoff);	1392	EXPORT_SYMBOL(do_mmap_pgoff);
1110		1393
1111	/*	1394	/*
1112	* handle mapping disposal for uClinux	1395	* split a vma into two pieces at address 'addr', a new vma is allocated either
		1396	* for the first part or the tail.
1113	*/	1397	*/
1114	static void put_vma(struct mm_struct mm, struct vm_area_struct vma)	1398	int split_vma(struct mm_struct mm, struct vm_area_struct vma,
		1399	unsigned long addr, int new_below)
1115	{	1400	{
1116	if (vma) {	1401	struct vm_area_struct *new;
1117	down_write(&nommu_vma_sem);	1402	struct vm_region *region;
		1403	unsigned long npages;
1118		1404
1119	if (atomic_dec_and_test(&vma->vm_usage)) {	1405	kenter("");
1120	delete_nommu_vma(vma);
1121		1406
1122	if (vma->vm_ops && vma->vm_ops->close)	1407	/* we're only permitted to split anonymous regions that have a single
1123	vma->vm_ops->close(vma);	1408	* owner */
		1409	if (vma->vm_file \|\|
		1410	atomic_read(&vma->vm_region->vm_usage) != 1)
		1411	return -ENOMEM;
1124		1412
1125	/* IO memory and memory shared directly out of the pagecache from	1413	if (mm->map_count >= sysctl_max_map_count)
1126	* ramfs/tmpfs mustn't be released here */	1414	return -ENOMEM;
1127	if (vma->vm_flags & VM_MAPPED_COPY) {
1128	realalloc -= kobjsize((void *) vma->vm_start);
1129	askedalloc -= vma->vm_end - vma->vm_start;
1130	kfree((void *) vma->vm_start);
1131	}
1132		1415
1133	realalloc -= kobjsize(vma);	1416	region = kmem_cache_alloc(vm_region_jar, GFP_KERNEL);
1134	askedalloc -= sizeof(*vma);	1417	if (!region)
		1418	return -ENOMEM;
1135		1419
1136	if (vma->vm_file) {	1420	new = kmem_cache_alloc(vm_area_cachep, GFP_KERNEL);
1137	fput(vma->vm_file);	1421	if (!new) {
1138	if (vma->vm_flags & VM_EXECUTABLE)	1422	kmem_cache_free(vm_region_jar, region);
1139	removed_exe_file_vma(mm);	1423	return -ENOMEM;
1140	}	1424	}
1141	kfree(vma);
1142	}
1143		1425
1144	up_write(&nommu_vma_sem);	1426	/* most fields are the same, copy all, and then fixup */
		1427	new = vma;
		1428	region = vma->vm_region;
		1429	new->vm_region = region;
		1430
		1431	npages = (addr - vma->vm_start) >> PAGE_SHIFT;
		1432
		1433	if (new_below) {
		1434	region->vm_top = region->vm_end = new->vm_end = addr;
		1435	} else {
		1436	region->vm_start = new->vm_start = addr;
		1437	region->vm_pgoff = new->vm_pgoff += npages;
1145	}	1438	}
		1439
		1440	if (new->vm_ops && new->vm_ops->open)
		1441	new->vm_ops->open(new);
		1442
		1443	delete_vma_from_mm(vma);
		1444	down_write(&nommu_region_sem);
		1445	delete_nommu_region(vma->vm_region);
		1446	if (new_below) {
		1447	vma->vm_region->vm_start = vma->vm_start = addr;
		1448	vma->vm_region->vm_pgoff = vma->vm_pgoff += npages;
		1449	} else {
		1450	vma->vm_region->vm_end = vma->vm_end = addr;
		1451	vma->vm_region->vm_top = addr;
		1452	}
		1453	add_nommu_region(vma->vm_region);
		1454	add_nommu_region(new->vm_region);
		1455	up_write(&nommu_region_sem);
		1456	add_vma_to_mm(mm, vma);
		1457	add_vma_to_mm(mm, new);
		1458	return 0;
1146	}	1459	}
1147		1460
1148	/*	1461	/*
1149	* release a mapping	1462	* shrink a VMA by removing the specified chunk from either the beginning or
1150	* - under NOMMU conditions the parameters must match exactly to the mapping to	1463	* the end
1151	* be removed
1152	*/	1464	*/
1153	int do_munmap(struct mm_struct *mm, unsigned long addr, size_t len)	1465	static int shrink_vma(struct mm_struct *mm,
		1466	struct vm_area_struct *vma,
		1467	unsigned long from, unsigned long to)
1154	{	1468	{
1155	struct vm_list_struct vml, *parent;	1469	struct vm_region *region;
1156	unsigned long end = addr + len;
1157		1470
1158	#ifdef DEBUG	1471	kenter("");
1159	printk("do_munmap:\n");
1160	#endif
1161		1472
1162	for (parent = &mm->context.vmlist; parent; parent = &(parent)->next) {	1473	/* adjust the VMA's pointers, which may reposition it in the MM's tree
1163	if ((*parent)->vma->vm_start > addr)	1474	* and list */
1164	break;	1475	delete_vma_from_mm(vma);
1165	if ((*parent)->vma->vm_start == addr &&	1476	if (from > vma->vm_start)
1166	((len == 0) \|\| ((*parent)->vma->vm_end == end)))	1477	vma->vm_end = from;
1167	goto found;	1478	else
		1479	vma->vm_start = to;
		1480	add_vma_to_mm(mm, vma);
		1481
		1482	/* cut the backing region down to size */
		1483	region = vma->vm_region;
		1484	BUG_ON(atomic_read(&region->vm_usage) != 1);
		1485
		1486	down_write(&nommu_region_sem);
		1487	delete_nommu_region(region);
		1488	if (from > region->vm_start) {
		1489	to = region->vm_top;
		1490	region->vm_top = region->vm_end = from;
		1491	} else {
		1492	region->vm_start = to;
1168	}	1493	}
		1494	add_nommu_region(region);
		1495	up_write(&nommu_region_sem);
1169		1496
1170	printk("munmap of non-mmaped memory by process %d (%s): %p\n",	1497	free_page_series(from, to);
1171	current->pid, current->comm, (void *) addr);	1498	return 0;
1172	return -EINVAL;	1499	}
1173		1500
1174	found:	1501	/*
1175	vml = *parent;	1502	* release a mapping
		1503	* - under NOMMU conditions the chunk to be unmapped must be backed by a single
		1504	* VMA, though it need not cover the whole VMA
		1505	*/
		1506	int do_munmap(struct mm_struct *mm, unsigned long start, size_t len)
		1507	{
		1508	struct vm_area_struct *vma;
		1509	struct rb_node *rb;
		1510	unsigned long end = start + len;
		1511	int ret;
1176		1512
1177	put_vma(mm, vml->vma);	1513	kenter(",%lx,%zx", start, len);
1178		1514
1179	*parent = vml->next;	1515	if (len == 0)
1180	realalloc -= kobjsize(vml);	1516	return -EINVAL;
1181	askedalloc -= sizeof(*vml);
1182	kfree(vml);
1183		1517
1184	update_hiwater_vm(mm);	1518	/* find the first potentially overlapping VMA */
1185	mm->total_vm -= len >> PAGE_SHIFT;	1519	vma = find_vma(mm, start);
		1520	if (!vma) {
		1521	printk(KERN_WARNING
		1522	"munmap of memory not mmapped by process %d (%s):"
		1523	" 0x%lx-0x%lx\n",
		1524	current->pid, current->comm, start, start + len - 1);
		1525	return -EINVAL;
		1526	}
1186		1527
1187	#ifdef DEBUG	1528	/* we're allowed to split an anonymous VMA but not a file-backed one */
1188	show_process_blocks();	1529	if (vma->vm_file) {
1189	#endif	1530	do {
		1531	if (start > vma->vm_start) {
		1532	kleave(" = -EINVAL [miss]");
		1533	return -EINVAL;
		1534	}
		1535	if (end == vma->vm_end)
		1536	goto erase_whole_vma;
		1537	rb = rb_next(&vma->vm_rb);
		1538	vma = rb_entry(rb, struct vm_area_struct, vm_rb);
		1539	} while (rb);
		1540	kleave(" = -EINVAL [split file]");
		1541	return -EINVAL;
		1542	} else {
		1543	/* the chunk must be a subset of the VMA found */
		1544	if (start == vma->vm_start && end == vma->vm_end)
		1545	goto erase_whole_vma;
		1546	if (start < vma->vm_start \|\| end > vma->vm_end) {
		1547	kleave(" = -EINVAL [superset]");
		1548	return -EINVAL;
		1549	}
		1550	if (start & ~PAGE_MASK) {
		1551	kleave(" = -EINVAL [unaligned start]");
		1552	return -EINVAL;
		1553	}
		1554	if (end != vma->vm_end && end & ~PAGE_MASK) {
		1555	kleave(" = -EINVAL [unaligned split]");
		1556	return -EINVAL;
		1557	}
		1558	if (start != vma->vm_start && end != vma->vm_end) {
		1559	ret = split_vma(mm, vma, start, 1);
		1560	if (ret < 0) {
		1561	kleave(" = %d [split]", ret);
		1562	return ret;
		1563	}
		1564	}
		1565	return shrink_vma(mm, vma, start, end);
		1566	}
1190		1567
		1568	erase_whole_vma:
		1569	delete_vma_from_mm(vma);
		1570	delete_vma(mm, vma);
		1571	kleave(" = 0");
1191	return 0;	1572	return 0;
1192	}	1573	}
1193	EXPORT_SYMBOL(do_munmap);	1574	EXPORT_SYMBOL(do_munmap);
1194		1575
1195	asmlinkage long sys_munmap(unsigned long addr, size_t len)	1576	asmlinkage long sys_munmap(unsigned long addr, size_t len)
1196	{	1577	{
1197	int ret;	1578	int ret;
1198	struct mm_struct *mm = current->mm;	1579	struct mm_struct *mm = current->mm;
1199		1580
1200	down_write(&mm->mmap_sem);	1581	down_write(&mm->mmap_sem);
1201	ret = do_munmap(mm, addr, len);	1582	ret = do_munmap(mm, addr, len);
1202	up_write(&mm->mmap_sem);	1583	up_write(&mm->mmap_sem);
1203	return ret;	1584	return ret;
1204	}	1585	}
1205		1586
1206	/*	1587	/*
1207	* Release all mappings	1588	* release all the mappings made in a process's VM space
1208	*/	1589	*/
1209	void exit_mmap(struct mm_struct * mm)	1590	void exit_mmap(struct mm_struct *mm)
1210	{	1591	{
1211	struct vm_list_struct *tmp;	1592	struct vm_area_struct *vma;
1212		1593
1213	if (mm) {	1594	if (!mm)
1214	#ifdef DEBUG	1595	return;
1215	printk("Exit_mmap:\n");
1216	#endif
1217		1596
1218	mm->total_vm = 0;	1597	kenter("");
1219		1598
1220	while ((tmp = mm->context.vmlist)) {	1599	mm->total_vm = 0;
1221	mm->context.vmlist = tmp->next;
1222	put_vma(mm, tmp->vma);
1223		1600
1224	realalloc -= kobjsize(tmp);	1601	while ((vma = mm->mmap)) {
1225	askedalloc -= sizeof(*tmp);	1602	mm->mmap = vma->vm_next;
1226	kfree(tmp);	1603	delete_vma_from_mm(vma);
1227	}	1604	delete_vma(mm, vma);
1228
1229	#ifdef DEBUG
1230	show_process_blocks();
1231	#endif
1232	}	1605	}
		1606
		1607	kleave("");
1233	}	1608	}
1234		1609
1235	unsigned long do_brk(unsigned long addr, unsigned long len)	1610	unsigned long do_brk(unsigned long addr, unsigned long len)
1236	{	1611	{
1237	return -ENOMEM;	1612	return -ENOMEM;
1238	}	1613	}
1239		1614
1240	/*	1615	/*
1241	* expand (or shrink) an existing mapping, potentially moving it at the same	1616	* expand (or shrink) an existing mapping, potentially moving it at the same
1242	* time (controlled by the MREMAP_MAYMOVE flag and available VM space)	1617	* time (controlled by the MREMAP_MAYMOVE flag and available VM space)
1243	*	1618	*
1244	* under NOMMU conditions, we only permit changing a mapping's size, and only	1619	* under NOMMU conditions, we only permit changing a mapping's size, and only
1245	* as long as it stays within the hole allocated by the kmalloc() call in	1620	* as long as it stays within the region allocated by do_mmap_private() and the
1246	* do_mmap_pgoff() and the block is not shareable	1621	* block is not shareable
1247	*	1622	*
1248	* MREMAP_FIXED is not supported under NOMMU conditions	1623	* MREMAP_FIXED is not supported under NOMMU conditions
1249	*/	1624	*/
1250	unsigned long do_mremap(unsigned long addr,	1625	unsigned long do_mremap(unsigned long addr,
1251	unsigned long old_len, unsigned long new_len,	1626	unsigned long old_len, unsigned long new_len,
1252	unsigned long flags, unsigned long new_addr)	1627	unsigned long flags, unsigned long new_addr)
1253	{	1628	{
1254	struct vm_area_struct *vma;	1629	struct vm_area_struct *vma;
1255		1630
1256	/* insanity checks first */	1631	/* insanity checks first */
1257	if (new_len == 0)	1632	if (old_len == 0 \|\| new_len == 0)
1258	return (unsigned long) -EINVAL;	1633	return (unsigned long) -EINVAL;
1259		1634
		1635	if (addr & ~PAGE_MASK)
		1636	return -EINVAL;
		1637
1260	if (flags & MREMAP_FIXED && new_addr != addr)	1638	if (flags & MREMAP_FIXED && new_addr != addr)
1261	return (unsigned long) -EINVAL;	1639	return (unsigned long) -EINVAL;
1262		1640
1263	vma = find_vma_exact(current->mm, addr);	1641	vma = find_vma_exact(current->mm, addr, old_len);
1264	if (!vma)	1642	if (!vma)
1265	return (unsigned long) -EINVAL;	1643	return (unsigned long) -EINVAL;
1266		1644
1267	if (vma->vm_end != vma->vm_start + old_len)	1645	if (vma->vm_end != vma->vm_start + old_len)
1268	return (unsigned long) -EFAULT;	1646	return (unsigned long) -EFAULT;
1269		1647
1270	if (vma->vm_flags & VM_MAYSHARE)	1648	if (vma->vm_flags & VM_MAYSHARE)
1271	return (unsigned long) -EPERM;	1649	return (unsigned long) -EPERM;
1272		1650
1273	if (new_len > kobjsize((void *) addr))	1651	if (new_len > vma->vm_region->vm_end - vma->vm_region->vm_start)
1274	return (unsigned long) -ENOMEM;	1652	return (unsigned long) -ENOMEM;
1275		1653
1276	/* all checks complete - do it */	1654	/* all checks complete - do it */
1277	vma->vm_end = vma->vm_start + new_len;	1655	vma->vm_end = vma->vm_start + new_len;
1278
1279	askedalloc -= old_len;
1280	askedalloc += new_len;
1281
1282	return vma->vm_start;	1656	return vma->vm_start;
1283	}	1657	}
1284	EXPORT_SYMBOL(do_mremap);	1658	EXPORT_SYMBOL(do_mremap);
1285		1659
1286	asmlinkage unsigned long sys_mremap(unsigned long addr,	1660	asmlinkage
1287	unsigned long old_len, unsigned long new_len,	1661	unsigned long sys_mremap(unsigned long addr,
1288	unsigned long flags, unsigned long new_addr)	1662	unsigned long old_len, unsigned long new_len,
		1663	unsigned long flags, unsigned long new_addr)
1289	{	1664	{
1290	unsigned long ret;	1665	unsigned long ret;
1291		1666
1292	down_write(&current->mm->mmap_sem);	1667	down_write(&current->mm->mmap_sem);
1293	ret = do_mremap(addr, old_len, new_len, flags, new_addr);	1668	ret = do_mremap(addr, old_len, new_len, flags, new_addr);
1294	up_write(&current->mm->mmap_sem);	1669	up_write(&current->mm->mmap_sem);
1295	return ret;	1670	return ret;
1296	}	1671	}
1297		1672
1298	struct page follow_page(struct vm_area_struct vma, unsigned long address,	1673	struct page follow_page(struct vm_area_struct vma, unsigned long address,
1299	unsigned int foll_flags)	1674	unsigned int foll_flags)
1300	{	1675	{
1301	return NULL;	1676	return NULL;
1302	}	1677	}
1303		1678
1304	int remap_pfn_range(struct vm_area_struct *vma, unsigned long from,	1679	int remap_pfn_range(struct vm_area_struct *vma, unsigned long from,
1305	unsigned long to, unsigned long size, pgprot_t prot)	1680	unsigned long to, unsigned long size, pgprot_t prot)
1306	{	1681	{
1307	vma->vm_start = vma->vm_pgoff << PAGE_SHIFT;	1682	vma->vm_start = vma->vm_pgoff << PAGE_SHIFT;
1308	return 0;	1683	return 0;
1309	}	1684	}
1310	EXPORT_SYMBOL(remap_pfn_range);	1685	EXPORT_SYMBOL(remap_pfn_range);
1311		1686
1312	int remap_vmalloc_range(struct vm_area_struct vma, void addr,	1687	int remap_vmalloc_range(struct vm_area_struct vma, void addr,
1313	unsigned long pgoff)	1688	unsigned long pgoff)
1314	{	1689	{
1315	unsigned int size = vma->vm_end - vma->vm_start;	1690	unsigned int size = vma->vm_end - vma->vm_start;
1316		1691
1317	if (!(vma->vm_flags & VM_USERMAP))	1692	if (!(vma->vm_flags & VM_USERMAP))
1318	return -EINVAL;	1693	return -EINVAL;
1319		1694
1320	vma->vm_start = (unsigned long)(addr + (pgoff << PAGE_SHIFT));	1695	vma->vm_start = (unsigned long)(addr + (pgoff << PAGE_SHIFT));
1321	vma->vm_end = vma->vm_start + size;	1696	vma->vm_end = vma->vm_start + size;
1322		1697
1323	return 0;	1698	return 0;
1324	}	1699	}
1325	EXPORT_SYMBOL(remap_vmalloc_range);	1700	EXPORT_SYMBOL(remap_vmalloc_range);
1326		1701
1327	void swap_unplug_io_fn(struct backing_dev_info bdi, struct page page)	1702	void swap_unplug_io_fn(struct backing_dev_info bdi, struct page page)
1328	{	1703	{
1329	}	1704	}
1330		1705
1331	unsigned long arch_get_unmapped_area(struct file *file, unsigned long addr,	1706	unsigned long arch_get_unmapped_area(struct file *file, unsigned long addr,
1332	unsigned long len, unsigned long pgoff, unsigned long flags)	1707	unsigned long len, unsigned long pgoff, unsigned long flags)
1333	{	1708	{
1334	return -ENOMEM;	1709	return -ENOMEM;
1335	}	1710	}
1336		1711
1337	void arch_unmap_area(struct mm_struct *mm, unsigned long addr)	1712	void arch_unmap_area(struct mm_struct *mm, unsigned long addr)
1338	{	1713	{
1339	}	1714	}
1340		1715
1341	void unmap_mapping_range(struct address_space *mapping,	1716	void unmap_mapping_range(struct address_space *mapping,
1342	loff_t const holebegin, loff_t const holelen,	1717	loff_t const holebegin, loff_t const holelen,
1343	int even_cows)	1718	int even_cows)
1344	{	1719	{
1345	}	1720	}
1346	EXPORT_SYMBOL(unmap_mapping_range);	1721	EXPORT_SYMBOL(unmap_mapping_range);
1347		1722
1348	/*	1723	/*
1349	* ask for an unmapped area at which to create a mapping on a file	1724	* ask for an unmapped area at which to create a mapping on a file
1350	*/	1725	*/
1351	unsigned long get_unmapped_area(struct file *file, unsigned long addr,	1726	unsigned long get_unmapped_area(struct file *file, unsigned long addr,
1352	unsigned long len, unsigned long pgoff,	1727	unsigned long len, unsigned long pgoff,
1353	unsigned long flags)	1728	unsigned long flags)
1354	{	1729	{
1355	unsigned long (get_area)(struct file , unsigned long, unsigned long,	1730	unsigned long (get_area)(struct file , unsigned long, unsigned long,
1356	unsigned long, unsigned long);	1731	unsigned long, unsigned long);
1357		1732
1358	get_area = current->mm->get_unmapped_area;	1733	get_area = current->mm->get_unmapped_area;
1359	if (file && file->f_op && file->f_op->get_unmapped_area)	1734	if (file && file->f_op && file->f_op->get_unmapped_area)
1360	get_area = file->f_op->get_unmapped_area;	1735	get_area = file->f_op->get_unmapped_area;
1361		1736
1362	if (!get_area)	1737	if (!get_area)
1363	return -ENOSYS;	1738	return -ENOSYS;
1364		1739
1365	return get_area(file, addr, len, pgoff, flags);	1740	return get_area(file, addr, len, pgoff, flags);
1366	}	1741	}
1367	EXPORT_SYMBOL(get_unmapped_area);	1742	EXPORT_SYMBOL(get_unmapped_area);
1368		1743
1369	/*	1744	/*
1370	* Check that a process has enough memory to allocate a new virtual	1745	* Check that a process has enough memory to allocate a new virtual
1371	* mapping. 0 means there is enough memory for the allocation to	1746	* mapping. 0 means there is enough memory for the allocation to
1372	* succeed and -ENOMEM implies there is not.	1747	* succeed and -ENOMEM implies there is not.
1373	*	1748	*
1374	* We currently support three overcommit policies, which are set via the	1749	* We currently support three overcommit policies, which are set via the
1375	* vm.overcommit_memory sysctl. See Documentation/vm/overcommit-accounting	1750	* vm.overcommit_memory sysctl. See Documentation/vm/overcommit-accounting
1376	*	1751	*
1377	* Strict overcommit modes added 2002 Feb 26 by Alan Cox.	1752	* Strict overcommit modes added 2002 Feb 26 by Alan Cox.
1378	* Additional code 2002 Jul 20 by Robert Love.	1753	* Additional code 2002 Jul 20 by Robert Love.
1379	*	1754	*
1380	* cap_sys_admin is 1 if the process has admin privileges, 0 otherwise.	1755	* cap_sys_admin is 1 if the process has admin privileges, 0 otherwise.
1381	*	1756	*
1382	* Note this is a helper function intended to be used by LSMs which	1757	* Note this is a helper function intended to be used by LSMs which
1383	* wish to use this logic.	1758	* wish to use this logic.
1384	*/	1759	*/
1385	int __vm_enough_memory(struct mm_struct *mm, long pages, int cap_sys_admin)	1760	int __vm_enough_memory(struct mm_struct *mm, long pages, int cap_sys_admin)
1386	{	1761	{
1387	unsigned long free, allowed;	1762	unsigned long free, allowed;
1388		1763
1389	vm_acct_memory(pages);	1764	vm_acct_memory(pages);
1390		1765
1391	/*	1766	/*
1392	* Sometimes we want to use more memory than we have	1767	* Sometimes we want to use more memory than we have
1393	*/	1768	*/
1394	if (sysctl_overcommit_memory == OVERCOMMIT_ALWAYS)	1769	if (sysctl_overcommit_memory == OVERCOMMIT_ALWAYS)
1395	return 0;	1770	return 0;
1396		1771
1397	if (sysctl_overcommit_memory == OVERCOMMIT_GUESS) {	1772	if (sysctl_overcommit_memory == OVERCOMMIT_GUESS) {
1398	unsigned long n;	1773	unsigned long n;
1399		1774
1400	free = global_page_state(NR_FILE_PAGES);	1775	free = global_page_state(NR_FILE_PAGES);
1401	free += nr_swap_pages;	1776	free += nr_swap_pages;
1402		1777
1403	/*	1778	/*
1404	* Any slabs which are created with the	1779	* Any slabs which are created with the
1405	* SLAB_RECLAIM_ACCOUNT flag claim to have contents	1780	* SLAB_RECLAIM_ACCOUNT flag claim to have contents
1406	* which are reclaimable, under pressure. The dentry	1781	* which are reclaimable, under pressure. The dentry
1407	* cache and most inode caches should fall into this	1782	* cache and most inode caches should fall into this
1408	*/	1783	*/
1409	free += global_page_state(NR_SLAB_RECLAIMABLE);	1784	free += global_page_state(NR_SLAB_RECLAIMABLE);
1410		1785
1411	/*	1786	/*
1412	* Leave the last 3% for root	1787	* Leave the last 3% for root
1413	*/	1788	*/
1414	if (!cap_sys_admin)	1789	if (!cap_sys_admin)
1415	free -= free / 32;	1790	free -= free / 32;
1416		1791
1417	if (free > pages)	1792	if (free > pages)
1418	return 0;	1793	return 0;
1419		1794
1420	/*	1795	/*
1421	* nr_free_pages() is very expensive on large systems,	1796	* nr_free_pages() is very expensive on large systems,